Core Concepts
OmniJet-α is a groundbreaking foundation model for particle physics, enabling transfer learning between different classes of tasks and demonstrating the potential for significant advancements in physics performance and computational efficiency.
Abstract
I. Introduction:
Foundation models are multi-dataset and multi-task machine learning methods.
They aim to improve physics performance while reducing training time and data requirements.
II. Methods and Dataset:
A. Dataset:
Utilizes the JetClass dataset containing jet-level and constituent-level features.
B. Jet Constituent Token Creation:
Explores binned, conditional, and unconditional tokenization approaches.
C. Transformer Backbone:
OmniJet-α utilizes a transformer backbone based on GPT transformer decoder model.
III. Results:
A. Token Quality:
Conditional tokenization with 8192 tokens shows improved resolution over other approaches.
B. Jet Generation:
Generated jets match truth level tokens well, with slight discrepancies in pT spectrum tails.
C. Transfer Learning from Generation to Classification:
Fine-tuning the generative model leads to significant gains in classification accuracy compared to training from scratch.
IV. Conclusion:
OmniJet-α represents a crucial step towards building foundation models for particle physics, showcasing its potential for generalization across tasks and classes.
Stats
OmniJet-αは、物理学の基礎モデルとして画期的な進歩を示し、異なるタスク間での転移学習を可能にする。