
Smart-Infinity addresses storage bandwidth bottleneck in large language model training using near-storage processing devices.


coremsg

smart-infinity-fast-large-language-model-training-with-near-storage-processing


Smart-Infinity: Fast Large Language Model Training with Near-Storage Processing


title_rewrite


This paper introduces a novel training paradigm for updating large language models (LLMs) that balances pre-training performance with reduced training cost by strategically switching learning rates.


a-learning-rate-path-switching-training-paradigm-for-efficient-version-updates-of-large-language-models


A Learning Rate Path Switching Training Paradigm for Efficient Version Updates of Large Language Models



Programmatically generated training data based on simple patterns over random tokens can effectively improve the generative capabilities of large language models (LLMs) on various natural language tasks.


cookbook-a-framework-for-enhancing-large-language-model-generative-abilities-using-programmatically-generated-data-templates


COOKBOOK: A Framework for Enhancing Large Language Model Generative Abilities Using Programmatically Generated Data Templates
