TeenyTinyLlama: Open-Source Compact Language Models for Brazilian Portuguese Text Generation
This study documents the development of two open-source compact language models, the TeenyTinyLlama (TTL) pair, tailored for low-resource settings and trained solely on Brazilian Portuguese text.
Efficient Encoder Models for Closely-Related Languages via Additional Pretraining of Multilingual Language Models
Comparable performance to dedicated from-scratch models can be obtained by additionally pretraining available multilingual models even with a limited amount of computation.
Enhancing Cross-Lingual Transfer for Low-Resource Languages in Large Language Models through Translation-Assisted Chain-of-Thought Processes
The paper proposes a novel method called TaCo (Translation-Assisted Cross-Linguality) that utilizes translations in a chain-of-thought process to efficiently instruction-tune large language models on new languages, especially low-resource ones, through a curriculum-learning approach.
Ziya2: A Data-Centric Approach to Enhance Large Language Models' Capabilities in Chinese, Mathematics, and Programming
Ziya2, a 13-billion-parameter language model, is developed through a data-centric approach that focuses on optimizing the use of pre-training data to enhance the model's capabilities in Chinese, mathematics, and programming tasks, while maintaining or improving its performance on general English benchmarks.
Sailor: Open Language Models Tailored for South-East Asian Languages
Sailor is a family of open language models ranging from 0.5B to 7B parameters, designed to perform well across South-East Asian languages including English, Chinese, Vietnamese, Thai, Indonesian, Malay, and Lao.
TeleChat: A Comprehensive Large Language Model with Extensive Pretraining and Supervised Fine-Tuning for Conversational AI
TeleChat is a suite of large language models (LLMs) with 3 billion, 7 billion, and 12 billion parameters, developed through extensive pretraining on a diverse corpus and supervised fine-tuning to align with human preferences for conversational AI applications.
HyperCLOVA X: A Powerful Korean-Centric Language Model with Multilingual Capabilities
HyperCLOVA X is a family of large language models tailored to the Korean language and culture, while also exhibiting strong performance in English, math, and coding.
© 2024 by Linnk AI