Evaluation of Large Language Models for Programming Code Generation
GPT-4 outperforms other models in generating code, showcasing potential as a reliable programming assistant.
LIEDER: Linguistically-Informed Evaluation for Discourse Entity Recognition
The author proposes the LIEDER dataset to assess language models' knowledge of semantic properties in discourse entity recognition, revealing deficiencies in understanding the NOVELTY requirement. Despite mastering EXISTENCE, UNIQUENESS, and PLURALITY, large language models lack awareness of NOVELTY.
LIEDER: Linguistically-Informed Evaluation for Discourse Entity Recognition
言語モデルのDE認識能力を評価する新しい方法を提案しました。
Analyzing Language Models for Text Classification: In-Context Learning Evaluation
Large language models outperform few-shot approaches in text classification, highlighting the importance of prompt design and model size.
Assessing Large Language Models in Translating Formal Specifications
Large Language Models struggle in translating formal specifications accurately, limiting their utility in system design.
Rigorous Test and Evaluation Methodology for Assessing Language Model Capabilities
A principled methodology called TEL'M (Test and Evaluation of Language Models) is proposed to rigorously assess the capabilities of current and future language models across high-value commercial, government and national security applications.
Prometheus 2: An Open-Source Language Model for Evaluating Other Language Models
Prometheus 2 is a state-of-the-art open-source language model that closely mirrors human and proprietary language model judgments for evaluating the quality of responses from various language models.
© 2024 by Linnk AI