논문 리뷰/초록 찍먹
-
초록) Probing LLMs for Joint Encoding of Linguistic Categories논문 리뷰/초록 찍먹 2023. 11. 12. 18:30
Large Language Models (LLMs) exhibit impressive performance on a range of NLP tasks, due to the general-purpose linguistic knowledge acquired during pretraining. Existing model interpretability research (tenney et al., 2019) suggests that a linguistic hierarchy emerges in the LLM layers, with lower layers better suited to solving syntactic tasks and higher layers employed for semantic processing..
-
초록) Are Human Explanations Always Helpful? Towards Objective Evaluation of Human Natural Language Explanations (2023)논문 리뷰/초록 찍먹 2023. 7. 21. 19:22
Human-annotated labels and explanations are critical for training explainable NLP models. However, unlike human-annotated labels whose quality is easier to calibrate (e.g., with a majority vote, human-crafted free-form explanations) can be quite subjective. Before blindly using them as ground truth to train ML models, a vital question needs to be asked: How do we evaluate a human-annotated expla..
-
초록) A fine-grained comparison of pragmatic language understanding in humans and language models논문 리뷰/초록 찍먹 2023. 7. 21. 16:59
Pragmatics and non-literal language understanding are essential to human communication, and present a long-standing challenge for artificial language models. We perform a fine-grained comparison of language models and humans on seven pragmatic phenomena, using zero-shot prompting on a expert-curated set of English materials. We ask whether models (1) select pragmatic interpretations of speaker u..
-
초록) Toward Human-Like Evaluation for Natural Language Generation withError Analysis (2023)논문 리뷰/초록 찍먹 2023. 7. 21. 15:37
The pretrained language model (PLM) based metrics have been successfully used in evaluatign language generation tasks. Recent studies of the human evaluation community show that considering both major errors (e.g. mistranslated tokens) and minor errors (e.g. imperfections in fluency) can produce high-quality judgments. This inspires us to approach the final goal of the automatic metrics (human-l..
-
초록) Can Large Language Models Be an Alternative to Human Evaluation?(2023)논문 리뷰/초록 찍먹 2023. 7. 21. 13:29
Human evaluation is indispensable and inevitable for assessing the quality of texts generated by machine learning models or written by humans. However, human evaluation is very difficult to reproduce and its quality is notoriously unstable, hindering fair comparisons among different natural language processing(NLP) models and algorithms. Recently, large language models (LLMs) have demonstrated e..
-
초록) Eight Things to Know about Large Language Models논문 리뷰/초록 찍먹 2023. 4. 18. 21:08
원문 The widespread public deployment of large languaeg models (LLMs) in recent moths has prompted a wave of new attention and engagement from advocates, policymakers, and scholars from many fields. This attention is a timely response to the many urgent questions that this technology raises, but it can sometimes miss important considerations. This paper surveys the evidence for eight potentially s..
-
초록) Training language models to follow instructions with human feedback논문 리뷰/초록 찍먹 2023. 4. 17. 10:41
* InstructGPT 이야기 Making language models bigger does not inherently make them better at following a user's intent. For example, large language models can generate outputs that are untruthful, toxic, or simply not helpful to ther user. In other words, there models are not aligned with theri users. In this paper, we show an avenue for aligning language models with user intent on a wide range of ta..