분류 전체보기
-
초록) Toward Human-Like Evaluation for Natural Language Generation withError Analysis (2023)논문 리뷰/초록 찍먹 2023. 7. 21. 15:37
The pretrained language model (PLM) based metrics have been successfully used in evaluatign language generation tasks. Recent studies of the human evaluation community show that considering both major errors (e.g. mistranslated tokens) and minor errors (e.g. imperfections in fluency) can produce high-quality judgments. This inspires us to approach the final goal of the automatic metrics (human-l..
-
초록) Can Large Language Models Be an Alternative to Human Evaluation?(2023)논문 리뷰/초록 찍먹 2023. 7. 21. 13:29
Human evaluation is indispensable and inevitable for assessing the quality of texts generated by machine learning models or written by humans. However, human evaluation is very difficult to reproduce and its quality is notoriously unstable, hindering fair comparisons among different natural language processing(NLP) models and algorithms. Recently, large language models (LLMs) have demonstrated e..
-
치매(Dementia)로 인한 언어장애, 경남도민신문, 2013.01.09음성/dementia ASR 2023. 7. 12. 21:09
알츠하이머병 치매의 언어특징은 초기에 언어는 인지에 비해 영향을 적게 받는다. 대부분 언어의 양상들이 말소리의 조음, 음성에 영향을 주지만 자동 구어와 같은 구문론적인 양상들은 영향을 받지 않는다. 명명하기(자주 사용하지 않은 단어에서 더 현저함)어려움을 보이거나, 구어 착어증, 장황함, 추상적 명칭, 함축된 의미, 유머의 이해 등의 곤란, 그림묘사의 어려움, 주제 유지의 어려움, 말의 반복, 언어 이해의 손상 등이 나타난다. 중기에는 음소적 착어증, 내용 없는 말이 늘어나고 자곤이 나타난다. 말의 속도가 빨라지는 경향이 있으며 대화가 빨리 진행되면 단어 사용의 혼란을 나타내고 청지각 능력과 읽기 이해의 손상이 나타나나 소리 내서 읽기에는 어려움을 나타내지 않는다. 반향어, 단어의 마지막 음절을 반복, ..
-
-
초록) Eight Things to Know about Large Language Models논문 리뷰/초록 찍먹 2023. 4. 18. 21:08
원문 The widespread public deployment of large languaeg models (LLMs) in recent moths has prompted a wave of new attention and engagement from advocates, policymakers, and scholars from many fields. This attention is a timely response to the many urgent questions that this technology raises, but it can sometimes miss important considerations. This paper surveys the evidence for eight potentially s..
-
초록) Training language models to follow instructions with human feedback논문 리뷰/초록 찍먹 2023. 4. 17. 10:41
* InstructGPT 이야기 Making language models bigger does not inherently make them better at following a user's intent. For example, large language models can generate outputs that are untruthful, toxic, or simply not helpful to ther user. In other words, there models are not aligned with theri users. In this paper, we show an avenue for aligning language models with user intent on a wide range of ta..
-
NLG) BERTScoreNLP Evaluation/metrics 2023. 3. 7. 11:55
Huggingface에 metrics를 묶어서 소개하는 페이지입니다. * https://huggingface.co/evaluate-metric evaluate-metric (Evaluate Metric) 🤗 Evaluate provides access to a wide range of evaluation tools. It covers a range of modalities such as text, computer vision, audio, etc. as well as tools to evaluate models or datasets. It has three types of evaluations: Metric: measures the performan huggingface.co 1. BERTScore 개요..
-
GLUE(General Language Understanding Evaluation) 소개NLP Evaluation/Benchmarks 2022. 11. 6. 17:27
1. GLUE * paper: https://openreview.net/pdf?id=rJ4km2R5t7 * site: https://gluebenchmark.com * huggingface: https://huggingface.co/datasets/glue GLUE Benchmark The General Language Understanding Evaluation (GLUE) benchmark is a collection of resources for training, evaluating, and analyzing natural language understanding systems gluebenchmark.com 2. GLUE의 세부 task(9) 1) Single Sentence Task (1) Co..