NLP Evaluation/Benchmarks
-
CoLA) Neural Network Acceptability JudgmentsNLP Evaluation/Benchmarks 2023. 11. 12. 18:36
CoLA: Corpus of Linguistic Acceptability - 10,657개 영어 문장: 여러 언어학 문헌(Ex. 문법책)에서 발췌 & grammatical / ungrammatical로 라벨링 1. Introduction ●“Acceptability judgements”: 사람의 문법적 지식을 관찰하기 위한 생성문법학자들의 가장 기본적이고 행동적인 측정 방법(Chomsky 1957; Schuetze, 1996) ● 신경망에서의 acceptability judgements: 신경망이 **문법적 개념을 익혔는지**를 판단, 이때 문법적 개념은 **인간의 언어적 능력 측면**에서의 개념) 2. Acceptability Judgements 2.1. In Linguistics ● 주로 생성문법(g..
-
GLUE(General Language Understanding Evaluation) 소개NLP Evaluation/Benchmarks 2022. 11. 6. 17:27
1. GLUE * paper: https://openreview.net/pdf?id=rJ4km2R5t7 * site: https://gluebenchmark.com * huggingface: https://huggingface.co/datasets/glue GLUE Benchmark The General Language Understanding Evaluation (GLUE) benchmark is a collection of resources for training, evaluating, and analyzing natural language understanding systems gluebenchmark.com 2. GLUE의 세부 task(9) 1) Single Sentence Task (1) Co..