From 92d6c444993beabc47630cdb18c27e086291ecb0 Mon Sep 17 00:00:00 2001 From: Nazneen Rajani Date: Thu, 5 May 2022 18:21:25 +0000 Subject: [PATCH] Autoeval config (#4234) * autoeval config added * autoeval config added * Added autonlp config changes https://github.com/huggingface/autonlp-backend/issues/414 * multi-input text classification as task id instead of category * improve metadata validation: - support YAML keys with dashes - add train-eval-index validation * revert debugging stuff * fix tests * style * Update metadata.py * Update metadata.py Co-authored-by: Quentin Lhoest Co-authored-by: Quentin Lhoest <42851186+lhoestq@users.noreply.github.com> Commit from https://github.com/huggingface/datasets/commit/6af556b64d54374133fe7bf3f1ffba1d82c49269 --- README.md | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/README.md b/README.md index bd2acd0..a9ca8f3 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,55 @@ task_categories: task_ids: - sentiment-classification paperswithcode_id: imdb-movie-reviews +train-eval-index: +- config: plain_text + task: text-classification + task_id: binary_classification + splits: + train_split: train + eval_split: test + col_mapping: + text: text + label: target + metrics: + - type: accuracy + - name: Accuracy + - type: f1 + name: F1 macro + args: + average: macro + - type: f1 + name: F1 micro + args: + average: micro + - type: f1 + name: F1 weighted + args: + average: weighted + - type: precision + name: Precision macro + args: + average: macro + - type: precision + name: Precision micro + args: + average: micro + - type: precision + name: Precision weighted + args: + average: weighted + - type: recall + name: Recall macro + args: + average: macro + - type: recall + name: Recall micro + args: + average: micro + - type: recall + name: Recall weighted + args: + average: weighted --- # Dataset Card for "imdb"