From 12e974610ba9083ed95f3edf08d7e899581f4de4 Mon Sep 17 00:00:00 2001 From: Xiao Date: Tue, 19 Mar 2024 09:26:24 +0000 Subject: [PATCH] Upload README.md with huggingface_hub --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 91d5b5f..1a57918 100644 --- a/README.md +++ b/README.md @@ -325,7 +325,8 @@ torchrun --nproc_per_node {number of gpus} \ --target_modules q_proj k_proj v_proj o_proj \ --start_layer 8 \ --head_multi True \ ---head_type simple +--head_type simple \ +--lora_extra_parameters linear_head ``` Our rerankers are initialized from [google/gemma-2b](https://huggingface.co/google/gemma-2b) (for llm-based reranker) and [openbmb/MiniCPM-2B-dpo-bf16](https://huggingface.co/openbmb/MiniCPM-2B-dpo-bf16) (for llm-based layerwise reranker), and we train it on a mixture of multilingual datasets: