add text-embeddings-inference tag for easier endpoint deployment (#19)

- add text-embeddings-inference tag for easier endpoint deployment (79c481748842b7efa0a12db59915db91731f0b93)


Co-authored-by: Moritz Laurer <MoritzLaurer@users.noreply.huggingface.co>

README.md

@@ -4,6 +4,7 @@ pipeline_tag: text-classification
 tags:
 - transformers
 - sentence-transformers
+- text-embeddings-inference
 language:
 - multilingual
 ---
@@ -30,8 +31,8 @@ And the score can be mapped to a float value in [0,1] by sigmoid function.
 | [BAAI/bge-reranker-base](https://huggingface.co/BAAI/bge-reranker-base) | [xlm-roberta-base](https://huggingface.co/xlm-roberta-base) | Chinese and English |     -     | Lightweight reranker model, easy to deploy, with fast inference. |
 | [BAAI/bge-reranker-large](https://huggingface.co/BAAI/bge-reranker-large) | [xlm-roberta-large](https://huggingface.co/FacebookAI/xlm-roberta-large) | Chinese and English |     -     | Lightweight reranker model, easy to deploy, with fast inference. |
 | [BAAI/bge-reranker-v2-m3](https://huggingface.co/BAAI/bge-reranker-v2-m3) | [bge-m3](https://huggingface.co/BAAI/bge-m3) |    Multilingual     |     -     | Lightweight reranker model, possesses strong multilingual capabilities, easy to deploy, with fast inference. |
-| [BAAI/bge-reranker-v2-gemma](https://huggingface.co/BAAI/bge-reranker-v2-gemma) | [google/gemma-2b](https://huggingface.co/google/gemma-2b) |    Multilingual     |     -     | Suitable for multilingual contexts, performs well in both English proficiency and multilingual capabilities. |
-| [BAAI/bge-reranker-v2-minicpm-layerwise](https://huggingface.co/BAAI/bge-reranker-v2-minicpm-layerwise) | [openbmb/MiniCPM-2B-dpo-fp16](https://huggingface.co/openbmb/MiniCPM-2B-dpo-fp16/tree/main) |    Multilingual     |   8-40    | Suitable for multilingual contexts, performs well in both English and Chinese proficiency, allows freedom to select layers for output, facilitating accelerated inference. |
+| [BAAI/bge-reranker-v2-gemma](https://huggingface.co/BAAI/bge-reranker-v2-gemma) |      [gemma-2b](https://huggingface.co/google/gemma-2b)      |    Multilingual     |     -     | Suitable for multilingual contexts, performs well in both English proficiency and multilingual capabilities. |
+| [BAAI/bge-reranker-v2-minicpm-layerwise](https://huggingface.co/BAAI/bge-reranker-v2-minicpm-layerwise) | [MiniCPM-2B-dpo-bf16](https://huggingface.co/openbmb/MiniCPM-2B-dpo-bf16) |    Multilingual     |   8-40    | Suitable for multilingual contexts, performs well in both English and Chinese proficiency, allows freedom to select layers for output, facilitating accelerated inference. |
 
 
 You can select the model according your senario and resource. 
@@ -77,26 +78,28 @@ print(scores) # [0.00027803096387751553, 0.9948403768236574]
 
 ```python
 from FlagEmbedding import FlagLLMReranker
-reranker = FlagLLMReranker('BAAI/bge-reranker-v2-gemma', use_bf16=True) # Setting use_bf16 to True speeds up computation with a slight performance degradation
+reranker = FlagLLMReranker('BAAI/bge-reranker-v2-gemma', use_fp16=True) # Setting use_fp16 to True speeds up computation with a slight performance degradation
+# reranker = FlagLLMReranker('BAAI/bge-reranker-v2-gemma', use_bf16=True) # You can also set use_bf16=True to speed up computation with a slight performance degradation
 
 score = reranker.compute_score(['query', 'passage'])
-print(score) # 2.15625
+print(score)
 
 scores = reranker.compute_score([['what is panda?', 'hi'], ['what is panda?', 'The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear or simply panda, is a bear species endemic to China.']])
-print(scores) # [-0.84765625, 10.625]
+print(scores)
 ```
 
 #### For LLM-based layerwise reranker
 
 ```python
 from FlagEmbedding import LayerWiseFlagLLMReranker
-reranker = LayerWiseFlagLLMReranker('BAAI/bge-reranker-v2-minicpm-layerwise', use_bf16=True) # Setting use_bf16 to True speeds up computation with a slight performance degradation
+reranker = LayerWiseFlagLLMReranker('BAAI/bge-reranker-v2-minicpm-layerwise', use_fp16=True) # Setting use_fp16 to True speeds up computation with a slight performance degradation
+# reranker = LayerWiseFlagLLMReranker('BAAI/bge-reranker-v2-minicpm-layerwise', use_bf16=True) # You can also set use_bf16=True to speed up computation with a slight performance degradation
 
 score = reranker.compute_score(['query', 'passage'], cutoff_layers=[28]) # Adjusting 'cutoff_layers' to pick which layers are used for computing the score.
-print(score) # -7.03125
+print(score)
 
 scores = reranker.compute_score([['what is panda?', 'hi'], ['what is panda?', 'The giant panda (Ailuropoda melanoleuca), sometimes called a panda bear or simply panda, is a bear species endemic to China.']], cutoff_layers=[28])
-print(scores) # [-10.0, 1.8203125]
+print(scores)
 ```
 
 ### Using Huggingface transformers
@@ -230,7 +233,7 @@ def get_inputs(pairs, tokenizer, prompt=None, max_length=1024):
             return_tensors='pt',
     )
 
-tokenizer = AutoTokenizer.from_pretrained('BAAI/bge-reranker-v2-minicpm-layerwise', trust_remote_code=True, torch_dtype=torch.bfloat16)
+tokenizer = AutoTokenizer.from_pretrained('BAAI/bge-reranker-v2-minicpm-layerwise', trust_remote_code=True)
 model = AutoModelForCausalLM.from_pretrained('BAAI/bge-reranker-v2-minicpm-layerwise', trust_remote_code=True, torch_dtype=torch.bfloat16)
 model = model.to('cuda')
 model.eval()
@@ -245,6 +248,20 @@ with torch.no_grad():
 
 ## Fine-tune
 
+### Data Format
+
+Train data should be a json file, where each line is a dict like this:
+
+```
+{"query": str, "pos": List[str], "neg":List[str], "prompt": str}
+```
+
+`query` is the query, and `pos` is a list of positive texts, `neg` is a list of negative texts, `prompt` indicates the relationship between query and texts. If you have no negative texts for a query, you can random sample some from the entire corpus as the negatives.
+
+See [toy_finetune_data.jsonl](https://github.com/FlagOpen/FlagEmbedding/tree/master/FlagEmbedding/llm_reranker/toy_finetune_data.jsonl) for a toy data file.
+
+### Train
+
 You can fine-tune the reranker with the following code:
 
 **For llm-based reranker**
@@ -253,7 +270,7 @@ You can fine-tune the reranker with the following code:
 torchrun --nproc_per_node {number of gpus} \
 -m FlagEmbedding.llm_reranker.finetune_for_instruction.run \
 --output_dir {path to save model} \
---model_name_or_path BAAI/bge-reranker-v2-gemma \
+--model_name_or_path google/gemma-2b \
 --train_data ./toy_finetune_data.jsonl \
 --learning_rate 2e-4 \
 --num_train_epochs 1 \
@@ -284,7 +301,7 @@ torchrun --nproc_per_node {number of gpus} \
 torchrun --nproc_per_node {number of gpus} \
 -m FlagEmbedding.llm_reranker.finetune_for_layerwise.run \
 --output_dir {path to save model} \
---model_name_or_path BAAI/bge-reranker-v2-minicpm-layerwise \
+--model_name_or_path openbmb/MiniCPM-2B-dpo-bf16 \
 --train_data ./toy_finetune_data.jsonl \
 --learning_rate 2e-4 \
 --num_train_epochs 1 \
@@ -309,10 +326,11 @@ torchrun --nproc_per_node {number of gpus} \
 --target_modules q_proj k_proj v_proj o_proj \
 --start_layer 8 \
 --head_multi True \
---head_type simple
+--head_type simple \
+--lora_extra_parameters linear_head
 ```
 
-Our rerankers are initialized from [google/gemma-2b](https://huggingface.co/google/gemma-2b) (for llm-based reranker) and [openbmb/MiniCPM-2B-dpo-fp16](https://huggingface.co/openbmb/MiniCPM-2B-dpo-fp16/tree/main) (for llm-based layerwise reranker), and we train it on a mixture of multilingual datasets:
+Our rerankers are initialized from [google/gemma-2b](https://huggingface.co/google/gemma-2b) (for llm-based reranker) and [openbmb/MiniCPM-2B-dpo-bf16](https://huggingface.co/openbmb/MiniCPM-2B-dpo-bf16) (for llm-based layerwise reranker), and we train it on a mixture of multilingual datasets:
 
 - [bge-m3-data](https://huggingface.co/datasets/Shitao/bge-m3-data)
 - [quora train data](https://huggingface.co/datasets/quora)

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "/share/cf/pycharm/reranker-finetune/bge_rerank/finetune_for_m3/m3_reranker",
+  "_name_or_path": "BAAI/bge-m3",
   "architectures": [
     "XLMRobertaForSequenceClassification"
   ],
@@ -31,4 +31,4 @@
   "type_vocab_size": 1,
   "use_cache": true,
   "vocab_size": 250002
-}
+}