groupuser/phi-1_5
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Adding _set_gradient_checkpointing for compatibility (#22)

Browse Source 
- Adding _set_gradient_checkpointing for compatibility (a30a931294ac0f344a0c1547877c692ceb17123c)


Co-authored-by: Vicente Rivera <vriveras@users.noreply.huggingface.co>
This commit is contained in:
Gustavo de Rosa 2023-10-17 12:11:30 +00:00 committed by system
parent b6a7e2fe15
commit 8091327f9e
1 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
modeling_mixformer_sequential.py
View File

@ -712,6 +712,10 @@ class MixFormerSequentialPreTrainedModel(PreTrainedModel):
"attention_mask": attention_mask, "attention_mask": attention_mask,
} }
def _set_gradient_checkpointing(self, module, value=False):
if isinstance(module, MixFormerSequentialPreTrainedModel):
module.gradient_checkpointing = value
class MixFormerSequentialForCausalLM(MixFormerSequentialPreTrainedModel): class MixFormerSequentialForCausalLM(MixFormerSequentialPreTrainedModel):
"""MixFormer (sequential for DeepSpeed) for Causal Language Modeling.""" """MixFormer (sequential for DeepSpeed) for Causal Language Modeling."""