v2.5.0

English Version

New Features:

1. Support for GPTQ & AWQ quantization of multimodal LLMs.
2. Support for dynamic addition of gradient checkpointing in the ViT section to reduce memory consumption.
3. Support for multimodal model pre-training.

New Models:

1. llama3.2, llama3.2-vision series
2. got-ocr2
3. llama3.1-omni
4. ovis1.6-gemma2
5. pixtral-12b
6. telechat2-115b
7. mistral-small-inst-2409

New Datasets:

1. egoschema

中文版

新特性：

1. 支持多模态LLM的gptq&awq量化.
2. 支持动态在vit部分增加gradient_checkpointing, 减少显存消耗.
3. 支持多模态模型预训练.

新模型：

1. llama3.2, llama3.2-vision系列
2. got-ocr2
3. llama3.1-omni
4. ovis1.6-gemma2
5. pixtral-12b
6. telechat2-115b
7. mistral-small-inst-2409

新数据集：

1. egoschema

What's Changed

• fix win32 quote by @tastelikefeet in #2065
• Fix yi template by @Jintao-Huang in #2067
• fix rlhf zero3 by @Jintao-Huang in #2072
• Update qwen2-vl最佳实践.md by @Digital2Slave in #2058
• fix RLHF & max_length by @Jintao-Huang in #2075
• Support Mistral-small-inst-2409 by @DaozeZhang in #2077
• dynamic vit gradient_checkpointing by @Jintao-Huang in #2071
• fix qwen2.5 template by @Jintao-Huang in #2081
• fix multiprocess remove_columns by @Jintao-Huang in #2088
• Support for fine-tuning Pixtral-12B. by @Jintao-Huang in #2090
• fix vllm tokenizer by @Jintao-Huang in #2099
• Fix the issue with media_offset in owl3 when batch_size > 1. by @LukeForeverYoung in #2100
• fix deploy openai compat by @Jintao-Huang in #2101
• fix dataset preprocess by @Jintao-Huang in #2102
• fix cpu infer device_map by @Jintao-Huang in #2103
• fix infer device_map by @Jintao-Huang in #2105
• Support for fine-tuning Llama 3.1 Omni. by @Jintao-Huang in #2106
• support vllm & qwen2-vl video by @Jintao-Huang in #2110
• Fix qwen2-vl zero2/3 by @Jintao-Huang in #2114
• fix qwen2-audio by @Jintao-Huang in #2116
• [TorchAcc] fix: fix find_labels and can_return_loss by @baoleai in #2120
• support got-ocr2 by @Jintao-Huang in #2123
• Support for fine-tuning and deployment of the Llama 3.2 series models. by @Jintao-Huang in #2130
• Support fine-tuning MLLama. by @Jintao-Huang in #2132
• fix not impl bug by @Jintao-Huang in #2134
• Compat vllm & qwen2-vl by @Jintao-Huang in #2136
• fix requirements by @Jintao-Huang in #2137
• fix model_type by @Jintao-Huang in #2138
• fix deploy vllm by @Jintao-Huang in #2141
• fix docs by @Jintao-Huang in #2142
• Fix VLM lora by @tastelikefeet in #2140
• support mllm pt by @Jintao-Huang in #2146
• [TorchAcc] fix: fix save config and additional file for swift and peft by @baoleai in #2149
• update quant_device_map by @Jintao-Huang in #2154
• fix qwen2-audio by @Jintao-Huang in #2157
• fix template by @Jintao-Huang in #2160
• compat trl==0.11 by @Jintao-Huang in #2169
• Support for Egoschema, a new video dataset by @DaozeZhang in #2173
• Update FAQ by @slin000111 in #2165
• fix mplug-owl3 infer by @Jintao-Huang in #2175
• Support quant mllm by @Jintao-Huang in #2177
• update setup.py by @Jintao-Huang in #2205
• fix bugs by @Jintao-Huang in #2207
• support telechat2 by @Jintao-Huang in #2210
• Support ovis 1.6 by @Jintao-Huang in #2211

New Contributors

• @Digital2Slave made their first contribution in #2058
• @LukeForeverYoung made their first contribution in #2100

Full Changelog: v2.4.2...v2.5.0