add finetune code for qwen2audio

README.md

@@ -241,6 +241,14 @@ generated_ids = model.generate(**inputs, max_length=256)
 generated_ids = generated_ids[:, inputs.input_ids.size(1):]
 response = processor.batch_decode(generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
 ```
+
+#### Finetuning
+We would like to thank the Hugging Face open-source community for their contributions, which have made it easy for us to implement model fine-tuning with Accelerate and DeepSpeed. We support both LoRA (Low-Rank Adaptation) and full-parameter fine-tuning, with the code provided by [Xiaoming Liu](https://github.com/Lollipop).
+
+```bash
+cd finetune && bash run.sh
+```
+
 #### 🤖 ModelScope
 We strongly advise users especially those in mainland China to use ModelScope. `snapshot_download` can help you solve issues concerning downloading checkpoints.
 ## Demo

README_CN.md

@@ -240,6 +240,14 @@ generated_ids = model.generate(**inputs, max_length=256)
 generated_ids = generated_ids[:, inputs.input_ids.size(1):]
 response = processor.batch_decode(generated_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
 ```
+
+#### Finetuning
+感谢 Hugging Face 开源社区的贡献，使我们能够轻松地使用 Accelerate 和 DeepSpeed 实现模型微调（finetuning）。脚本支持 LoRA（低秩适应）和全参数微调，相关代码来自[Xiaoming Liu](https://github.com/Lollipop)。
+
+```bash
+cd finetune && bash run.sh
+```
+
 #### 🤖 ModelScope
 我们强烈建议用户，特别是中国大陆地区的用户，使用 ModelScope。`snapshot_download` 可以帮助您解决下载检查点时遇到的问题。
 <br>

finetune/accelerate_configs/deepspeed_z1.yaml

@@ -0,0 +1,24 @@
+compute_environment: LOCAL_MACHINE
+debug: false
+deepspeed_config:
+  deepspeed_multinode_launcher: standard
+  gradient_accumulation_steps: 1
+  gradient_clipping: 1.0
+  zero3_init_flag: false
+  zero_stage: 1
+distributed_type: DEEPSPEED
+downcast_bf16: 'no'
+enable_cpu_affinity: false
+machine_rank: 0
+main_process_ip: localhost
+main_process_port: 9999
+main_training_function: main
+mixed_precision: bf16
+num_machines: 2
+num_processes: 16
+rdzv_backend: static
+same_network: true
+tpu_env: []
+tpu_use_cluster: false
+tpu_use_sudo: false
+use_cpu: false

finetune/accelerate_configs/deepspeed_z2.yaml

@@ -0,0 +1,26 @@
+compute_environment: LOCAL_MACHINE
+debug: false
+deepspeed_config:
+  deepspeed_multinode_launcher: standard
+  gradient_accumulation_steps: 1
+  gradient_clipping: 1.0
+  offload_optimizer_device: none
+  offload_param_device: none
+  zero3_init_flag: false
+  zero_stage: 2
+distributed_type: DEEPSPEED
+downcast_bf16: 'no'
+enable_cpu_affinity: false
+machine_rank: 0
+main_process_ip: localhost
+main_process_port: 9999
+main_training_function: main
+mixed_precision: bf16
+num_machines: 2
+num_processes: 16
+rdzv_backend: static
+same_network: true
+tpu_env: []
+tpu_use_cluster: false
+tpu_use_sudo: false
+use_cpu: false

finetune/accelerate_configs/deepspeed_z3.yaml

@@ -0,0 +1,27 @@
+compute_environment: LOCAL_MACHINE                                                                                                                                                                     
+debug: false                                                                                                                                                                                           
+deepspeed_config:
+  deepspeed_multinode_launcher: standard
+  gradient_accumulation_steps: 1
+  gradient_clipping: 1.0
+  offload_optimizer_device: cpu
+  offload_param_device: cpu
+  zero3_init_flag: true
+  zero3_save_16bit_model: false
+  zero_stage: 3
+distributed_type: DEEPSPEED
+downcast_bf16: 'no'
+enable_cpu_affinity: false
+machine_rank: 0
+main_process_ip: localhost
+main_process_port: 999
+main_training_function: main
+mixed_precision: bf16
+num_machines: 1
+num_processes: 8
+rdzv_backend: static
+same_network: true
+tpu_env: []
+tpu_use_cluster: false
+tpu_use_sudo: false
+use_cpu: false