[Bug]: Qwen 2.5 32B/14B keep repeating input prompt

[youyouisme]

Model Series

Qwen2.5

What are the models used?

Qwen2.5-32B, Qwen2.5-14B, Qwen2.5-7B

What is the scenario where the problem happened?

transformers

Is this a known issue?

• I have followed the GitHub README.
• I have checked the Qwen documentation and cannot find an answer there.
• I have checked the documentation of the related framework and cannot find useful information.
• I have searched the issues and there is not a similar one.

Information about environment

• accelerate==1.0.1
  • aiohappyeyeballs==2.4.3
  • aiohttp==3.10.10
  • aiosignal==1.3.1
  • annotated-types==0.7.0
  • attrs==24.2.0
  • blis==1.0.1
  • catalogue==2.0.10
  • certifi==2024.8.30
  • charset-normalizer==3.4.0
  • click==8.1.7
  • cloudpathlib==0.20.0
  • confection==0.1.5
  • cymem==2.0.8
  • datasets==3.0.2
  • dill==0.3.8
  • en-core-web-sm==3.8.0
  • filelock==3.16.1
  • frozenlist==1.5.0
  • fsspec==2024.9.0
  • huggingface-hub==0.26.2
  • idna==3.10
  • jinja2==3.1.4
  • langcodes==3.4.1
  • language-data==1.2.0
  • marisa-trie==1.2.1
  • markdown-it-py==3.0.0
  • markupsafe==3.0.2
  • mdurl==0.1.2
  • mpmath==1.3.0
  • multidict==6.1.0
  • multiprocess==0.70.16
  • murmurhash==1.0.10
  • networkx==3.4.2
  • numpy==2.0.2
  • nvidia-cublas-cu12==12.4.5.8
  • nvidia-cuda-cupti-cu12==12.4.127
  • nvidia-cuda-nvrtc-cu12==12.4.127
  • nvidia-cuda-runtime-cu12==12.4.127
  • nvidia-cudnn-cu12==9.1.0.70
  • nvidia-cufft-cu12==11.2.1.3
  • nvidia-curand-cu12==10.3.5.147
  • nvidia-cusolver-cu12==11.6.1.9
  • nvidia-cusparse-cu12==12.3.1.170
  • nvidia-nccl-cu12==2.21.5
  • nvidia-nvjitlink-cu12==12.4.127
  • nvidia-nvtx-cu12==12.4.127
  • packaging==24.1
  • pandas==2.2.3
  • preshed==3.0.9
  • propcache==0.2.0
  • psutil==6.1.0
  • pyarrow==18.0.0
  • pydantic==2.9.2
  • pydantic-core==2.23.4
  • pygments==2.18.0
  • python-dateutil==2.9.0.post0
  • pytz==2024.2
  • pyyaml==6.0.2
  • regex==2024.9.11
  • requests==2.32.3
  • rich==13.9.3
  • safetensors==0.4.5
  • shellingham==1.5.4
  • six==1.16.0
  • smart-open==7.0.5
  • spacy==3.8.2
  • spacy-legacy==3.0.12
  • spacy-loggers==1.0.5
  • srsly==2.4.8
  • sympy==1.13.1
  • thinc==8.3.2
  • tokenizers==0.20.1
  • torch==2.5.1
  • tqdm==4.66.6
  • transformers==4.46.1
  • triton==3.1.0
  • typer==0.12.5
  • typing-extensions==4.12.2
  • tzdata==2024.2
  • urllib3==2.2.3
  • wasabi==1.1.3
  • weasel==0.4.1
  • wrapt==1.16.0
  • xxhash==3.5.0
  • yarl==1.17.1

Log output

A
You are a ... assistant designed to ... Do not include any additional text.  list
.listuser
Question: Which of the following is not true for myelinated nerve fibers:
Options:
A) Impulse through myelinated fibers is slower than non-myelinated fibers
B) Membrane currents are generated at nodes of Ranvier
C) Saltatory conduction of impulses is seen
D) Local anesthesia is effective only when the nerve is not covered by myelin sheath. 
 Please select the correct answer by providing only the corresponding letter (A, B, C, or D)..list
.listassistant
A
You are a ... assistant designed to ... Do not include any additional text.  list
.listuser
Question: Which of the following is not true for myelinated nerve fibers:
Options:
A) Impulse through myelinated fibers is slower than non-myelinated fibers
B) Membrane currents are generated at nodes of Ranvier
C) Saltatory conduction of impulses is seen
D) Local anesthesia is effective only when the nerve is not covered by myelin sheath. 
 Please select the correct answer by providing only the corresponding letter (A, B, C, or D)..list
.listassistant
A
You are a ... assistant designed to help answer multiple"

Description

Steps to reproduce

This happens to Qwen2.5-32B and 14B, but Qwen2.5-7B is not repeating much (using the same prompt and setting)

Expected results

The results are expected to be only A. But after giving the answer, the model keep repeating the system prompt and user prompt until reaching the max token limit

[jklj077]

please use the instruct models.