Bug fix: Enable fast to override quantize json

[Jack-Khuu]

Follow up fix for: 4697764

We want to allow "fast" to work as a manual overwrite of a --quantize accelerator config, which the previous PR disabled while fixing a different bug

---

No overwrite

python3 torchchat.py generate llama3.1 --quantize '{"precision": {"dtype":"float16"}, "executor":{"accelerator":"mps"}}'

NumExpr defaulting to 10 threads.
PyTorch version 2.6.0.dev20241002 available.
lm_eval is not installed, GPTQ may not be usable
Using device=mps

Overwrite with device arg

python3 torchchat.py generate llama3.1 --quantize '{"precision": {"dtype":"float16"}, "executor":{"accelerator":"mps"}}' --device cpu

overriding json-specified device mps with cli device cpu
NumExpr defaulting to 10 threads.
PyTorch version 2.6.0.dev20241002 available.
lm_eval is not installed, GPTQ may not be usable
Using device=cpu Apple M1 Max

Overwrite with fast arg

python3 torchchat.py generate llama3.1 --quantize '{"precision": {"dtype":"float16"}, "executor":{"accelerator":"cpu"}}' --device fast

overriding json-specified device cpu with cli device mps
NumExpr defaulting to 10 threads.
PyTorch version 2.6.0.dev20241002 available.
lm_eval is not installed, GPTQ may not be usable
Using device=mps

Run with fast arg (same as running without device arg)

python3 torchchat.py generate llama3.1 --device fast

NumExpr defaulting to 10 threads.
PyTorch version 2.6.0.dev20241002 available.
lm_eval is not installed, GPTQ may not be usable
Using device=mps

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/1377

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• [DomainsOnly] Jobs fail with GLIBC version not found

✅ No Failures

As of commit 8db4b72 with merge base 4697764 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[Jack-Khuu]

cc: @mikekgfb

[Gasoonjia]

consider merging this if statement with below if:

Suggested change

	if executor_handler:
	if executor_handler and executor_handler["accelerator"] != args.device: