自部署大模型实验细节
huggingface 换源 1 export HF_ENDPOINT=https://hf-mirror.com 下载 1 2 3 4 5 6 7 8 9 10 nohup huggingface-cli download --resume-download Qwen/Qwen-72B-Chat-Int4 --local-dir ./Qwen-72B-Chat-Int4 > 72b.log & huggingface-cli download --resume-download Qwen/Qwen-14B --local-dir ./Qwen-14B huggingface-cli download --resume-download Qwen/Qwen2-7B-Instruct --local-dir ./Qwen-7B huggingface-cli download --resume-download Qwen/Qwen1.5-14B-Chat --local-dir ./Qwen1.5-14B 启动 vllm api 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20