项目地址

chatglm2-6b

下载模型

cd ~/Downloads
git clone --depth=1 https://www.modelscope.cn/ZhipuAI/chatglm2-6b.git

创建测试项目

cd ~/code
mkdir ai/chatglm2-6b -p
cd ai/chatglm2-6b
mkdir ZhipuAI
ln -s ~/Downloads/chatglm2-6b ZhipuAI/

创建虚拟环境安装依赖

python3 -m venv venv
source venv/bin/activate

pip install transformers modelscope cpm_kernels -U
pip install torch
pip install accelerate
pip install sentencepiece

完整的venv环境

(venv) yeqiang@yeqiang-MS-7B23:~/code/ai/chatglm2-6b$ pip list
Package                  Version
------------------------ ------------
accelerate               0.26.1
addict                   2.4.0
aiohttp                  3.9.1
aiosignal                1.3.1
aliyun-python-sdk-core   2.14.0
aliyun-python-sdk-kms    2.16.2
async-timeout            4.0.3
attrs                    23.2.0
certifi                  2023.11.17
cffi                     1.16.0
charset-normalizer       3.3.2
cpm-kernels              1.0.11
crcmod                   1.7
cryptography             41.0.7
datasets                 2.16.1
dill                     0.3.7
einops                   0.7.0
filelock                 3.13.1
frozenlist               1.4.1
fsspec                   2023.10.0
gast                     0.5.4
huggingface-hub          0.20.2
idna                     3.6
importlib-metadata       7.0.1
Jinja2                   3.1.3
jmespath                 0.10.0
MarkupSafe               2.1.3
modelscope               1.11.0
mpmath                   1.3.0
multidict                6.0.4
multiprocess             0.70.15
networkx                 3.2.1
numpy                    1.26.3
nvidia-cublas-cu12       12.1.3.1
nvidia-cuda-cupti-cu12   12.1.105
nvidia-cuda-nvrtc-cu12   12.1.105
nvidia-cuda-runtime-cu12 12.1.105
nvidia-cudnn-cu12        8.9.2.26
nvidia-cufft-cu12        11.0.2.54
nvidia-curand-cu12       10.3.2.106
nvidia-cusolver-cu12     11.4.5.107
nvidia-cusparse-cu12     12.1.0.106
nvidia-nccl-cu12         2.18.1
nvidia-nvjitlink-cu12    12.3.101
nvidia-nvtx-cu12         12.1.105
oss2                     2.18.4
packaging                23.2
pandas                   2.1.4
pillow                   10.2.0
pip                      22.0.2
platformdirs             4.1.0
psutil                   5.9.7
pyarrow                  14.0.2
pyarrow-hotfix           0.6
pycparser                2.21
pycryptodome             3.20.0
python-dateutil          2.8.2
pytz                     2023.3.post1
PyYAML                   6.0.1
regex                    2023.12.25
requests                 2.31.0
safetensors              0.4.1
scipy                    1.11.4
sentencepiece            0.1.99
setuptools               59.6.0
simplejson               3.19.2
six                      1.16.0
sortedcontainers         2.4.0
sympy                    1.12
tokenizers               0.15.0
tomli                    2.0.1
torch                    2.1.2
torchflow                0.0.3
tqdm                     4.66.1
transformers             4.36.2
triton                   2.1.0
typing_extensions        4.9.0
tzdata                   2023.4
urllib3                  2.1.0
xxhash                   3.4.1
yapf                     0.40.2
yarl                     1.9.4
zipp                     3.17.0

main.py

# 备注：最新模型版本要求modelscope >= 1.9.0
# pip install modelscope -U 

from modelscope.utils.constant import Tasks
from modelscope import Model
from modelscope.pipelines import pipeline
model = Model.from_pretrained('ZhipuAI/chatglm2-6b', device_map='auto', revision='v1.0.12')
pipe = pipeline(task=Tasks.chat, model=model)
inputs = {'text':'你好', 'history': []}
result = pipe(inputs)
inputs = {'text':'介绍下清华大学', 'history': result['history']}
result = pipe(inputs)

执行

python进程cpu使用率100.3%持续了很长时间，没有启用多线程？