大纲
Hugging-Face 介绍
Hugging-Face 大语言模型 LLM 管理
Transformers 机器学习框架
文本生成推理 (TGI)
Hugging Face
Hugging-Face -- 大语言模型界的 Github
Hugging Face 专门开发用于构建机器学习应用的工具。该公司的代表产品是其为自然语言处理应用构建的 transformers 库，以及允许用户共享机器学习模型和数据集的平台
大模型平台 hugging face
国内对标 -- 百度千帆
百度智能云千帆大模型平台（以下简称千帆或千帆大模型平台）是面向企业开发者的一站式大模型开发及服务运行平台。千帆不仅提供了包括文心一言底层模型和第三方开源大模型，还提供了各种 AI 开发工具和整套开发环境，方便客户轻松使用和开发大模型应用。支持数据管理、自动化模型 SFT 以及推理服务云端部署的一站式大模型定制服务，助力各行业的生成式 AI 应用需求落地。
百度千帆
国内对标 -- 魔搭社区
ModelScope 社区成立于 2022 年 6 月，是一个模型开源社区及创新平台，由阿里巴巴通义实验室（Institute for Intelligent Computing），联合 CCF 开源发展委员会，共同作为项目发起方。社区联合国内 AI 领域合作伙伴与高校机构， 致力于通过开放的社区合作，构建深度学习相关的模型开源社区，并开放相关模型创新技术，推动基于“模型即服务”（Model-as-a-Service）理念的模型应用生态的繁荣发展。
Hugging Face 是行业风向标
开发者仓库 Hub
机器学习模型 Hub
数据集 Hub
机器学习算法库与封装工具
解决方案
文档
大语言模型管理
通过 Git 下载大模型
git lfs install#模型会下载到本地目录，并不在hugging face的标准管理目录里，可能会导致后续Transformers库重新下载git clone https://huggingface.co/google/owlv2-base-patch16-ensemble
客户端工具 huggingface_hub
python -m pip install huggingface_hub# 需要开启代理huggingface-cli login
高速下载
python -m pip install huggingface_hub[hf_transfer]#缺点：目前还不支持代理，导致下载大文件可能有时候更慢。export HF_HUB_ENABLE_HF_TRANSFER=1download gpt2 config.json
huggingface-cli 用法
(base) hogwarts: llama seveniruby$ huggingface-cli -husage: huggingface-cli <command> [<args>]positional arguments:{env,login,whoami,logout,repo,upload,download,lfs-enable-largefiles,lfs-multipart-upload,scan-cache,delete-cache}huggingface-cli command helpersenv Print information about the environment.login Log in using a token from huggingface.co/settings/tokenswhoami Find out which huggingface.co account you are logged in as.logout Log outrepo {create, ls-files} Commands to interact with your huggingface.co repos.upload Upload a file or a folder to a repo on the Hubdownload Download files from the Hublfs-enable-largefilesConfigure your repository to enable upload of files > 5GB.lfs-multipart-uploadCommand will get called by git-lfs, do not call it directly.scan-cache Scan cache directory.delete-cache Delete revisions from the cache directory.options:-h, --help show this help message and exit
使用示例
huggingface-cli loginhuggingface-cli download gpt2huggingface-cli download gpt2 config.jsonhuggingface-cli download gpt2 --revision main
使用 Hub client library 管理
from huggingface_hub import hf_hub_downloadhf_hub_download(repo_id="google/pegasus-xsum", filename="config.json")from huggingface_hub import hf_hub_downloadhf_hub_download(repo_id="google/pegasus-xsum",filename="config.json",revision="4d33b01d79672f27f001f6abade33f22d993b151")
hugging face 的本地缓存文件结构
#文件快照./snapshots./snapshots/187d69aae616814bc1386bdfe38e256518d17637./snapshots/187d69aae616814bc1386bdfe38e256518d17637/added_tokens.json./snapshots/187d69aae616814bc1386bdfe38e256518d17637/tokenizer_config.json./snapshots/187d69aae616814bc1386bdfe38e256518d17637/special_tokens_map.json./snapshots/187d69aae616814bc1386bdfe38e256518d17637/config.json./snapshots/187d69aae616814bc1386bdfe38e256518d17637/README.md./snapshots/187d69aae616814bc1386bdfe38e256518d17637/merges.txt./snapshots/187d69aae616814bc1386bdfe38e256518d17637/.gitattributes./snapshots/187d69aae616814bc1386bdfe38e256518d17637/pytorch_model.bin./snapshots/187d69aae616814bc1386bdfe38e256518d17637/vocab.json./snapshots/187d69aae616814bc1386bdfe38e256518d17637/preprocessor_config.json#真正的文件缓存./blobs./blobs/76e821f1b6f0a9709293c3b6b51ed90980b3166b./blobs/69feda8b53b1c9e2a85ae756bf58c120c3c1b4b4a4d97d4876578c1809a63d76./blobs/39808ac074d69141c0b2f065d16f56dfa5194cac./blobs/1c4c373d81adc0c63bb3368a89f4a92574395c2c./blobs/b24bfd579e8682df31c7794a3e1c3ef6a9a22a7a./blobs/5ff4ac542d5e800dd3aadbdcda354868b5f24ddd./blobs/a6344aac8c09253b3b630fb776ae94478aa0275b./blobs/dfc1049705d082052a81e7d18bde6d11fc137a0d./blobs/ee790e862e5a47398da080893272d3c2b76cf0e5./blobs/469be27c5c010538f845f518c4f5e8574c78f7c8#分支，指定分支有助于解决重复下载问题./refs./refs/main
hugging face transformers 机器学习框架
Transformers 库
Transformers 是一个预训练的最先进模型库，用于自然语言处理 (NLP)、计算机视觉以及音频和语音处理任务。该库不仅包含 Transformer 模型，还包含非 Transformer 模型，例如用于计算机视觉任务的现代卷积网络。Transformers 提供 API 和工具，可轻松下载和训练最先进的预训练模型，它支持 PyTorch、TensorFlow 和 JAX 之间的框架互操作性。
Transformers 能力
自然语言处理：文本分类、命名实体识别、问答、语言建模、摘要、翻译、多项选择和文本生成。
计算机视觉：图像分类、对象检测和分割。
音频：自动语音识别和音频分类。
多模态：表格问答、光学字符识别、扫描文档信息提取、视频分类和视觉问答。
| Task | Description | Modality | Pipeline identifier |
|---|---|---|---|
| Text classification | assign a label to a given sequence of text | NLP | pipeline(task=“sentiment-analysis”) |
| Text generation | generate text given a prompt | NLP | pipeline(task=“text-generation”) |
| Summarization | generate a summary of a sequence of text or document | NLP | pipeline(task=“summarization”) |
| Task | Description | Modality | Pipeline identifier |
|---|---|---|---|
| Image classification | assign a label to an image | Computer vision | pipeline(task=“image-classification”) |
| Image segmentation | assign a label to each individual pixel of an image (supports semantic, panoptic, and instance segmentation) | Computer vision | pipeline(task=“image-segmentation”) |
| Object detection | predict the bounding boxes and classes of objects in an image | Computer vision | pipeline(task=“object-detection”) |
| Task | Description | Modality | Pipeline identifier |
|---|---|---|---|
| Audio classification | assign a label to some audio data | Audio | pipeline(task=“audio-classification”) |
| Automatic speech recognition | transcribe speech into text | Audio | pipeline(task=“automatic-speech-recognition”) |
| Task | Description | Modality | Pipeline identifier |
|---|---|---|---|
| Visual question answering | answer a question about the image, given an image and a question | Multimodal | pipeline(task=“vqa”) |
| Document question answering | answer a question about the document, given a document and a question | Multimodal | pipeline(task=“document-question-answering”) |
| Image captioning | generate a caption for a given image | Multimodal | pipeline(task=“image-to-text”) |
快速开始
pip install transformers datasetspip install torchpip install tensorflow#测试python -c "from transformers import pipeline; print(pipeline('sentiment-analysis')('we love you'))"pipeline api 使用pipe = pipeline("text-classification")pipe("This restaurant is awesome")#输出[{'label': 'POSITIVE', 'score': 0.9998743534088135}]
使用第三方库
pipe = pipeline(model="FacebookAI/roberta-large-mnli")pipe("This restaurant is awesome")#输出[{'label': 'NEUTRAL', 'score': 0.7313136458396912}]
图像识别
def test_zero_shot_object_detector():with open('coco_sample.png', mode='rb') as f:image = Image.open(f)object_detector = pipeline('object-detection', revision='main')predictions = object_detector(image)print(json.dumps(predictions, indent=2, ensure_ascii=False))#输出[{"score": 0.9982201457023621,"label": "remote","box": {"xmin": 40,"ymin": 70,"xmax": 175,"ymax": 117}},{"score": 0.9960021376609802,"label": "remote","box": {"xmin": 333,"ymin": 72,"xmax": 368,"ymax": 187}},{"score": 0.9954745173454285,"label": "couch","box": {"xmin": 0,"ymin": 1,"xmax": 639,"ymax": 473}},{"score": 0.9988006353378296,"label": "cat","box": {"xmin": 13,"ymin": 52,"xmax": 314,"ymax": 470}},{"score": 0.9986783862113953,"label": "cat","box": {"xmin": 345,"ymin": 23,"xmax": 640,"ymax": 368}}]
文本生成推理 Text Generation Inference
Text Generation Inference
文本生成推理 (TGI) 是一个用于部署和服务大型语言模型 (LLM) 的工具包。TGI 为最流行的开源 LLM 提供高性能文本生成，包括 Llama、Falcon、StarCoder、BLOOM、GPT-NeoX 和 T5。
model=tiiuae/falcon-7b-instructvolume=$PWD/data # share a volume with the Docker container to avoid downloading weights every rundocker run --gpus all --shm-size 1g -p 8080:80 -v $volume:/data ghcr.io/huggingface/text-generation-inference:1.4 --model-id $modelmodel=teknium/OpenHermes-2.5-Mistral-7Bvolume=$PWD/data # share a volume with the Docker container to avoid downloading weights every rundocker run --gpus all --shm-size 1g -p 8080:80 -v $volume:/data ghcr.io/huggingface/text-generation-inference:1.4 --model-id $model
TGI 接口
curl 127.0.0.1:8080/generate \-X POST \-d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":20}}' \-H 'Content-Type: application/json'
编程使用 TGI
import requestsheaders = {"Content-Type": "application/json",}data = {'inputs': 'What is Deep Learning?','parameters': {'max_new_tokens': 20,},}response = requests.post('http://127.0.0.1:8080/generate', headers=headers, json=data)print(response.json())# {'generated_text': '\n\nDeep Learning is a subset of Machine Learning that is concern
推荐学习
人工智能测试开发训练营，为大家提供全方位的人工智能测试知识和技能培训。行业专家授课，实战驱动，并提供人工智能答疑福利。内容包含ChatGPT与私有大语言模型的多种应用，人工智能应用开发框架 LangChain，视觉与图像识别自动化测试，人工智能产品质量保障与测试，知识图谱与模型驱动测试，深度学习应用，带你一站式掌握人工智能测试开发必备核心技能，快速提升核心竞争力！
