大家好，我是刘聪NLP。
Ta来了，Ta来了，Ta带着7个Size的开源模型迎面走来了。
是的，期待已久的Qwen2.5开源了，共有7个尺寸规模，包括：0.5B、1.5B、3B、7B、14B、32B和72B，分别有Base模型和Instruct模型。本次全是Dense模型，没有MoE模型。
同时还开源了Qwen2.5-Coder模型和Qwen2.5-Math模型。
还开了GGUF、GPTQ和AWQ 3种量化模型，别问，就是服务到位，主打一个“全”。
你有Llama3.1，我有Qwen2.5，请问阁下如何应对。
下面从模型说明、效果说明、Qwen2.5-72B实测、快速使用等几个方面来介绍一下刚刚开源的Qwen2.5系列模型。
Blog: https://qwenlm.github.io/blog/qwen2.5/
HF: https://huggingface.co/collections/Qwen/qwen25-66e81a666513e518adb90d9e
模型介绍
训练数据从Qwen2的7T Tokens 扩展到了Qwen2.5的18T tokens。 上下文最大长度128K，其中超过32K的部分利用YARN来进行外推处理，并且最大生成长度8K（从1K升级到 8K），更大的输出长度意味着可以更容易适配到cot输出上。 结构化数据（如表格）理解能力更强，并且输出json效果更好，同时适配多样的系统提示词，增强角色扮演能力。 更强的代码和数学能力，借助Qwen2-Math的数据到Qwen2.5上，让基模表现更好。 模型依旧采用RoPE、SwiGLU、RMSNorm和GQA。 依然是多语言模型：支撑29种语言，包括中文、英语、法语、西班牙语、葡萄牙语、德语、意大利语、俄语、日语、韩语、越南语、泰语、阿拉伯语等。 Coder模型，开源1.5B、7B共2个版本，基于Qwen2.5模型初始化，持续训练5.5T Tokens，包括源代码、文本代码混合数据、合成数据等。 Math模型，开源1.5B、7B、72B共3个版本，基于Qwen2.5模型初始化，利用Qwen2-Math-72B-Instruct模型合成更多高质量的数学预训练数据，同时从网络资源、书籍和代码中获取更多高质量的中文数学数据。
模型主要采用 Apache 2.0 开源许可协议，而 Qwen2.5-3B 和 Qwen2.5-72B 分别使用 Qwen Research 许可协议 和 Qwen 许可协议。
模型效果
先来看看Qwen2.5-72B模型效果，整体远超Llama3.1-70B模型，并且部分指标超过405B模型
还有增加的Qwen2.5-32B模型也是超越了之前的Qwen2-57B-A14B模型，并且部分指标上超过了GPT4o-mini模型。
Qwen2.5-3B版本模型也是在小型语言模型上崭露头角。
Qwen2.5-Coder全面领先Deepseek模型。
Qwen2.5-Math-72B模型超过GPT4o-2024-08-06。
Qwen2.5-72B Instruct测试
下面所有测试结果都是在lmsys上进行测试，https://lmarena.ai/
注意：可能是由于解码的原因，如果不加上step by step，直接问的话，会出现结果波动情况。
年度最佳测试，表露我真实的心情
9.9和9.11谁大的问题
strawberry有几个r的问题
弱智吧的问题
推理的问题
PS：加上step by step，模型输出会更稳定，并且效果会更好！！！
更多测试样例，欢迎留言测试。
HF快速使用：
模型下载有困难的同学，详见我之前写的一篇文章《大模型下载使我痛苦》
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "Qwen/Qwen2.5-7B-Instruct"
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.bfloat16,
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(model_name)
prompt = "将“I love Qwen2.5”的内容反过来写，请一步一步思考"
messages = [{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
generated_ids = model.generate(
**model_inputs,
max_new_tokens=512
)
generated_ids = [
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)
如果想进行o1的自动cot模型，可以尝试使用下面的系统提示词，来自https://github.com/codelion/optillm/blob/main/cot_reflection.py
You are an AI assistant that uses a Chain of Thought (CoT) approach with reflection to answer queries. Follow these steps:
1. Think through the problem step by step within the <thinking> tags.
2. Reflect on your thinking to check for any errors or improvements within the <reflection> tags.
3. Make any necessary adjustments based on your reflection.
4. Provide your final, concise answer within the <output> tags.
Important: The <thinking> and <reflection> sections are for your internal reasoning process only.
Do not include any part of the final answer in these sections.
The actual response to the query must be entirely contained within the <output> tags.
Use the following format for your response:
<thinking>
[Your step-by-step reasoning goes here. This is your internal thought process, not the final answer.]
<reflection>
[Your reflection on your reasoning, checking for errors or improvements]
</reflection>
[Any adjustments to your thinking based on your reflection]
</thinking>
<output>
[Your final, concise answer to the query. This is the only part that will be shown to the user.]
</output>
或来自https://github.com/bklieger-groq/g1的系统提示词：
You are an expert AI assistant that explains your reasoning step by step. For each step, provide a title that describes what you're doing in that step, along with the content. Decide if you need another step or if you're ready to give the final answer. Respond in JSON format with 'title', 'content', and 'next_action' (either 'continue' or 'final_answer') keys. USE AS MANY REASONING STEPS AS POSSIBLE. AT LEAST 3. BE AWARE OF YOUR LIMITATIONS AS AN LLM AND WHAT YOU CAN AND CANNOT DO. IN YOUR REASONING, INCLUDE EXPLORATION OF ALTERNATIVE ANSWERS. CONSIDER YOU MAY BE WRONG, AND IF YOU ARE WRONG IN YOUR REASONING, WHERE IT WOULD BE. FULLY TEST ALL OTHER POSSIBILITIES. YOU CAN BE WRONG. WHEN YOU SAY YOU ARE RE-EXAMINING, ACTUALLY RE-EXAMINE, AND USE ANOTHER APPROACH TO DO SO. DO NOT JUST SAY YOU ARE RE-EXAMINING. USE AT LEAST 3 METHODS TO DERIVE THE ANSWER. USE BEST PRACTICES.
Example of a valid JSON response:
json
{
"title": "Identifying Key Information",
"content": "To begin solving this problem, we need to carefully examine the given information and identify the crucial elements that will guide our solution process. This involves...",
"next_action": "continue"
}
PS：看到这里，如果觉得不错，可以来个点赞、在看、关注。给公众号添加【星标⭐️】不迷路！您的支持是我坚持的最大动力！
欢迎多多关注公众号「NLP工作站」，加入交流群，交个朋友吧，一起学习，一起进步！
