大家好！我们和多位专家一起在 ACM Transactions on Intelligent Systems and Technology (ACM TIST) 上组织了一期关于大模型评测方面的 special issue，主要征集「大模型评测」有关的论文。截稿日期为「2024年2月底」，欢迎大家赐稿！
期刊信息
期刊名称：ACM Transactions on Intelligent Systems and Technology (TIST) Special issue: Evaluation of large language models 详细信息：https://dl.acm.org/pb-assets/static_journal_pages/tist/pdf/TIST_CfP_LLMS-1696884403733.pdf 投稿链接：https://dl.acm.org/journal/tist
征稿范围
General evaluation, benchmarks, and protocols: New metrics and methods to measure the performance, fairness, and robustness of large language models. Logical evaluation: How well do these models follow rational thought? Robustness and trustworthiness evaluation: Are current LLMs robust and trustworthy to use? Causal evaluation: Assessing the ability of models to understand cause-effect relationships. Uncertainty, safety, and security evaluation: How safe is it to deploy existing LLMs to security-critical applications? Agents evaluation: Evaluate the advantages and disadvantages of using LLMs as agents. Multimodal and multilingual evaluation: broaden the application of LLMs in various modalities and languages. Deployment and mobile evaluation: How far is it to bring LLMs to more ubiquitous devices in daily life? Financial evaluation: Investigating the performance and reliability of language models in predicting, understanding, and advising on financial markets. Evaluation in other domains such as education, psychology, society, and others, with implications of evaluation results for real-world applications.
Guest editors
Jindong Wang, Microsoft Research Asia, jindong.wang@microsoft.com Linyi Yang, Westlake University, yanglinyi@westlake.edu.cn Sunayana Sitaram, Microsoft Research India, sunayana.sitaram@microsoft.com Qiang Yang, Hong Kong University of Science and Technology, qyang@cse.ust.hk Bhiksha Raj, Carnegie Mellon University, bhiksha@cs.cmu.edu
Important dates
Submission deadline: February 28, 2024 First-round review decisions: May 31, 2024 Deadline for Major and Minor revision submissions: June 30, 2024 – August 20, 2023 Second-round review decisions: September 30, 2024 Notification of final decisions: October 31, 2024 Tentative publication: Late 2024
