海归学者发起的公益学术平台
分享信息，整合资源
交流学术，偶尔风月
针对各种应用的机器学习势的开发涉及大量数据集的创建，其中包含不同理论水平的量子力学计算数据集。问题是，为每一个研究专门创建数据集并从头开始训练模型十分耗时，严重阻碍了机器学习势的进一步发展。为建立机器学习势的预训练模型，虽然可从现有文献中有效收集这些数据，但这些数据集出自不同的计算方式和具有不同的精度，甚至可能相互矛盾。元学习所寻求建立的模型虽然不是专门针对任何特定的任务，但可以快速地重新训练到许多新的相似任务，其中每一个任务都是一个特定的学习问题。即使新数据的量相对较少，这种再训练也可能是有效的。如何将元学习应用于创建庞大的机器学习原子间势数据集，自然成了急需打通的路径。
Fig. 1 | The Reptile algorithm.
来自美国洛斯阿拉莫斯国家实验室的Alice E. A. Allen等，通过创建各种系统的机器学习原子间势，从单个阿司匹林分子到ANI-1ccx数据集，展示了元学习的广泛适用性。通过对多个大型有机分子数据集进行模型预训练，研究者表明这些数据集可以组合在一起，并对模型进行了预训练。使用预训练模型的好处体现在了3BPA分子原子间势模型训练上，产生了更准确和更平滑的势函数。元学习极大地扩展了机器学习原子间势可用的拟合数据的多样性，并建立了为机器学习原子间势创建现成预训练的基础模型的可能性。
Fig. 2 | Meta-learning used for an aspirin interatomic potential.
元学习将多种不同数据集同时拟合，建立的预训练模型，从而限制了参数更新的空间，使模型可以被快速调整到具体任务的训练上。这一进步改变了对现有数据集的利用方式，并为机器学习原子间势的拟合开辟了新的途径。该文近期发表于npj Computational Materials 10: 154 (2024)，英文标题与摘要如下，点击左下角“阅读原文”可以自由获取论文PDF。
Learning together: Towards foundation models for machine learning interatomic potentials with meta-learning
Alice E. A. Allen, Nicholas Lubbers, Sakib Matin, Justin Smith, Richard Messerly, Sergei Tretiak & Kipton Barros
The development of machine learning models has led to an abundance of datasets containing quantum mechanical (QM) calculations for molecular and material systems. However, traditional training methods for machine learning models are unable to leverage the plethora of data available as they require that each dataset be generated using the same QM method. Taking machine learning interatomic potentials (MLIPs) as an example, we show that meta-learning techniques, a recent advancement from the machine learning community, can be used to fit multiple levels of QM theory in the same training process. Meta-learning changes the training procedure to learn a representation that can be easily re-trained to new tasks with small amounts of data. We then demonstrate that meta-learning enables simultaneously training to multiple large organic molecule datasets. As a proof of concept, we examine the performance of a MLIP refit to a small drug-like molecule and show that pre-training potentials to multiple levels of theory with meta-learning improves performance. This difference in performance can be seen both in the reduced error and in the improved smoothness of the potential energy surface produced. We therefore show that meta-learning can utilize existing datasets with inconsistent QM levels of theory to produce models that are better at specializing to new datasets. This opens new routes for creating pre-trained, foundation models for interatomic potentials.
扩展阅读
