PTBPlus

Multi-Fidelity Transfer Learning for Quantum Chemical Data Using a Robust DFTB Baseline

PTBPlus is the multi-fidelity transfer learning (MFTL) framework built on top of the PTBP parameter set. It uses cheap DFTB(PTBP) data for pre-training and a small high-fidelity dataset (DFT or CCSD(T)) for fine-tuning, yielding accurate machine-learning interatomic potentials at a fraction of the data-generation cost of single-fidelity approaches.

Introduction

Machine-learned interatomic potentials (MLIPs) typically rely on large, high-fidelity reference datasets that are expensive to compute. Coupled-cluster (CC) or hybrid-DFT references in particular are prohibitive for systematic dataset generation. Transfer learning (TL) addresses this by pre-training on a cheaper proxy dataset and fine-tuning on the small target dataset.

Two issues commonly arise:

  1. Negative transfer — features learned during pre-training can hurt fine-tuning when the proxy is too noisy.

  2. Catastrophic forgetting — fine-tuning overwrites all of the prior information, making pre-training irrelevant.

Both are particularly pertinent when the structural or fidelity overlap between the pre-training and fine-tuning data is insufficient. PTBPlus disentangles these two effects by using DFTB(PTBP) — an inexpensive, physically motivated, semi-empirical method with broad chemical coverage — to generate low-fidelity data with perfect structural overlap to the target application.

Method

The MFTL workflow:

  1. Pre-training data: generate a large dataset (10k–50k configurations) at the DFTB(PTBP) level, sampled from the same configuration space as the target.

  2. Pre-train an interatomic potential (MACE in this work) on the low-fidelity data.

  3. Fine-tune on a small high-fidelity dataset (e.g. 0.5k PBE0+MBD configurations).

  4. (Optional) iterate across additional fidelities — for example DFTB → DFT → CCSD(T) — to recover even higher accuracy.

A central practical finding is that early stopping during pre-training is critical. Training the low-fidelity model fully tends to teach the network details of the DFTB potential energy surface that cannot be corrected by the small fine-tuning set, leading to negative transfer. Stopping early acts as a regularizer.

Results

PTBPlus was benchmarked on three datasets:

QM7x — small organic molecules (PBE0+MBD reference)

  • MFTL pre-trained on 50k DFTB(PTBP) configurations and fine-tuned on 0.5k PBE0+MBD points reaches 6.1 meV/atom (energy) and 92.5 meV/Å (forces) RMSE on the high-fidelity test set.

  • Single-fidelity learning on the same 0.5k set: 11.8 meV/atom / 195.0 meV/Å.

  • Fine-tuned MACE-OFF23 foundation model: 7.17 meV/atom / 127.87 meV/Å.

MFTL outperforms both, with the gap most pronounced for the smallest fine-tuning sets. The advantage over the foundation model is attributed to better description of short interatomic distances in QM7x non-equilibrium structures, which are sparse in the foundation model’s training data.

Tungsten — transition-metal materials (PBE reference)

A 1.58k-configuration set covering bulk, surfaces, vacancies, gamma-surfaces, and dislocation cores.

  • For the smallest training set (100 configurations), MFTL gives an almost four-fold improvement in energy RMSE over single-fidelity learning.

  • Fine-tuning the MACE-MP-0 foundation model “as is” actually performs worse than single-fidelity learning for larger training sets — a clear example of negative transfer caused by the limited tungsten coverage in MPTraj.

  • A multi-head fine-tuning strategy on MACE-MP-0 mitigates this and brings it close to MFTL.

MD22 — CCSD(T) benchmark (Benzene, Malonaldehyde, Toluene)

A triple-fidelity workflow (DFTB → DFT → CCSD(T)) was tested using 50k DFTB and 10k DFT configurations as pre-training stages.

  • Triple-fidelity TL achieves 0.49 meV/atom (energy) and 25.59 meV/Å (forces) RMSE with only 100 CCSD(T) datapoints.

  • Improvements of 59.75% (energy) and 45.09% (forces) over single-fidelity learning at the same training-set size.

  • Triple-fidelity even outperforms double-fidelity (DFT-only pre-training), confirming that the cheap DFTB data still carries useful information for the CCSD(T) learning task.

Takeaways

  • Structural overlap matters as much as fidelity. A simple, cheap baseline with the right configuration coverage can beat a much more accurate foundation model trained on a different region of chemical space.

  • More pre-training data helps; more pre-training epochs do not. Early stopping is essential to avoid negative transfer from noisy low-fidelity labels.

  • MFTL is more versatile than Δ-ML. Δ-ML reaches similar accuracy on QM7x but requires evaluating the low-fidelity method at inference time and scales poorly with system size; MFTL needs only a single ML model at inference.

  • Multiple fidelities compound. Adding DFTB(PTBP) data on top of a DFT pre-training step still improves CCSD(T) accuracy.

These results suggest that inexpensive, broad-coverage models like DFTB(PTBP) — and small-basis DFT — can play an important role in training the next generation of foundation models for chemistry and materials.

Citation

If you use PTBPlus, please cite:

M. Cui, K. Reuter, J. T. Margraf. Multi-Fidelity Transfer Learning for Quantum Chemical Data Using a Robust Density Functional Tight Binding Baseline. ChemRxiv (2024). doi:10.26434/chemrxiv-2024-9734b