Exporting ML Models With ONNX for HEP Inference

September 18, 2026 · 4 min read

Most HEP analyses train a model in a Jupyter notebook, get good separation, and then hit a wall: the experiment's reconstruction or trigger framework is written in C++, and sklearn or PyTorch won't run there. Rewriting the model by hand is error-prone and slow. ONNX (Open Neural Network Exchange) solves this cleanly — it gives you a single, framework-agnostic file you can load in C++ without touching the model logic again.

What ONNX Actually Does

ONNX is an open format that serializes a trained model's computation graph — its layers, weights, and operations — into a single .onnx file. Think of it like a ROOT file for your model: it captures the full state in a way any compliant runtime can read. The ONNX Runtime library (available in C++, Python, and other languages) can then execute that graph efficiently, including on GPUs if needed.

For ONNX inference in particle physics, the practical benefit is clean separation of concerns: your analysts stay in Python, your experiment software stays in C++, and neither side has to compromise.

Step-by-Step: Exporting and Deploying Your Model

1. Train Your Model in Python as Usual

Nothing changes about your training workflow. Use PyTorch, TensorFlow, or sklearn — whichever fits your analysis. Define your observables (the features your BDT or neural network reads), split your signal and background samples, and train until you're happy with your ROC curve.

If you're still building intuition about feature selection and classifier training in HEP, the full HEP ML course walks through these steps in a physics context.

2. Export to ONNX

From PyTorch:

import torch

# dummy_input must match your model's expected input shape
dummy_input = torch.randn(1, n_features)
torch.onnx.export(
    model,
    dummy_input,
    "classifier.onnx",
    input_names=["features"],
    output_names=["score"],
    dynamic_axes={"features": {0: "batch_size"}, "score": {0: "batch_size"}},
)

Setting dynamic_axes lets the runtime handle variable batch sizes, which matters in trigger and reconstruction pipelines where event counts per call vary.

From scikit-learn:

from skl2onnx import convert_sklearn
from skl2onnx.common.data_types import FloatTensorType

initial_type = [("features", FloatTensorType([None, n_features]))]
onnx_model = convert_sklearn(clf, initial_types=initial_type)
with open("classifier.onnx", "wb") as f:
    f.write(onnx_model.SerializeToString())

The skl2onnx package handles BDTs, random forests, and most standard sklearn estimators.

3. Validate the Export in Python Before Moving to C++

Always check that the ONNX model reproduces your Python model's outputs on a held-out sample before handing it to the C++ side. A mismatch here usually signals an unsupported operation or a data-type issue.

import onnxruntime as rt
import numpy as np

sess = rt.InferenceSession("classifier.onnx")
input_name = sess.get_inputs()[0].name
onnx_scores = sess.run(None, {input_name: X_test.astype(np.float32)})[0]

Compare onnx_scores against your original model's predictions. If they agree, you're ready to hand off the file.

4. Load and Run the Model in C++

Add ONNX Runtime as a dependency in your CMakeLists or equivalent build system, then:

#include <onnxruntime/core/providers/cpu/cpu_provider_factory.h>
#include <onnxruntime_cxx_api.h>

Ort::Env env(ORT_LOGGING_LEVEL_WARNING, "hep-inference");
Ort::Session session(env, "classifier.onnx", Ort::SessionOptions{});

// Prepare input tensor from your event's observables
std::vector<float> input_values = { /* your feature values */ };
// ... shape and run inference ...

The runtime handles all the numerical operations internally. From your C++ analysis code's point of view, you pass in a vector of floats (your observables per event) and get back a score. No Python interpreter, no dependency on your training framework.

5. Integrate With Your Experiment Framework

Most large experiment frameworks (Athena, CMSSW, and others in the HEP ecosystem) already have either official or community-supported ONNX Runtime integration layers. Check your experiment's software documentation before writing your own wrapper — there may already be a standard interface for deploying ML model C++ physics workflows, which keeps your code maintainable and auditable.

Common Pitfalls

• Data type mismatches. ONNX Runtime often expects float32; make sure your C++ and Python inputs agree.
• Unsupported ops. Some custom layers or activations don't have ONNX equivalents. Test the export immediately after training, not after months of downstream development.
• Preprocessing. Scalers and normalizations applied in Python need to travel with the model — either fold them into the ONNX graph or document them explicitly for the C++ side.

If you want deeper grounding in how classifiers and their outputs should be interpreted before you deploy them, the complete HEP ML course covers evaluation, calibration, and systematic uncertainties in a physics-native way.

Train once in Python, validate once at the boundary, and let ONNX model export in HEP do the rest — your C++ framework gets a reliable, tested classifier without anyone rewriting a line of model logic.