Convolutional Neural Networks for Jet Images

August 28, 2026 · 5 min read

If you have spent time computing jet substructure variables — trimmed mass, N-subjettiness ratios, energy correlation functions — you already know the craft: choose observables that capture the physics you care about, tune them carefully, and feed them to a classifier. That pipeline works well, but it forces you to decide upfront which features matter. Convolutional neural networks offer a different deal: represent the jet as an image and let the network discover the relevant structure on its own.

What Is a Jet Image?

A jet image is a two-dimensional pixel grid built from the particles inside a jet. The axes are typically pseudorapidity η and azimuthal angle φ — the same coordinates you already use on detector plots. Each pixel records some deposited quantity, usually transverse momentum p_T, summed over all particles that fall inside that cell. The result looks like a calorimeter readout: a sparse grid of energy deposits arranged around the jet axis.

Because CNN jet images live in a space the network can treat like a photograph, you can borrow the entire machinery of computer vision — convolutional filters, pooling layers, learned feature hierarchies — and apply it directly to jet physics.

Preprocessing: The Steps That Actually Matter

Raw jet images vary in overall scale, orientation, and reflection symmetry. Before training, three preprocessing steps are standard practice.

1. Center the Jet

Translate the image so the leading-p_T particle (or the jet axis) sits at the origin. This removes position dependence that has nothing to do with jet structure.

2. Rotate and Flip

Rotate the image so the principal axis of the p_T distribution points in a fixed direction — typically toward positive η or φ. Then reflect so the bulk of radiation sits in the same half of the image. Without this step, the network wastes capacity learning orientation invariance instead of substructure.

3. Normalize the Pixel Intensities

Divide each pixel by the total p_T of the image. This removes overall energy scale, so the classifier focuses on the pattern of deposits rather than the absolute energy. You can think of this the same way you would think about normalizing a template before a chi-square fit.

Getting preprocessing right is more important than architectural choices. Spend time here before tuning anything else.

Building the CNN

A convolutional neural network for jet image classification applies a series of learned filters that scan across the pixel grid. Early layers pick up local patterns — isolated deposits, two-prong splittings — and later layers combine those into global structure. This hierarchy mirrors how you might build substructure variables manually, except the network learns both the filters and how to combine them.

A workable starting point:

• Two or three convolutional layers, each followed by a ReLU activation and max-pooling
• A flatten step, then one or two dense layers
• A softmax output for classification (quark vs. gluon, signal vs. background, W vs. QCD)

Start simple. A small network trained carefully almost always beats a large network trained carelessly. If you want a structured path through architecture choices and training strategy, the complete HEP ML course covers these decisions in the context of real analyses.

When CNN Jet Images Outperform Hand-Crafted Variables

The honest answer is: it depends on what you are trying to discriminate.

For tasks where the discriminating physics is diffuse and high-dimensional — for example, distinguishing highly boosted objects whose decay products produce subtle radiation patterns — CNN jet images can capture correlations that are hard to encode in a small set of substructure variables. The network sees the full p_T map and can learn combinations of deposits that no single observable would expose.

For tasks where one or two observables already saturate the available information, a simple BDT on those variables will be competitive and far easier to understand, calibrate, and defend to collaborators.

A practical rule: if your hand-crafted BDT is still improving as you add more and more substructure variables, you are likely leaving information on the table that a convolutional neural network jet physics approach can recover.

Practical Considerations Before You Commit

• Simulation quality matters more for images than for high-level variables. Low-level pixel deposits are sensitive to particle multiplicity and soft radiation, which are harder to model and harder to reweight.
• Images are large inputs. Training is slower than a BDT on a handful of variables. Start with a coarser grid during development.
• Interpretability is harder. You can visualize the average image per class and the network's activation maps, but explaining why the network fires on a given event requires extra work.

If you are still getting comfortable with neural network basics, free Module 1 is a good place to build the foundation before tackling image-based models.

Takeaway

Turn your jet into a centered, rotated, normalized pixel grid, apply a modest CNN, and compare its performance against your best set of substructure variables — the gap between them is a direct measurement of how much physics your hand-crafted features were leaving behind.

References

Guest, D., Cranmer, K., & Whiteson, D. (2018). Deep Learning and its Application to LHC Physics. Annual Review of Nuclear and Particle Science, 68, 161-181. arXiv:1806.11484.