Feed-Forward Neural Networks for HEP Classification

August 10, 2026 · 5 min read

If you have spent time fitting Boosted Decision Trees or cut-based selections, the jump to neural networks can feel larger than it really is. At its core, a feed-forward neural network is just a sequence of weighted sums followed by simple nonlinear functions — the same building blocks you already use when you fit a polynomial or compute a likelihood. Understanding that structure makes the whole framework feel far less mysterious.

What a Feed-Forward Network Actually Does

A multilayer perceptron (the classical name for a feed-forward net) takes a vector of input features — detector observables, kinematic variables, whatever you have — and passes them through a series of layers. Each layer computes a weighted sum of its inputs, adds a bias term, and then applies a nonlinearity. That nonlinearity is what prevents the whole stack from collapsing into a single linear transformation.

Think of it this way: one layer is a linear classifier, which is roughly equivalent to a single cut in a projected variable. Stack two or more layers with nonlinearities between them, and the network can carve out curved, non-convex decision boundaries in feature space — boundaries that no single linear cut or even a simple polynomial could describe.

The Key Components

• Input layer: Each node corresponds to one observable. Transverse momentum, pseudorapidity, invariant mass, isolation variables — these are your features. Normalization matters here; large dynamic ranges can destabilize training.
• Hidden layers: Each neuron computes a weighted sum of the previous layer's outputs, passes it through an activation function (commonly ReLU, a simple piecewise-linear function), and sends the result forward.
• Output layer: For binary signal-versus-background classification, you typically end with a single neuron using a sigmoid activation, which maps any real number to the interval (0, 1). You can read this output as an approximate posterior probability that the event is signal.

Training as Minimization

Training a neural network is an optimization problem — you are minimizing a loss function over the network's weights. For binary classification, the standard loss is binary cross-entropy, which you can recognize as the negative log-likelihood of a Bernoulli model. If you have ever maximized a likelihood by hand, you already understand what the optimizer is doing; here it just operates in a very high-dimensional weight space using gradient descent.

The gradient is computed via backpropagation, an efficient application of the chain rule that propagates the loss gradient backward through the layers. In practice, you never implement this yourself; frameworks like PyTorch and TensorFlow handle it automatically.

Watching for Overtraining

Overtraining in a neural network is the same disease you know from fitting too many polynomial terms to a histogram: the model learns the statistical fluctuations in the training sample rather than the underlying physics. The symptom is a loss that improves on training data but worsens on held-out validation data.

Standard defenses include:

• Hold-out validation: always keep a fraction of your data out of training and monitor the validation loss epoch by epoch.
• Early stopping: halt training when the validation loss stops improving.
• Dropout: randomly zero a fraction of neurons during each training step, which forces the network to distribute its learned representation across many pathways.
• Weight regularization: add an L2 penalty on the weights to discourage large parameter values.

From Network Output to Physics Quantities

The network output score behaves like a likelihood ratio — it orders events from most background-like to most signal-like. You can sweep a threshold on this score to produce an efficiency-versus-purity curve, which is exactly the ROC curve familiar from cut optimization. A perfect classifier hugs the upper-left corner; a random one falls on the diagonal.

One important habit: always check what your network has actually learned. If signal and background differ in a variable you did not intend to exploit — such as a reconstruction artifact or a variable correlated with pile-up — the network will find it. Plotting the distribution of input features separately for high-score and low-score events is a fast sanity check that has caught many subtle problems before they reached a paper.

A Minimal Starting Point for Your Analysis

For a typical HEP binary classification task, a reasonable starting architecture is a small network with two or three hidden layers and a modest number of neurons per layer. Begin with the set of high-level kinematic observables you would have used in a cut-based analysis. Train, check the validation loss curve, inspect the score distributions for signal and background, and then iterate.

You do not need a large or exotic architecture to get useful results. The most impactful decision is usually the choice of input features, not the depth of the network.

For a full treatment of neural network classification in particle physics — including hands-on exercises, common failure modes, and how to connect network outputs to statistical inference — the HEP ML full course covers these topics in detail. You can also explore the free Module 1 to get a feel for the approach before committing. Related techniques like gradient boosting are covered in our blog if you want to benchmark a neural network against a classical method first.

A feed-forward neural network is, in the end, a flexible function approximator trained to distinguish signal from background — start simple, validate carefully, and let the data tell you when you need more complexity.

References

Baldi, P., Sadowski, P., & Whiteson, D. (2014). Searching for exotic particles in high-energy physics with deep learning. Nature Communications, 5, 4308. arXiv:1402.4735.