How a Single Decision Tree Works, From Root to Leaf

August 7, 2026 · 5 min read

If you have heard the phrase "boosted decision trees" in a journal club and nodded along, you are not alone. Most introductions jump straight to the ensemble and skip the single tree underneath. That is a shame, because understanding one tree — how it splits, what it optimises, and where it breaks down — is the foundation for everything that comes after, including gradient boosting and random forests. This article walks you through a single decision tree, step by step, in language that maps onto what you already know from HEP analysis.

What a Decision Tree Actually Is

A decision tree is a sequence of binary cuts on your input variables, arranged hierarchically. You start with all your events at the top — the root — and at each step the algorithm chooses one variable and one threshold to divide the sample into two groups. Each division is called a node, and the process repeats on each sub-sample until you reach a stopping condition. The final groups are called leaves, and every event that lands in a leaf gets the same prediction: signal or background, or a continuous score if you are doing regression.

Think of it as an automated rectangular cut-flow. Instead of you choosing pT > 30 GeV and |η| < 2.4 by hand, the tree searches over all your observables — your features, in ML language — and all possible thresholds to find the cut that best separates signal from background at each step. The decision tree splitting rule is what governs that search.

The Splitting Rule: Purity as the Objective

At every node the algorithm asks: given all the events sitting here right now, which single cut produces the purest daughter samples? Purity, in this context, means one class dominates. A node containing only signal events is pure signal; a node split evenly between signal and background is maximally impure.

The standard measure of impurity is the Gini index. If you call the signal fraction in a node p, then the Gini index is p(1 − p). It reaches zero when the node is pure (all signal or all background) and peaks when the sample is half-and-half. The algorithm evaluates every possible variable and every possible threshold, computes the weighted Gini index of the two daughter nodes that would result, and picks the split that achieves the largest reduction in impurity. This reduction is called the information gain, and choosing the split that maximises it is the decision tree splitting rule in practice.

For physicists, this maps naturally onto optimising signal-to-noise. A pure leaf is a high-purity control or signal region. The Gini criterion is simply a smooth, differentiable proxy for the kind of purity you already care about in cut-based selections.

Growing the Tree, Step by Step

Step 1 — Present the full training sample at the root

Every event, signal and background alike, sits at the top node.

Step 2 — Search for the best split

Loop over every feature and every unique threshold value. Compute the Gini gain for each candidate split. Select the winner.

Step 3 — Divide and recurse

Send events left or right according to the winning cut. Now treat each daughter node as an independent sub-problem and repeat Step 2 on each.

Step 4 — Stop when a criterion is met

Common stopping conditions include: the node contains fewer than some minimum number of events, the tree has reached a maximum depth, or no split improves purity by more than a small threshold. These hyperparameters control tree complexity and are your main handles against overtraining.

Step 5 — Assign leaf scores

Once the tree stops growing, each leaf gets a score — typically the signal fraction of the training events that landed there. A new event is scored by dropping it down the tree and reading off its leaf's value.

Why One Tree Is Not Enough

A single decision tree explained in this way has an obvious virtue: it is completely interpretable. You can print the splits, read them like a cut table, and understand exactly why any event was classified the way it was. That interpretability is real and valuable.

The problem is variance. A deep tree memorises the training sample. Shift the training set slightly and the tree changes dramatically — analogous to overfitting a chi-square with too many free parameters. A shallow tree avoids that instability but misses real structure in the data. There is no single depth that solves both problems simultaneously.

This is precisely why ensembles were invented. Boosting trains many shallow trees in sequence, each one correcting the errors of the previous, so the final model is stable and expressive without any single component being overfit. If you want to understand how gradient boosting and BDTs build on what you just learned, or go deeper into the full workflow from preprocessing to deployment, the complete HEP ML course covers all of it in a physics-native context. You can also start with the free Module 1 to get a feel for the approach before committing.

A decision tree in a single physics analysis gives you a transparent, auditable baseline — and knowing exactly how it works is what lets you trust, debug, and ultimately improve it.

The single decision tree is not the destination; it is the vocabulary you need to read everything else.

References

Roe, B. P., Yang, H.-J., Zhu, J., Liu, Y., Stancu, I., & McGregor, G. (2005). Boosted decision trees as an alternative to artificial neural networks for particle identification. Nuclear Instruments and Methods in Physics Research A, 543, 577-584. arXiv:physics/0408124.