High-energy physics has a data problem, or rather, a data volume problem. The four major LHC experiments collectively accumulated over 2 exabytes of ROOT data through Runs 1–3. With the High-Luminosity LHC (HL-LHC) expected to begin data taking by the end of this decade, that number is projected to balloon beyond 10 exabytes under management. To put this in perspective, 90% of all LHC data will be produced during the HL-LHC era. Storage alone consumes roughly half of the total HEP computing budget. Every percentage point of compression matters. Every microsecond of read latency, multiplied across billions of events, adds up to real compute hours and real cost.
For a quarter of a century, TTree, ROOT's columnar data storage format, has been the backbone of essentially every HEP analysis pipeline. It stores the events, branches, and leaves that physicists query, plot, and fit every day. But TTree was designed in the late 1990s, before there was even a ratified C++ standard, before parallel computing was mainstream, and before SSDs, NVMe drives, and object stores existed. The computing landscape of 2025 is unrecognizable compared to 1995. The question is no longer can we keep TTree alive? but rather can we afford not to replace it?
Then we have RNTuple, ROOT's next-generation I/O subsystem, built from the ground up to be the designated data format for LHC data starting in Run 4.
What Exactly Is TTree, and Why Has It Worked So Well?
Before discussing RNTuple, it's worth appreciating what TTree got right. TTree is a columnar storage system, meaning that instead of writing complete event records one after another (row-wise), it stores all values of a given data member ("branch") contiguously on disk. If you only want to plot the transverse momentum () of muons across 100 million events, TTree lets you read just the column without touching anything else, charges, vertex positions and isolation variables, none of it gets loaded into memory.
This columnar design was ahead of its time. TTree also provided seamless C++ integration: you could store arbitrary C++ objects, including nested structures and collections, without explicitly defining data schemas. You just handed it your classes, and ROOT's dictionary system took care of serialization. TTree organized data into "baskets" (compressed blocks of column data) and "clusters" (groups of entries that form a self-contained unit for I/O). The combination of columnar layout, transparent compression, and deep C++ integration made TTree the de facto standard for HEP event data.
But TTree also accumulated 25 years of design debt.
Where TTree Shows Its Age
Several fundamental limitations have become increasingly painful as the data volumes and hardware landscape have evolved:
Big-endian byte ordering.
TTreeserializes data in big-endian format, a legacy choice from an era when SPARC and PowerPC architectures were common. Today, virtually every system running HEP workloads,x86andARMis little-endian. This means that every read operation incurs a byte-swap overhead, and uncompressed TTree data cannot be directly memory-mapped without a transformation step. It's a constant, invisible tax on every I/O operation.No data checksumming.
TTreehas no built-in integrity verification for the data pages themselves. Bit flips, silent corruption during network transfer, or partial writes can go undetected. When you're managing exabytes of data across a global distributed infrastructure, the absence of systematic checksumming is a real operational risk. Problems may only surface as mysterious analysis anomalies months after the corruption occurred.Limited parallel write scalability.
TTreewas designed for single-threaded writing. While ROOT has bolted on some multi-threading support over the years through implicit multi-threading and basket-level parallelism, the fundamental architecture was never designed for the kind of many-core scalability that modern hardware demands. Writing a singleTTreefrom multiple threads requires careful synchronization, and the result is often far from linear scalability.Suboptimal on modern storage hardware. NVMe SSDs can deliver gigabytes per second of sequential throughput and tens of thousands of IOPS. Object stores (S3, DAOS, EOS) represent a fundamentally different storage paradigm.
TTreewas designed for spinning disks accessed through POSIX file semantics over NFS or XRootD. Its I/O patterns, relatively small, synchronous reads, leave significant performance on the table when running on modern storage. There's no support for asynchronous I/O by default, no Direct I/O support, and no native object store integration.Compression inefficiencies.
TTree's on-disk layout and basket structure, while functional, don't maximize compression ratios for modern algorithms. The way it handles variable-length collections (likestd::vector<float>) mixes offset data with payload data in ways that reduce the compressor's ability to find patterns. The default compression algorithm waszlib, which is robust but slow and not state-of-the-art for compression ratio.API brittleness. The
TTreeAPI predates modern C++ idioms. Raw pointers, manual memory management, runtime type resolution that can silently produce wrong results if you mistype a branch name or use the wrong type, these are all real sources of bugs in analysis code. TheSetBranchAddresspattern is notoriously error-prone.
None of these are flaws in the context of when TTree was designed. They are consequences of a format that has been remarkably successful for 25 years but was never designed for the world we now inhabit.
RNTuple: A Clean-Sheet Redesign
RNTuple (short for "nested tuple") is not an incremental improvement to TTree. It is a backwards-incompatible redesign of both the binary on-disk format and the C++ API. The ROOT team, led by Jakob Blomer at CERN, made the deliberate decision to break compatibility with TTree in order to fully exploit optimization opportunities that would be impossible under the constraints of backward compatibility. This was not a decision taken lightly, but the HL-LHC data challenge made it necessary.
The design of RNTuple draws on the ROOT team's quarter-century of experience with TTree, combined with best practices from modern industry columnar formats like Apache Parquet and Arrow. The result is purpose-built for the unique characteristics of HEP data: statistically independent events, complex nested data structures, columnar access patterns, and write-once-read-many workflows.
RNTuple's internal architecture is organized into four well-separated layers:
Event Iteration Layer: The user-facing API. Provides convenient, type-safe interfaces for looping over events, creating models, reading and writing entries. This is where
RNTupleReader,RNTupleWriter, andRNTupleModellive.Logical Layer: Maps complex C++ types (including nested collections,
std::variant,std::optional, user-defined classes) onto flat columns of fundamental types. This is where the recursive decomposition of C++ objects into serializable fields happens.Primitives Layer: Groups ranges of column elements into pages, the basic unit of compression and I/O. A page contains a contiguous range of values for a single column.
Storage Layer: Handles the actual I/O of pages and metadata (headers, footers, page lists). This layer abstracts over different storage backends: local files, XRootD, object stores (DAOS, S3), NVMe with Direct I/O.
This layered design has practical consequences. Adding support for a new C++ type means only touching the logical layer. Adding support for a new storage backend means only touching the storage layer. The concerns are genuinely separated.
On-Disk Format: What Changed and Why
The first production version of the RNTuple binary format specification was released in November 2024 as part of ROOT 6.34. This is a formally specified format with a public specification document, a first for ROOT, and a significant step for interoperability with third-party tools (like Uproot in Python).
Here are the key architectural differences from TTree:
Little-endian byte ordering.
RNTupleuses the native byte order of modern architectures. Uncompressed data can be directly memory-mapped without byte-swapping, eliminating an entire class of overhead.Strict separation of offset and payload data. For variable-length collections (e.g.,
std::vector<float>),RNTuplestores the index/offset column separately from the payload column. InTTree, these are interleaved within the same basket, which hurts both compression and random access.RNTuple's separation allows the compressor to work on homogeneous data, dramatically improving compression ratios.Pages instead of baskets. The basic unit of compressed data in
RNTupleis a page, a contiguous range of values for a single column. Pages within a cluster are reordered on write to ensure locality of pages belonging to the same column. This is in contrast toTTree's baskets, which have more complex and less predictable layout patterns.Clusters with structured metadata. Like
TTreeclusters,RNTupleclusters are self-contained blocks of consecutive entries. But RNTuple stores cluster metadata (page locations, element ranges) in a compressed footer, separate from the data itself. The header contains the schema (field and column definitions), and the footer contains the cluster index. This clean separation enables efficient metadata access without touching the data pages.64-bit xxHash3 checksums everywhere. Every page, the header, the footer, and the anchor are checksummed. Data integrity is verified on every read. Silent corruption is caught immediately, not months later when an analysis produces a mysterious spike in a distribution. This alone is a transformative improvement for an experiment managing petabytes across a worldwide grid.
Forward and backward compatibility mechanisms. The specification includes explicit versioning and feature flags to allow future ROOT versions to add new capabilities (new metadata fields, new column types) while old ROOT versions can still read the core data. Schema evolution, the ability to add, remove, or change the type of fields, is supported.
Native support for modern types.
std::variant,std::optional, half-precision floats (std::float16_t), and other modern C++ types are first-class citizens, not afterthoughts bolted on through workarounds.
The Numbers: Storage and Throughput
The performance claims are backed by extensive benchmarking across real experiment data models. Here's what the ROOT team has demonstrated:
- Storage efficiency: For CMS NanoAODs and ATLAS PHYSLITE files,
RNTupleproduces files that are 10–50% smaller than equivalentTTreefiles with the same compression algorithm. A CMS NanoAOD study showed a 39% size reduction withzstdcompression. For ATLASDAOD_PHYS, the savings are 20–35%.RNTuplealso switched the default compression fromzlib(TTree's default) tozstd, which provides a better compression ratio and faster decompression.
When you multiply these savings across exabytes, the cost implications for storage and network bandwidth are staggering.
Read throughput:
RNTupleconsistently achieves multiple times faster read throughput compared toTTree. The speedup factors vary by workload and storage medium, but factors of 2–5x are typical for analysis benchmarks using RDataFrame with real experiment data on SSD storage. The main contributors to higher throughput are: smaller input data to begin with, asynchronous reading by default, parallel I/O to SSDs, and fewer instructions in the hot I/O code path.Write performance and scalability:
RNTuplesupports efficient multithreaded writing with near-linear scalability. The parallel writer design uses buffered pages and vector writes, when a cluster is committed, all pages are sealed and written in a single operation. This is fundamentally more scalable than TTree's approach, where parallel writing requires careful coordination around basket buffers.Comparison with industry formats: The ROOT team has also benchmarked
RNTupleagainstHDF5and Apache Parquet.RNTupleconsistently produces smaller files and higher throughput for HEP workloads. This is not because HDF5 or Parquet are poor formats , they're excellent for their target use cases. But HEP data has unique characteristics (deeply nested variable-length collections, columnar access to a small subset of hundreds of branches) thatRNTupleis specifically optimized for.
The API: Modern, Safe, and Familiar (Enough)
RNTuple's C++ API follows modern core guidelines: type-safe templates, smart pointer ownership semantics, and RResult<> error handling that avoids silent failures. Compare:
// TTree: runtime type matching, raw pointers, silent failure modes
float pt;
tree->SetBranchAddress("muon_pt", &pt); // typo? wrong type? you'll only know at runtime
// RNTuple: compile-time type safety, clear ownership
auto model = RNTupleModel::Create();
auto pt = model->MakeField<float>("muon_pt");
auto reader = RNTupleReader::Open(std::move(model), "Events", "data.root");For analysis physicists, the most important thing is that RDataFrame works identically with both TTree and RNTuple. If your analysis is written using RDataFrame, switching from TTree to RNTuple is essentially a one-line change , or in ROOT 6.32+, RDataFrame auto-detects the format entirely. Your cuts, histograms, and column definitions don't change at all.
For framework developers (reconstruction, simulation, derivation), RNTuple provides the RNTupleWriter interface with model creation, field definition, and entry-based filling that maps naturally onto existing TTree-based workflows.
The Migration Path
The transition from TTree to RNTuple is already underway. ATLAS, CMS, and LHCb have all integrated initial RNTuple support into their experiment frameworks. The timeline looks like this:
- ROOT 6.34 (November 2024): First production release of the on-disk binary format specification. All future ROOT versions will read data written with this format.
- ROOT 6.36 (Q2 2025): First production APIs following an external review by the HEP Center for Computational Excellence.
- Run 4 (HL-LHC, late 2020s):
RNTupleis the designated format for new LHC data.
For existing data, ROOT provides RNTupleImporter, a tool that converts TTree datasets to RNTuple:
auto importer = RNTupleImporter::Create("input.root", "Events", "output.root");
importer->Import();And critically: TTree is not going away. It will remain in ROOT indefinitely. The exabytes of existing TTree data will continue to be readable. The transition is about new data and new analyses, not about obsoleting the past.
Why This Matters Beyond the Numbers
The move from TTree to RNTuple is not just a performance optimization. It represents a fundamental rethinking of how HEP stores and accesses data, informed by both 25 years of TTree experience and the realities of modern computing:
Robustness for the HL-LHC era. Strict checksumming, formal specification, and well-defined compatibility guarantees mean that the exabytes of HL-LHC data will be stored in a format designed for long-term reliability. When experiments need to access Run 4 data in 2045, the format's formal specification ensures that third-party readers can be written even if the ROOT codebase has evolved significantly.
Hardware co-evolution. NVMe drives with Direct I/O, distributed object stores, analysis facilities with fast local caches , the storage landscape is diversifying. RNTuple's storage layer abstraction is designed to exploit these systems, not just tolerate them. Asynchronous, parallel I/O is the default, not an afterthought.
Interoperability. The formal binary format specification enables implementations in languages beyond C++, Julia, Rust, Python (through Uproot), without reverse-engineering the source code. This is crucial as the field increasingly adopts polyglot analysis ecosystems.
Cost. When storage is half your computing budget and you're looking at 10+ exabytes, a 30% reduction in file size translates directly to millions of dollars (or Swiss francs) in savings. Faster throughput means shorter analysis turnaround and fewer CPU-hours consumed.
The ROOT team is continuing to develop RNTuple with several areas of active work: precision cascades for separating lossy compression levels into different files, deeper integration with analysis facilities, GPU-friendly data transfer mechanisms, and schema evolution maturity. The collaboration with experiments is bidirectional, early adopters provide feedback that shapes the API, and the ROOT team ensures that the format supports the full breadth of experiment data models.
For anyone in the field: try it out. Convert a TTree to RNTuple with the importer, run your RDataFrame analysis, and see the difference yourself. The format is production-ready on disk, the APIs are stabilizing rapidly, and the performance improvements are real and measured.
After 25 years of TTree, the future of HEP I/O is being written, in little-endian, checksummed, and very, very fast.
References and further reading:
- ROOT Blog: First Release of the RNTuple On-Disk Format (Nov 2024)
- ROOT Blog: RNTuple, Where Are We Now and What's Next? (Jan 2024)
- J. Blomer et al., "ROOT's RNTuple I/O Subsystem: The Path to Production," EPJ Web Conf. 295, 06020 (2024)
- J. Lopez-Gomez, J. Blomer, "RNTuple Performance: Status and Outlook," J. Phys. Conf. Ser. 2438, 012118 (2023)
- J. Blomer et al., "Evolution of the ROOT Tree I/O," EPJ Web Conf. 245, 02030 (2020)
- RNTuple Binary Format Specification v1.0.0.0
- ROOT RNTuple and EOS: The Next Generation of Event Data I/O (CHEP 2024)