Work

Columnar data processing for HEP analysis

Year: 2019
Type: conference-paper
Abstract: In the last stages of data analysis, physicists are often forced to choose between simplicity and execution speed. In High Energy Physics (HEP), high-level languages like Python are known for ease of use but also very slow execution. However, Python is used in speed-critical data analysis in other fields of science and industry. In those fields, most operations are performed on Numpy arrays in an array programming style; this style can be adopted for HEP by introducing variable-sized, nested data structures. We describe how array programming may be extended for HEP use-cases and an implementation known as awkward-array. We also present integration with ROOT, Apache Arrow, and Parquet, as well as preliminary performance results.
Language: English
Subfield: Software
Open Access status: diamond
OpenAlex ID: W2974132935
FWCI: 0.882
Cites: 3
Cited by: 6
Related to: 10