Work

An Efficient Format for Nearly Constant-Time Access to Arbitrary Time Intervals in Large Trace Files

Year: 2008

Type: article

Abstract: A powerful method to aid in understanding the performance of parallel applications uses log or trace files containing time-stamped events and states (pairs of events). These trace files can be very large, often hundreds or even thousands of megabytes. Because of the cost of accessing and displaying such files, other methods are often used that reduce the size of the tracefiles at the cost of sacrificing detail or other information. This paper describes a hierarchical trace file format that provides for display of an arbitrary time window in a time independent of the total size of the file and roughly proportional to the number of events within the time window. This format eliminates the need to sacrifice data to achieve a smaller trace file size (since storage is inexpensive, it is necessary only to make efficient use of bandwidth to that storage). The format can be used to organize a trace file or to create a separate file ofannotationsthat may be used with conventional trace files. We present an analysis of the time to access all of the events relevant to an interval of time and we describe experiments demonstrating the performance of this file format.

Source: Scientific Programming

DOI: 10.1155/2008/749874

Authors: Anthony Chan, William Gropp, Ewing Lusk

Institutions: Argonne National Laboratory, University of Illinois Urbana-Champaign

Language: English

Topic: Parallel Computing and Optimization Techniques

Subfield: Hardware and Architecture

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Estimated APC paid: $2,000

Funders: U.S. Department of Energy, Office of Science, Advanced Scientific Computing Research