Glossary
Application Programming Interface (API)
An Application Programming Interface (API) specifies how software components interact with each other. In practice, an API is typically provided in the form of a library that includes specifications for functions, data structures, object classes, constants, and variables. A good API makes it easier to develop programs by providing all the building blocks.
Attribute
An (HDF5) attribute is a metadata object describing the nature and/or intended usage of a primary data object. A primary data object may be a group, dataset, or committed data type. Attributes are assumed to be very small compared with primary data objects, so storing them as standard (HDF5) datasets would be inefficient.
Cursor
A cursor is a control structure used to iterate through the results returned by a previously executed query. It can be seen as an effective means of abstracting the programmer from low-level implementation details involved in accessing data stored in specific structures. Besides offering several ways to traverse result sets according to specific needs, cursors in HDFql also store result sets returned by DATA QUERY LANGUAGE (DQL) and DATA INTROSPECTION LANGUAGE (DIL) operations.
Dataset
A (HDF5) dataset is an object composed of a collection of data elements and metadata that describes the data elements, data layout, and all other information necessary to write and read the data. A dataset may be a multidimensional array of data elements, and it may have zero or more attributes.
Data type
A data type is a classification identifying one of various types of data such as integer, floating-point, or string, which determines the possible values for that type, the operations that can be done on values of that type, the meaning of the data, and the way values of that type can be stored. In other words, a data type is a classification of data that tells HDFql how the user intends to use it.
Endianness
Endianness refers to the ordering of bytes when packing words for storage in memory. In big endian format, whenever addressing memory or storing words bytewise, the most significant byte – i.e. the byte containing the most significant bit – is stored first (has the lowest address); subsequently, the following bytes are stored in order of decreasing significance, with the least significant byte – i.e. the one containing the least significant bit – stored last (having the highest address). The little endian format reverses this order: the least significant byte is addressed/stored first (lowest address) and the most significant byte is addressed/stored last (highest address).
Group
A (HDF5) group is a container structure that can hold zero or more objects (i.e. datasets, (soft) links, external links, and other groups) and can have zero or more attributes (attached to it). Every object must be a member of at least one group, except the root group (i.e. “/”), which may not belong to any group.
Hierarchical Data Format (HDF)
The Hierarchical Data Format (HDF) is a set of file formats and libraries designed to store large amounts of numerical data. It is a versatile data model that can represent complex data objects and a wide variety of metadata. HDF is supported by The HDF Group, whose mission is to ensure the continued development of HDF technologies and the continued accessibility of data currently stored in HDF.
Hyperslab
A hyperslab allows reading or writing a slice (portion) of a dataset (as opposed to its entirety). It can be a selection of a logically contiguous collection of points in a dataspace, or it can be a regular pattern of points or blocks in a dataspace.
Member
A member is an element that composes an HDF5 dataset or attribute of data type HDFQL_ENUMERATION or HDFQL_COMPOUND. It has a descriptive name that uniquely identifies it among other members at the same (nested) level (i.e. cannot be repeated) and stores a certain value in accordance with its nature (i.e. data type).
Message Passing Interface (MPI)
The Message Passing Interface (MPI) is a standardized means of exchanging messages between multiple computers running a parallel program across distributed memory. It was designed by a group of researchers from academia and industry to work on a wide variety of parallel computing architectures. MPI fosters the development of a parallel software industry and encourages the development of portable and scalable large-scale parallel applications.
Operation
An operation is an action that can be performed in HDFql, such as creating an HDF5 file or reading data from a dataset. Operations can be seen as the HDFql language itself. In HDFql, many operations exist, and they are categorized as DATA DEFINITION LANGUAGE (DDL), DATA MANIPULATION LANGUAGE (DML), DATA QUERY LANGUAGE (DQL), DATA INTROSPECTION LANGUAGE (DIL), and DATA MISCELLANEOUS LANGUAGE (DMISCL).
Parallel HDF5 (PHDF5)
The Parallel HDF5 (PHDF5) is a parallel version of HDF5, which is a set of file formats and libraries designed to store large amounts of numerical data. It leverages MPI to effectively manipulate HDF5 files in parallel across multiple computers. In HDFql, PHDF5 can be explicitly used through the CREATE FILE, USE FILE, INSERT, and SELECT operations.
Processing
Processing options enable transforming results of a query according to the programmer’s needs, such as ordering or truncating. These options are optional and may be used to create a (linear) pipeline to further process result sets returned by DATA QUERY LANGUAGE (DQL) and DATA INTROSPECTION LANGUAGE (DIL) operations.
Redirecting
Redirecting options enable reading data from the cursor in use, a (text, binary, or Excel) file, or memory (i.e. a user-defined variable) and writing it into an HDF5 dataset or attribute through a CREATE DATASET, CREATE ATTRIBUTE, or INSERT operation. They also enable writing result sets (i.e. data) returned by DATA QUERY LANGUAGE (DQL) and DATA INTROSPECTION LANGUAGE (DIL) operations into the cursor in use, a (text, binary, or Excel) file, or memory (i.e. a user-defined variable).
Result set
A result set stores the results returned by DATA QUERY LANGUAGE (DQL) and DATA INTROSPECTION LANGUAGE (DIL) operations (which can have data type HDFQL_TINYINT, HDFQL_UNSIGNED_TINYINT, HDFQL_SMALLINT, HDFQL_UNSIGNED_SMALLINT, HDFQL_INT, HDFQL_UNSIGNED_INT, HDFQL_BIGINT, HDFQL_UNSIGNED_BIGINT, HDFQL_FLOAT, HDFQL_DOUBLE, or HDFQL_VARCHAR).
Result subset
A result subset stores the results returned by DATA QUERY LANGUAGE (DQL) and DATA INTROSPECTION LANGUAGE (DIL) operations (which can have data type HDFQL_CHAR, HDFQL_VARTINYINT, HDFQL_UNSIGNED_VARTINYINT, HDFQL_VARSMALLINT, HDFQL_UNSIGNED_VARSMALLINT, HDFQL_VARINT, HDFQL_UNSIGNED_VARINT, HDFQL_VARBIGINT, HDFQL_UNSIGNED_VARBIGINT, HDFQL_VARFLOAT, HDFQL_VARDOUBLE, or HDFQL_OPAQUE).
Subcursor
A subcursor is meant to complement (i.e. help) cursors in the task of storing data of type HDFQL_CHAR, HDFQL_VARTINYINT, HDFQL_UNSIGNED_VARTINYINT, HDFQL_VARSMALLINT, HDFQL_UNSIGNED_VARSMALLINT, HDFQL_VARINT, HDFQL_UNSIGNED_VARINT, HDFQL_VARBIGINT, HDFQL_UNSIGNED_VARBIGINT, HDFQL_VARFLOAT, HDFQL_VARDOUBLE, and HDFQL_OPAQUE. In practice, when a result set is of one of these data types, only the first element of the result set is stored in the cursor (as expected), while all elements of the result set are stored in the subcursor. In other words, each position of the cursor stores the first element of the result set and also points to a subcursor that in turn stores all the elements of the result set. Similar to cursors, HDFql subcursors offer several ways to traverse result subsets.