# 6 Language

HDFql is a high-level (declarative) language to manage HDF5 files in a simple and natural way. It was designed to be similar to SQL (wherever possible), so that its learning effort is kept to a minimum while still providing great power and flexibility to the programmer. This chapter describes data types, processing options to further transform result sets, redirecting options to read/write data/result sets from/into disparate input/output sources, and operations (i.e. the language itself) available in HDFql. It also introduces the text formatting conventions used throughout this chapter to describe HDFql operations ([Table 6.1](#table-6-1)), as well as a summary of existing operations ([Table 6.2](#table-6-2)). Before continuing, it is recommended to first read the HDF5 User’s Guide available at <https://support.hdfgroup.org/documentation/hdf5/latest/_u_g.html> to facilitate understanding of the current chapter.

| **Convention** | **Description** | **Example** |
|---|---|---|
| **Bold** | Keyword that must be typed exactly as shown | **CREATE** |
| _Italic_ | Value that the programmer must supply | _dataset_name_ |
| Between brackets ([]) | Optional keyword/value | [**DATASET**] |
| Between braces ({}) | Logical grouping of keywords/values (to ease understanding) | {[**TRUNCATE**] **BINARY FILE** _file_name_} |
| Separated with a pipe (\|) | Set of keywords/values from which one must be chosen | **GROUP** \| **DATASET** \| **ATTRIBUTE** |
| Asterisk (\*) | Keyword/value that may be supplied zero or more times | _group_name_ [**,** _group_name_]\* |
| Plus (+) | Keyword/value that must be supplied one or more times | {**,** [_chunk_dim_]}+ |

Table 6.1 – HDFql operations text formatting conventions

| **Operation** | **Description** |
|---|---|
| [CREATE DIRECTORY](6-4-1-create-directory.md) | Create a directory |
| [CREATE FILE](6-4-2-create-file.md) | Create an HDF5 file |
| [CREATE GROUP](6-4-3-create-group.md) | Create an HDF5 group |
| [CREATE DATASET](6-4-4-create-dataset.md) | Create an HDF5 dataset |
| [CREATE ATTRIBUTE](6-4-5-create-attribute.md) | Create an HDF5 attribute |
| [CREATE \[SOFT \| HARD\] LINK](6-4-6-create-soft-hard-link.md) | Create an HDF5 soft or hard link |
| [CREATE EXTERNAL LINK](6-4-7-create-external-link.md) | Create an HDF5 external link |
| [ALTER DIMENSION](6-4-8-alter-dimension.md) | Alter (i.e. change) the dimensions of an existing HDF5 dataset |
| [RENAME DIRECTORY](6-4-9-rename-directory.md) | Rename (or move) an existing directory |
| [RENAME FILE](6-4-10-rename-file.md) | Rename (or move) an existing file |
| [RENAME](6-4-11-rename.md) | Rename (or move) an existing HDF5 group, dataset, attribute, (soft) link, or external link |
| [COPY FILE](6-4-12-copy-file.md) | Copy an existing file |
| [COPY](6-4-13-copy.md) | Copy an existing HDF5 group, dataset, attribute, (soft) link, or external link |
| [DROP DIRECTORY](6-4-14-drop-directory.md) | Drop (i.e. delete) an existing directory |
| [DROP FILE](6-4-15-drop-file.md) | Drop (i.e. delete) an existing file |
| [DROP](6-4-16-drop.md) | Drop (i.e. delete) an existing HDF5 group, dataset, attribute, (soft) link, or external link |
| [INSERT](6-5-1-insert.md) | Insert (i.e. write) data into an HDF5 dataset or attribute |
| [SELECT](6-6-1-select.md) | Select (i.e. read) data from an HDF5 dataset or attribute |
| [SHOW FILE VALIDITY](6-7-1-show-file-validity.md) | Show (i.e. get) the validity of a file (i.e. whether it is a valid HDF5 file or not) |
| [SHOW USE DIRECTORY](6-7-2-show-use-directory.md) | Show (i.e. get) the working directory currently in use |
| [SHOW USE FILE](6-7-3-show-use-file.md) | Show (i.e. get) the HDF5 file currently in use or check if a certain HDF5 file is used (i.e. opened) |
| [SHOW ALL USE FILE](6-7-4-show-all-use-file.md) | Show (i.e. get) all HDF5 files in use (i.e. open) |
| [SHOW USE GROUP](6-7-5-show-use-group.md) | Show (i.e. get) the HDF5 group currently in use |
| [SHOW](6-7-6-show.md) | Show (i.e. get) HDF5 objects (i.e. groups, datasets, attributes, (soft) links, and external links) or check for the existence of an object |
| [SHOW TYPE](6-7-7-show-type.md) | Show (i.e. get) the type of an HDF5 object (i.e. group, dataset, or attribute) |
| [SHOW DATA TYPE](6-7-8-show-data-type.md) | Show (i.e. get) the data type of an HDF5 dataset or attribute or of its members |
| [SHOW MEMBER](6-7-9-show-member.md) | Show (i.e. get) the members of an HDF5 dataset or attribute |
| [SHOW MASK](6-7-10-show-mask.md) | Show (i.e. get) the (filter) mask of an HDF5 dataset |
| [SHOW ENDIANNESS](6-7-11-show-endianness.md) | Show (i.e. get) the endianness of an HDF5 dataset or attribute or of its members |
| [SHOW CHARSET](6-7-12-show-charset.md) | Show (i.e. get) the charset of an HDF5 dataset or attribute or of its members |
| [SHOW STORAGE TYPE](6-7-13-show-storage-type.md) | Show (i.e. get) the storage type (layout) of an HDF5 dataset |
| [SHOW STORAGE ALLOCATION](6-7-14-show-storage-allocation.md) | Show (i.e. get) the storage allocation of an HDF5 dataset |
| [SHOW STORAGE DIMENSION](6-7-15-show-storage-dimension.md) | Show (i.e. get) the storage dimensions of an HDF5 dataset |
| [SHOW DIMENSION](6-7-16-show-dimension.md) | Show (i.e. get) the dimensions of an HDF5 dataset or attribute |
| [SHOW ORDER](6-7-17-show-order.md) | Show (i.e. get) the (creation) order strategy of an HDF5 group or dataset |
| [SHOW PAD](6-7-18-show-pad.md) | Show (i.e. get) the padding of an HDF5 dataset or attribute or of its members |
| [SHOW TAG](6-7-19-show-tag.md) | Show (i.e. get) the tag of an HDF5 dataset or attribute or of its members |
| [SHOW OFFSET](6-7-20-show-offset.md) | Show (i.e. get) the member offsets of an HDF5 dataset or attribute |
| [SHOW FILL TYPE](6-7-21-show-fill-type.md) | Show (i.e. get) the fill type of an HDF5 dataset |
| [SHOW FILL](6-7-22-show-fill.md) | Show (i.e. get) the fill values of an HDF5 dataset |
| [SHOW FILE SIZE](6-7-23-show-file-size.md) | Show (i.e. get) the size (in bytes) of a file or of the HDF5 file currently in use |
| [SHOW USERBLOCK SIZE](6-7-24-show-userblock-size.md) | Show (i.e. get) the size (in bytes) of a user-defined block of data stored within an HDF5 file or within the HDF5 file currently in use |
| [SHOW USERBLOCK](6-7-25-show-userblock.md) | Show (i.e. get) the user-defined block of data stored within an HDF5 file or within the HDF5 file currently in use |
| [SHOW ALIGNMENT](6-7-26-show-alignment.md) | Show (i.e. get) the alignment parameter values for accessing HDF5 files |
| [SHOW SIZE](6-7-27-show-size.md) | Show (i.e. get) the size (in bytes) of an HDF5 dataset or attribute |
| [SHOW HDFQL VERSION](6-7-28-show-hdfql-version.md) | Show (i.e. get) the version of the HDFql library |
| [SHOW HDF5 VERSION](6-7-29-show-hdf5-version.md) | Show (i.e. get) the version of the HDF5 library used by HDFql |
| [SHOW MPI VERSION](6-7-30-show-mpi-version.md) | Show (i.e. get) the version of the MPI library used by HDFql |
| [SHOW DIRECTORY](6-7-31-show-directory.md) | Show (i.e. get) the directories within a directory or check for the existence of a directory |
| [SHOW FILE](6-7-32-show-file.md) | Show (i.e. get) the files within a directory or check for the existence of a file |
| [SHOW EXECUTE STATUS](6-7-33-show-execute-status.md) | Show (i.e. get) the status of the last executed operation |
| [SHOW LIBRARY BOUNDS](6-7-34-show-library-bounds.md) | Show (i.e. get) the library bound values for creating or opening HDF5 files |
| [SHOW CACHE](6-7-35-show-cache.md) | Show (i.e. get) the cache parameter values for accessing HDF5 files or datasets |
| [SHOW ATOMIC](6-7-36-show-atomic.md) | Show (i.e. get) the atomicity for accessing HDF5 files in an MPI environment |
| [SHOW MPI SIZE](6-7-37-show-mpi-size.md) | Show (i.e.get) the number (i.e. size) of processes associated with the default MPI communicator (MPI_COMM_WORLD) or with a user-defined MPI communicator |
| [SHOW MPI RANK](6-7-38-show-mpi-rank.md) | Show (i.e. get) the rank of the calling process associated with the default MPI communicator (MPI_COMM_WORLD) or with a user-defined MPI communicator |
| [SHOW MPI ROOT](6-7-39-show-mpi-root.md) | Show (i.e. get) the MPI root process rank (for gathering processed (HDF5) data by other MPI processes) |
| [SHOW MPI HINT](6-7-40-show-mpi-hint.md) | Show (i.e. get) MPI hints for accessing HDF5 files in an MPI environment |
| [SHOW EXTERNAL LINK PREFIX](6-7-41-show-external-link-prefix.md) | Show (i.e. get) the prefix to prepend to file names stored in HDF5 external links |
| [SHOW FLUSH](6-7-42-show-flush.md) | Show (i.e. get) the status of automatic flushing |
| [SHOW THREAD](6-7-43-show-thread.md) | Show (i.e. get) the number of (CPU) threads to use when executing operations that support parallelism |
| [SHOW PLUGIN PATH](6-7-44-show-plugin-path.md) | Show (i.e. get) the path where plugins (in the form of shared libraries) are searched for and dynamically loaded by HDFql/HDF5 library |
| [SHOW DECIMAL SCALE](6-7-45-show-decimal-scale.md) | Show (i.e. get) the decimal scale (i.e. number of decimal places) to use when writing fractional numbers into a text or Excel file |
| [SHOW DEBUG](6-7-46-show-debug.md) | Show (i.e. get) the status of the debug mechanism |
| [USE DIRECTORY](6-8-1-use-directory.md) | Use (i.e. open) a directory for subsequent operations |
| [USE FILE](6-8-2-use-file.md) | Use (i.e. open) an HDF5 file for subsequent operations |
| [USE GROUP](6-8-3-use-group.md) | Use (i.e. open) an HDF5 group for subsequent operations |
| [FLUSH](6-8-4-flush.md) | Flush the entire virtual HDF5 file (global) or the specific HDF5 file (local) currently in use |
| [CLOSE FILE](6-8-5-close-file.md) | Close the HDF5 file currently in use or a certain HDF5 file in use (i.e. opened) |
| [CLOSE ALL FILE](6-8-6-close-all-file.md) | Close all HDF5 files in use (i.e. opened) |
| [CLOSE GROUP](6-8-7-close-group.md) | Close the HDF5 group currently in use (i.e. opened) |
| [SET USERBLOCK](6-8-8-set-userblock.md) | Set (i.e. write) a user-defined block of data stored within an HDF5 file or within the HDF5 file currently in use |
| [SET ALIGNMENT](6-8-9-set-alignment.md) | Set the alignment for accessing HDF5 files |
| [SET LIBRARY BOUNDS](6-8-10-set-library-bounds.md) | Set the library bound values for creating and opening HDF5 files |
| [SET CACHE](6-8-11-set-cache.md) | Set the cache parameter values for accessing HDF5 files or datasets |
| [SET ATOMIC](6-8-12-set-atomic.md) | Set the atomicity for accessing HDF5 files in an MPI environment to enabled or disabled |
| [SET MPI ROOT](6-8-13-set-mpi-root.md) | Set the MPI root process rank for gathering processed (HDF5) data by other MPI processes |
| [SET MPI HINT](6-8-14-set-mpi-hint.md) | Set MPI hints for accessing HDF5 files in an MPI environment |
| [SET EXTERNAL LINK PREFIX](6-8-15-set-external-link-prefix.md) | Set the prefix to prepend to file names stored in HDF5 external links |
| [SET FLUSH](6-8-16-set-flush.md) | Set the automatic flushing of the entire virtual HDF5 file (global) or only the HDF5 file (local) currently in use to enabled or disabled |
| [SET THREAD](6-8-17-set-thread.md) | Set the number of (CPU) threads to use when executing operations that support parallelism |
| [SET PLUGIN PATH](6-8-18-set-plugin-path.md) | Set the path where plugins (in the form of shared libraries) are searched for and dynamically loaded by HDFql/HDF5 library |
| [SET DECIMAL SCALE](6-8-19-set-decimal-scale.md) | Set the decimal scale (i.e. number of decimal places) to use when writing fractional numbers into a text or Excel file |
| [SET DEBUG](6-8-20-set-debug.md) | Set the debug mechanism to enabled or disabled |

Table 6.2 – HDFql operations

## In this section

- [6.1 Data Types](6-1-data-types.md)
- [6.2 Processing](6-2-processing.md)
- [6.3 Redirecting](6-3-redirecting.md)
- [6.4 Data Definition Language (DDL)](6-4-data-definition-language-ddl.md)
- [6.5 Data Manipulation Language (DML)](6-5-data-manipulation-language-dml.md)
- [6.6 Data Query Language (DQL)](6-6-data-query-language-dql.md)
- [6.7 Data Introspection Language (DIL)](6-7-data-introspection-language-dil.md)
- [6.8 Data Miscellaneous Language (DMISCL)](6-8-data-miscellaneous-language-dmiscl.md)