6.3.1FROM

Syntax

FROM {cursor | text_file | binary_file | excel_file | memory}

cursor := CURSOR [SKIP skip_value]

text_file := [DOS | UNIX] [TEXT] FILE file_name [NO SEPARATOR | SEPARATOR {separator_value | {[separator_value], [subseparator_value]}}] [SKIP skip_value]

binary_file := BINARY FILE file_name [SKIP skip_value]

excel_file := EXCEL [XLS | XLSX] FILE file_name [SHEET sheet_name] [SKIP skip_value]

memory := MEMORY variable_number [SIZE variable_size] [OFFSET (member_offset | {[member_offset] {, [member_offset]}+})] [MAX max_elements]

Description

Read data from the cursor in use (default behavior when no redirecting option is specified), a (text, binary, or Excel) file, or memory (i.e. a user-defined variable), and write it into an HDF5 userblock, dataset, or attribute through a CREATE FILE, CREATE DATASET, CREATE ATTRIBUTE, INSERT, or SET USERBLOCK operation. In detail, this procedure (which is known as input redirecting) can be performed from:

The cursor in use. Example: “CREATE DATASET my_dataset AS FLOAT VALUES FROM CURSOR” or “INSERT INTO my_dataset VALUES FROM CURSOR”.
A text file. Example: “CREATE DATASET my_dataset AS FLOAT VALUES FROM TEXT FILE my_file.txt” or “INSERT INTO my_dataset VALUES FROM TEXT FILE my_file.txt”.
A binary file. Example: “CREATE DATASET my_dataset AS FLOAT VALUES FROM BINARY FILE my_file.bin” or “INSERT INTO my_dataset VALUES FROM BINARY FILE my_file.bin”.
An Excel file (in XLS or XLSX format). Example: “CREATE DATASET my_dataset AS FLOAT VALUES FROM EXCEL XLS FILE my_file.xls” or “INSERT INTO my_dataset VALUES FROM EXCEL FILE input.xlsx”. Of note, for HDFql to be able to read data from an Excel file it needs to find and dynamically load a shared library named “libxl” (which is responsible for handling files of this type). The specification of the path where HDFql/HDF5 library may find shared libraries is done through the operation SET PLUGIN PATH.
A user-defined variable that was previously registered through the function hdfql_variable_register. Example: “CREATE DATASET my_dataset AS FLOAT VALUES FROM MEMORY 0” or “INSERT INTO my_dataset VALUES FROM MEMORY 2”. Of note, when working in Java, HDFql has to copy each element of the Java variable into the HDF5 dataset or attribute (managed by the underlying HDFql C library) as the JVM does not provide a direct access to the memory associated to the variable, which induces a performance penalty. This penalty is not present when working in other programming languages supported by HDFql – namely C, C++, Python, C#, Fortran, R, and MATLAB – as these provide a way for the underlying HDFql C library to access the memory of the variable directly.

Parameter(s)

file_name – optional string that specifies the name of a text, binary, or Excel file to read data from.

sheet_name – optional string that specifies the name of an Excel sheet to read data from. If not specified, the sheet to read data from is the one currently selected for viewing/editing within the Excel file.

separator_value – optional string that specifies the separator to use between elements (of the data) when reading these from a text file. If not specified, its default value is a comma (,). In case the keyword NO SEPARATOR is specified, no separator is used between elements (of the data) when reading these from a text file.

subseparator_value – optional string that specifies the subseparator to use between elements (of the data) when reading these from a text file. The subseparator is only applicable when the data type of the HDF5 dataset or attribute is either HDFQL_VARTINYINT, HDFQL_UNSIGNED_VARTINYINT, HDFQL_VARSMALLINT, HDFQL_UNSIGNED_VARSMALLINT, HDFQL_VARINT, HDFQL_UNSIGNED_VARINT, HDFQL_VARBIGINT, HDFQL_UNSIGNED_VARBIGINT, HDFQL_VARFLOAT, HDFQL_VARDOUBLE, or HDFQL_OPAQUE, and ignored otherwise. If not specified, its default value is a space.

skip_value – optional integer that specifies the number of initial elements to skip (i.e. ignore) in case the input redirecting is the cursor in use, the number of initial lines (delimited by an end-of-line (EOL) terminator or a separator) to skip in case the input redirecting is a text file, the number of initial bytes to skip in case the input redirecting is a binary file, or the number of initial cells in the sheet to skip in case the input redirecting is an Excel file. If not specified, nothing is skipped. variable_number – optional integer that specifies the number of the variable whose data will be written into the HDF5 dataset or attribute. The number is returned by the function hdfql_variable_register upon registering the variable or, subsequently, returned by the function hdfql_variable_get_number.

variable_size – optional integer that specifies the size (in bytes) of the variable whose data will be written into the HDF5 dataset or attribute. Of note, the specification of a size only has effect for a dataset or attribute of data type HDFQL_COMPOUND (for any other data type the specification is ignored – i.e. has no effect).

member_offset – optional integer that specifies the (memory) member offsets that compose the variable whose data will be written into the HDF5 dataset or attribute. If specified, the variable is assumed to be a C padded struct data type (i.e. its members may not be contiguous in memory due to padding between them) and is used as such by HDFql. If not specified, the variable is assumed to be a C primitive or packed struct data type (i.e. its members are contiguous in memory and have no padding between them) and is used as such by HDFql. Of note, the specification of an offset only has effect for a dataset or attribute of data type HDFQL_COMPOUND (for any other data type the specification is ignored – i.e. has no effect).

max_elements – optional integer that specifies the maximum number of elements of the data stored in the variable to write into the HDF5 dataset or attribute. In other words, only the first max_elements of the data will be written into the dataset or attribute. Of note, max_elements may be smaller than the number of elements that the dataset or attribute may store (in this case, the remainder of the dataset or attribute will be zeroed if a number or emptied if a string).

Example(s)

# use (i.e. open) an HDF5 file named "my_file.h5" 
USE FILE my_file.h5

# show (i.e. get) HDF5 file currently in use and populate cursor in use with it 
SHOW USE FILE

# create an HDF5 dataset named "my_dataset0" of data type variable-length char with initial values from the cursor in use
CREATE DATASET my_dataset0 AS VARCHAR VALUES FROM CURSOR

# select (i.e. read) data from dataset "my_dataset0" and populate cursor in use with it (should return "my_file.h5")
SELECT FROM my_dataset0

# create an HDF5 dataset named "my_dataset1" of data type char of one dimension (size 3) 
CREATE DATASET my_dataset1 AS TINYINT(3)

# insert (i.e. write) values from a text file named "my_file0.txt" into dataset "my_dataset1" (assuming that the file "my_file0.txt" exists and contains "65,66,67")
INSERT INTO my_dataset1 VALUES FROM FILE my_file0.txt

# select (i.e. read) data from dataset "my_dataset1" and populate cursor in use with it (should return 65, 66, 67)
SELECT FROM my_dataset1

# insert (i.e. write) values from a text file named "my_file1.txt" into dataset "my_dataset1" (assuming that the file "my_file1.txt" exists and contains "90**92**94")
INSERT INTO my_dataset1 VALUES FROM TEXT FILE my_file1.txt SEPARATOR **

# select (i.e. read) data from dataset "my_dataset1" and populate cursor in use with it (should return 90, 92, 94)
SELECT FROM my_dataset1

# insert (i.e. write) values from a binary file named "my_file.bin" into dataset "my_dataset1" (assuming that the file "my_file.bin" exists and contains "ABC")
INSERT INTO my_dataset1 VALUES FROM BINARY FILE my_file.bin

# select (i.e. read) data from dataset "my_dataset1" and populate cursor in use with it (should return 65, 66, 67)
SELECT FROM my_dataset1

# insert (i.e. write) values from an Excel file named "my_file0.xlsx" stored in the sheet currently selected for viewing/editing within the file into dataset "my_dataset1" (assuming that the file "my_file0.xlsx" exists and contains three cells with values 10, 20, and 30)
INSERT INTO my_dataset1 VALUES FROM EXCEL FILE my_file0.xlsx

# select (i.e. read) data from dataset "my_dataset1" and populate cursor in use with it (should return 10, 20, 30)
SELECT FROM my_dataset1

# insert (i.e. write) values from an Excel file named "my_file1.xlsx" stored in a sheet named "my_sheet" into dataset "my_dataset1" (assuming that the file "my_file1.xlsx" exists and contains three cells with values 5, 6, and 7)
INSERT INTO my_dataset1 VALUES FROM EXCEL FILE my_file1.xlsx SHEET my_sheet

# select (i.e. read) data from dataset "my_dataset1" and populate cursor in use with it (should return 5, 6, 7)
SELECT FROM my_dataset1

// declare variables 
char script[1024];
double data[3][2];
int x;
int y;

// create an HDF5 dataset named "my_dataset2" of data type double of two dimensions (size 3x2)
hdfql_execute("CREATE DATASET my_dataset2 AS DOUBLE(3, 2)");

// populate variable "data" with certain values 
data[0][0] = 3.2; 
data[0][1] = 1.3; 
data[1][0] = 0; 
data[1][1] = 0.2; 
data[2][0] = 9.1; 
data[2][1] = 6.5;

// register variable "data" for subsequent use (by HDFql) 
hdfql_variable_register(data);

// prepare script to insert (i.e. write) values from variable "data" into dataset "my_dataset2"
sprintf(script, "INSERT INTO my_dataset2 VALUES FROM MEMORY %d", hdfql_variable_get_number(data));

// execute script 
hdfql_execute(script);

// unregister variable "data" as it is no longer used/needed (by HDFql) 
hdfql_variable_unregister(data); 

// select (i.e. read) data from dataset "my_dataset2" and populate cursor in use with it 
hdfql_execute("SELECT FROM my_dataset2");

// display content of cursor in use (should be 3.2, 1.3, 0, 0.2, 9.1, 6.5) 
while(hdfql_cursor_next(NULL) == HDFQL_SUCCESS)
{
    printf("%f\n", *hdfql_cursor_get_double(NULL));
}

// declare variables 
char script[1024];
HDFQL_VARIABLE_LENGTH data[3];

// create an HDF5 dataset named "my_dataset3" of data type variable-length double of one dimension (size 3)
hdfql_execute("CREATE DATASET my_dataset3 AS VARDOUBLE(3)");

// allocate memory in variable "data" 
data[0].address = malloc(2 * sizeof(double)); 
data[0].count = 2;
data[1].address = malloc(3 * sizeof(double)); 
data[1].count = 3;
data[2].address = malloc(1 * sizeof(double)); 
data[2].count = 1;

// populate variable "data" with certain values 
*((double *) data[0].address + 0) = 3.2; 
*((double *) data[0].address + 1) = 1.3; 
*((double *) data[1].address + 0) = 0; 
*((double *) data[1].address + 1) = 0.2; 
*((double *) data[1].address + 2) = 9.1; 
*((double *) data[2].address + 0) = 6.5; 

// register variable "data" for subsequent use (by HDFql) 
hdfql_variable_register(data);

// prepare script to insert (i.e. write) values from variable "data" into dataset "my_dataset3"
sprintf(script, "INSERT INTO my_dataset3 VALUES FROM MEMORY %d", hdfql_variable_get_number(data));

// execute script 
hdfql_execute(script);

// unregister variable "data" as it is no longer used/needed (by HDFql) 
hdfql_variable_unregister(data); 

// select (i.e. read) data from dataset "my_dataset3" and populate cursor in use with it 
hdfql_execute("SELECT FROM my_dataset3");

// display content of cursor in use (should be 3.2, 1.3, 0, 0.2, 9.1, 6.5) 
while(hdfql_cursor_next(NULL) == HDFQL_SUCCESS)
{
    while(hdfql_subcursor_next(NULL) == HDFQL_SUCCESS)
    {
       printf("%f\n", *hdfql_subcursor_get_double(NULL));
    }
}

// release memory allocated in variable "data" 
free(data[0].address);
free(data[1].address);
free(data[2].address);

// declare variables 
char script[1024];
char *data[3];

// create an HDF5 dataset named "my_dataset4" of data type variable-length char of one dimension (size 3)
hdfql_execute("CREATE DATASET my_dataset4 AS VARCHAR(3)");

// allocate memory in variable "data" 
data[0] = malloc(13 * sizeof(char)); 
data[1] = malloc(5 * sizeof(char)); 
data[2] = malloc(7 * sizeof(char)); 

// populate variable "data" with certain values 
strcpy(data[0], "Hierarchical"); 
strcpy(data[1], "Data"); 
strcpy(data[2], "Format"); 

// register variable "data" for subsequent use (by HDFql) 
hdfql_variable_register(data);

// prepare script to insert (i.e. write) values from variable "data" into dataset "my_dataset4"
sprintf(script, "INSERT INTO my_dataset4 VALUES FROM MEMORY %d", hdfql_variable_get_number(data));

// execute script 
hdfql_execute(script);

// unregister variable "data" as it is no longer used/needed (by HDFql) 
hdfql_variable_unregister(data); 

// select (i.e. read) data from dataset "my_dataset4" and populate cursor in use with it 
hdfql_execute("SELECT FROM my_dataset4");

// display content of cursor in use (should be "Hierarchical", "Data", "Format") 
while(hdfql_cursor_next(NULL) == HDFQL_SUCCESS)
{
    printf("%s\n", hdfql_cursor_get_char(NULL));
}

// release memory allocated in variable "data" 
free(data[0]);
free(data[1]);
free(data[2]);

// declare structure 
struct data
{ 
   char description[7]; 
   int index; 
};

// declare variables 
char script[1024];
struct data cities[3]; 
int number;

// create an HDF5 dataset named "my_dataset5" of data type compound of one dimension (size 3) composed of two members named "description" (of data type char) and "index" (of data type int)
hdfql_execute("CREATE DATASET my_dataset5 AS COMPOUND(description AS CHAR(7), index AS INT)(3)");

// populate variable "cities" with certain values 
memcpy(cities[0].description, "Toronto", 7);
cities[0].index = 10; 
memcpy(cities[1].description, "Nairobi", 7);
cities[1].index = 12; 
memcpy(cities[2].description, "Caracas", 7);
cities[2].index = 11; 

// register variable "cities" for subsequent use (by HDFql) 
number = hdfql_variable_register(cities);

// prepare script to insert (i.e. write) values from variable "cities" into dataset "my_dataset5"
sprintf(script, "INSERT INTO my_dataset5 VALUES FROM MEMORY %d SIZE %d OFFSET(%d, %d)", number, sizeof(struct data), offsetof(struct data, description), offsetof(struct data, index));

// execute script 
hdfql_execute(script); 

// unregister variable "cities" as it is no longer used/needed (by HDFql) 
hdfql_variable_unregister(cities);

6.3.1FROM

#Syntax

#Description

#Parameter(s)

#Example(s)

Syntax

Description

Parameter(s)

Example(s)