Skip to content

netCDF4 cannot read from little-endian HDF4 files #113

@timburgess

Description

@timburgess

Using OS X 10.9.5, the following example code will produce a little-endian HDF4 file which contains a simple incrementing 5x5 array. Specifically, binary ORing the defined datatype with DNFT_LITEND creates a little-endian file.

#include <stdio.h>
#include "mfhdf.h"

#define DIM1 5
#define DIM0 5
#define RANK 2
#define FILENAME "hdf_little-endian.hdf"
#define SDSNAME "data"

int main() {
  
    int32 sd_id, sds_id, istat, sd_index;
    int32 dims[2], start[2], edges[2], rank;
    int16 array_data[DIM0][DIM1];
    intn i, j, count;  
  
    start[0] = 0;
    start[1] = 0;
    edges[0] = DIM1;
    edges[1] = DIM0;
  
    // populate data array
    count = 0;
    for (j = 0; j < DIM0; j++) 
    {
        for (i = 0; i < DIM1; i++)
            array_data[j][i] = count++;
    }
  
    sd_id = SDstart(FILENAME, DFACC_CREATE);
    if (sd_id != FAIL)
        printf ("Opened %s\n", FILENAME);

    sds_id = SDcreate(sd_id, SDSNAME, DFNT_LITEND|DFNT_INT16, RANK, edges);
//  sds_id = SDcreate(sd_id, SDSNAME, DFNT_INT16, RANK, edges);

    istat = SDendaccess(sds_id);

    istat = SDend(sd_id);
    if (istat != FAIL)
        printf("closed\n");

       sd_id = SDstart(FILENAME, DFACC_WRITE);
    if (sd_id != FAIL)
        printf ("Opened %s\n", FILENAME);
  
    sd_index = 0;
    sds_id = SDselect(sd_id, sd_index);
  
    istat = SDwritedata(sds_id, start, NULL, edges, (VOIDP)array_data);
  
    istat = SDendaccess(sds_id);

    istat = SDend(sd_id);
    if (istat != FAIL)
        printf("closed\n");       
  
    return 0;
}

Using hdp dumpsds filename shows

File name: hdf_little-endian.hdf 

Variable Name = data
     Index = 0
     Type= little-endian format 16-bit signed integer
     Ref. = 2
     Compression method = NONE
     Rank = 2
     Number of attributes = 0
     Dim0: Name=fakeDim0
         Size = 5
         Scale Type = number-type not set
         Number of attributes = 0
     Dim1: Name=fakeDim1
         Size = 5
         Scale Type = number-type not set
         Number of attributes = 0
     Data : 
                0 1 2 3 4 
                5 6 7 8 9 
                10 11 12 13 14 
                15 16 17 18 19 
                20 21 22 23 24 

However ncdump fails to read the file with a NetCDF: Bad type ID.

This issue also impacts the python netCDF4 library as it fails with the same error, producing:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "netCDF4.pyx", line 1503, in netCDF4.Dataset.__init__ (netCDF4.c:24382)
RuntimeError: NetCDF: Bad type ID

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions