Extract a String from a HDF5 using mdf5-metno

[hai-nguyen-van]

Hello,

I've been trying to use hdf5-metno to extract data from a V7.3 .mat file. I successfully extracted data f64 arranged in 2D and 3D matrices/tensors. However, I have an association (X2, MY_STRING) where MY_STRING is a string in MATLAB.

MAIN
|_ X1
|_ X2
   |_ MY_STRING

I fail to extract MY_STRING the right way in Rust. I tried the following:

let v0 = file.group("MAIN").unwrap().dataset("X2")?.read::<VarLenAscii>()?;

I get the error:

v0.as_ref() = Err(
    no conversion paths found,
)

Interestingly, the dimension is 2 🤔

v0.ndim() = 2

Did I do something wrong?

[magnusuMET]

Could be the string is unicode, in which case you must use VarLenUnicode. What is the result of h5dump -H filename.h5?

[hai-nguyen-van]

Changing from VarLenAscii to VarLenUnicode did not help much. In this specific example, the data I would like to query is in /PARAM/LBVERSION and the expected string as output is v46. It's not supposed to be complicated to extract this string I can't do it.

Here's the .mat file : h5_test.mat.txt
Here's the result from h5dump : h5dump_res.txt

[magnusuMET]

matlab does have quite the complex structure. I can do

    let group = file.group("PARAM").unwrap();
    let var = group.dataset("LBVERSION").unwrap();

    let s1 = var.read_2d::<u16>().unwrap();

    for c in s1 {
        println!("{:?}", char::from(c as u8));
    }

to read the variable. You will have to read the datatype as u16 and then convert to a string yourself, here I made a guess that the values were just ascii chars giving the output v46\0

[peterfpeterson]

I have similar issues. They are fixed length strings, but I don't know their length until runtime. Once it is figured out, it would be great to add to the examples and/or docs.

[CramBL]

I have similar issues. They are fixed length strings, but I don't know their length until runtime. Once it is figured out, it would be great to add to the examples and/or docs.

If you're reading e.g. attributes you can use this approach for any kind of string, and if it's in an array of a dataset, you can use the same approach for each index.

// Get the data type descriptor for the attribute
let dtype = attr.dtype()?.to_descriptor()?;
match &dtype {
    // Handle variable-length ASCII string
    TypeDescriptor::VarLenAscii => {
        let value: VarLenAscii = attr.read_scalar()?;
        Ok(value.as_str().to_owned())
    }
    // Handle variable-length UTF-8 string
    TypeDescriptor::VarLenUnicode => {
        let value: VarLenUnicode = attr.read_scalar()?;
        Ok(value.as_str().to_owned())
    }
    // Handle fixed-length ASCII or UTF-8 string
    TypeDescriptor::FixedAscii(_) | TypeDescriptor::FixedUnicode(_) => {
        let buf = attr.read_raw()?;
        let string = String::from_utf8_lossy(&buf)
            .trim_end_matches('\0')
            .to_owned();
        Ok(string)
    }

This is a lazy approach where the raw bytes are read and the string is then parsed from up until the 0-termination, you could use the length in e.g.TypeDescriptor::FixedUnicode(strlen) to parse it in a more proper way.