Misalignment between the prediction and the Ground Truth

[tylin7111095022]

Thanks for awesome contribution about UNETR! I met the problem that my predict result is mismatch with ground truth. like

I think there is a axis in predict result is opposite with my ground truth. But I do multiple preprocess with monai pipeline so I can't get the clear affine matrix. And I refer the example by Invertd and SaveImaged to save result to nifti file, and find the problem. Is there any suggestion? Thanks!

I get the warning transform info of image is not available or no InvertibleTransform applied., and when I loop my dataloader, I get the keys of every batch only 'image', 'foreground_start_coord', 'foreground_end_coord'. seems no 'image_meta_dict'. Is the reason get the above warning?

I check the metadata both predict and ground truth and found their direction matrix is different.

direction matrix of predict:
[[-1, 0, 0 ],
[ 0, -1, 0 ],
[ 0, 0, 1 ]]

direction matrix of ground truth:
[[ 1, 0, 0 ],
[ 0, -1, 0 ],
[ 0, 0, 1 ]]

following is my transform pipeline:

pre_transform = transforms.Compose(
            [
                transforms.LoadImaged(keys=["image"]),
                transforms.EnsureChannelFirstd(keys=["image"]),
                transforms.Orientationd(keys=["image"], axcodes="RAS"),
                transforms.Spacingd(
                    keys=["image"], pixdim=(args.space_x, args.space_y, args.space_z), mode=("bilinear")
                ),
                transforms.ScaleIntensityRanged(
                    keys=["image"], a_min=args.a_min, a_max=args.a_max, b_min=args.b_min, b_max=args.b_max, clip=True
                ),
                transforms.CropForegroundd(keys=["image"], source_key="image", allow_smaller=True),
                transforms.ToTensord(keys=["image"]),
            ]
        )

postTransforms = Compose([
                Invertd(
                    keys="pred",                     # invert the `pred` data field, also support multiple fields
                    transform=preTransform,        
                    orig_keys="image",               # get the previously applied pre_transforms information on the `image` data field,
                    meta_keys="image_meta_dict",     # key include meta data
                    orig_meta_keys="image_meta_dict",
                    nearest_interp=False,  # don't change the interpolation mode to "nearest" when inverting transforms
                    # to ensure a smooth output, then execute `AsDiscreted` transform
                    to_tensor=True, # convert to PyTorch Tensor after inverting
                ),
                AsDiscreted(keys="pred", threshold=0.5),
                
                SaveImaged(
                    keys="pred",
                    output_dir="./output",
                    output_postfix="seg",
                    resample=False,
                    print_log=True,
                )
            ])

[tylin7111095022]

I solve the problem because my version of monai is 1.5. So the metadata had included in the image tag already! Following is my code and it can successfully predict the label with correct metadata.

from monai.transforms import Compose, Invertd, SaveImaged, AsDiscreted
from monai.inferer import sliding_window_inference
from monai.data.meta_tensor import MetaTensor

postTransforms = Compose([
                Invertd(
                    keys="pred",                     # invert the `pred` data field, also support multiple fields
                    transform=preTransform,        
                    orig_keys="image",           # get the previously applied pre_transforms information on the `image` data field,
                    nearest_interp=False,  
                    to_tensor=True, # convert to PyTorch Tensor after inverting
                ),
                AsDiscreted(keys="pred", threshold=0.5),
                
                SaveImaged(
                    keys="pred",
                    output_dir="./output",
                    output_postfix="seg",
                    resample=False,
                    print_log=True,
                )
            ])

for d in loader:
    input_data = d['image].cuda()
    out = sliding_window_inference(input_data, (64, 64, 64), 1, model, overlap= 0.5)
    prob = torch.softmax(out, 1).cpu().numpy()
    predict = np.argmax(prob, axis = 1).astype(np.uint8)
    predict_tensor = torch.from_numpy(predict.astype(np.float32))
    d["pred"] = MetaTensor(predict_tensor, meta=d["image"].meta)
    postTransform(d)