mask, moving_mask, and mask_all_stages

[amirhszd]

hey there, I am doing very simple coregistration, between two datasets that I have, VNIR and SWIR. in each dataset there are parts that are no-data-value=0. initially I would get the compounding map and set those to zero, which would give me correct results, but I want to exclude those parts from metric calculation as setting them to zero would not do it. However, the code seems to fail and would not do a correct job moving one to the other. I am assuming the data type of the mask I need to pass to the image is a binary, 1 for where I want calculation to be done, 0 for where I don't want it to be done. correct? Am I missing something?

thank you

output by simply setting zeros in the image

output by passing mask

code:
final_mask = np.invert(np.logical_or(mean_swir_uint8 == 0, mean_vnir_uint8 == 0)).astype(np.uint8)

mask_ants = ants.from_numpy(final_mask, is_rgb=False, has_components=False)

# Convert VNIR and SWIR images to ANTs images
vnir_image_ants = ants.from_numpy(mean_vnir_uint8, is_rgb=False, has_components=False)
swir_image_ants = ants.from_numpy(mean_swir_uint8, is_rgb=False, has_components=False)

# Register images with specified parameters
mtx = ants.registration(
    fixed=vnir_image_ants,
    moving=swir_image_ants,
    type_of_transform=method,
    mask = mask_ants,
    moving_mask=mask_ants,
    flow_sigma=flow_sigma,
    syn_metric=syn_metric,
    smoothing_in_mm=smoothing_mm,
    mask_all_stages=True
)

# Apply the transformation to three selected bands for RGB representation
bands_warped = []
for i in [30, 60, 90]:  # Replace these indices with the desired bands
    band_warped = apply_registration_to_band(swir_arr[..., i], mean_vnir_uint8, mtx['fwdtransforms'])
    bands_warped.append(to_uint8(band_warped))

bands_warped_rgb = np.dstack(bands_warped)

[cookpa]

You need two separate binary mask images. The mask for the mask argument needs to be in the fixed space, and the mask for the moving_mask needs to be in the moving space.

Using masks requires good initialization - if the foreground of the fixed and moving images don't overlap well, the registration can fail. You can do an initial rigid or affine registration without the masks, then use that transform to initialize a registration with the masks.

[amirhszd]

I am posting the two images I have from two platforms. As you can see each of them has areas where there is no data thus there is no need to to register one to another, and I am only interested in the overlap area. so I want to get the combined areas that I have in both, so I develop a mask that captures what is non-zero in both of them. vnir != 0 OR swir !=0. this resulting mask would be used for both fixed and moving. this logic makes sense, if I treat them separately it is going to push areas that exist in moving (SWIR) but not the in fixed (VNIR). previous to this I performed manual coregistration by hand, so the pixels are very close, using elastic registration to compensate for areas that shadows and heights are perceived differently.

VNIR

VNIR mask

SWIR

SWIR mask

[cookpa]

OK. If the images are pre-aligned, then it might make sense to use the same mask for both. You can probably just use mask in this case and dispense with moving_mask. Note, however, that all ANTs registration is done in the physical space. When you convert numpy to antsImage it assumes some defaults for physical space, and you have to ensure that your images and masks have the correct information.

Also, you will need to add an initial_transform argument. If you've done the initial alignment by hand, you would add initial_transform='Identity'. This stops the registration trying to initialize by the image centers of mass.

I don't know what method you are using for type_of_transform but I think you would need SyNOnly, the Elastic options all do affine registration, which you don't want.

[amirhszd]

fantastic, thank you very much!

[cookpa]

The initial_transform option has caught out a few users, might be too late to change but I added a documentation note #739