Q&A: How to find an appropriate segmentation method for my data sets?
See also a similar Q&A for a different image here:
- How to find an appropriate segmentation method for my data sets (Example 2)?
- How to find an appropriate segmentation method for my data sets (Example 3)?
Answer provided on 8th of October 2020.
CiliaQ Preparator offers multiple methods to perform image segmentation:
- Segmentation based on a standard intensity threshold (e.g. determined by an automatic algorithm)
- Using a hysteresis threshold
- Using a custom developed method that we coined “Canny3D”
The different methods are described in detail in the CiliaQ publication (https://doi.org/10.1140/epje/s10189-021-00031-y).
In the following, I document how I searched for and optimized the settings for an exemplary image from a CiliaQ User.
Usually, we first try to establish the segmentation based on a standard threshold algorithm. To manually search for the best algorithm, I usually generate a maximum-intensity-projection in FIJI (Image > Stacks > Z project) and then investigate it.
maximum-intensity-projection image - original image provided by (c) Eirini Tsekitsidou
What do we see in this image?
- There is blurred background structures (circle in upper right).
- There is large intensity differences within cilia.
This means we need to cope with the following challenges:
- The range of intensities that differentiates the background structures from cilia is low and not homogeneous in all image regions. How to find a good threshold that separates cilia from background in all regions?
- How to normalize the intensities of cilia so that we can get continuous cilia while applying the same threshold everywhere?
We first try to solve these issue by normalizing intensities in the image. We can e.g. normalize intensities in the image based on the size of an object. If we subtract the local background with a small radius (e.g. 10 px) we will remove larger blurred areas, while cilia that are fine structures will stay. If we further decrease the radius we might be possible to even normalize intensities within cilia (e.g. 5px). To test this approach, duplicate the maximum projection image (Image > Duplicate) and then subtract the background (Image > Subtract Background > select 5 px and press OK). In the resulting image (see below), we see that the image is reduced to the cilia structures, background / defocused background is mostly gone, intensities of cilia are a little bit more similar. This makes segmentation easier. However there are still intensity differences between cilia.
Background-subtracted maximum-intensity-projection image - original image provided by (c) Eirini Tsekitsidou
Next, we try to find a good threshold algorithm and check whether we can find a threshold that allows to separate cilia from background, even though the intensities within cilia largely vary.
To test how the different available threshold methods in FIJI cope with the image, we go to Image > Adjust > Threshold.... Here we can select different threshold methods and check what they pick up as cilia live in the image. The methods presented here are the identical methods that you can select later in CiliaQ Preparator, when you want to perform automated analysis of your images.
We see that, when using the method Huang, many cilia are not continuously detected.
Thresholding of the background-subtracted maximum-intensity-projection image - original image provided by (c) Eirini Tsekitsidou
With the method Triangle it looks much better but we face the problem that some cilia are so bright, that the detection gets them as big blobs. While others are fine and well detected. If we used that threshold algorithm fro segmentation, we will face a lot of bias in the morphological output parameters from CiliaQ.
Thresholding of the background-subtracted maximum-intensity-projection image - original image provided by (c) Eirini Tsekitsidou
The method Li seems to work best among all thresholding methods. However, still also some background structures and some cilia are discontinuous. With a little bit of editing the analysis pipeline would work.
Thresholding of the background-subtracted maximum-intensity-projection image - original image provided by (c) Eirini Tsekitsidou
Nonetheles, we can still try a second, alternative way to cope with the intensity differences in the image. Let’s try other alternatives offered by CiliaQ Preparator, i.e. the Canny3D method that relies on edge detection and hysteresis thresholding. Edge detection is more robust to detect cilia even when the intensity levels are inhomogeneous. This however we may not manually try (because performing all the steps manually would take quite some steps) but using CiliaQ Preparator. To this end, we feed the original image into CiliaQ Preparator, select the Subtract Background 5 px option (as we found taht this helps - see above), and select the Canny 3D method.
Setting up CiliaQ Preparator to employ background subtraction and Canny 3D
In the next step, CiliaQ Preparator allows several options for Canny3D. To understand what the individual options mean in more detail, you may read the paragraph in the CiliaQ preprint. Usually, I recommend to first go with the standard settings and see how they perform.
Setting up the Canny 3D settings in CiliaQ Preparator
When CiliaQ Preparator is done, open the generated ...CQP.tif file. You will not immediately see the segmentation results because CiliaQ Preparator keeps cilia intensities. However, if you adjust the Brightness/Contrast, you can see the results better (Image > Adjust > Brightness Contrast; drag down the maximum to display the generated mask)
Individual slice of the 3D image segmented using Canny 3D - original image provided by (c) Eirini Tsekitsidou
Here we see that the cilia are actually all equally well detected. Some are empty in the core. This happens when the ciliary segmentations touch the Z borders. One could circumvent that during imaging by recording a wider z stack. But one can also ignore this here, because for drawing the ciliary skeletons, CiliaQ allows to blur the image before skeletonization. Blurring will basically fill the gaps later. For example, if we just run this CQP file in CiliaQ and select a Gaussian XY Blur of 2.0 in the Skeletonization settings, you get nice ciliary skeletons in the CQ_SKL_3D output file meaning that the cilia were well tracked.
3D Skeleton visualization by CiliaQ for the image segmented using Canny 3D - original image provided by (c) Eirini Tsekitsidou
Still there are some cilia that lie so close that they are detected as a merged object. These you should cut apart CiliaQ Editor into their individual domains (you see in the SKL_3D picture above that such closely connected multiple cilia incorrectly appear as ultra long cilia in the image).
Multiple connected cilia objects in an individual slice of the 3D image segmented using Canny 3D - original image provided by (c) Eirini Tsekitsidou
Lastly, you can always optimize it further. What you can still try is to play a bit with the Canny settings. E.g. one could try to get finer cilia by reducing the blur in the canny settings but this could also make the cilia objects less connected. One could also play with the alpha from the sobel filter and see whether it gets still better.
There will probably still be possibilities to optimize this even further also in CiliaQ. E.g. you can compare to switch on or switch off the „Increase range“ option in CiliaQ to better detect the in homogeneously stained cilia if they have gaps. Or you may further play with the Subtract Background radius that we selected in CiliaQ Preparator.
At some point one needs to decide when the setting is sufficient and few manual editing does not matter any more.
Of course, one should also try the same setting for a few more images in the data set to see whether it is appropriate for all images.