Activity 8: Morphological Operations

For the past few activities, we dwell on the internal properties which can be observed in the Fourier space, the corresponding color distribution and chromaticity of an image, and similar characteristics. In this activity, we will dwell more on the structural aspect of an object in an image and we apply image processing by modifying or morphing certain structural elements. Doing these morphological operations requires some basic knowledge about Set Theory. I wont dwell much on the said theory but I'll just give a short summary which can be seen in the table below [1].

Table 1. Set Theory symbols and definitions [1].

A. Erosion and Dilation

Erosion and Dilation are two morphological operations that we will discuss first in this activity. Erosion, by concept and experience, is a well known fact that is occurs when soil, rock, or any basic piece of the Earth's landmass is displaced or moved from one point to another. When it comes to image processing, an image $A$ can be eroded by another image $B$, and the result is an image containing the set of all points $x$, such that $B$ when translated by $x$ is contained in $A$. This is a morphological process that includes image size reduction of $A$ with $B$ being the Structuring Element. Dilation, on the other hand, is commonly attributed to enlargement or expansion. Similar with Erosion, Dilation can be done in an image $A$ through a Structuring Element $B$; and occurs when the intersection of $A$ and the reflection of $B$ is not an empty set. For the first part of the activity, these two morphological operations were done manually through the use of this site http://polarski.cz/graph_paper/. These manual outputs were then compared with simulated operations. Four test images were both dilated and eroded with 5 structuring elements found in Figure 1.

Figure 1. Structuring elements used for erosion and dilation. The red dots are the reference points for the said processes. a) 2x2 square, b) 2x1 rectangle, c) 1x2 rectangle, d) cross of length 3 pixels, and e) a diagonal of length 2 pixels starting from bottom-right to top-left. 

Observing through the simulated morphological operations, I can say that I've done quite well in this part. This can be seen in the similarity of the dilated and eroded images for both the manual process (Fig.2) and the simulation (Fig.3).

Figure 2. Manual processing of erosion (colored violet) and dilation (colored green) for five test images namely, (from left to right) a 5x5 square, a triangle of base 4 and height 3 pixels, a 10x10 hollow square having a width of 2 pixels, and a cross of length 5 pixels. Structuring elements are read from top to bottom (a to e found in figure 1). 

Figure 3. Simulated erosion (upper part) and dilation (lower part) morphological operations through scilab. The test images are read from top to bottom (5x5 square, 4x3 base-height triangle, 10x10 2 pixel thickness hollow square, and a 5px length cross), and Sturcturing Elements are read from left to right (a to e in figure 1).

For the simulation, each structural element was created using the CreateStructureElement() function of Scilab. Eroding and Dilating with these structural elements can be done digitally through the ErodeImage() and DilateImage() functions, respectively. Through doing the manual part was quite rigorous and took me about 3 hours to finish, it is essential to understand the process of these operations. Also, it felt satisfying to produce quite good results. The similarity and notable preciseness of the manual and simulated morphological process infers to us about the robustness and straightforward approach of morphological operations. Now that we finished the basic morphological process, it's time to dig a bit deeper.

B. Closing, Opening, and Top-Hat Operators

In this part of the activity, we introduce operations which include integrations of both dilation and erosion. The Closing operator tends to dilate first and then erode an image with the same structural element; while the Opening operator erodes the image first and then dilates it afterward. These two processes can be used to morph and segment images. The closing operator tends to "close" certain gaps in objects based on the structural element. Similarly, the opening operator "opens" these gaps for object isolation based on the structural element. This can be seen in Figure 4 below wherein a binarized test image of cells is opened (closed) using the OpenImage() (CloseImage()). We can see here the said "opening" and "closing" of gaps. In this case, gaps of certain hypothetical cells are either opened or closed. Note that the whole image was processed due to the initial input stacksize('max') for Scilab, that puts the stacksize to be analyzed into maximum possible memory value.

Figure 4. Morphological operations done in test cell images see in the leftmost binarized image.

The Top-Hat Operator, on the other hand, is modification of the two operators. This operator has two types and one of which is executed in Scilab.  These two types are the white and black top-hat functions. The white top-hat (Scilab executed) opens an image first and then subtracts it from the original image, while the black top-hat does the same operation but with the closing function instead of the opening function.  In figure 4, we see the white-top hat outlining the edges and parts omitted from opening an image with circle diameters of 8 pixels. Note that the binarized image had a threshold of >218 pixels as seen in histogram plot (Figure 5) below.

Figure 5. Threshold determination of the hypothetical test cells image.

C. Cancer Cell Detection

In this activity, we also apply these said operations in complex and vital processes like Cancer cell detection. We know that cancer cells are generally larger in size compared to normal cells. Here, we do morphological operations to automatically deduce cancer cells from an image.

First, we want to have a basis whether a cell is a cancer cell or not. Having knowledge about the variation in sizes, we could use the diameter or the cell size/area in a image as basis. As seen in Fig. 4 above, the opening operation works quite well for cleaning the image for processing, but I noticed that it deletes some important information. Due to this, I made my own specialized function. This specialized function includes opening the difference of two sub-functions $X$ and $Y$. Here $X$ is the opened of the closed image with threshold diameter of 5px; and $Y$ is the closed of the top-hat image with similar threshold diameter of 5px. The differences were obtained and were opened with a threshold diameter of 10px. It can be seen that less details were lost in the specialized function. Blobs were then obtained assigned using SearchBlobs() function. Their corresponding areas were then noted and plotted, as seen in the image below.

Figure 6. Histogram plots of the areas of the cell blobs before (top) and after (bottom) the application of the interquartile range (IQR) 

It can be seen that there are outliers in the image so we use the interquartile range (IQR) method wherein values within the difference of the 1st quartile with1.5 times the IQR ($x > (1^{st}Q - 1.5IQR$), and the sum of the third quartile with 1.5 times the IQR ($x < (3^{rd}Q + 1.5IQR$), the the ones we take into account ($IQR = 3^{rd}Q - 1^{st}Q$). The result can be seen in the lower histogram in Fig. 6., wherein the range varied significantly from 365-5986 to 365-697. The mean = 469.185 and standard deviation =  66.606 are obtained using the mean() and stdev() functions, respectively. The range of values are therefore between about 400-540pixels.

Now that we have this range, we can now determine the cancer cells by terms of area variation. We apply this to a hypothetical organ with cancer cells image seen in the figure below. We first binarize the image and then apply the Watershed() function to separate the connected blobs to omit area anomalies brought by connected cells. We do this by having a gradient for the edge detection from the Sobel algorithm.

Figure 7. (From left to right) Original hypothetical cancer cells image, binarized image with normalized threshold  = 0.88, and a binarized image with the application of the Watershed function.

We then apply the said filter range obtained from the statistical process done with the normal cell images by using FilterBySize() and the result is shown in figure 8. It can be seen that there are still connected cells that was not segmented by the applied watershed function. This is in the limitation of the Sobel edge-detection algorithm and SearchBlobs function. In the midst of two errors, the process still significantly increases cancer cell detection.

Figure 8. Area-based (left) and Diameter-based (right) cell detection method.

Also, due to the structural element diameter dependence of the cells upon image opening, we can determine the cancer cells using a Diameter-based method. It was observed that the maximum diameter for normal cells was only 13pixels, therefore we implement a structural element with diameter greater than 13 and then open the image and the result is quite remarkable for its simplicity!

This has been the most tiring activity. Huhu. But yeah, the struggle really payed off. I learned so much from this activity and I had fun! I like the thought that I'm learning and growing as a soon-to-be-physicist. Also, believe that I did quite great in this activity: from explaining the points, presenting the figures, and extending for specialized and researched functions, I grade myself a 12/10 here. :)

I would like to thank Fatima Garcia, Audrey Arcilla, Miguel Panagsagan, and Denise Musni for being with me as I do this activity. I also would like to thank Recovery Food in UP town center and Kuya Barteezy's blog for guidance. Up to the next activity!

[1] Set Theory symbols and definitions: http://media-cache-ec0.pinimg.com/1200x/11/3d/46/113d4693d4edf453430927b4558f6640.jpg, last accessed: December 8, 2016.
[2] M. Soriano, "Activity 8 - Morphological Operations," Applied Physics 186. 2014.