Friday, December 9, 2016
Activity 8: Morphological Operations
For the past few activities, we dwell on the internal properties which can be observed in the Fourier space, the corresponding color distribution and chromaticity of an image, and similar characteristics. In this activity, we will dwell more on the structural aspect of an object in an image and we apply image processing by modifying or morphing certain structural elements. Doing these morphological operations requires some basic knowledge about Set Theory. I wont dwell much on the said theory but I'll just give a short summary which can be seen in the table below [1].
Table 1. Set Theory symbols and definitions [1].
A. Erosion and Dilation
Erosion and Dilation are two morphological operations that we will discuss first in this activity. Erosion, by concept and experience, is a well known fact that is occurs when soil, rock, or any basic piece of the Earth's landmass is displaced or moved from one point to another. When it comes to image processing, an image $A$ can be eroded by another image $B$, and the result is an image containing the set of all points $x$, such that $B$ when translated by $x$ is contained in $A$. This is a morphological process that includes image size reduction of $A$ with $B$ being the Structuring Element. Dilation, on the other hand, is commonly attributed to enlargement or expansion. Similar with Erosion, Dilation can be done in an image $A$ through a Structuring Element $B$; and occurs when the intersection of $A$ and the reflection of $B$ is not an empty set. For the first part of the activity, these two morphological operations were done manually through the use of this site http://polarski.cz/graph_paper/. These manual outputs were then compared with simulated operations. Four test images were both dilated and eroded with 5 structuring elements found in Figure 1.
 |
| Figure 1. Structuring elements used for erosion and dilation. The red dots are the reference points for the said processes. a) 2x2 square, b) 2x1 rectangle, c) 1x2 rectangle, d) cross of length 3 pixels, and e) a diagonal of length 2 pixels starting from bottom-right to top-left. |
Observing through the simulated morphological operations, I can say that I've done quite well in this part. This can be seen in the similarity of the dilated and eroded images for both the manual process (Fig.2) and the simulation (Fig.3).
 |
| Figure 2. Manual processing of erosion (colored violet) and dilation (colored green) for five test images namely, (from left to right) a 5x5 square, a triangle of base 4 and height 3 pixels, a 10x10 hollow square having a width of 2 pixels, and a cross of length 5 pixels. Structuring elements are read from top to bottom (a to e found in figure 1). |
 |
| Figure 3. Simulated erosion (upper part) and dilation (lower part) morphological operations through scilab. The test images are read from top to bottom (5x5 square, 4x3 base-height triangle, 10x10 2 pixel thickness hollow square, and a 5px length cross), and Sturcturing Elements are read from left to right (a to e in figure 1). |
For the simulation, each structural element was created using the CreateStructureElement() function of Scilab. Eroding and Dilating with these structural elements can be done digitally through the ErodeImage() and DilateImage() functions, respectively. Through doing the manual part was quite rigorous and took me about 3 hours to finish, it is essential to understand the process of these operations. Also, it felt satisfying to produce quite good results. The similarity and notable preciseness of the manual and simulated morphological process infers to us about the robustness and straightforward approach of morphological operations. Now that we finished the basic morphological process, it's time to dig a bit deeper.
B. Closing, Opening, and Top-Hat Operators
In this part of the activity, we introduce operations which include integrations of both dilation and erosion. The Closing operator tends to dilate first and then erode an image with the same structural element; while the Opening operator erodes the image first and then dilates it afterward. These two processes can be used to morph and segment images. The closing operator tends to "close" certain gaps in objects based on the structural element. Similarly, the opening operator "opens" these gaps for object isolation based on the structural element. This can be seen in Figure 4 below wherein a binarized test image of cells is opened (closed) using the OpenImage() (CloseImage()). We can see here the said "opening" and "closing" of gaps. In this case, gaps of certain hypothetical cells are either opened or closed. Note that the whole image was processed due to the initial input stacksize('max') for Scilab, that puts the stacksize to be analyzed into maximum possible memory value.
 |
| Figure 4. Morphological operations done in test cell images see in the leftmost binarized image. |
The Top-Hat Operator, on the other hand, is modification of the two operators. This operator has two types and one of which is executed in Scilab. These two types are the white and black top-hat functions. The white top-hat (Scilab executed) opens an image first and then subtracts it from the original image, while the black top-hat does the same operation but with the closing function instead of the opening function. In figure 4, we see the white-top hat outlining the edges and parts omitted from opening an image with circle diameters of 8 pixels. Note that the binarized image had a threshold of >218 pixels as seen in histogram plot (Figure 5) below.
 |
| Figure 5. Threshold determination of the hypothetical test cells image. |
C. Cancer Cell Detection
In this activity, we also apply these said operations in complex and vital processes like Cancer cell detection. We know that cancer cells are generally larger in size compared to normal cells. Here, we do morphological operations to automatically deduce cancer cells from an image.
First, we want to have a basis whether a cell is a cancer cell or not. Having knowledge about the variation in sizes, we could use the diameter or the cell size/area in a image as basis. As seen in Fig. 4 above, the opening operation works quite well for cleaning the image for processing, but I noticed that it deletes some important information. Due to this, I made my own specialized function. This specialized function includes opening the difference of two sub-functions $X$ and $Y$. Here $X$ is the opened of the closed image with threshold diameter of 5px; and $Y$ is the closed of the top-hat image with similar threshold diameter of 5px. The differences were obtained and were opened with a threshold diameter of 10px. It can be seen that less details were lost in the specialized function. Blobs were then obtained assigned using SearchBlobs() function. Their corresponding areas were then noted and plotted, as seen in the image below.
 |
| Figure 6. Histogram plots of the areas of the cell blobs before (top) and after (bottom) the application of the interquartile range (IQR) |
It can be seen that there are outliers in the image so we use the interquartile range (IQR) method wherein values within the difference of the 1st quartile with1.5 times the IQR ($x > (1^{st}Q - 1.5IQR$), and the sum of the third quartile with 1.5 times the IQR ($x < (3^{rd}Q + 1.5IQR$), the the ones we take into account ($IQR = 3^{rd}Q - 1^{st}Q$). The result can be seen in the lower histogram in Fig. 6., wherein the range varied significantly from 365-5986 to 365-697. The mean = 469.185 and standard deviation = 66.606 are obtained using the mean() and stdev() functions, respectively. The range of values are therefore between about 400-540pixels.
Now that we have this range, we can now determine the cancer cells by terms of area variation. We apply this to a hypothetical organ with cancer cells image seen in the figure below. We first binarize the image and then apply the Watershed() function to separate the connected blobs to omit area anomalies brought by connected cells. We do this by having a gradient for the edge detection from the Sobel algorithm.
 |
| Figure 7. (From left to right) Original hypothetical cancer cells image, binarized image with normalized threshold = 0.88, and a binarized image with the application of the Watershed function. |
We then apply the said filter range obtained from the statistical process done with the normal cell images by using FilterBySize() and the result is shown in figure 8. It can be seen that there are still connected cells that was not segmented by the applied watershed function. This is in the limitation of the Sobel edge-detection algorithm and SearchBlobs function. In the midst of two errors, the process still significantly increases cancer cell detection.
 |
| Figure 8. Area-based (left) and Diameter-based (right) cell detection method. |
Also, due to the structural element diameter dependence of the cells upon image opening, we can determine the cancer cells using a Diameter-based method. It was observed that the maximum diameter for normal cells was only 13pixels, therefore we implement a structural element with diameter greater than 13 and then open the image and the result is quite remarkable for its simplicity!
This has been the most tiring activity. Huhu. But yeah, the struggle really payed off. I learned so much from this activity and I had fun! I like the thought that I'm learning and growing as a soon-to-be-physicist. Also, believe that I did quite great in this activity: from explaining the points, presenting the figures, and extending for specialized and researched functions, I grade myself a 12/10 here. :)
I would like to thank Fatima Garcia, Audrey Arcilla, Miguel Panagsagan, and Denise Musni for being with me as I do this activity. I also would like to thank Recovery Food in UP town center and Kuya Barteezy's blog for guidance. Up to the next activity!
I would like to thank Fatima Garcia, Audrey Arcilla, Miguel Panagsagan, and Denise Musni for being with me as I do this activity. I also would like to thank Recovery Food in UP town center and Kuya Barteezy's blog for guidance. Up to the next activity!
[2] M. Soriano, "Activity 8 - Morphological Operations," Applied Physics 186. 2014.
Wednesday, December 7, 2016
Activity 7: Image Segmentation
Previous activities allow us to analyze and enhance images trough various methods and image processing applications. Though these processes are quite efficient and reliable, sometimes, in the process of noise reduction, color detection, and similar concepts, it is easier to use morphological operations such as threshold applications and color-histogram mapping to remove certain image imperfections. Generally, these are processes of image segmentation. Here, image segmentation was done for both gray-scaled and colored images.
A. Grayscale Image Segmentation
Image segmentation and processing usually deals with separating or technically segmenting specific parts of the entire image that we want to focus on and obtain. Here, we highlight a region of interest (ROI) for further image analysis and processing. Taking note that each image pixel has a certain color value, we can easily segment a grayscaled image (such as one below) due to the simplicity and monotonicity of the values [1].
Notice that this image tend to have darker and lighter parts. If we want to obtain the handwritten and printed parts of the check, we can do grayscaled image thresholding as done in the previous activities. Also, it is evident that lighter colors dominate most of the image area. We can check this by obtaining a histogram of the said image and segmentation can be done by omitting the area where the pixels tend to be at maximum and grayscale values tend to cluster. This can be seen in the image below.
In Fig. 2, the ones omitted and the highlighted parts. It can be seen that the image pixels tend to peak and cluster at values between 160 and 230 (highlighted in orange). These values are for the light-colored grayscale values. We also omit values beyond 230 because we know that these are lighter parts of the image (highlighted in yellow). With this, we apply image thresholding for values greater than 160 and try to see the effect upon decreasing the threshold value with increments of 40 as seen in the figure below.
It can be seen in Fig. 3 above that as we decrease the pixel grayscale threshold value, we tend to lose more and more details. Based on personal observation, the most efficient and effective threshold value in obtaining the printed and handwritten texts is the grayscale threshold value of 120. The value of 160 still has unwanted features while the value of 80 lacks certain important features.
As you can see, grayscale image segmentation can be quite easy as long as you know the correct threshold value for the parts that you need to segment; but when it comes to colored images, segmenting isn't that easy.
B. Colored Image Segmentation
The difficulty in dealing with colored image segmentation as opposed to grayscale image segmentation is a matter of contrast and monochrome similarity. For colored images, grayscale values can be similar for different colored images. This results to confusions when trying to segment colored images by the use of grayscale pixel value thresholding. Thus, we need not only the color brightness variations of the image pixels, but also the pure chromaticity of the image details. With this, we can segment the image for specified colors. According to the handout given by Dr. Soriano, we can utilize the Normalized Chromaticity Coordinates (NCC) of the image, given by:
$$r = \dfrac{R}{R + G + B},\quad g = \dfrac{G}{R + G + B},\quad b = \dfrac{B}{R + G + B}$$
with $R$, $G$, and $B$ being the red, green, and blue image spaces, respectively [1]. It can also be implied in the above equation that the sum of $r$, $g$, and $b$ is equal to 1. With this in mind, we choose $b$ to be equal to $1-r-g$ so that we will only deal with two variables [1]. Here is a representation of the NCC using two variables $r$ and $g$:
As we can see in the figure above, primary colors can be obtained when either $r$, $g$, or $b$ would be equal to one. Similarly, other colors can be represented by the combination of the $r$ and $g$ coordinates in the given color space. Here, we do image segmentation using two proposed methods: the Parametric and Non-parametric Probability Distribution Estimation method.
B.1. Parametric Probability Distribution Estimation
In this method, a region of interest (ROI) is chosen and its corresponding color distribution is taken into account. Segmentation is done upon determining the probability that the pixel belongs to the said color distribution, which can be obtained by normalizing the histogram of the ROI with the total number of pixels. We then obtain the probability that a pixel chromaticity, either $r$ or $g$, belongs to the obtained ROI through the equation:
$$p(r) = \dfrac{1}{\sigma_r \sqrt{2\pi}} \exp \left \lbrace - \dfrac{( r-\mu_r )^2}{2\sigma_r ^{2}}\right \rbrace$$
Here, assume independent Gaussian distributions for $r$ and $g$, and obtain $\mu_r$, $\sigma_r$ and $\mu_g$, $\sigma_g$ from the cropped pixel $r$-$g$ values, respectively. We then take the product of $p(r)$ and $p(g)$ to obtain the probability that a certain color pixel belongs to the color distribution of the ROI [1].
B.2. Non-parametric Probability Distribution Estimation: Histogram Backprojection
In contrast to the parametric method, which relies on the probability that the pixel belongs to the chosen ROI's color distribution, the non-parametric method uses the histogram itself for pixel tagging [1]. This method is specifically called, Histogram Backprojection, wherein the pixel location is assigned a value based on the color histogram value in the chromaticity space. Here, $r$ and $g$ values are converted into integers and are binned into a matrix to obtain a calibrated 2D histogram[1].
C. Color Image Segmentation Application: Parametric Segmentation vs. Histogram Backprojection
Here, we apply the two methods for three images shown in the figure below. The leftmost image is by observation the simplest type wherein color values tend to be distinct and do not overlap unlike the image in the middle wherein the color patterns are more complex with discernible color gradients and distinguishable transitions. The rightmost image, on the other hand, is an image of personal choice to show validity, mastery, and personal touch on the said techniques. For all three images, three distinct ROIs were chosen and tested for segmentation.
C.1. Color Image Segmentation for Simple-type Colored Image
The figure below shows the image with three ROIs, namely, the pink, yellow, and blue colored spherical objects. It is quite evident that for segmenting images with simple color palettes, the Histogram Backprojection method is much more efficient for it is able to obtain with ease the desired region of interest. This is due to the effect of binning the histogram values. Note that the binning used in the program was 32 bins. Furthermore, because of this process, all the color values with the certain bin will be listed as a pixel within the ROI. Thus, even values of different lighting and shading can still be listed within the ROI. On the other hand, though Parametric Segmentation also produces significantly valuable results, it fails to return the whole region of interest and is limited to the color distribution with the same brightness and lighting as the chosen cropped palette. Here, we give 1pt for Histogram Backprojection.
C.2. Color Image Segmentation for Complex-type Colored Image
For this part of the activity, the image of choise has much more complex color palettes that transition from one color to another. Similarly, we select three ROIs, specifically, blue-green to yellow-green, violet to indigo, and yellow-orange to red, that also transition from one color to another. Here, it is easily discernible that Parametric Segmentation granted much more precise, complete, and promising results. This is due to the fact that Parametric segmentation takes into account all the pixel values within the cropped ROI and obtain the probability distribution of the pixel in the image if it is within the said ROI. This makes the Parametric method more dynamic and efficient for images having ROIs of transitioning color palettes.
On the other hand, due to the binning process of Histogram Backprojection, it misses out on the specific details needed to return the chosen ROI. This can be seen in the image above wherein the Non-parametric method returned images with missing blots. This may be accounted for by increasing the number of bins to allow the non-parametric method to be more specific. This time, 1pt goes to the Parametric Method.
C.3. Color Image Segmentation for Colored Image of Personal Choice
Now we go to the tie-breaker round. In this part, we choose an image of personal choice with three ROIs namely, the sky, the clouds, and the sand. Here, we can further note the differences of the two methods. Similar to the results we saw above, the Parametric method focuses more on the precision of the cropped ROI, while histogram Backprojection focuses on obtaining the chosen ROI regardless of the brightness differences. It is quite nice to see that even cloud reflections visible on the surface of the sea are still segmented. Overall, in the chosen image, Histogram Backprojection gave better results due to its versatility when it comes to the lighting of the chosen ROI. But if this were a different image, and the chose ROI would be like the transition of the clouds to the sky, Parametric Segmentation would definitely be better than the Non-parametric one.
Overall, both methods can give quite remarkable results depending on the chosen ROI to be segmented. It is just up to the knowledge and familiarity of the one doing image processing on what to use to have higher efficiency and better results. It is always better to gain more experience and at the same time study about the topic. This thought is also applicable when it comes to life decisions. We have to gauge well on what method to use and on how to approach a certain situation. We acquire knowledge through experience but at the same time, researching and gaining ample knowledge first could be essential.
Lastly, I would like to acknowledge and thank my father for taking the picture and providing me good food while I do the activity. I would also like to thank Tin Santos for introducing to me Econ library. It's quite a convenient place to do papers. The noise and AC is just right for me to be productive. I thank Denise Musni for pushing me to finish this blog. I would give myself a 12/10 for the extra effort done in presenting the results well. I used Microsoft Powerpoint to join together and decrease the number of figures. Extra work was also done in applying segmentation methods for real-life images. But the most important part is that I learned a lot and had quite some fun while doing it in the midst of the hell week. :)
[1] M. Soriano, "A7-Image Segmentation," Applied Physics 186. 2014.
[2] Simple colored image: http://imanada.com/daut/as/m/a/abstract-archives-page-5-of-8-canvas-print-art-colourful-rainbow-balls-modern-design-20x16-free-uk-pp_colourful-abstract-designers_home-decor_home-decoration-ideas-decorator-decore-decorators-collecti_797x797.jpg
[3] Complex colored image: https://hdwallpaperpoint.com/colorful-blocks-rainbow-3d-graphics-background-4k-hd-wallpaper
 |
| Figure 1. Grayscaled image of a check obtained from Dr. Soriano |
 |
| Figure 2. Histogram of the grayscaled image in Figure 1. |
 |
| Figure 3. Segmented image with the application of grayscale value thresholding. Only grayscale values less then 160 were taken with decreasing increments of 40. |
As you can see, grayscale image segmentation can be quite easy as long as you know the correct threshold value for the parts that you need to segment; but when it comes to colored images, segmenting isn't that easy.
B. Colored Image Segmentation
The difficulty in dealing with colored image segmentation as opposed to grayscale image segmentation is a matter of contrast and monochrome similarity. For colored images, grayscale values can be similar for different colored images. This results to confusions when trying to segment colored images by the use of grayscale pixel value thresholding. Thus, we need not only the color brightness variations of the image pixels, but also the pure chromaticity of the image details. With this, we can segment the image for specified colors. According to the handout given by Dr. Soriano, we can utilize the Normalized Chromaticity Coordinates (NCC) of the image, given by:
$$r = \dfrac{R}{R + G + B},\quad g = \dfrac{G}{R + G + B},\quad b = \dfrac{B}{R + G + B}$$
with $R$, $G$, and $B$ being the red, green, and blue image spaces, respectively [1]. It can also be implied in the above equation that the sum of $r$, $g$, and $b$ is equal to 1. With this in mind, we choose $b$ to be equal to $1-r-g$ so that we will only deal with two variables [1]. Here is a representation of the NCC using two variables $r$ and $g$:
 |
| Figure 4. Normalized Chromaticity Coordinate 2D space with axes $r$ and $g$. Image is from Wikipedia |
B.1. Parametric Probability Distribution Estimation
In this method, a region of interest (ROI) is chosen and its corresponding color distribution is taken into account. Segmentation is done upon determining the probability that the pixel belongs to the said color distribution, which can be obtained by normalizing the histogram of the ROI with the total number of pixels. We then obtain the probability that a pixel chromaticity, either $r$ or $g$, belongs to the obtained ROI through the equation:
$$p(r) = \dfrac{1}{\sigma_r \sqrt{2\pi}} \exp \left \lbrace - \dfrac{( r-\mu_r )^2}{2\sigma_r ^{2}}\right \rbrace$$
Here, assume independent Gaussian distributions for $r$ and $g$, and obtain $\mu_r$, $\sigma_r$ and $\mu_g$, $\sigma_g$ from the cropped pixel $r$-$g$ values, respectively. We then take the product of $p(r)$ and $p(g)$ to obtain the probability that a certain color pixel belongs to the color distribution of the ROI [1].
B.2. Non-parametric Probability Distribution Estimation: Histogram Backprojection
In contrast to the parametric method, which relies on the probability that the pixel belongs to the chosen ROI's color distribution, the non-parametric method uses the histogram itself for pixel tagging [1]. This method is specifically called, Histogram Backprojection, wherein the pixel location is assigned a value based on the color histogram value in the chromaticity space. Here, $r$ and $g$ values are converted into integers and are binned into a matrix to obtain a calibrated 2D histogram[1].
C. Color Image Segmentation Application: Parametric Segmentation vs. Histogram Backprojection
Here, we apply the two methods for three images shown in the figure below. The leftmost image is by observation the simplest type wherein color values tend to be distinct and do not overlap unlike the image in the middle wherein the color patterns are more complex with discernible color gradients and distinguishable transitions. The rightmost image, on the other hand, is an image of personal choice to show validity, mastery, and personal touch on the said techniques. For all three images, three distinct ROIs were chosen and tested for segmentation.
 |
| Figure 5. Images to be segmented. (From left to right) Simple colored image[2], Complex colored image[3], and personal image of choice from my own personal photobunk. |
C.1. Color Image Segmentation for Simple-type Colored Image
The figure below shows the image with three ROIs, namely, the pink, yellow, and blue colored spherical objects. It is quite evident that for segmenting images with simple color palettes, the Histogram Backprojection method is much more efficient for it is able to obtain with ease the desired region of interest. This is due to the effect of binning the histogram values. Note that the binning used in the program was 32 bins. Furthermore, because of this process, all the color values with the certain bin will be listed as a pixel within the ROI. Thus, even values of different lighting and shading can still be listed within the ROI. On the other hand, though Parametric Segmentation also produces significantly valuable results, it fails to return the whole region of interest and is limited to the color distribution with the same brightness and lighting as the chosen cropped palette. Here, we give 1pt for Histogram Backprojection.
 |
| Figure 6. Color segmentation for the simple-type colored image showing three different ROIs and image comparisons for the two proposed methods (Parametric and Non-parametric). |
C.2. Color Image Segmentation for Complex-type Colored Image
For this part of the activity, the image of choise has much more complex color palettes that transition from one color to another. Similarly, we select three ROIs, specifically, blue-green to yellow-green, violet to indigo, and yellow-orange to red, that also transition from one color to another. Here, it is easily discernible that Parametric Segmentation granted much more precise, complete, and promising results. This is due to the fact that Parametric segmentation takes into account all the pixel values within the cropped ROI and obtain the probability distribution of the pixel in the image if it is within the said ROI. This makes the Parametric method more dynamic and efficient for images having ROIs of transitioning color palettes.
 |
| Figure 7. Color segmentation for the complex-type colored image showing three different ROIs having observable color gradients and image comparisons for the two proposed methods (Parametric and Non-parametric). |
C.3. Color Image Segmentation for Colored Image of Personal Choice
Now we go to the tie-breaker round. In this part, we choose an image of personal choice with three ROIs namely, the sky, the clouds, and the sand. Here, we can further note the differences of the two methods. Similar to the results we saw above, the Parametric method focuses more on the precision of the cropped ROI, while histogram Backprojection focuses on obtaining the chosen ROI regardless of the brightness differences. It is quite nice to see that even cloud reflections visible on the surface of the sea are still segmented. Overall, in the chosen image, Histogram Backprojection gave better results due to its versatility when it comes to the lighting of the chosen ROI. But if this were a different image, and the chose ROI would be like the transition of the clouds to the sky, Parametric Segmentation would definitely be better than the Non-parametric one.
 |
| Figure 8. Color segmentation for the image of personal choice: a beach scenery in Sipalay City, Negros Occidental, with three different ROIs, and image comparisons for the two proposed methods (Parametric and Non-parametric). |
Lastly, I would like to acknowledge and thank my father for taking the picture and providing me good food while I do the activity. I would also like to thank Tin Santos for introducing to me Econ library. It's quite a convenient place to do papers. The noise and AC is just right for me to be productive. I thank Denise Musni for pushing me to finish this blog. I would give myself a 12/10 for the extra effort done in presenting the results well. I used Microsoft Powerpoint to join together and decrease the number of figures. Extra work was also done in applying segmentation methods for real-life images. But the most important part is that I learned a lot and had quite some fun while doing it in the midst of the hell week. :)
[1] M. Soriano, "A7-Image Segmentation," Applied Physics 186. 2014.
[2] Simple colored image: http://imanada.com/daut/as/m/a/abstract-archives-page-5-of-8-canvas-print-art-colourful-rainbow-balls-modern-design-20x16-free-uk-pp_colourful-abstract-designers_home-decor_home-decoration-ideas-decorator-decore-decorators-collecti_797x797.jpg
[3] Complex colored image: https://hdwallpaperpoint.com/colorful-blocks-rainbow-3d-graphics-background-4k-hd-wallpaper
Subscribe to:
Posts (Atom)