Previous activities allow us to analyze and enhance images trough various methods and image processing applications. Though these processes are quite efficient and reliable, sometimes, in the process of noise reduction, color detection, and similar concepts, it is easier to use morphological operations such as threshold applications and color-histogram mapping to remove certain image imperfections. Generally, these are processes of image segmentation. Here, image segmentation was done for both gray-scaled and colored images.
A. Grayscale Image Segmentation
Image segmentation and processing usually deals with separating or technically segmenting specific parts of the entire image that we want to focus on and obtain. Here, we highlight a region of interest (ROI) for further image analysis and processing. Taking note that each image pixel has a certain color value, we can easily segment a grayscaled image (such as one below) due to the simplicity and monotonicity of the values [1].
Notice that this image tend to have darker and lighter parts. If we want to obtain the handwritten and printed parts of the check, we can do grayscaled image thresholding as done in the previous activities. Also, it is evident that lighter colors dominate most of the image area. We can check this by obtaining a histogram of the said image and segmentation can be done by omitting the area where the pixels tend to be at maximum and grayscale values tend to cluster. This can be seen in the image below.
In Fig. 2, the ones omitted and the highlighted parts. It can be seen that the image pixels tend to peak and cluster at values between 160 and 230 (highlighted in orange). These values are for the light-colored grayscale values. We also omit values beyond 230 because we know that these are lighter parts of the image (highlighted in yellow). With this, we apply image thresholding for values greater than 160 and try to see the effect upon decreasing the threshold value with increments of 40 as seen in the figure below.
It can be seen in Fig. 3 above that as we decrease the pixel grayscale threshold value, we tend to lose more and more details. Based on personal observation, the most efficient and effective threshold value in obtaining the printed and handwritten texts is the grayscale threshold value of 120. The value of 160 still has unwanted features while the value of 80 lacks certain important features.
As you can see, grayscale image segmentation can be quite easy as long as you know the correct threshold value for the parts that you need to segment; but when it comes to colored images, segmenting isn't that easy.
B. Colored Image Segmentation
The difficulty in dealing with colored image segmentation as opposed to grayscale image segmentation is a matter of contrast and monochrome similarity. For colored images, grayscale values can be similar for different colored images. This results to confusions when trying to segment colored images by the use of grayscale pixel value thresholding. Thus, we need not only the color brightness variations of the image pixels, but also the pure chromaticity of the image details. With this, we can segment the image for specified colors. According to the handout given by Dr. Soriano, we can utilize the Normalized Chromaticity Coordinates (NCC) of the image, given by:
$$r = \dfrac{R}{R + G + B},\quad g = \dfrac{G}{R + G + B},\quad b = \dfrac{B}{R + G + B}$$
with $R$, $G$, and $B$ being the red, green, and blue image spaces, respectively [1]. It can also be implied in the above equation that the sum of $r$, $g$, and $b$ is equal to 1. With this in mind, we choose $b$ to be equal to $1-r-g$ so that we will only deal with two variables [1]. Here is a representation of the NCC using two variables $r$ and $g$:
As we can see in the figure above, primary colors can be obtained when either $r$, $g$, or $b$ would be equal to one. Similarly, other colors can be represented by the combination of the $r$ and $g$ coordinates in the given color space. Here, we do image segmentation using two proposed methods: the Parametric and Non-parametric Probability Distribution Estimation method.
B.1. Parametric Probability Distribution Estimation
In this method, a region of interest (ROI) is chosen and its corresponding color distribution is taken into account. Segmentation is done upon determining the probability that the pixel belongs to the said color distribution, which can be obtained by normalizing the histogram of the ROI with the total number of pixels. We then obtain the probability that a pixel chromaticity, either $r$ or $g$, belongs to the obtained ROI through the equation:
$$p(r) = \dfrac{1}{\sigma_r \sqrt{2\pi}} \exp \left \lbrace - \dfrac{( r-\mu_r )^2}{2\sigma_r ^{2}}\right \rbrace$$
Here, assume independent Gaussian distributions for $r$ and $g$, and obtain $\mu_r$, $\sigma_r$ and $\mu_g$, $\sigma_g$ from the cropped pixel $r$-$g$ values, respectively. We then take the product of $p(r)$ and $p(g)$ to obtain the probability that a certain color pixel belongs to the color distribution of the ROI [1].
B.2. Non-parametric Probability Distribution Estimation: Histogram Backprojection
In contrast to the parametric method, which relies on the probability that the pixel belongs to the chosen ROI's color distribution, the non-parametric method uses the histogram itself for pixel tagging [1]. This method is specifically called, Histogram Backprojection, wherein the pixel location is assigned a value based on the color histogram value in the chromaticity space. Here, $r$ and $g$ values are converted into integers and are binned into a matrix to obtain a calibrated 2D histogram[1].
C. Color Image Segmentation Application: Parametric Segmentation vs. Histogram Backprojection
Here, we apply the two methods for three images shown in the figure below. The leftmost image is by observation the simplest type wherein color values tend to be distinct and do not overlap unlike the image in the middle wherein the color patterns are more complex with discernible color gradients and distinguishable transitions. The rightmost image, on the other hand, is an image of personal choice to show validity, mastery, and personal touch on the said techniques. For all three images, three distinct ROIs were chosen and tested for segmentation.
C.1. Color Image Segmentation for Simple-type Colored Image
The figure below shows the image with three ROIs, namely, the pink, yellow, and blue colored spherical objects. It is quite evident that for segmenting images with simple color palettes, the Histogram Backprojection method is much more efficient for it is able to obtain with ease the desired region of interest. This is due to the effect of binning the histogram values. Note that the binning used in the program was 32 bins. Furthermore, because of this process, all the color values with the certain bin will be listed as a pixel within the ROI. Thus, even values of different lighting and shading can still be listed within the ROI. On the other hand, though Parametric Segmentation also produces significantly valuable results, it fails to return the whole region of interest and is limited to the color distribution with the same brightness and lighting as the chosen cropped palette. Here, we give 1pt for Histogram Backprojection.
C.2. Color Image Segmentation for Complex-type Colored Image
For this part of the activity, the image of choise has much more complex color palettes that transition from one color to another. Similarly, we select three ROIs, specifically, blue-green to yellow-green, violet to indigo, and yellow-orange to red, that also transition from one color to another. Here, it is easily discernible that Parametric Segmentation granted much more precise, complete, and promising results. This is due to the fact that Parametric segmentation takes into account all the pixel values within the cropped ROI and obtain the probability distribution of the pixel in the image if it is within the said ROI. This makes the Parametric method more dynamic and efficient for images having ROIs of transitioning color palettes.
On the other hand, due to the binning process of Histogram Backprojection, it misses out on the specific details needed to return the chosen ROI. This can be seen in the image above wherein the Non-parametric method returned images with missing blots. This may be accounted for by increasing the number of bins to allow the non-parametric method to be more specific. This time, 1pt goes to the Parametric Method.
C.3. Color Image Segmentation for Colored Image of Personal Choice
Now we go to the tie-breaker round. In this part, we choose an image of personal choice with three ROIs namely, the sky, the clouds, and the sand. Here, we can further note the differences of the two methods. Similar to the results we saw above, the Parametric method focuses more on the precision of the cropped ROI, while histogram Backprojection focuses on obtaining the chosen ROI regardless of the brightness differences. It is quite nice to see that even cloud reflections visible on the surface of the sea are still segmented. Overall, in the chosen image, Histogram Backprojection gave better results due to its versatility when it comes to the lighting of the chosen ROI. But if this were a different image, and the chose ROI would be like the transition of the clouds to the sky, Parametric Segmentation would definitely be better than the Non-parametric one.
Overall, both methods can give quite remarkable results depending on the chosen ROI to be segmented. It is just up to the knowledge and familiarity of the one doing image processing on what to use to have higher efficiency and better results. It is always better to gain more experience and at the same time study about the topic. This thought is also applicable when it comes to life decisions. We have to gauge well on what method to use and on how to approach a certain situation. We acquire knowledge through experience but at the same time, researching and gaining ample knowledge first could be essential.
Lastly, I would like to acknowledge and thank my father for taking the picture and providing me good food while I do the activity. I would also like to thank Tin Santos for introducing to me Econ library. It's quite a convenient place to do papers. The noise and AC is just right for me to be productive. I thank Denise Musni for pushing me to finish this blog. I would give myself a 12/10 for the extra effort done in presenting the results well. I used Microsoft Powerpoint to join together and decrease the number of figures. Extra work was also done in applying segmentation methods for real-life images. But the most important part is that I learned a lot and had quite some fun while doing it in the midst of the hell week. :)
[1] M. Soriano, "A7-Image Segmentation," Applied Physics 186. 2014.
[2] Simple colored image: http://imanada.com/daut/as/m/a/abstract-archives-page-5-of-8-canvas-print-art-colourful-rainbow-balls-modern-design-20x16-free-uk-pp_colourful-abstract-designers_home-decor_home-decoration-ideas-decorator-decore-decorators-collecti_797x797.jpg
[3] Complex colored image: https://hdwallpaperpoint.com/colorful-blocks-rainbow-3d-graphics-background-4k-hd-wallpaper
 |
| Figure 1. Grayscaled image of a check obtained from Dr. Soriano |
 |
| Figure 2. Histogram of the grayscaled image in Figure 1. |
 |
| Figure 3. Segmented image with the application of grayscale value thresholding. Only grayscale values less then 160 were taken with decreasing increments of 40. |
As you can see, grayscale image segmentation can be quite easy as long as you know the correct threshold value for the parts that you need to segment; but when it comes to colored images, segmenting isn't that easy.
B. Colored Image Segmentation
The difficulty in dealing with colored image segmentation as opposed to grayscale image segmentation is a matter of contrast and monochrome similarity. For colored images, grayscale values can be similar for different colored images. This results to confusions when trying to segment colored images by the use of grayscale pixel value thresholding. Thus, we need not only the color brightness variations of the image pixels, but also the pure chromaticity of the image details. With this, we can segment the image for specified colors. According to the handout given by Dr. Soriano, we can utilize the Normalized Chromaticity Coordinates (NCC) of the image, given by:
$$r = \dfrac{R}{R + G + B},\quad g = \dfrac{G}{R + G + B},\quad b = \dfrac{B}{R + G + B}$$
with $R$, $G$, and $B$ being the red, green, and blue image spaces, respectively [1]. It can also be implied in the above equation that the sum of $r$, $g$, and $b$ is equal to 1. With this in mind, we choose $b$ to be equal to $1-r-g$ so that we will only deal with two variables [1]. Here is a representation of the NCC using two variables $r$ and $g$:
 |
| Figure 4. Normalized Chromaticity Coordinate 2D space with axes $r$ and $g$. Image is from Wikipedia |
B.1. Parametric Probability Distribution Estimation
In this method, a region of interest (ROI) is chosen and its corresponding color distribution is taken into account. Segmentation is done upon determining the probability that the pixel belongs to the said color distribution, which can be obtained by normalizing the histogram of the ROI with the total number of pixels. We then obtain the probability that a pixel chromaticity, either $r$ or $g$, belongs to the obtained ROI through the equation:
$$p(r) = \dfrac{1}{\sigma_r \sqrt{2\pi}} \exp \left \lbrace - \dfrac{( r-\mu_r )^2}{2\sigma_r ^{2}}\right \rbrace$$
Here, assume independent Gaussian distributions for $r$ and $g$, and obtain $\mu_r$, $\sigma_r$ and $\mu_g$, $\sigma_g$ from the cropped pixel $r$-$g$ values, respectively. We then take the product of $p(r)$ and $p(g)$ to obtain the probability that a certain color pixel belongs to the color distribution of the ROI [1].
B.2. Non-parametric Probability Distribution Estimation: Histogram Backprojection
In contrast to the parametric method, which relies on the probability that the pixel belongs to the chosen ROI's color distribution, the non-parametric method uses the histogram itself for pixel tagging [1]. This method is specifically called, Histogram Backprojection, wherein the pixel location is assigned a value based on the color histogram value in the chromaticity space. Here, $r$ and $g$ values are converted into integers and are binned into a matrix to obtain a calibrated 2D histogram[1].
C. Color Image Segmentation Application: Parametric Segmentation vs. Histogram Backprojection
Here, we apply the two methods for three images shown in the figure below. The leftmost image is by observation the simplest type wherein color values tend to be distinct and do not overlap unlike the image in the middle wherein the color patterns are more complex with discernible color gradients and distinguishable transitions. The rightmost image, on the other hand, is an image of personal choice to show validity, mastery, and personal touch on the said techniques. For all three images, three distinct ROIs were chosen and tested for segmentation.
 |
| Figure 5. Images to be segmented. (From left to right) Simple colored image[2], Complex colored image[3], and personal image of choice from my own personal photobunk. |
C.1. Color Image Segmentation for Simple-type Colored Image
The figure below shows the image with three ROIs, namely, the pink, yellow, and blue colored spherical objects. It is quite evident that for segmenting images with simple color palettes, the Histogram Backprojection method is much more efficient for it is able to obtain with ease the desired region of interest. This is due to the effect of binning the histogram values. Note that the binning used in the program was 32 bins. Furthermore, because of this process, all the color values with the certain bin will be listed as a pixel within the ROI. Thus, even values of different lighting and shading can still be listed within the ROI. On the other hand, though Parametric Segmentation also produces significantly valuable results, it fails to return the whole region of interest and is limited to the color distribution with the same brightness and lighting as the chosen cropped palette. Here, we give 1pt for Histogram Backprojection.
 |
| Figure 6. Color segmentation for the simple-type colored image showing three different ROIs and image comparisons for the two proposed methods (Parametric and Non-parametric). |
C.2. Color Image Segmentation for Complex-type Colored Image
For this part of the activity, the image of choise has much more complex color palettes that transition from one color to another. Similarly, we select three ROIs, specifically, blue-green to yellow-green, violet to indigo, and yellow-orange to red, that also transition from one color to another. Here, it is easily discernible that Parametric Segmentation granted much more precise, complete, and promising results. This is due to the fact that Parametric segmentation takes into account all the pixel values within the cropped ROI and obtain the probability distribution of the pixel in the image if it is within the said ROI. This makes the Parametric method more dynamic and efficient for images having ROIs of transitioning color palettes.
 |
| Figure 7. Color segmentation for the complex-type colored image showing three different ROIs having observable color gradients and image comparisons for the two proposed methods (Parametric and Non-parametric). |
C.3. Color Image Segmentation for Colored Image of Personal Choice
Now we go to the tie-breaker round. In this part, we choose an image of personal choice with three ROIs namely, the sky, the clouds, and the sand. Here, we can further note the differences of the two methods. Similar to the results we saw above, the Parametric method focuses more on the precision of the cropped ROI, while histogram Backprojection focuses on obtaining the chosen ROI regardless of the brightness differences. It is quite nice to see that even cloud reflections visible on the surface of the sea are still segmented. Overall, in the chosen image, Histogram Backprojection gave better results due to its versatility when it comes to the lighting of the chosen ROI. But if this were a different image, and the chose ROI would be like the transition of the clouds to the sky, Parametric Segmentation would definitely be better than the Non-parametric one.
 |
| Figure 8. Color segmentation for the image of personal choice: a beach scenery in Sipalay City, Negros Occidental, with three different ROIs, and image comparisons for the two proposed methods (Parametric and Non-parametric). |
Lastly, I would like to acknowledge and thank my father for taking the picture and providing me good food while I do the activity. I would also like to thank Tin Santos for introducing to me Econ library. It's quite a convenient place to do papers. The noise and AC is just right for me to be productive. I thank Denise Musni for pushing me to finish this blog. I would give myself a 12/10 for the extra effort done in presenting the results well. I used Microsoft Powerpoint to join together and decrease the number of figures. Extra work was also done in applying segmentation methods for real-life images. But the most important part is that I learned a lot and had quite some fun while doing it in the midst of the hell week. :)
[1] M. Soriano, "A7-Image Segmentation," Applied Physics 186. 2014.
[2] Simple colored image: http://imanada.com/daut/as/m/a/abstract-archives-page-5-of-8-canvas-print-art-colourful-rainbow-balls-modern-design-20x16-free-uk-pp_colourful-abstract-designers_home-decor_home-decoration-ideas-decorator-decore-decorators-collecti_797x797.jpg
[3] Complex colored image: https://hdwallpaperpoint.com/colorful-blocks-rainbow-3d-graphics-background-4k-hd-wallpaper
No comments:
Post a Comment