Content + Image Segmentation * Types of Image Segmentation . Semantic Segmentation Instance Segmentation * Types of Image Segmentation Techniques based on the image properties: . . Threshold Method. Edge Based Segmentation. Region Based Segmentation. Clustering Based Segmentation. Watershed Based Method. Artificial Neural Network Based Segmentation.Image Segmentation Image Segmentation is finding the group of pixels that are similar. It is partitioning the image into various subgroups (of pixels) called Image Objects. Assigning labels to pixels, and the pixels with the same label fall under single category. It can reduce the complexity of the image to analyse the image becomes simpler. Using these labels, User can specify boundaries, draw lines, and separate the most required objects in an image from unnecessary pixels label. Algorithm detects an instances, which provides about individual objects, and hence the all the persons have different colors. Image 1Types of Image Segmentation 1. Semantic Segmentation - In image 1, every pixel belongs to a particular class (either background or person). Also, all the pixels belonging to a particular class are represented by the same color (background as black and person as pink). Instance Segmentation - Image 2 has also assigned a particular class to each pixel of the image. However, different objects of the same class have different colors (Person 1 as red, Person 2 as green, background as black, etc.). Semantic Segmentation Instance SegmentationWhy Image Segmentation is Needed? Identify the objects from image. 2. Not only identifying, Also Provides more insight information about the objects. Applications facial recognition - identifying an employee to mark attendance automatically. Medical industry - faster diagnosis, detecting diseases, tumors, cell and tissue patterns from radiography, MRI, endoscopy, thermography, ultrasonography, etc. Satellite images - identify various patterns, objects, geographical contours, soil information etc., which can be used for agriculture, mining, geo-sensing, ete. Robotics, like Robotic process automation (RPA), self-driving cars, etc.Types of Image Segmentation Techniques based on the image properties: Two approaches: 1. Similarity Detection (Region based Approach) 1. Detecting similar pixels in an image based on a threshold, region growing, region spreading, and region merging. 2. Machine learning algorithms like clustering relies on this approach of similarity detection on an unknown set of features. It detects similarity based on a pre-defined (known) set of features. 2. Discontinuity Detection (Boundary / Edge based Approach) 1. Searches for discontinuity. 2. Here, edges detected based on various metrics of discontinuity like intensity, color ete. 3. E.g. Image Segmentation Algorithms like Edge Detection, Point Detection, Line DetectionTypes of Image Segmentation Techniques based on the image properties: 1. Threshold Method. 2. Edge Based Segmentation. 3. Region Based Segmentation. 4. Clustering Based Segmentation. 5. Watershed Based Method. 6. Artificial Neural Network Based Segmentation.uo based Segmentation Threshold value T is considered as a constant. But that approach may be ineffective considering the amount of noise of the image. So, we can either keep it constant or change it dynamically based on the image properties to obtain better results. * Global Threshold - If we want to divide the image into two regions (object and background), we define a single threshold value. * Local Threshold - If we have multiple objects along with the background, we must define multiple thresholds. These thresholds are collectively known as local threshold. Based on that, thresholding is of the following types: 1. Simple Thresholding + This technique replaces the pixels in an image with either black or white. + If the intensity of a pixel (1;;) at position (i, j) is less than the threshold (T), then replace that with black and if it is more, then we replace that pixel with white. This is a binary approach to thresholding.Threshold based Segmentation 2. Otsu’s Binarization In global thresholding, we had used an arbitrary value for threshold value and it remains a constant. How can we define and determine the correctness of the selected threshold? A simpler but rather inept method is to trial and see the error. Take an image histogram that has two peaks (bimodal image), one for the background and one for the foreground. According to Otsu binarization, for that image, we can approximately take a value in the middle of those peaks as the threshold value. So, it automatically calculates a threshold value from image histogram for a bimodal image. The disadvantage is for images that are not bimodal, the image histogram has multiple peaks, or one of the classes (peaks) present has high variance. However, Otsu’s Binarization is widely used in document scans, removing unwanted colors from a document, pattern recognition etc.Threshold based Segmentation 3. Adaptive Thresholding A global value as threshold value may not be good in all the conditions where an image has different background and foreground lighting conditions in different actionable areas. We need an adaptive approach that can change the threshold for various components of the image. In this, the algorithm divides the image into various smaller portions and calculates the threshold for those portions of the image. Hence, we obtain different thresholds for different regions of the same image. This in turn gives us better results for images with varying illumination. The algorithm can automatically calculate the threshold value. The threshold value can be the mean of neighborhood area or it can be the weighted sum of neighborhood values where weights are a Gaussian window (a window function to define regions).Edge based Segmentation Edge detection is the process of locating edges in an image based on various discontinuities in grey level, colour, texture, brightness, saturation, contrast etc. To further enhance the results, supplementary processing steps must follow to concatenate all the edges into edge chains that correspond better with borders in the image. Edges consist of meaningful features and contains significant information. It significantly reduces the size of the image and filters out information that may be regarded as less relevant, preserving and focusing solely on the important structural properties of an image for a business problem.Edge based Segmentation Edge detection algorithms are two categories: Gradient based methods and Gray Histograms. Basic edge detection operators like sobel operator, canny, Robert’s variable etc. These operators aid in detecting the edge discontinuities and hence mark the edge boundaries. The end goal is to reach at least a partial segmentation using this process, where we group all the local edges into a new binary image where only edge chains that match the required existing objects or image parts are present.Region based Segmentation Creating segments by dividing the image into various components having similar characteristics. Region-based image segmentation techniques initially search for some seed points — either smaller parts or considerably bigger chunks in the input image. Next, Either add more pixels to the seed points or further diminish or shrink the seed point to smaller segments, and merge with other smaller seed points. Hence, there are two basic techniques based on this method. 1. Region Growing It’s a bottom to up method where begin with a smaller set of pixel and start accumulating or iteratively merging it based on certain pre-determined similarity constraints. Region growth algorithm starts with choosing an arbitrary seed pixel in the image and compare it with its neighboring pixels.Region based Segmentation If there is a match or similarity in neighboring pixels, then they are added to the initial seed pixel, thus increasing the size of the region. When we reach the saturation, the growth of that region cannot proceed further. So, the algorithm now chooses another seed pixel, which necessarily does not belong to any region(s) that currently exists and start the process again. Region growing methods often achieve effective Segmentation that corresponds well to the observed edges. But sometimes, when the algorithm lets a region grow completely before trying other seeds, that usually biases the segmentation in favour of the regions which are segmented first. To counter this effect, most of the algorithms begin with the user inputs of similarities first, no single region is allowed to dominate and grow completely and multiple regions are allowed to grow simultaneously.Region based Segmentation Region growth, also a pixel based algorithm like thresholding but the major difference is thresholding extracts a large region based out of similar pixels, from anywhere in the image whereas region-growth extracts only the adjacent pixels. Region growing techniques are preferable for noisy images, where it is highly difficult to detect the edges. . Region Splitting and Merging The splitting and merging based segmentation methods use two basic techniques done together in conjunction — region splitting and region merging — for segmenting an image. Splitting involves iteratively dividing an image into regions having similar characteristics and merging employs combining the adjacent regions that are somewhat similar to each other.Region based Segmentation + A region split, unlike the region growth, considers the entire input image as the area of business interest. + Then, it would try matching a known set of parameters or pre-defined similarity constraints and picks up all the pixel areas matching the criteria. * This is a divide and conquers method as opposed to the region growth algorithm. Multiple regions with Local Thresholds Background & Foreground with Global ThresholdClustering based Segmentation * Clustering is dividing the population (data points) into a number of groups, such that data points in the same groups are more similar to other data points in that same group than those in other groups (clusters). k-means clustering + The k represents the number of clusters. 5 FPP SP First, randomly select k initial clusters Randomly assign each data point to any one of the k clusters Calculate the centers of these clusters Calculate the distance of all the points from the center of each cluster Depending on this distance, the points are reassigned to the nearest cluster Calculate the center of the newly formed clusters Finally, repeat steps (4), (5) and (6) until either the center of the clusters does not change or we reach the set number of iterations + The key advantage of using k-means algorithm is that it is simple and easy to understand.Watershed based Segmentation + Watershed is a ridge approach, also a region-based method, which follows the concept of topological interpretation. + It considers the analogy of geographic landscape with ridges and valleys for various components of an image. The slope and elevation of the topography are distinctly quantified by the gray values of the respective pixels — called the gradient magnitude. + Based on this 3D representation which is usually followed for Earth landscapes, the watershed transform decomposes an image into regions that are called “catchment basins”. + For each local minimum, a catchment basin comprises all pixels whose path of steepest descent of gray values terminates at this minimum. The algorithm considers the pixels as a “local topography” (elevation), often initializing itself from user-defined markers. Then, the algorithm defines “basins” which are the minima points and hence, basins are flooded from the markers until basins meet on watershed lines. + The watersheds are formed that separate basins from each other. Hence the picture gets decomposed because we have pixels assigned to each such region or watershed Overlapping Objects Distances Sperated Objects “esArtificial Neural Network based Segmentation ANN uses AI to automatically process and identify the components of an image like objects, faces, text, hand-written text etc. Convolutional Neural Networks are specifically used to identify and process high-definition image data. + An image is considered either as a set of vectors (colour annotated polygons) or a raster (a table of pixels with numerical values for colors). The vector or raster is turned into simpler components that represent the constituent physical objects and features in an image. Multispectral Image Mask Channel Infrarred Channel 1 Infrarred Channel 2 Labeled Image be Infrarred Channel 3 ror RGB Image ry (3 Colour Sementic rieeeten ey Segmantation aMorphological based Segmentation It is for analysing the geometric structure inherent within an image. In this, the output image pixel values are based on similar pixels of input image with is neighbours and produces a new binary image. This method is also used in foreground background separation. The base of the morphological operation is dilation, erosion, opening, closing expressed in logical AND, OR. This technique is mainly used in shape analysis and noise removal after thresholding an image. Example: watershed algorithm.References * Richard Szeliski, Computer Vision: Algorithms and Applications, Springer 2010