International Journal of Advanced Network, Monitoring and Controls Volume 05, No.

01, 2020

Review of 3D Point Cloud Data Segmentation Methods

Ruan Xiaoyi Liu Baolong

School of Computer Science and Engineering School of Computer Science and Engineering
Xi’an Technological University Xi’an Technological University
Xi’an, 710032, China Xi’an, 710032, China
E-mail: [email protected] E-mail: [email protected]

Abstract—3D point cloud segmentation is one of the key steps set of point sets corresponding to a certain characterist-
in point cloud processing, which is the technology and process ic surface in F , which represents a region obtained by
of dividing the point cloud data set into several specific regions
with unique properties and proposing interesting targets. It segmentation. If the following conditions are met, R is
has important applications in medical image processing, called a partition of the point set S :
industrial inspection, cultural relic’s identification and 3D n
visualization. Despite widespread use, point cloud segmenta- 1)
R i S, indicates that the union of the divi-ded
tion still faces many challenges because of uneven sampling i 1

density, high redundancy, and lack explicit structure of point regions is the measured point set S , that is, each
cloud data. The main goal of this paper is to analyse the most measurement point is divided into a certain region.
popular algorithms and methodologies to segment point clouds.
To facilitate analysis and summary, according to the principle
of segmentation we divide the 3D point cloud segmentation
2) Ri  R j  Ø, indicates that the point sets obta-
methods into edge-based methods, region-based methods,
graph-based methods, model-based methods, and machine ined by segmentation do not intersect each other, and
learning-based methods. Then analyze and discuss the
advantages, disadvantages and application scenarios of these each measurement point cannot belong to two different
segmentation methods. For some algorithms the results of the
segmentation and classification is shown. Finally, we outline regions at the same time.
the issues that need to be addressed and important future
research directions. 3) The points in each region have the same
characteristics, and any two adjacent regions do not
Keywords-Line Point Cloud; Segmentation; Classification have the same characteristics.
I. INTRODUCTION 4) Vi (i  1,2,..., n) , R are all connected regions.
Image segmentation is one of the basic research
directions of computer vision, and its purpose is to METHODS SUMMARY
II.
subdivide a digital image into multiple regions with We divide the 3D point cloud segmentation method
similar properties[1]. Segmentation of 2D images has into:edge-based methods, region-based methods,
more than 50 years of research history, but 3D point graph-based methods, model-based methods, and
cloud data is a highly redundant and irregularly ordered machine learning-based methods based on the basis of
structure, point cloud segmentation also faces many the current segmentation.
challenges. The segmentation of point clouds into
foreground and background is a fundamental step in A. Edge-based methods
processing 3D point clouds. Given the set of point The edge-based segmentation method is currently
clouds, the 3D point cloud segmentation can be defined the most studied method[2]. Edges are the basic
with the sets: features that describe the shape of point cloud objects
(Figure 1). The edge-based segmentation method first
Let the surface set F  {F1 , F2 ,...Fn } reconstructed detects the geometric boundary points of the data
from the point set S, and the set R  {R1 , R2 ,...Rn } be points. The edge of the point cloud is usually composed
of the normal vector of the point cloud or the boundary
a subset of the power set of S . Each element in R is a points with abrupt curvature. These boundary points

DOI: 10.21307/ijanmc-2020-010 66
 International Journal of Advanced Network, Monitoring and Controls Volume 05, No.01, 2020

are connected and then the entire data set is divided Figure 2. Firstly, a small piece of seed region is given
into the independent multiple point sets achieve the to the segmentation target, and the seed region is used
purpose of segmenting the point cloud. Therefore, the as a starting point to search for the neighborhood; the
edge detection technology is the key to edge-based curvature, normal vector, and geometric features of the
segmentation. point cloud are used as standards; if a point meets the
criteria for seed growth, the point is incorporated into
Bhanu[3] first used gradients to detect edges, fitting
the seed region, and the process is repeated as a new
3D lines to set points, and detecting changes in the
seed point, until all points are detected, a growth region
direction of the unit normal vector on the surface.
is formed. The segmentation results are shown in
Subsequently, Jiang[4] proposed a fast segmentation
Figure 3. However, this method has the problems of
method using scanline grouping technology. The scan
being easily affected by noise and easily causing the
lines of the distance image are divided into curves and
problem of segmentation holes and excessive segmen-
then they are clustered to represent the surface. This
tation. [8-9].
method is better than Bhanu 's method in segmentation
quality and running time , but it is not suitable for point
clouds with uneven density. Sappa[5] proposed a new
edge detection method. By performing edge detection
on binary maps, extracting closed contours for fast
segmentation; Wang et al.[6] proposed a fast point
cloud edge detection algorithm. First, the point cloud
data is grid-organized to exclude non-edge discrete
points. Finally, AlphaShapes judgment conditions are
used to fficiently extract edges, which can effectively
avoids the problem of extracting outer boundaries and
holes.

(a)original point cloud (b) the edge of data a.

Figure 2. Area-based methods flow chart
Figure 1. Point Cloud and Its Edge

The advantage of edge-based segmentation is that it

has good segmentation results for data with strong
contrast in different regions, and can detect the edges
of different regions very intuitively. The disadvantage
is that although it detects all the edges, it is difficult to
determine the detection, the relationship between the
edge of the target area and the boundary of the target (a)original point cloud (b)segmented by Region Grow
area. In addition, such algorithms are very sensitive to
Figure 3. Example of Region grow segmentation
noise and are not suitable for objects with smooth
surface changes. Many scholars then the algorithm is improved.
B. Area-based methods Zhang[10] use Otsu determining the optimal segmetati-
on threshold as a constraint condition of growth, better
The region-based method classifies nearby points
than the traditional segmentation methods, but is
with similar attributes by searching for neighborhood
susceptible to noise; Angelina et al.[11] improved the
information to achieve the purpose of segmentation. In
region growing method with region merging and
1976 Zucker[7] proposed a " Region Grow " should be
genetic algorithms, and its segmentation efficiency has
split for 2D images, the researchers used after being 3D
been improved to some extent, but the boundary
point cloud. The segmentation as shown in follows the
retention is poor; Xiao et al.[12] proposed clustering-

67
 International Journal of Advanced Network, Monitoring and Controls Volume 05, No.01, 2020

based adaptive region growth for road image has good performance. Noise immunity to segment
segmentation Method, but its scope of application is millions of point clouds of a sample in less than a minute.
limited; Besl et al.[13] divide the image into 8 regions Hough transform the detection points into Hough
based on the curvature characteristics by calculating space[21]. Then a voting algorithm is used to detect
the Gaussian and average curvature of the point cloud objects with a specific shape, so that the problem of
surface, set seed nodes for these 8 regions, and then use detecting arbitrary shapes is transformed into a
polynomial simulation This method is highly suscep- statistical peak problem[22], which is mostly used for
tible to noise and is time-consuming; Chen et al.[14] the detection of circles and ellipses. Document[23]
calculated the ratio of the minimum eigenvalue of each compared RANSAC and 3D Hough transform results
point and the sum of the three eigenvalues by decom- show 3D Hough transform segment parameter values
posing the covariance matrix, and the ratio was slower and more sensitive, RANSAC the detection
minimum point as a seed point, the process is mainly result in the time segments and run more efficiently.
used in the building plane extraction rule, the high cost However, when RANSAC is applied to plane detection,
of time; Vosselman et al.[15] use randomly selecting the “pseudo-plane” problem often occurs. To solve this
seed sampling points, is determined these seed point problem, an improved RANSAC method[24] based on
whether neighborhood can be fitted to a predetermined Normal Distribution Transfor-mation (NDT) units,
model, but the method is prone to false segmentation; which is accurate. The rate can reach more than 88.5 %,
Koster[16] using the generated information in a and the plane integrity exceeds 85.0 %. In most cases,
relatively irregular pyramid FIG between the storage the method based on model fitting it needs to provide
area, for comparing and merging adjacent regions; Pu the geometric model to be detected. Zhang et al.[25]
[17] planar surface of the growth surface segmentation provide a multi-model fitting method based on splitting
algorithm laser transactions; Ning seats[18] proposed a and merging, which can eliminate the need to set the
two-step method based on the coarse segmentation and number of models in advance. In this case, automatic
fine segmentation, for extracting the main subject in segmentation of point cloud is realized.
the scene and finer details.
The region-based method is more accurate than the
edge-based method. For the seed method, the segmen-
tation effect is directly related to the selection of the
seed points. The quality of the seeds and the merge
rules determine the segmentation effect, and this
method is extremely susceptible to noise, causes over
or under segmentation.
(a) Least squares fitting (b) RANSAC
C. Model-based approach
Figure 4. Comparison of Least Squares Fitting and RANSAC
This method is based on geometric shapes such as
spheres, cones, planes, and cylinders to group point The model-based method has pure mathematical
clouds. According to these shapes, points with the same principles, is fast and powerful, and has heterogeneity.
mathematical model will be divided into the same The main limitation of this method is the inaccuracy of
region. The most typical algorithm is Fischer[19] dealing with different point clouds. This method has
random sample consensus algorithm (RANSAC) and been implemented in point cloud libraries based on
the improvement of the method. RANSAC achieves various models based on lines, planes, circles, and so
segmentation by detecting features with regular geome- on.
tric shapes such as planes, spheres, and cylinders. The
basic principle of the algorithm is to calculate the D. Graph theory-based approach
mathematical model parameters of the data based on a Graph theory image segmentation uses the principle
set of sample data sets containing abnormal data to of graph segmentation to segment the point cloud. This
obtain valid sample data. Figure 4 compares the least type of algorithm models the point cloud into a graph
squares method with the RANSAC algorithm. The model composed of nodes and edges that reflect the
results are shown in noisy data In addition, RANSAC relationship between the nodes. Graph theory-based
can more effectively fit the target. optimal segmentation is typified by the FH algorithm
proposed by Felzensawalb and Huttenloch[26] . This
RANSAC is widely used in the extraction of method constructs a weighted undirected graph using
building surfaces; Li[20] proposed an efficient RAN- point clouds as vertices, calculates RGB differences
SAC based algorithm on traditional RANSAC, which

68
 International Journal of Advanced Network, Monitoring and Controls Volume 05, No.01, 2020

between different vertices to construct weights, and multi-scale features and PointNet for feature classifi-
combines regions. Chen[27] achieves the goal of auto- cation of LiDAR point cloud data in complex scenes.
matic segmentation by building a k-nearest neighbor This method improves the local feature rxtraction of
graph, adding background constraints, and finding the PointNet. Ability to realize the automatic classification
minimum secant line. Compared with several segmen- of LiDAR point clouds in complex scenes; Niu[36]
tation methods, the results show that this method is improved the problem of the lack of local topological
suitable for segmenting foreground and background, information of the generated features, and proposed a
suitable for specified target extraction, or implementing method that uses bisymmetric functions and spatial
multi-objective extraction in a supervised classification transformation networks to obtain more robust and
manner. Ural et al.[28] suppose that a conditional stronger discrimination Compared with PointNet++,
random field model exists, and use graph cut algorithm the training time is reduced by 20% with the same
to disconnect part of the graph model to form indepen- accuracy.
dent regions according to the maximum posterior
The advantages of machine learning-based methods
estimation criterion. Li et al.[29] proposed a progre-
are good classification results and high accuracy. But
ssive form of a two-level optimal segmentation
these algorithms require the training process large
algorithm. First, the topological relationship and the
amounts of data have been tagged to do the training
distance measurement characteristics of points were
sample, and dividing the object is defined as the target
calculated under the Riemann geometry frame. The k-
has been trained, it is difficult to point to complete the
means clustering method was used to obtain the
modeling of the relationship, so that the algorithm is
segmented voxels as the bottom layer. Segmentation
difficult to comprehensively promote.
result. Then, the voxels of the point cloud are modeled
as nodes, the minimum spanning tree is constructed, Comparison of various point cloud segmentation
and the high-level feature information of the nodes is methods is shown in Table Ⅰ.
extracted. The region segmentation effect of the point
TABLE I. COMPARISON OF VARIOUS POINT CLOUD
cloud details is obtained by using graph optimization. SEGMENTATION METHODS
Much of the work of graph-based methods has been
invested in probabilistic inference models, such as the segmentation
Advantage Disadvantage
conditional random field (CRF) method, which uses methods
CRFs to label points with different geometric surface
sensitive to noise and
primitives. edge-based
Can detect the edges of
not suitable for objects
different areas very
methods with smooth surface
Graph theory-based methods can still efficiently intuitively for point cloud.
changes.
segment point clouds in complex backgrounds, but
these methods often lack real-time performance[30]. The segmentation
result depends on the
E. Machine learning-based methods quality of the seeds
region-based More accurate than edge-
Machine learning treats point cloud segmentation as and the merging rules.
methods based methods.
There will be over-
classifying point cloud data. YU[31] proposed a neural segmentation or under-
network learning method that combines segmentation segmentation.
and classification simultaneously for target
recognition;Yang[32] uses support vector machines to Fast segmentation speed,
model-based and heterogeneous,suitable Difficult to use in
classify geometric features; Guan et al.[33] converts methods for simple geometric complex scenarios.
point clouds into voxels data to extract features and models.
then use AdaBoost for training and target extraction;
currently the most well-known is Charles et al.[34] use graph-based Suitable for complex
Lack of real-time.
neural network for Point Set for 3D classification and methods scenes.
segmentation-PointNet. Although PointNet can discri-
minate the type of object, it lacks robustness to noise machine
Point cloud segmentation
learning-
and data. based
has high accuracy, good lack of real-time.
recognition effect.
methods.
To solve this problem, Charles et al. Proposed
PointNet++, which can extract local features at
different scales and obtain deep features through a
multilayer network structure. In addition, Zhao et al.
[35] proposed a deep neural network model based on

69
 International Journal of Advanced Network, Monitoring and Controls Volume 05, No.01, 2020

CONCLUSION
III. [10] Zhang L, Guo LM, He W, et al. An image segmentation algorithm
based on maximal variance between-class and region growing [J] .
Point cloud segmentation is a necessary link for Information and Electronic Engineering. 2005 .:45-48
target detection and 3D reconstruction. This paper [11] Angelina S, Suresh LP, Veni SH K. Image segmentation based on
genetic algorithm for region growth and region merging [C].
investigates some classic point cloud segmentation Computing, Electronics and Electrical Technologies, 2012.
methods and analyzes their advantages and [12] Xiao Xiaoming, Ma Zhi, Cai Zixing, et al. An adaptive region
disadvantages. As can be seen from this article, growing algorithm for road segmentation [J]. Control Engineering,
although point cloud segmentation has undergone a 2011, 18(3): 364.
long period of research, there are still limitations. [13] Besl P J, Jain R C. Segmentation through variable-order surface
fitting [J]. IEEE Transactions on pattern analysis and machine
Segmentation methods based on edges and regions intelligence, 1988, 10(2): 167-192.
have good real-time performance but are sensitive to [14] Chen J, Chen BQ. Architectural modeling from sparsely scanned
noise, and the segmentation effect is often not ideal. range data [J]. International Journal of Computer Vision, 2008.
The method based on model fitting has great [15] Vosselman G, Gorte B G H, Sithole G, et al. Recognising structure in
limitations and it is difficult to adapt to the laser scanner point clouds[J]. International archives of
photogrammetry, remote sensing and spatial information sciences,
segmentation requirements of complex environments. 2004, 46(8): 33-38.
Although graph-based and machine learning-based [16] Koster K, Spann M. MIR: An approach to robust clustering-
methods can overcome noise and efficiently segment application to range image segmentation [J]. IEEE Transactions on
Pattern Analysis and Machine Intelligence, 2000, 22(5): 430-444.
point cloud data, it is difficult to meet real-time system
[17] Pu S, Vosselman G. Automatic extraction of building features from
requirements. Therefore, how to overcome the terrestrial laser scanning [J]. International Archives of
influence of noise and how to improve the real-time Photogrammetry, Remote Sensing and Spatial Information Sciences,
performance of the segmentation system is still the 2006, 36(5): 25-27.
focus of point cloud segmentation research. [18] X. Ning, X. Zhang, Y. Wang, M. Jaeger, Segmentation of architecture
shape information from 3D point cloud[J]. In VRCAI, 2009:77-81
ACKNOWLEDGMENT [19] Nguyen A, Le B. 3D point cloud segmentation: A survey[C].2013 6th
IEEE conference on robotics, automation and mechatronics (RAM).
The completion of this paper is inseparable from the IEEE, 2013.
patient guidance of Prof. Liu Baolong, Prof. Chen Hua, [20] Yan Li, Xie Hong, Hu Xiaobin, et al. A new hybrid point cloud plane
Teacher Wu Qiong, and Teacher Lipeng Si. I also segmentation method [J]. Journal of Wuhan University (Information
Science Edition), 2013:128-130.
thank other teachers and students in this laboratory for [21] Schnabel, Ruwen. Efficient RANSAC for point ‐ cloud shape
their help. Finally, I would like to thank the science and detection [J]. Computer graphics forum, 2007, 8(06): 1259-1284.
technology program of Weiyang District of Xi'an City [22] PVC Hough (1962) Method and Means for Recognizing Complex
(Project No.: 2018036) for its fund support. Patterns, US Patent 3069654.
[23] Tarsha-Kurdi, Fayez, Tania Landes, and Pierre Grussenmeyer.
Hough-transform and extended ransac algorithms for automatic
detection of 3d building roof planes from lidar data [J], 2007.
REFERENCES [24] Li L, Yang F, Zhu H, et al. An improved RANSAC for 3D point
cloud plane segmentation based on normal distribution transformation
[1] Shapiro L, Stockman G. Computer Vision Prentice Hall [J].
cells [J]. Remote Sensing, 2017:160-163.
2001:62(3): 271-279.
[25] ZHANG Liangpei, Z Yun, C Zhenzhong, et al. Splitting and Merging
[2] Luo Xiping, Tian Jie, Zhuge infants, and the like. Summary of image
Based Multi-model Fitting for Point Cloud Segmentation. Acta
segmentation [D]. 1999.
Geodaetica et Cartographica Sinica, 2018.
[3] B. Bhanu, S. Lee, C. Ho, and T. Henderson, Range data processing:
[26] Felzenszwalb P F, Huttenlocher D P. Efficient graph-based image
Representation of surfaces by edges [J]. In proc.int. Pattern
segmentation [J]. International journal of computer vision, 2004,
recognition conf. 1896: 236-238.
59(2): 167-181.
[4] XY Jiang, H. Bunke, and U. Meier, Fast range image segmentation
[27] Chen X, Golovinskiy A, Funkhouser T. A benchmark for 3D mesh
using high-level segmentation primitives [J], In 3rd IEEE Workshop
segmentation [J]. Acm transactions on graphics (tog), 2009, 28(3): 1-
on Applications of Computer Vision. 1996.
12.
[5] A. Sappa, M. Devy, Fast range image segmentation by an edge
[28] URAL S, SHAN J. Min-cut Based Segmentation of Airborne LiDAR
detection strategy. In 3D Digital Imaging and Modeling. 2001.
Point Clouds[C]. International Archives of the Photogrammetry,
[6] Wang Zongyue, Ma Hongchao, Xu Honggen, et al. Research on water Remote Sensing and Spatial Information Sciences, XXXIX-B3.
body contour extraction method based on LiDAR point cloud data [J]. Melbourne, Australia:ISPRS, 2012:167-172.
Wuhan University Journal of Information Science. 2010:31-26.
[29] Li Minglei, Liu Shaochuang, Yang Huan, etc. A Two-level
[7] Zucker SW. Regiorowing: childhood and adolescence [J], Computer Optimized Lidar Point Cloud Scene Segmentation Method. Journal of
Graphics and Image Processing. 1976, 17(1): 99-125. Surveying and Mapping, 2018, 47 (2): 269-274.
[8] Li Renzhong, Liu Yangyang, Yang Man, et al. 3D point cloud [30] ANDONI A, INDYK P. Near-optimal Hashing Algorithms for
segmentation based on improved region growth [J]. Chinese Journal Approximate Nearest Neighbor in High Dimensions[C]//Proceedings
of Optics, 2018:23-26. of the 47th Annual IEEE Symposium on Foundations of Computer
[9] Gao F. 375 cases of MATLAB image processing [M]. Beijing: Posts Science. Berkeley, CA, USA: IEEE, 2006:459-468.
and Telecommunications Press, 2015. [31] YU Yongtao, LI J, GUAN Haiyan, et al. Learning Hierarchical
Features for Automated Extraction of Road Markings from 3-D
Mobile LiDAR Point Clouds[J]. IEEE Journal of Selected Topics in

70
 International Journal of Advanced Network, Monitoring and Controls Volume 05, No.01, 2020

Applied Earth Observations and Remote Sensing, 2015, 8(2): 709– conference on computer vision and pattern recognition. 2017: 652-
726. 660.
[32] PANG Guan, NEUMANN U. Training-based Object Recognition in [35] Zhao Zhongyang, Cheng Yinglei, Shi Xiaosong, et al. LiDAR point
Cluttered 3D Point Clouds[C]. Proceedings of 2013 International cloud feature classification method based on multi-scale features and
Conference on 3D Vision-3DV 2013. Seattle, WA, USA: IEEE, PointNet [J]. Progress in Laser and Optoelectronics, 2019, 56 (5):
2013:87-94. 052804.
[33] Yang Bisheng, Mei Baoyan. Research on 3D City Model [36] Niu Chengeng, Liu Yujie, Li Zongmin, et al. Method of 3D target
Visualization [D]. 2000. recognition and model segmentation based on point cloud data [J].
[34] Qi C R, Su H, Mo K, et al. Pointnet: Deep learning on point sets for Journal of Graphics, 2019, 40 (2): 274-281.
3d classification and segmentation[C]. Proceedings of the IEEE

71