2016 12th IAPR Workshop on Document Analysis Systems

A Table Detection Method for PDF Documents

Based on Convolutional Neural Networks
Leipeng Hao, Liangcai Gao∗ , Xiaohan Yi, Zhi Tang
Institute of Computer Science and Technology
Peking University
Beijing, China
Email: {haoleipeng, glc, chlxyd, tangzhi}@pku.edu.cn

Abstract—Because of the better performance of deep learning lines or the tables that has complex layouts. More often, (flow)
on many computer vision tasks, researchers in the area of charts with intersected vertical and horizontal lines are usually
document analysis and recognition begin to adopt this technique detected as faked tables in prior methods.
into their work. In this paper, we propose a novel method for
table detection in PDF documents based on convolutional neutral Meanwhile, in the recent years, deep learning techniques
networks, one of the most popular deep learning models. In have greatly improved the results of many computer vision
the proposed method, some table-like areas are selected first by tasks and information processing work. In order to improve
some loose rules, and then the convolutional networks are built table detection performance and make up for the limitations of
and refined to determine whether the selected areas are tables prior methods, this paper proposes a method of table detection
or not. Besides, the visual features of table areas are directly
extracted and utilized through the convolutional networks, while based on deep learning techniques. In more details, according
the non-visual information (e.g. characters, rendering instruc- to the knowledge of the diversity of tables, several table-like
tions) contained in original PDF documents is also taken into areas first are selected by some loose rules. An area is chosen
consideration to help achieve better recognition results. The according to the loose rules that if it shares similarity with a
primary experimental results show that the approach is effective table area, even the similarity is a little, which means almost all
in table detection.
the table areas are detected and a lot of areas that are prone to
Keywords—table detection; convolutional neural net-
be detected as tables such as (flow) charts and matrixes are also
works; deep learning; document analysis
collected. Then, this paper adopts and refines the convolutional
I. I NTRODUCTION neural networks, one of the popular deep learning models,
Tables are widely used in many domains to present and as the basic element of networks to determine whether the
communicate structured information to human readers since selected areas are tables or not. In this step, the areas that
tables enable readers to rapidly search, compare and under- used to be detected as faked tables are discarded. Meanwhile,
stand facts and draw conclusions. Hence, automatically de- in the deep learning model, the non-visual information (e.g. the
tecting tables from documents and extracting the information coordinates of characters and the start point and end point of
contained in tables are of significant importance in the field ruling lines) contained in original PDF documents is extracted
of document recognition and analysis and have attracted a lot and taken advantage of to further improve the performance of
of research efforts in the past two decades. This paper focuses table detection. In addition, because training the convolutional
on the table detection in PDF documents. networks requires a large number of sample images, this paper
PDF (Portable Document Format) has become more and collects a dataset with more than 7000 PDF pages containing
more popular because of its consistency of presentation be- tables with labeled ground-truth. And the dataset would be
tween different underlying platforms, screens of handheld publicly available for the researching purpose.
devices. However, there is little or no structure information in The rest of the paper is organized as follows. Section II
PDF documents, which makes the information extraction and summarizes the relevant researches on table recognition, es-
document understanding a challenging problem even though pecially those carried out directly on PDF documents. Section
later PDF supports tagging. III describes the framework of the proposed method in detail,
As far as table detection is concerned, many researches have including selecting table-like areas, adopted structure of the
been carried out on scanned document images and web pages, convolutional neural networks etc. Section IV introduces the
but no work can handle all the tables well due to the diversity dataset and analyses the experimental results. Conclusion and
of table layouts and variety of encodings. The existing methods future works are discussed in Section V.
on table recognition still suffer from some limitations. For II. R ELATED WORK
example, the image-based methods are prone to fail when
directly carried out on PDF pages, while most of rule-based Table detection has attracted a good number of research
methods are hardly able to recognize the tables without ruling efforts so far. Zanibbi et al.[1] and Silva et al.[2] have given
a comprehensive survey of table recognition methods. Most
*Liangcai Gao is the corresponding author of the early researches on table recognition concentrated on

978-1-5090-1792-8/16 $31.00 © 2016 IEEE 287

DOI 10.1109/DAS.2016.23
image-based documents, such s the T-Recs and T-Recs++ have been utilized in a wide range of signal and information
systems proposed by Kieninger et al.[3][4]. However, those processing tasks during the past several years, such as the
image-based methods don’t perform well directly on PDF hand-written characters recognition and picture classification.
documents. For example, LeNet, proposed by LeCun et al.[10], has
As far as we know, the pdf2table system, proposed by Burcu greatly improved the performance of hand-written character
Yildiz et al.[5], is the first relevant research that deals with recognition. Wang et al.[11] propose a method based on auto-
PDF documents directly. Their work is based on the data re- encoder to recognize handwritten Chinese characters which
turned by the pdftohtml (http://pdftohtml.sourceforge.net) tool, outperforms traditional methods using hand-crafted features
which returns text chunks and their absolute coordinates in the and convolutional neural networks. The table detection method
PDF document. The pdf2table consists of two components: based on deep learning has not been proposed yet. Since deep
table detection and table decomposition. Firstly, it merges learning has greatly enhanced the results of many computer
text segments into lines. Then it determines a portion of text vision tasks, this paper attempts an adoption of deep learning
elements as a table only by means of the knowledge of the on the task of table detection.
absolute coordinates of the text elements. At last, it returns the
identified table headers, the spanning behavior of the headers III. P ROPOSED M ETHOD
and the assigning of data cells. Oro et al.[6] propose a similar The proposed method consists of three main procedures:
table recognition approach which classifies text lines into three table-like areas proposal, convolutional neural networks and
categories: text lines, table lines and unknown lines according adding information contained in original PDF documents. This
to the number of segments in the lines. After the classification section will discuss each procedure in detail. The work flow
process, the table lines and unknown lines are combined to of the proposed method is illustrated in Fig. 1.
form tables. But these methods are all based on the assumption
that the pages are of single-column. A. Table-like areas proposal
Liu et al.[7] develop a table search engine system called This method deals with tables in three categories: tables
TableSeer. It craws scientific PDF documents online, such with horizontal and vertical rule lines, tables with horizontal
as DBLP and CiteSeer, finds out the documents with tables, rule lines and tables with no rule lines. The table-like areas
recognizes the table regions, extracts the table contents and proposal procedure begins with examining horizontal and
indexes them. Their approach of table detection is mainly vertical lines on the page from top to bottom. In order to obtain
based on merging sparse lines which are similar to the methods the lines from the PDF, we use PDF parser to generate graphic
of Burcu Yildiz[5]. However, TableSeer, as a searching system, objects for each PDF instruction, each with its respective
depends too much on the precision of table detection. What’s coordinates and attributes. For different categories of tables,
more, it makes the assumption that all the tables in the PDF different strategies are applied.
documents have a caption, discarding those without a caption Tables with horizontal and vertical rule lines. When
and leading to a low recall rate. horizontal lines are crossed (or almost crossed) by vertical
The methods described above purely take the content layout lines, this may represent a tabular grid. This method collects
features into consideration and are based on the observation all the regions that contain vertical lines intersecting with a set
that table lines contain more than one text segments. On of horizontal lines as candidate table areas. Apparently, some
one hand, text line segmentation, as well as spanning cells areas that correspond to graphic objects such as (flow) charts
detection, is sensitive to predefined thresholds. On the other and boxes are selected as well. Those non-table areas will be
hand, two or more tables on the same page and irregular tables discarded by convolutional neural networks next.
cannot be handled very well by those methods. As a result, Tables with horizontal rule lines. When horizontal lines
some table regions are under-segmented or over-segmented. delineate rows in a table, the horizontal lines are practically
From the essence of tables, the graphic ruling lines should always the entire width of the table, so we attempt to check
also be treated as importantly as content layout to spot table whether the lines delineate table rows or not. First, this method
regions. Hassan et al.[8] detect tables in PDF documents collects the line objects not contained in the areas that are
utilizing both ruling lines and content layout. However, their proposed as horizontally and vertically ruled tables, and then
method utilizes these two sources separately. And lots of some pre-processing are performed to merge any dotted or
false positive tables are detected because the detected graphic touching lines that have been written in parts to the document
lines are not verified first in their method. Similarly, Fang et page. Second, to ensure that the lines represent ruling lines
al.[9] propose a method via both visual separators and tabular of a table, not the underline of headings, or separators for
structures of contents. The separators refer to not only graphic headers or footers on the page, this method applies the trained
lines but also white spaces to handle unruled tables. This convolutional networks to do the checking and generate a
method detects page columns in the first place to assist table boolean matrix, G[N ][N ], where N denotes the number of
detection in multi-column pages, and achieves a satisfactory the lines. If G[i][j] is true, the region from line i to line j
accuracy rate. represents a table.
Deep structured learning has emerged as a new area of ma- This method intends to search for large integrated table areas
chine learning. The techniques developed from deep learning between those horizontal lines. In order to settle this problem,

288
 After removing the vertical rectangle chains
PDF page
parsing
x
11...1 00 11...1 00 111 00 11...1 00 11...1 00 11...1

Horizontally V1 1000111000110000
and vertically OR
ruled table V2 1100111001100001
area proposal V 1100111001110001
2 2 3

sim(V1, V2) = 2 + 2 + 32 = 17
2 2

Horizontally Non-ruled table

CNN
ruled table areas proposal
Fig. 2. Example of generating binarized vector and similarity calculation

only deals with the non-ruled tables within one column and
Generate Calculating this step begins with column detection and separation.
matrix G[i][j] similarity
As shown in Fig. 1, a line-finding algorithm is carried out
in every column. The horizontal text lines in a table area are
practically share the same alignment. Furthermore, the gaps
in a horizontal table line are much larger than that in a text
dynamic
Text line finding line of paragraphs. To detect the table area with no rule lines,
programming
and sorting for text each line, this method calculates its vertical projection
algorithm Output
and generates a binarized vector. The n-th bit of the vector is
1 if the projection of the corresponding text line is not 0 in
x-coordinate. Fig. 2 shows how to generate a binarized vector
Table areas corresponding to a text line.
Non-ruled table
proposal The text lines are sorted according to their vertical position
from top to bottom. Then a measure of similarity is calculated
between adjacent text lines. Fig. 2 also shows the process
Fig. 1. The work flow of the proposed method of calculating the similarity. First, a vector is generated by
calculating or operation bit by bit between two text line
vectors. Second, this method counts the lengths of consecutive
a dynamic programming algorithm is applied on the horizontal 0 in the new vector. The measure of similarity is the sum of
lines. Let V al(T ) denote the value of table T which contains the square of the lengths counted in last step.
m horizontal lines. We define Apparently, the similarity between table text lines from a
V al(T )  m2 same table is much larger because they have the same align-
ment and larger overlapped gap in the horizontal direction,
Apparently, the larger the value of a table is, the larger and while the similarity between paragraph text lines is smaller.
more complete the table area is. Let maxval[i][j] denote the After the calculation of text line similarity, this method selects
max value that if we search for j tables between the first line the consecutive text lines that has roughly equal and larger
and the i-th line. Then similarity as non-ruled table areas.
maxval[i][j] = max(maxval[i − 1][j], maxval[k][j − 1] The candidate table areas of all the three categories are put
as input into the adopted and refined convolutional neural net-
+(i − k)2 ) (if G[i][k] == true) works, a judgement will be made at the output of convolutional
networks whether the input area is a table or not.
At the same time, this method uses route[i][j] to mark
which the value of maxval[i][j] is from. At the end of algo- B. Convolutional neural networks
rithm, this method selects the max value from maxval[N ][1]
Convolutional Networks are trainable multi-stage architec-
to maxval[N ][N ] and calculates the table areas with hori-
tures. The input and output of each stage are sets of arrays
zontal rule lines in the document page according to matrix
called feature maps[10]. Each feature map at the output
route[i][j].
represents a particular feature extracted through a specific filter
Tables with no rule lines. Given the fact that all the tables
on the input. The feature maps in higher stages are usually
with no rule lines in our pre-training dataset are within one
smaller than that in lower stages, representing more abstract
single column and non-ruled tables are scarce, this method

289
 feature feature
maps maps
classification
module
feature feature
maps maps u
ut
output
outp

input

sub-sampling convolutions full connection

convolutions
sub-sampling
convolutions

Fig. 4. The architecture of the convolutional neural networks

by a max-pooling layer with sampling stride of 3. The third

convolutional layer consists of 300 filters of size 5 × 5 with
stride of 1, not followed by a sub-sampling layer. The fifth
and sixth layer are fully-connected layers with 1024 and 2048
Filter Ķ Max nodes respectively.
The convolution operation is formulated as
feature   
bias map 
l l−1 l l
input
parameter yKj = f xi ∗ Kij + bj
Mi
l
where yKj is the j-th convolutional result of the j-th layer
l−1
Sigmoid Ķ and xi is the i-th output feature map of (l − 1)-th layer.
l
Kij is the convolutional kernel between i-th input feature
feature
map bias weight map and the j-th output feature map of l-th layer. blj is
parameter the bias of j-the output layer. ∗ denotes the convolutional
operation and f () denotes the activation function. This method
Fig. 3. The operation process of 3-layer stage in convolutional networks takes the hyperbolic tangent as the activation function. So the
convolution operation is actually expressed as
  

features. Basically, each stage is composed of three layers: a l l−1 l l
yKj = tanh xi ∗ Kij + bj
filter bank layer, a non-linearity layer and a feature pooling Mi
layer. Fig. 3 shows the detailed operations on a feature map
The max-pooling function is
in such 3-layer stage. The architecture is similar to LeNet by  
LeCun[10] but with some improvements. yjl = max xlj(k)
k∈(s×s)
A typical convolutional networks consist of one or more
such 3-layer stages mentioned above, followed by more than where yjl is the pooling output of the j-th feature map of the
one fully-connected layers and a classification module. The l-th layer. s × s denotes the max pooling region. Thus the max
architecture of the convolutional networks adopted in this pooling output is the maximum value of convolution map in
paper is illustrated in Fig. 4, which consists of 7 layers: three the region.
convolutional layers, two sub-sampling layers, and tow fully- The fully connected layer takes the function
connected layers. 
yjl = yil−1 · ωi,j
l
+ blj
Each input image is down-sampled to 76 × 76 × 3. The first
i
convolutional layer consists of 100 trainable filter kernels of
size 5 × 5 with stride of 1, followed by a max-pooling layer where yjl denotes the j-th node in the l-th layer, ωi,j
l
denotes
l−1 l l
with sampling stride of size 2. The second convolutional layer the weights between yj and yj , and bj is the bias of the
has 200 filter kernels of size 7 × 7 with stride of 1, followed l-th layer.

290
 The proposed method takes SoftMax as the classification TABLE I
module. The networks are then trained under a log loss (or E XPERIMENTAL RESULTS AND COMPARISON WITH OTHER METHODS
cross-entropy) regime, giving a non-linear variant of multino- Participant methods Precision Recall F1-measure
mial logistic regression. Since the function maps a vector and proposed method 0.9846 0.8366 0.9046
a specific index i to a real value, the derivative needs to take proposed method(X) 0.9724 0.9215 0.9463
Silva 0.9292 0.9831 0.9554
the index into account: Nitro 0.9397 0.9323 0.9360
∂ Nurminen 0.9210 0.9077 0.9143
(q, i) = σ(q, i)(δik − σ(q, k)) Yildiz 0.6399 0.8530 0.7313
∂qk Stoffel 0.7536 0.6991 0.7253
Here, the Kronecker delta is used for simplicity. The process
of training is terminated when the loss function converges.
IV. E XPERIMENT AND R ESULT
C. Adding information of original PDF Documents A. Dataset
The convolutional networks take the image of the proposed This paper collects over 7000 pages of PDF documents
areas as input. There is much information contained in the that contain tables and labels the table areas. The pages are
original PDF documents that could also be utilized, such as from both English and Chinese e-Books, conferences and
the coordinates of the characters and the start point and end journals. The dataset shows good variety in table styles, from
point of a line in tables. In this paper, those features are horizontally and vertically ruled tables to horizontally ruled
extracted from PDF pages and transformed into vectors in and non-ruled tables, from horizontal tables to vertical tables,
order to improve the results further. from inside-column tables to span-column tables. We choose
PDF documents are described by low-level structural objects 5000 table areas randomly and convert them to images to
such as a group of characters, lines, curves, images, etc., and train the convolutional networks. Another 5000 images of
associated style attributes such as font, color, stroke, fill, and other objects of the document pages (e.g. paragraphs, charts,
shape, etc.[12]. To parse those low-level objects, this paper matrixes) are selected as well. Both the table area and non-
utilizes the PDF parser provided by Founder Corporation, table area images are cut out from the pages manually and the
which is developed according to the PDF specification[13]. 10000 images constitute the training dataset.
Both text objects and graph objects are parsed. For text objects, The test dataset this method uses is the dataset of table com-
the character attributes such as font and the bounding-box petition of ICDAR 2013[14]. It consists of 156 tables overall.
of the characters are taken into consideration. This method Non-ruled tables, tables with complex header structures and
also takes the graphic objects including drawing and clipping small tables, with fewer than five rows are very common in
instructions into account. this dataset, which used to cause difficulties for most of the
Based on the architecture shown in Fig. 4, there are two proposed methods.
layers we can put the information vector in: the input layer
and the output layer. First, this paper transforms the text B. Results and analysis
and graphic objects features extracted from the proposed area The proposed method is compared with some academic
in the PDF page into line vectors. Second, the line vector participant methods of the table competition of ICDAR 2013.
can be put into either the input layer or the output layer of Their experimental results are shown in literature[14].
convolutional networks. For the input layer, enlarge the line This paper use three performance metrics: precision (the
vector to the same dimension of input image, and catenate it to percentage of the objects that are in fact true), recall (the
the edge of the image; for the output layer, catenate the vector percentage of the true objects that the method finds) and F1-
to the end of output vector of convolutional networks after measure. The experimental results are shown in Table I. Fig.
normalization and the enlarged vector will be connected to 5 shows some detected table areas examples of the proposed
the classification module. This paper attempts both approaches method. As shown in Table I (proposed method(X) means the
mentioned above and adding the vector into the input layer results after adding the information extracted from original
achieves better results. PDF pages into the input layer of the convolutional networks),
In summary, the framework of the proposed method is the proposed method surpasses most of the academic partic-
as follows: First, this method trains the convolutional neural ipant methods of table competition of ICDAR 2013 in F1-
networks with the training dataset, and selects some table-like measure metric. Unlike some other methods, our method is
areas based on the loose rules. Then the trained convolution not adapted specifically for the competition dataset. Also, the
networks tests the collected areas and makes a judgement recall and F1-measure metrics are all notably improved after
whether the area is a table or not, after which some post adding the information extracted from PDF pages, indicating
processing is carried out to output the detection results of the information of original PDF document makes a good
table area on a document page. Furthermore, the information contribution. The proposed method is fairly effective, while
contained in the PDF documents are extracted and added to the examples illustrated in Fig. 5 also shows some limitations
the input layer and output layer of convolutional networks of this method: some selected and detected as true table areas
respectively to improve the performance. include extra region that belongs to other object in the page.

291
 Fig. 5. Experimental example illustrations

In fact, the loose rules this method uses to collect table-like [2] A. C. e Silva, A. M. Jorge, and L. Torgo, “Design of an end-to-end method
areas may cause three limitations: (a) proposing some areas to extract information from tables,” International Journal of Document
Analysis and Recognition (IJDAR), vol. 8, no. 2-3, pp. 144–171, 2006.
that contains intersected lines such as figures and (flow) charts; [3] T. Kieninger and A. Dengel, “A paper-to-html table converting system,”
(b) merging horizontally ruled tables that are in the same page in Proceedings of Document Analysis Systems (DAS), vol. 98, 1998.
and quite close to each other; (c) the selected area contains [4] ——, “Applying the t-recs table recognition system to the business letter
domain,” in Document Analysis and Recognition, 2001. Proceedings.
extra region that belongs to other objects in the page. As Sixth International Conference on. IEEE, 2001, pp. 518–522.
described in Section IV, the convolutional neural networks [5] B. Yildiz, K. Kaiser, and S. Miksch, “pdf2table: A method to extract table
can discard the areas of (a). The proposed method applies information from pdf files,” in IICAI, 2005, pp. 1773–1785.
[6] E. Oro and M. Ruffolo, “Pdf-trex: An approach for recognizing and
effective post process (by defining the value of table and extracting tables from pdf documents,” in Document Analysis and Recog-
dynamic programming algorithm) to separate the tables in (b). nition, 2009. ICDAR’09. 10th International Conference on. IEEE, 2009,
However, the propose method still suffers from limitation (c). pp. 906–910.
[7] Y. Liu, K. Bai, P. Mitra, and C. L. Giles, “Tableseer: automatic table
We will deal with it in future work. metadata extraction and searching in digital libraries,” in Proceedings of
the 7th ACM/IEEE-CS joint conference on Digital libraries. ACM, 2007,
V. C ONCLUSION pp. 91–100.
[8] T. Hassan and R. Baumgartner, “Table recognition and understanding
This paper proposes a table detection method by combining from pdf files,” in Document Analysis and Recognition, 2007. ICDAR
loose rules to collect table-like areas and convolutional neural 2007. Ninth International Conference on, vol. 2. IEEE, 2007, pp. 1143–
networks to determine whether the chosen areas are table or 1147.
[9] J. Fang, L. Gao, K. Bai, R. Qiu, X. Tao, and Z. Tang, “A table
not. Experimental results show that the proposed method is detection method for multipage pdf documents via visual seperators and
effective, and the information contained in original PDF pages tabular structures,” in Document Analysis and Recognition (ICDAR), 2011
makes a good contribution to the performance, indicating that International Conference on. IEEE, 2011, pp. 779–783.
[10] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning
the information is valuable and cannot be ignored. applied to document recognition,” Proceedings of the IEEE, vol. 86,
The proposed method still suffers from some limitations no. 11, pp. 2278–2324, 1998.
such as the detected table area contains extra region that [11] M. Wang, Y. Chen, and X. Wang, “Recognition of handwritten characters
in chinese legal amounts by stacked autoencoders,” in Pattern Recognition
belongs other objects in the page which will be dealt with (ICPR), 2014 22nd International Conference on. IEEE, 2014, pp. 3002–
next. Also, table structure recognition and understanding will 3007.
be carried out in the future. [12] J. Fang, X. Tao, Z. Tang, R. Qiu, and Y. Liu, “Dataset, ground-truth and
performance metrics for table detection evaluation,” in Document Analysis
ACKNOWLEDGMENT Systems (DAS), 2012 10th IAPR International Workshop on. IEEE, 2012,
pp. 445–449.
This work is supported by the projects of National Natural [13] PDF Reference 1.7.
[14] R. Girshick, J. Donahue, T. Darrell, and J.Malik, “ICDAR 2013 Table
Science Foundation of China (No. 61573028), the Natural Competition,” in Document Analysis and Recognition (ICDAR), 2013 12th
Science Foundation of Beijing (No. 4142023) and the Beijing International Conference on. IEEE, 2013, pp. 1449–1453.
Nova Program(XX2015B010). We also thank the anonymous
reviewers for their valuable comments.
R EFERENCES
[1] T. M. Breuel, “Two geometric algorithms for layout analysis,” in Docu-
ment analysis systems v. Springer, 2002, pp. 188–199.

292