Structure and Infrastructure Engineering

Maintenance, Management, Life-Cycle Design and Performance

ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/nsie20

Deep learning-based detection of structural

damage using time-series data

Hung V. Dang , Mohsin Raza , Tung V. Nguyen , T. Bui-Tien & Huan X. Nguyen

To cite this article: Hung V. Dang , Mohsin Raza , Tung V. Nguyen , T. Bui-Tien & Huan X.
Nguyen (2020): Deep learning-based detection of structural damage using time-series data,
Structure and Infrastructure Engineering, DOI: 10.1080/15732479.2020.1815225

To link to this article: https://doi.org/10.1080/15732479.2020.1815225

Published online: 03 Sep 2020.

Submit your article to this journal

Article views: 10

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

https://www.tandfonline.com/action/journalInformation?journalCode=nsie20
STRUCTURE AND INFRASTRUCTURE ENGINEERING
https://doi.org/10.1080/15732479.2020.1815225

Deep learning-based detection of structural damage using time-series data

Hung V. Danga,e , Mohsin Razab, Tung V. Nguyenc, T. Bui-Tiend and Huan X. Nguyena
a
Faculty of Science and Technology, Middlesex University, London, UK; bDepartment of Computer Science, Edge Hill University, Ormskirk,
UK; cModeling & Simulation Team, Schlumberger, Clamart, France; dDepartment of Bridge and Tunnel Engineering, Faculty of Civil
Engineering, University of Transport and Communications, Hanoi, Vietnam; eFaculty of Building and Industrial Construction, National
University of Civil Engineering, Hanoi, Vietnam

ABSTRACT ARTICLE HISTORY

Previously, it was nearly impossible to use raw time series sensory signals for structural health moni- Received 5 March 2020
toring due to the inherent high dimensionality of measured data. However, recent developments in Revised 8 June 2020
deep learning techniques have overcome the need of complex preprocessing in time series data. This Accepted 7 July 2020
study extends the applicability of four prominent deep learning algorithms: Multi-Layer Perceptron,
KEYWORDS
Long Short Term Memory network, 1D Convolutional Neural Network, and Convolutional Neural Vibration; machine learning;
Network to structural damage detection using raw data. Three structures ranging from relatively small finite element methods;
structures to considerably large structures are extensively investigated, i.e., 1D continuous beam under signal processing; structural
random excitation, a 2D steel frame subjected to earthquake ground motion, and a 3D stayed-cable health monitoring; neural
bridge under vehicular loads. In addition, a modulated workflow is designed to ease the switch of dif- network; time-series;
ferent DL algorithms and the fusion of data from sensors. The results provide a more insightful picture structural analysis
of the applicability of Deep Learning algorithms in performing structural damage detection via quanti-
tative evaluations of detection accuracy, time complexity, and required data storage in multi-damage
scenarios. Moreover, these results emphasize the high reliability of 2DCNN, as well as the good bal-
ance between accuracy and complexity of Long Short Term Memory and 1D Convolutional
Neural Network.

1. Introduction Recently, deep learning (DL) techniques have gained con-

siderable prominence. In comparison with conventional
Civil structures, whether they are buildings, bridges, dams,
machine learning methods, Deep Learning possesses two
monuments, offshore platforms, nuclear build-outs, have superior properties: feature learning and scalability. The per-
been constantly subjected to unpredictable and rather vio- formance of the former depends substantially on handcraft
lent excitations during their lifetime. Vehicular loads, wind features requiring domain knowledge and additional prepro-
forces, environmental changes, impact loads, and earth- cessing data, while the latter is able to automatically extract
quakes can compromise structure integrity, making it vul- meaningful features from raw data thanks to its hierarchical
nerable to any future harsh conditions and increasing multi-layer architecture where lower layers identify basic
chances of total collapse. Therefore, it is necessary to sys- features, and deeper layers synthesize higher-level patterns
tematically monitor operational states to early recognize in terms of learned lower-level ones (Brownlee, 2019).
structural deficiencies, prevent catastrophic failures, and Moreover, it is acknowledged that bigger data resort to
optimize maintenance strategies. Structural Health larger architectures to maximize benefits and vice versa.
Monitoring (SHM) is achieved using a network of sensors That is why this artificial intelligent paradigm has become
installed across structures for measurement and assessment an exciting research direction, achieving impressive results
of current condition of structure under observation. With in diverse research areas such as object recognition, machine
translation, cancer detection, fault diagnostics, and so forth.
the development of sensor technology such as Wireless
The reasons for the success of the Deep Learning algorithms
Sensor Networks (WSNs), Internet of Things (IoT), the
can be attributed to: 1) the development of high-perform-
measurement data can be collected in regular intervals to
ance computers with Graphic Processing Units (GPUs) and
ensure safety of involved personnel and sustainability of the Tensor Processing Units (TPUs), 2) the availability of huge
structure. However, turning massive measurement data into datasets due to the increasing digitalization of society
actionable information is a challenging task for the engin- (Goodfellow, Bengio, & Courville, 2016).
eering community not only because of the high dimension- For SHM applications, deep learning-based methods offer
ality of data but also inevitable uncertainties such as promising tools to achieve end-to-end system as it does not
computational errors, signal noise, environmental effects. require any additional signal preprocessing steps. Moreover,

CONTACT Hung V. Dang [email protected]

ß 2020 Informa UK Limited, trading as Taylor & Francis Group
2 H. V. DANG ET AL.

the DL models are applicable for various tasks such as dam- architectures such as Long Short Term Memory (LSTM) and
age detection, damage localization, damage severity, etc. by its simplified version Gated Recurrent Unit were developed by
fine-tuning a few last layers of the architecture or using scientists. Zhao, Wang, Yan, and Mao (2016) developed two
transfer learning techniques. Therefore, researchers in the LSTM-based methods, namely basic LSTMs, and Deep LSTMs,
engineering community have recently applied different for SHM of high-speed CNC machines using sensory data.
neural networks models for SHM, such as multi-layer per- Their results confirmed that LSTM network could perform
ceptron (MLP), Recurrent Neural Networks (RNN), much better than a number of baseline methods. Yuan, Wu,
Convolution Neural Networks (CNN). and Lin (2016) investigated the remaining useful life of aero-
A MLP consists of an input layer, single or multiple hid- engine utilizing LSTM under various operation modes and sev-
den layers, and an output layer. The number of hidden eral degradation scenarios. They found that the standard ver-
layers and the number of neurons per layer are hyperpara- sion of LSTM itself has a strong ability to achieve accurate
meters of the network, usually determined via some search (both long term and short term) prediction during the degrad-
strategies such as grid search or random search. To detect ation process. Lei, Liu, and Jiang (2019) developed LSTM-based
faults in rotary machinery components, Lu, Wang, and method for fault diagnosis of wind turbines based on multi-
Zhou (2017) developed a stacked autoencoder network com- sensor time-series signals. In their study, LSTM achieved best
posed of three hidden layers, providing direct health state performance among neural network architectures, including
classification from raw time series dataset without any prior the vanilla RNN, MLP, and Deep CNN.
signal processing techniques. Being interested in fault diag- The 2DCNN is a well-known architecture in computer
nosis of rotating machinery, Ma, Sun, and Chen (2018) built vision which helps to build several deep learning applica-
a novel coupling deep neural networks which could capture tions surpassing human experts in image classification and
joint information among multimodal sensory data to achieve object recognition. In order to leverage the advances of
high accurate prediction results. Galloway, Catterson, Fay, 2DCNNs, researchers have developed various techniques to
Robb, and Love (2016) studied fault diagnosis of tidal tur- convert one-dimensional sensory data into two-dimensional
bines by implementing a deep network. It was found that images, before feeding to a 2DCNN model, designed for
their deep learning approach achieved better results com- damage detection tasks. Wang and Gao (2017) tackled the
pared to feature-based methods involving k-nearest neigh- fault classification of the gearbox with the help of a
bor classifier, support vector machine, and decision trees. 2DCNN-based method and time-scale images received from
Another deep learning architecture for handling time-ser- wavelet transformations of vibration signals. The proposed
ies data is one-dimensional Convolutional Neural Network method could attain an accuracy of more than 99% in clas-
(1DCNN), which applies the convolution operator on sifying different fault types in the tested gears. Lu et al.
sequences of data points to extract informative features. (2017) investigated bearing fault diagnosis by means of a
Abdeljaber et al. (2018) proposed a 1DCNN model to detect 2DCNN using 2D feature maps of monitoring data. In their
changes in structural properties of a steel frame using meas- study, the 1D time-series signal was rearranged into 2D
ured acceleration signals. J. Li, Dackermann, Xu, and Samali maps via a matrix reconstruction method. The comparison
(2011) published promising results for structural damage experiment results are competitive and promising as classifi-
detection of Euler-Bernoulli beams by combining 1DCNN cation accuracies were more than 90% on average, and vari-
and original waveform signals in lieu of handcrafted fea- ation ranges were less than 2%. Recently, Tang, Chen, Bao,
tures. Avci, Abdeljaber, Kiranyaz, Hussein, and Inman and Li (2019) applied 2DCNN to the anomaly detection
(2018) addressed the loss of connection stiffness of a steel problem of a long-span cable-stayed bridge, utilizing 2D
frame structure via a novel SHM method using 1DCNN representation of accelerometer sensor signals. Their results
and WSNs. Zhang, Miyamori, Mikami, and Saito (2019) suggest that 2DCNN approach fared better than the existing
developed a 1D-CNN method for vibration SHM (VSHM) method in structural anomaly detection, and was scalable to
of bridge structures and successfully tested on both a simpli- include more signal data from multiple measurants.
fied laboratory model and a real steel bridge. Ince (2019) A notable trend towards DL techniques can be seen in
demonstrated that the 1DCNN architecture was highly various application areas, but there are still limited studies
effective in real-time monitoring of motor conditions on the applicability of DL in structural health monitoring,
because their model took only 1.0 ms per classification, and especially for full-scale structures. Moreover, reviewed works
the experimental accuracy was more than 97%. Recently, only focused on one specific DL algorithm, but no extensive
Liu and Zhang (2020) applied the CNN model to investigate study comparing different DL algorithms in the civil engin-
the Maryland and Delaware highway bridges using historical eering field on multi-facets, including detection accuracy,
data and achieves a high accuracy of more than 85%. time complexity, and required data storage. This study is
The recurrent neural network is a special architecture thus motivated by the need for a more insightful and com-
among Deep learning algorithms designed for capturing time- plete picture of the applicability of DL algorithms to the
dependent characteristics. Thus, RNNs are naturally proposed Structural Damage Detection (SDD). The main contribu-
for feature learning of sensor measurements. However, the sen- tions of this work are summarized as follows:
sor data usually consist of long sequential samples, therefore,
the vanilla RNN suffers either gradient exploding or vanishing.
Design of a modulated workflow flexible in switching
To cope with this long-range dependencies, some derived RNN different DL algorithms and fusing data from multiple
 STRUCTURE AND INFRASTRUCTURE ENGINEERING 3

sensors, thus enabling to 1) compare the practical effect- The time series sensory signal can also be represented in
iveness of various DL algorithms, 2) perform multiple frequency domain via Fourier Transformation (FT). Any
structural damage detection tasks such as damage local- signal can be expressed as a sum of infinite series of sinus-
ization, damage severity, and 3) handle time series data oids. The FT will decompose a signal into its constituent
polluted by noises through the noise injection learn- frequencies together with their associated coefficients. For
ing method. civil structure, the frequency range of interest depends on

Evaluation of strengths and weaknesses of each technique both dynamic characteristics of structures and external
through three case studies of structures ranging from excitations.
relatively small structures (i.e., a reinforced concrete Moreover, it is possible to observe the evolution of fre-
beam) to considerably large structures (i.e. a stayed-cable quency components overtime using the short-time Fourier
bridge). It was found that 2DCNN constantly achieves Transform (STFT) technique, which results in a 2D repre-
the highest accuracy prediction, but it requires additional sentation of the signal. A long signal is divided into shorter
storage for images, while LSTM and 1DCNN demon- segments of equal length; then FT is applied to each seg-
strate a fair balance between accuracy and computa- ment. Mathematically, the STFT is formulated as follows:
tion complexity. ð1
STFTðs, f Þ ¼ xðtÞwðtsÞe2pift dt: (2)
1
The rest of the paper is organized as follows: Section 2
briefs basic vibration signal analysis, including its time- where x(t) is the signal to be transformed, wðtÞ is the win-
domain representation, frequency-domain representation, dow function, i is the imaginary unit, f and s are frequency
and 2D time-frequency representation. Section 3 presents and time index of the STFT, respectively. Applying time dis-
the architectures of four investigated algorithms: MLPs, cretization, the formulation is rewritten as follows:
LSTM, 1DCNN, and 2DCNN. In Section 4, the performance X
nþN1
of these algorithms is compared through three case studies STFTn, f ¼ xk wkn e2pifk , (3)
involving a 1D continuous beam, 2D steel frame, and 3D k¼n

stayed-cable bridge. Finally, the conclusions are drawn in in which n stands for the segment number, N denotes the
Section 5. length of a segment. The 2D representation of STFT ampli-
tude is called the spectrogram of the signal. Figure 1 illus-
trates an example of acceleration signal with its Fourier
2. Deep learning-based structural damage detection
transformation and spectrogram.
Currently, there is a vast number of techniques developed
to improve the performance of DL, yet making these techni-
2.2. Multi-Layer perceptron-based algorithm
ques practical and applicable to real scenarios is not a trivial
task. One of the vital elements in building a successful DL The first investigated neural network is the multi-layer per-
application is to select an appropriate DL architecture, ceptron (MLP) composed of input layer, hidden layer and
understand its impact on the application performance as output layer. Measured signal data are fed into the input
well as its tradeoffs. Thereby, in this part, we investigate the layer, which is then synthesized through one or multiple
applicability of the most widely used architectures across a hidden layers before emerging at the output layer as damage
spectrum of sectors (Jensen, Pedersen, & Thomsen, 2017) to diagnostic predictions. Mathematically, this process can be
SHM, namely, MLP, 1DCNN, LSTM, and 2DCNN after a formulated as (Burkov, 2019):
brief introduction to the structural dynamic response. Y ¼ FðXjWÞ ¼ fL ð:::f2 ðf1 ðXjW1 ÞjW2 ÞWL Þ, (4)
where L is the total number of layers in the network includ-
2.1. Introduction to structural dynamic response ing the input and output layer, W is the matrix of parame-
In this work, acceleration signals serve as input for DL algo- ters, X and Y denote the input and output vectors,
rithms. A set of essential points across the structure is pre- respectively. The connection between two consecutive layers
defined; then, dynamic responses are measured and is performed by combining a linear matrix operation and a
collected. It is common to apply a simple and rapid stand- nonlinear activation function as follows:
ardization processing on measured signals before feeding fl ðXl jWl Þ ¼ hðWl  Xl þ bl Þ with l ¼ 1, 2, :::, L (5)
data into the DL model. The technique rescales data to a
where  is the element-wise product, Xl is the input of the
range having zero mean and unit variance. The mathemat-
l-layer, bl is bias vector, and h denotes the activation func-
ical formula of the standardization technique is expressed as
tion that is a sigmoid function for hidden layers and the
follows:
softmax function for the output layer. For this study, struc-
xl
xnew ¼ , (1) tural damage detection tasks are posed as multi-classification
r problems, e.g., detect damaged/undamaged state (0/1 class),
where x denotes the original data, xnew is the standardized assess damage severity (healthy, minor, medium, major
data, l and r are the mean value and the standard deviation class), etc.; thus the softmax function is selected at the out-
of the original data, respectively. put layer as a permanent part of the method. Softmax
4 H. V. DANG ET AL.

Figure 1. Vibration signal a) time-domain representation, b) frequency-domain representation, c) Time-frequency representation.

function normalizes evaluated values into a form of prob- dataset is significantly lower than training accuracy) and
ability falling between [0, 1], probabilities of all classes are requires a high computational cost. In this work, the num-
added up to 100%, and the class with the highest probability ber of output classes is basically under 100 therefore, num-
will be selected as predicted class. The softmax activation ber of perceptrons in hidden layer are set to 1000. Figure 2
function is formulated as (François, 2017): shows the architecture of the implemented MLP with 1000-
2 3 2 1 3 perceptron hidden layer.
y1 ewL XL
6 . 7 6 7
6 . 7
6 . 7 6 ... 7
6 7 1 6 k 7 2.3. Long short term memory-based algorithm
Y ¼ fL ðXL jWL Þ ¼ 6 yk 7 ¼ PK wi X 6 w XL 7
6 e L 7, (6)
6 . 7 e L L 6 .. 7
6 . 7 k¼1
4 . 5 In this section, LSTM network is reviewed and adapted for
4 . 5 K structural damage detection purposes. The fundamental the-
yK ewL XL
ory of LSTM refers to the work of (Hochreiter &
where K is the number of output classes, yk is the predicted Schmidhuber, 1997). The structure of LSTM consists of
probability of the class k, wkL is the weight row vector k of jointly connected repeating cells. Each cell has three gates,
the parameter matrix HL associated with the kth output namely forget gate, input gate, and output gate for control-
class, i.e.: WL ¼ ½w1L . . . wkL . . . wKL T : ling information flow. Outputs of LSTM sequences are fed
Herein, the network parameters are computed in a super- into a fully connected layer with softmax activation func-
vised fashion, which means the structural state is labeled in tion, which further provides the probability for each pre-
advance, next the database are divided into three groups, dicted class (see Figure 3).
namely training, validation, and testing datasets. The value The mathematical formulation of this model is described
of network parameters are randomly initialized, then they as follows. Given xt, the input at time step t, ht1 the output
are iteratively updated via the backpropagation algorithm to of LSTM cell at time step t  1, a linear transformation of
minimize the deviation between prediction values and anno- the combination of xt and ht1 is expressed by:
tated structural states. At a step t of the training process, Lðht1 , xt Þ ¼ W ½ht1 , xt  þ b, (9)
the parameter values are updated by:
where W and b are the weight matrix and bias vector of
ij ij j
wl ðtÞ ¼ wl ðt1Þ þ gdil xðl1Þ ðtÞ, (7) the network.
Then, the three gates inside each cell of LSTM are writ-
where l denotes the layer number, dli is the error at node i ten (Olah, 2015):
of layer l, which is calculated backward from the deviation
ff ¼ rðLf ðht1 , xt ÞÞ
at output layer dL and g stands for the learning rate. The
fi ¼ rðLi ðht1 , xt ÞÞ (10)
training process terminates when a tolerable error level is
fo ¼ rðLo ðht1 , xt ÞÞ,
met or the number of iterations reaches to a limit.
The deviation between the prediction and actual struc- in which the subscripts f, i, o indicate the forget gate, input
tural states is computed via the cross-entropy function gate and output gate, respectively. r is the sigmoid activa-
(Janocha & Czarnecki, 2017) as follows: tion function.
X At every time step t, the new candidate of information is
Loss ¼ dL ¼  yk log pð^y k Þ, (8) firstly created by applying the tanh activation function on a
k
linear transformation of the combination ½ht1 ; xt  :
where k denotes the class number in output, p is the prob-
ability estimate, y is the actual label and ^y stands for predic- Ct ¼ tanhðLc ðht1 , xt ÞÞ, (11)
tion value. where Lc is a creation linear operator whose formula is
Theoretically, MLP network with one hidden layer can similar to Eq. (9). Then the flow of information from previ-
approximate any functions in the training process on the ous step is updated with the new candidate by the gates of
condition of a sufficiently large hidden layer (Goodfellow LSTM cell:
et al., 2016). However, such a large hidden layer usually
leads to the over-fitting phenomena (accuracy on test st ¼ ðff  ht1 Þ丣ðfi  Ct Þ, (12)
 STRUCTURE AND INFRASTRUCTURE ENGINEERING 5

(Kiranyaz et al., 2019):

hk ¼ conv1Dðwk , XÞ þ bk , (14)
where hk, wk and bk are respectively the output vector,
weight vector and bias parameter of the kernel k. X is the
input vector, and conv1D is the 1D convolution operator
whose ith output is calculated as follows:
X
Nk
conv1Dðwk , XÞðiÞ ¼ wk  XðiÞ ¼ wkj xij , (15)
j¼1

where Nk is the length of the kernel k, wkj is the jth element

of vector wk.
In fact, this 1DCNN architecture can be extended to a
Figure 2. Architecture of the Multi-Layer Perceptron Network dedicated deep CNN configuration involving multiple consecutive
for VSHM.
convolutional layer to perform complex tasks requiring vari-
ous feature abstraction levels.

2.5. 2D Convolution neural network-based algorithm

The fourth deep learning algorithm is based on the deep
2DCNN, which utilizes images instead of temporal signals
as input data. The vibration signal is translated into the
time-frequency representation through the short-time
Fourier Transform technique, as presented in Section 2.
Then, a powerful image classifier in the Artificial Intelligent
literature is employed to detect abnormally structural cases.
Here we adopted the ResNet-18 classifier and employed
Figure 3. Architecture of the Long Short Term Memory Neural Network (Lei
et al., 2019).
transfer learning technique to fine-tune the model for struc-
tural damage identification tasks. The ResNet-18 is chosen
due to its competitive performance, and explicit feed-for-
where  and 丣 are element-wise product and element-wise
ward architecture which is depicted in Figure 5. There are
addition. Next, the output of the cell at time step t is calcu-
in total 18 layers of 2D convolution neural layers divided
lated based on the updated information and the output gate:
into four blocks, and shortcut connections are inserted
ht ¼ fo  st : (13) between every two CNN layers (He, Zhang, Ren, &
Sun, 2016).
Besides, the transfer learning technique allows exploiting
2.4. 1D Convolution neural network-based algorithm general knowledge pretrained with a vast amount of data of
different categories in a particular domain. For further
It is commonly acknowledged that the convolutional neural
details of the transfer learning technique, refer to (Pan &
networks (CNNs) can provide outstanding performance on
Yang, 2010) for its widely cited unified definition and recent
signal classification and pattern recognition because 1) its
works of (Howard & Gugger, 2020) for its practical imple-
architecture is especially suitable for discovering local rela- mentation API. The reasons of using this technique before-
tionships in space; 2) it reduces the number of network hand are two folds: 1) the total number of parameters of
parameters, thus leading to a lower computational complex- ResNet-18 is more than 11 million, thus training the whole
ity compared to conventional neural networks. Once vibra- model from scratch requires impractically large volumes of
tion data enter into the network, the 1DCNN layers will data and time. 2) There exists a broad consensus that for
extract inner relationships between measured points and computer vision, features extracted from base models well-
their higher derivatives before feeding to the last fully con- performing on general tasks benefit the performance of tar-
nected layers serving as a classifier. Figure 4 depicts the get models in specific domains because features are more
detailed configuration of the applied CNN for structural generic in early layers and more task-specific in later layers
damage identification, which is developed from the work of (Lee, Karpathy, & Johnson, 2018). Herein, the parameters of
(Lin, Nie, & Ma, 2017). Its configuration included a the ResNet-18 model are pretrained with millions of images
sequence of layers involving an input layer, two convolu- from the ImageNet dataset from Google.
tional layers, a max-pooling layer, two fully connected (FC) The workflow of the algorithm is illustrated in Figure 6.
layers, and an output layer. Firstly, original vibration signals are fed into a preprocess
The hyperparameters of a 1D convolution layer comprise module to be converted into images of time-frequency rep-
the number of kernels, the kernel length, and the stride resentation, prior to entering the ResNet-18 models.
value. The relation for one kernel is expressed as follows Afterwards, computed data from the ResNet is fed into a
6 H. V. DANG ET AL.

Figure 4. Architecture of 1DCNN for structural damage detection adapted from (Lin et al., 2017).

Figure 5. Architecture of the ResNet-18 Network (He et al., 2016).

Figure 6. Workflow of 2DCNN-based method for structural damage detection.

fully connected layer before giving the classification results length of 9 m (Figure 7). Its cross-section dimensions are
at the output layer using the softmax activation function. 40  60 cm2, Young’s modulus E is 30 GPa with 95% confi-
dence interval being E95 ¼ ð160:05Þ=E (Noguchi & Nemati,
1995), Poisson’s ratio is 0.3, mass density q is between 2200
3. Performance evaluation
and 2400 kg/m3 (Kosmatka, Kerkhoff, & Panarese, 2019),
In this section, the implementation of the DL algorithms is and the modal ratio is 2%. The beam is modeled in the
investigated through three case studies from a simple 1D Finite Element software Abaqus Standard (Dassault, 2016a),
structure to a full-scale 3D structure. Their performances in using the three dimensional beam element. The element has
terms of accuracy, time complexity, required storage as well six degrees of freedom (dofs) at each node, including three
as the robustness in dealing with noisy data are quantified translations and three rotations around the x, y, and z direc-
and compared based on calculated results. The next subsec- tions. Section properties and material properties are sum-
tion will go through three case studies from a simple one- marized in Table 1. The beam is subjected to a time-varying
dimensional beam structure to a two-dimensional frame uniform load modeled as a Gaussian random process with
structure to a three-dimensional full-scale stayed- amplitude falling between 1 and 20 kN/m, and the coeffi-
cable bridge. cient of variation is 0.1. Figure 7(b) shows a representative
time history of load amplitude at the middle of the beam.
The self-weight is implicitly accounted for in the simulation
3.1. Case study 1: 1D continuous beam excited by
through the given material mass density and gravitational
random force
acceleration.
The first numerical experiment is a continuous concrete Each span of the continuous beam is divided into nine
beam usually occurring in buildings subjected to time-vary- segments of 1 m length. The damage is randomly assigned
ing uniformly distributed loads reflecting machinery excita- to these segments. The damage is modeled as a reduction in
tion, for example. The beam consists of three spans of equal the effective height of the section, leading to element
 STRUCTURE AND INFRASTRUCTURE ENGINEERING 7

Figure 7. Three-span continuous beam under time-varying uniformly distributed load. a) Schematic representation of the beam. b) Example of a time-varying
amplitude loading at the middle of the beam.

Table 1. Numerical input parameters for the continuous beam case study.
Number f1 f10 Duration of Time step Mesh size Element Analysis
of mode (Hz) (Hz) simulation (s) (s) (m) type method
10 11.67 169.67 120 1e-3 0.2 B21 Modal analysis

Table 2. Random parameters for Monte Carlo simulations of the continuous beam structure.
E q Loading Damage Damage
Parameter (kN/m2) (kg/m3) (kN/m) level (%) location
Distribution Normal Uniform Uniform Uniform uniform
Value E95 ¼ [28.5  31.5] [2200–2400] [1–20] [0, 5, 15, 25] [s1-s9]
0: No damage.

stiffness loss, thus resulting in changes in the dynamic satisfied accuracy but requires less computational effort than
behavior of the structure. Three damage levels are investi- the direct integration, therefore it is suitable to generate an
gated, corresponding to 5%, 15%, and 25% reduction in the extensive dataset for DL algorithms. The number of modes
effective height. In reality, this type of damage is often included the modeling is selected through a conventional
observed in reinforced concrete elements prone to cracking, convergence analysis, resulting in a value of 10 for the
spalling, and so forth. The monitoring system is simulated present simply supported beams. The simulation duration is
by using 9 virtual accelerometers measuring vertical acceler- T ¼ 120 s, and the sampling frequency is f ¼ 200 Hz.
ation responses of the beam; three vrtual sensors are Figure 8 illustrates an example of time-series sensor data
mounted on each span (at 1/4, 1/2, and 3/4 span). The setup obtained from numerical simulations.
configuration is schematically depicted in Figure 7. It is sup- Intuitively, using time-series from only one sensor hardly
posed that virtual sensors work in ideal conditions, which provides accurate results about damage localization across
means there is no noise included in the obtained acceler- the whole structure, while taking multiple sensor data as
ation time-series. The variation of responses only results input for the neural network model can benefit the model
from uncertainties in input parameters and various damage performance. There are two ways to handle multiple time
scenarios introduced. The inputs for the Monte Carlo simu- series: i) combining these time series into a 3D time-ampli-
lation are summarized in Table 2, other parameters remain tude-sensor tensor or even 4D time-frequency-amplitude-
identical between calculations. The setup configuration is sensor tensor before feeding to a neural network ii) each
schematically depicted in Figure 7. time series enters to a separate model of the same architec-
Due to the uncertainty of external load, material proper- ture. The former way needs to increase the dimension of
ties’ variation, and various damage scenarios, we use the input data, while the latter one requires to repeat training
Monte Carlo method to generate an input database of 10000 process by the number of networks. To cope with these
samples and run extensive finite element simulations. The issues, one revisits the second approach by concatenating
finite element simulation is realized by the use of the mode- the output of each neural network into a long vector before
based dynamic method in Abaqus Standard, which provides passing through a common output layer with the softmax
the time-dependent response of the structure via the prin- activation function to provide damage classification output,
ciple of superposition. At first, the eigenvalue analysis pro- as schematically illustrated in Figure 9. In this way, the
cedure (subspace projection method) is conducted to extract resulting unified model has the capability of using the ori-
modal characteristics, the number of extracted modes ginal multiple time series as input without increasing input
should be sufficient to reflect the real structural behavior dimensions or repeating the training process.
closely. After that, the amplitude coefficient associated with The first structural damage detection task is to estimate
each mode in response to external excitations is determined; damage severity. The data are labeled by the four introduced
the response of the structure is obtained by superposing damage levels, namely, healthy, minor, medium, and major,
modal responses. More details of the mode-based dynamic as listed in Table 3. The damage classes are evenly distrib-
analysis can be found in (Dassault, 2016c). In general, uted, i.e., there are 2500 cases per damage level. Such uni-
modal dynamic procedure could provide results with form distribution simplifies the training process by avoiding
8 H. V. DANG ET AL.

Figure 8. Time history of acceleration recorded at virtual sensors from s1 to s9.

Given a specific structural class denoted C, True Positive

(TP) is the number of outcomes where the model correctly
predicts the class C. True Negative (TN) is obtained when a
time series comes from another class, and the model cor-
rectly avoids assigning C for the time series. Inversely, False
Positive (FP) is where the model mislabels an input as C
while it is not, and False Negative (FN) is when the model
ignores the true label C of the input. Then, the precision P,
recall R and F1 metric are derived as follows:
Figure 9. Concatenating calculation outputs of different neural networks into
final classification results. Acceleration signal retrieved by different virtual sen- TP
sor is fed to the corresponding neural network. P ¼
TP þ FP
TP
Table 3. Categorization of damage severities. R ¼ (16)
TP þ FN
No Minor Medium Severe 2
damage damage damage damage F1 ¼ :
Damage severity (Label “O”) (Label “I”) (Label “M”) (Label “S”)
1=P þ 1=R
Height reduction 0% 5% 10% 15% Figure 11 highlights the F1-score obtained with four deep
learning algorithms on testing data. It is noted that all deep
learning algorithms can discriminate between healthy states
the class imbalance problem. Afterward, they are divided and minor damaged states (5% section loss), which are visu-
into three groups: training, validation, and testing with a ally difficult to recognize. Moreover, the best testing results
ratio of 80/10/10 for training and testing the performance of are obtained with 2DCNN algorithms, followed by LSTM,
presented DL algorithms. The training process aims to then 1DCNN and MLP.
determine the network’s parameters, which minimize the The second structural damage detection task is damage
deviation between the predicted outputs and true labels localization. In reality, structures consist of a huge number
measured herein by the Cross-Entropy loss function of elements, therefore, the sensors are strategically placed at
(Goodfellow et al., 2016) by means of the Adam optimizer essential elements or distributed uniformly across the struc-
algorithm (Kingma & Ba, 2014). The learning rate standing ture, and damage localization tasks are focused on the ele-
for the relative amount of DL model weights updated after ments associated with sensor placement. The process of the
each optimization iteration is set to lr ¼ 0.0001, the batch damage localization in this example is realized as follows. At
size, i.e., the number of data utilized in one optimization first, the fictive damage is introduced to an element corre-
iteration is 32. These values are commonly selected in sponding to sensor locations (9 possible cases), the dataset is
Machine Learning practices and were found effective in further labeled by respective damage locations. The number
this work. of neurons in the output layer of the network is set to 9. The
Figure 10 shows the evolution of training loss and valid- training process is then carried out similarly as damage
ation accuracy of four neural network algorithms obtained severity task.
during the training process. Apparently, 2DCNN model Figure 12 details the testing results obtained with four
achieves the best validation accuracy, followed by LSTM, deep learning algorithms represented via confusion matrices,
IDCNN and MLP respectively. On the other hand, 1DCNN in which the horizontal axis denotes classes predicted by
and MLP have fast convergence speeds, the curves follow a models, the vertical axis represents the actual ones, and the
stable trend after around 100 epochs. number of each class is visualized by color intensity. The
 STRUCTURE AND INFRASTRUCTURE ENGINEERING 9

Figure 10. Evolution of training loss and validation accuracy in function of the number of epochs in the training process.

by four air mount isolators, enabling the structure to move

horizontally.
The above setting of the structure is considered as the
undamaged condition. Thereafter, various modifications are
introduced to the structure to generate different damaged
states, including loosening or completely removing column-
plate connections at a corner of the first floor or third floor,
or at both locations. Structural states and corresponding
labels are enumerated in Table 5. There are 5 undamaged
states, 3 damages locations, and 2 damage levels, which
Figure 11. Comparison of F1-score obtained by different DL algorithms on the result in total 11 structural states, where L1C, L3A, and L13
test database for damage severity task. denote damage locations on the first floor, third floor, and
at both floors, respectively. DB0 refers to the removal of the
2DCNN and LSTM provide highly accurate damage identifi- bolt, and DBB represents the bracket was com-
cation output. Although, some false predictions are pletely removed.
observed, which are related mostly to the location s1 and s9, The structure is excited through an electrodynamic
near two extremities of the beam. shaker at the corner of the base floor, the vibration of each
Table 4 summarizes the main parameters of four algo- story is measured by using 8 single-axis accelerometers with
rithms, including training times, testing accuracies for both 1 V/g sensitivity and a sampling frequency of 1600 Hz, two
damage severity, and damage localization tasks. All the sensors installed for each column-plate joints (in x and y
experiments are carried out on a high-end machine directions), resulting in total 24 accelerometers. Each struc-
equipped with 6 CPU Intel Xeon, 32-GB RAM, and 2 tural state is measured multiple times with different levels of
Geforce GTX 1080Ti GPU. shaker excitation. For each sensor, there are in total 1080
Remark 1: The fusion DL workflow described above facil- time-series of length 2048 corresponding 11 structural states.
iates the detection of the damage occurring across the whole Figure 14 shows a representative example of vibration sig-
structure, which is infeasible if using time series from a sin- nals from a sensor on floor 2. It is difficult to visually dis-
gle sensor, especially when the sensor is far apart from dam- tinguish damaged states from undamaged ones. The
age locations. One also observed that 2DCNN achieves the vibration database is split into training/validation/testing
best performance (99.2%) but requires the most computa- datasets with a ratio of 80/10/10. Next, one adapts the pre-
tional time, i.e., 1.6 times higher than that of LSTM, which sent algorithms to perform three damage detection tasks,
also yields good identification results (95%). The MLP and meaning identify undamaged/damaged state, damage local-
1DCNN are notably suitable for a quick assessment of struc- ization, and damage severity. Input for neural network mod-
tural conditions due to its rapid implementation and con- els is 24 time-series of length 2048, labeled by
vergence speed, whose training time is only one-third of corresponding structural states as mentioned in Table 5.
that of 2DCNN. Damage detection of investigated methods is presented in
Table 6 and in Figure 15 in terms of efficiency and accur-
acy. As expected, the 2DCNN achieves the highest accuracy
3.2. Case study 2: experimental dataset
in all three tasks: 99% for damage detection, 95% for dam-
In this case study, these investigated neural network archi- age localization, and 94.6% for damage severity, following
tectures are applied to an experimentally measured vibration by LSTM, 1DCNN, and MLP. In the figure, the efficiency is
data from a three-story frame structure realized at Los estimated based on training time, required storage and
Alamos National Laboratory (2002) as shown in Figure 13 number of trainable parameters; results are normalized
with detail dimensions. The reasons for selecting this dataset between 0 and 1 for a better comparison, where higher
are its validity, clarity, and particularly a sufficient number value denotes a better efficiency. It is noticed that MLP has
of available data for building deep learning applications. a number of trainable parameters close to those of 1DCNN
The frame consists of Unistrut columns, and aluminum and LSTM, but its architecture is highly suitable for parallel
plates with 1.3 cm thickness and joined together using bolt- calculation then the training time is significantly lower than
to-bracket connections. The base of the frame is supported that of LSTM’s recurrent architecture.
10 H. V. DANG ET AL.

Figure 12. Confusion matrix for damage localization obtained by different DL algorithms.

Table 4. Damage detection results obtained with different Deep learning algorithms.
Damage severity Damage localization
Task
Model MLP 1DCNN LSTM 2DCNN MLP 1DCNN LSTM 2DCNN
Number of epoch 300 300 300 100 300 300 300 100
Total traing time (s) 900 1188 2250 3650 712 807 1918 2100
Testing accuracy 78.8% 87.5% 93.0% 98.9% 83.9% 86.0% 95.0 % 99.2%

Remark 2: In summary, the validity of the present meth- 1 xi

fi ¼ a0 þ a1 : (17)
ods is successfully proved through experimental data from 2xi 2
laboratory testing. Different SDD tasks can be performed by
where xi is the circular frequency of mode i, a0, a1 are two
reassigning respective labels for time-series input, fine-tun-
parameters to be determined. Setting a damping ratio f ¼
ing models’ parameters, without the need of redesigning
0:02 for the first two modes (f1 ¼ 3:36Hz, f2 ¼ 10:31),
entire neural network architectures.
results in the constants a0 and a1 to be 0.31828 and
0.000466, respectively.
3.3. Case study 3: 2D frame subjected to seismic motion The column-beam connection in frame structures is a
critical element which significantly affects structure behav-
The second case study is a two-bay, five-story steel frame
ior. In this work, the connection is modeled as a semi-rigid
structure, as highlighted in Figure 16(a) with the dimension
link with rotational spring, which has been proven to reflect
details and the element numbering (Fragiacomo, Amadio, &
Macorini, 2004). The characteristics of the frame are as fol- the dynamic characteristics of structures accurately
lows: the span length is 6 m, the story height is 3.5 m except (Csebfalvi, 2007). Value of rotational stiffness of the semi-
for the ground story whose height is 4 m. The steel material rigid connection is estimated as follows (Sekulovic & Salatic,
has modulus of elasticity E ¼ 210 GPa, strength fu ¼ 360 Pa, 2001):
mass density q ¼ 7850 kg/m3. The beams are made from  
3EI k
European steel profiles: IPE400 for those of the first three c¼ , (18)
l 1k
floors, IPE360 for the fourth and fifth floors, HEB260 for
outer columns, and HEB320 for middle columns. Damping where c denotes the connection rotational stiffness, l is the
is modelled by using the Rayleigh approach, the damping length of the structure element, k is a fixity factor ranged
ratio for the vibration mode i is calculated as follows (Cruz from 0 to 1, I is the moment of inertia of the element
& Miranda, 2017): cross-section.
 STRUCTURE AND INFRASTRUCTURE ENGINEERING 11

Figure 13. Three-story frame structure experiment (Los Alamos National Laboratory, 2002).

Table 5. Structural states of the three-story frame structure.

State 1 2 3 4 5 6 7 8 9 10
Damage 0 0 0 0 0 1 1 1 1 1
Location 0 0 0 0 0 L1C L1C L1C L1C L1C
Level 0 0 0 0 0 DB0 DBB D05 D10 DHT
State 11 12 13 14 15 16 17 18 19 20
Damage 1 1 1 1 1 1 1 1 1 1
Location L3A L3A L3A L3A L3A L13 L13 L13 L13 L13
Level DB0 DBB D05 D10 DHT DB0 DBB D05 D10 DHT

Figure 14. Acceleration time series measured at the second floor for 11 structural states.

Table 6. Damage detection results on experimental database of a 3-story frame. using vibration signal measured during and shortly after
Task MLP 1DCNN LSTM 2DCNN excitation is an appealing alternative because obtained sig-
of trucks (m/s) loss (%) location nals are rich in information in terms of amplitude, variabil-
Detection accuracy (%) 97 97 99 99 ity, and frequency contents. Since resulting models can
Localization accuracy (%) 87 93 93.5 95
Severity accuracy (%) 85 91 92.6 94.6 depend on specific excitation, e.g., earthquake records, it is
Training time (s) 350 470 930 1250 beneficial to consider different earthquake records, enriching
Number of trainable 389806 342670 326606 540550 the diversity of database and robustness of DL methods.
parameters
Storage (Mb) 1010 1010 1010 1945 Specifically, for this case study, the frame is subjected to
three simulated ground motion with characteristics of the
1940 El Centro earthquake, the 1999 Chi-Chi earthquake,
In fact, it is convenient to train a model able to detect and the 2001 El Salvador earthquake. The ground acceler-
underlying damages from free or ambient vibration meas- ation time histories of these earthquakes are highlighted in
urements because of their accessibility. On the other hand, Figure 16(c). Virtual accelerometer sensors were placed at
12 H. V. DANG ET AL.

the frame connections to capture frame movements during will be advantageous for classifiers, especially those having
ground motions. the capability of exploiting time-varying dependence such as
In general, damages do not take place before earthquake LSTM and 2DCNN. A procedure using only a portion of
events, but during or shortly after. Therefore simulations signals recorded after the occurrence of damage is also feas-
are performed in two phases: The first phase corresponding ible, but it will require corresponding undamaged data to
to the first half 30 s of earthquake records where the struc- identify property shifts, especially when working with differ-
ture is supposedly still intact, then in the second phase, con- ent seismic spectrums triggering different eigenmodes of
nection stiffness reductions of different levels, i.e., 0% the structure.
(healthy), 30%, 60%, and 90%, are arbitrarily introduced to Dynamic analysis is performed with the help of the
two connections (multiple damage scenarios) by reducing mode-based dynamic analysis (Awkar & Lui, 1999) and
rotational stiffness c via Eq. (18), other connections are kept modeling transfer technique in which the outputs of the first
unchanged. The selection of 30 s adopted here comes from stage are considered as inputs of the second stage. More
the observation that most significantly strong part of EQ details about seismic analyses of a frame structure via
(Figure 16) records falls between [0–30 s], specifically the mode-based dynamic analysis could be found in (Dassault,
root mean square (RMS) value of the first 30 s is greater 2016b). In engineering practice, using linear dynamic ana-
than 8 times that of the 30 s second part for all three con- lysis method still a preferable choice compared to nonlinear
sidered EQ records, thereby it is supposed that the second counterpart thanks to its practicality, accuracy of results can
part does not incur further damage. The recorded 60 s- be adjusted further via a safety factor. Tables 7 and 8 enclo-
length acceleration is used as input data to train the deep ses numerical parameters of the finite element model.
learning models further. As changes in connection stiffness The same procedure presented for the first example is
could lead to changes in global stiffness of the structure, applied for this 2D frame to build deep learning models:
using complete signals recorded from before to after damage performing Monte Carlo simulation, involving parameters’
event, could store property shift in a single data. Thus, it variation, realization of extensive finite element analysis,
labelling data by structural states, training deep learning
models for each sensors, combining output of all sensors
into a final classification result and evaluation of the mod-
els’ performance. As steel is a homogeneous material and
usually manufactured with a highly controlled process, then
their properties are more reliable than those of concrete,
which is heterogeneous material and can be manually cast
in situ. Therefore, one excludes the variation of steel proper-
ties from the Monte Carlo simulation in this case study.
The main uncertainty sources for Monte Carlo simulations
Table 7. Numerical input parameters for the 2D steel frame case study.
Time Mesh
Number f1 f20 Duration of step size Element Analysis
of mode (Hz) (Hz) simulation (s) (s) (m) type method
20 3.36 93.43 120 1e-3 0.2 B21 Modal
analysis
Figure 15. Damage detection results in terms of accuracy and efficiency.

Figure 16. a) Schematic representation of the two-bay five-story steel frame subjected to earthquake ground motion. b) 2D model of the steel frame in ABAQUS.
c) Ground accelerations of three investigated earthquake: Elcentro, Elsalvado and Chichi, respectively (Haddadi et al., 2008).
 STRUCTURE AND INFRASTRUCTURE ENGINEERING 13

Table 8. Random parameters for Monte Carlo simulations of the 2D steel frame structure.
Damage location Stiffness
Parameter Earthquake Vertical attribute Horizontal attribute reduction (%)
Distribution Uniform Uniform Uniform uniform
Value [El Salvador,] 2-combination 2-combination [0, 30,60, 90]
ElCentro, Chi-Chi] of [r, c, l] of [1–5]
0: No damage.

Table 9. Label of frame connection.

Connection Label
Connection number 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Horizontal attribute 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5
Vertical attribute l l l l l c c c c c r r r r r

of this example are degrees of stiffness reduction, damage points of different damage scenarios mix together or data
locations, and earthquake records drawn from uniform dis- points belonging to the same damage class but are not prop-
tributions as listed in Table 8. erly grouped, the utilized algorithm needs to be improved.
For this case study, ones investigate the double damage Figure 17 demonstrates the results of the stiffness reduc-
localization tasks. The multiple damage detection is much tion identification for steel frame under earthquake ground
more challenging than single damage detection due to the motion. The rows of the figure correspond to MLP, LSTM,
increasing number of scenarios, especially for complex struc- 1DCNN, and 2DCNN algorithms, respectively. The left side
tures. There are 15 frame connections in total; then, the num- of each row presents the results of detecting the building
ber of scenarios is only 15 for single damage, but rises to 105 storey of damaged connection pairs, including the percent-
cases for double damage, will be 455 cases for triple damage age accuracy and their tSNE representation. The right-hand
scenarios, and so forth. With a given database, when the num- side highlights results related to the vertical attribute. For
ber of classes increases, the training sub-dataset for each class vertical attributes, all four deep learning algorithms show
will be fewer, thus the overall accuracy decreases. To over- high accuracies of more than 97%. The tSNE indicates that
come this limitation, the damage scenarios are divided into obtained results for different groups are well separated, but
different subcategories of a smaller number of classes by MLP is not as good as the three other algorithms in cluster-
incorporating more complementary attributes. ing data, for example, the data of the pair (r,r) are subdi-
For the considered scenario, further information, i.e., the vided into three locations. For horizontal attributes of
story number (horizontal attribute), and the related frame col- damage localization, as the number of output classes
umns (vertical attribute), are integrated into the frame con- increases to 15, the superiority of LSTM and 2DCNN is well
nection information apart from node number (see Table 9). demonstrated. Their average accuracies are 97% and 99%
There are 5 stories; thus, the number of double damage scen- respectively, whereas, for MLP and 1DCNN algorithms,
arios with respect to the horizontal attribute reduces to 15 some pairs of damaged connections obtain under 60%,
cases. The label for each case is denoted by corresponding sto- (such as the pair (2, 5) for MLP and pair (3,4) for 1DCNN).
ries; for example, label (3, 5) indicates damages occurring in The tSNE also highlights this observation. The tSNE of
the beams of the third and fifth stories. Whereas with 3 MLP’s results shows an area where data points of several
groups of columns: left, center, and right column (denoted by classes overlap together at the bottom left corner. In the
l, c, r), there are only 9 possible cases which are annotated by tSNE of 1DCNN, the pair (2, 5) and (3, 4) are not well split.
a pair of numbers, for instance, (l, c) means damage happens In contrast, the tSNE of LSTM displays good clustering
in left and center columns. Then, a two-head deep neural net- results, but there are some data points far apart from the
work is trained, one head for detecting the story of damaged others. The tSNE of 2DCNN has the best clustering image.
connections, another for their vertical attribute. By combining Remark 3: The proposed DL workflow has the ability to
these two results, the damaged connections are accurately diagnose the multi-damage scenarios across the structure with
localized. This multi-head strategy is more amenable in prac- high accuracy. 2DCNN consistently provides the best detection
tice than the direct multi-classification approach when deal- results, followed by LSTM and 1DCNN. In contrast, the per-
ing with a large number of output classes and unchanged formance of MLP algorithm needs to be improved further by
database volumes. modifying the network’s hyperparameters. Theoretically, MLP
To visualize the classification results, t-Distributed architecture can attain an accuracy as high as that of 2DCNN,
Stochastic Neighbor Embedding (t-SNE) technique is but the hyperparameter optimization is another laborious task,
adopted due to its ability of dimensionality reduction in vis- and change radically for different SDD tasks, thus it is more
ualizing high-dimensional datasets (Maaten & Hinton, practical to invoke other DL algorithms.
2008). Since deep learning models will extract multiple pat-
terns from raw data, thus using t-SNE provides a graphical
3.4. Case study 3: 3D bridge under moving load
presentation about how efficient each algorithm in cluster-
ing different damage scenarios. If input signals of the same For the third case study, the DL approaches are applied to a
damage class are well grouped and separated from other full-scale bridge structure. In stayed-cable bridges, cable is
classes, the DL algorithm is effective. In contrast, if data one of the most crucial components (Montoya, Deodatis,
14 H. V. DANG ET AL.

Figure 17. Damage localization accuracy for double damaged scenarios obtained by different Deep Learning algorithms and the corresponding tSNE visualizations.

Betti, & Waisman, 2015), having a great impact on the prestressed girders of 2.6 m depth and I-shaped transverse
dynamic behavior of the structure. If there is a loss in the steel beams spaced at 4.4 m intervals. The structure is sus-
prestressed force in cables, the structural rigidity will be sig- pended by a system of cables composed of 48 stays con-
nificantly reduced. The common method to examine the nected to either side of two outstanding pylons. The pylon
operational condition of cables is measuring vibrations of has the H shape with a height of 150 m, including two legs
each cable, calculating its eigenfrequency values then deriv- whose cross-section dimension varies gradually from
ing the actual prestressed force in cables from empirical for- 12  7.7 m at the base to 5.5  4.7 m at the top, linked by a
mulae. However, this method does not allow for directly transverse beam at the 87 m level. The details of the geometric
assessing the interaction between global dynamic responses dimensions of the bridge are illustrated in Figure 18(b, c). In
of the structure and the cable system. Alternatively, by using terms of cable parameters, the cross-section of each cable
the presented deep learning methods, one attempts to detect is 0.019 m2, and an average value of the effective cable ten-
the location of cable prone to prestressed force reduction sion is selected as 6256 kN after measurements of (Cruz &
via bridge vibration time series, which can be obtained from Miranda, 2017). The material parameters of the bridges are
accelerometer sensors installed across the bridge. as follows: concrete of C45/55 grade with Ec¼44.17 GPa,
The presented methodologies are applied to the Vasco da volumic density of 2400 kg/m3; structural steel S355J2G3
Gama cable-stayed bridge in Lisbon, Portugal (Figure 18(a)), with Es¼210 GPa and volumic density of 7850 kg/m3. For
for assessing tension reductions in cables based on dynamic the structural damping of the Vasco da Gama Bridge, it is
responses of the structure to crossing vehicles. The main assumed that a damping ratio of 0.3% is applied for all
structure of the bridge consists of a central span of 420 m vibration modes.
length, and three lateral spans of 62, 70.7 and 72 m either A finite element model of the bridge is developed to
side, leading to a total length of 829 m (Pedro, Oliveira, & simulate the structural behavior when a vehicle travels
Reis, 2010). The bridge deck is 31 m wide, made by a 25 cm across, (see Figure 19(a)) using Abaqus software. The bridge
thick concrete slab supported by two longitudinal deck is modeled by four noded S4R shell elements from the
 STRUCTURE AND INFRASTRUCTURE ENGINEERING 15

Figure 18. a) Vasco da Gama bridge located in Lisbon, Portugal (Vasco de Gama bridge, 2011) b) Schematic representation of the bridge and measurement points’
location c) Pylon elevation with dimension in place.

Figure 19. a) Finite element model of the Vasco da Gama bridge built in Abaqus and b) a standard vehicular load (HS20 truck) according to American Association
of State Highway and Transportation Officials (AASHTO, 1998).

Table 10. Numerical input parameters for the 3D stayed-cable bridge case study.
Duration of Time step Mesh size Beam Cable Deck Connection Analysis
simulation (s) (s) (m) ele. type ele. type ele. type type method
60 1e4 1.0 B31 T3D2 S4R Zero-length Explicit
connector analysis

Abaqus element library. The pylons of the cable-stayed stressed action. The pre-stressed value per strand is obtained
bridges are modeled using 3D beam elements. The element by dividing the tension force by the strand area. In FEM,
has six degrees of freedom (dofs) at each node, including the tension is modeled as an initial condition, directly
three translations and three rotations around the x, y and z applied on cable elements at the beginning of the analysis
directions. The stay cables are modeled using two-noded using predefined field, mechanical category, where the type
T3D2 truss elements (Olamigoke, 2018) subjected to pre- is selected as stress. The pre-stressed value is the same for
16 H. V. DANG ET AL.

all cables except for the damaged one, whose value is virtu- below 30% tension loss in one stay-cable in detecting either
ally reduced by a prescribed damage level. Table 10 lists damage location or damage severity. Therefore, three levels
numerical input parameters for the 3D FEM of the stayed- of tension reduction investigated are 30%, 60%, and 100%
cable bridge in Abaqus. (total loss of tension). Furthermore, when working with
In terms of vehicular load, a standard HS20 truck, added-noise data, preliminary calculation with low-level ten-
according to the American Association of State Highway sion loss (about 5%) did not yield satisfying results then was
and Transportation Officials (AASHTO, 1998) specifications, not reported in this manuscript. The damage is arbitrarily
is selected in this study. The truck weighs 36 tons, has three introduced to one of 128 cables, at the beginning of the
axles with configuration and loading details highlighted as simulation. In total, 10000 simulations are realized, includ-
in Figure 19(b). Multiple cases of vehicular loads are consid- ing different damage scenarios, and various vehicular load
ered in the calculation, such as one or multiples vehicles cases, as summarized in Table 11. The learning process is
traveling at different velocities. Each truck is modeled by 3 conducted in a supervised fashion in which each series of
concentrated loads of 35, 145 and 145 kN, respectively, mov- dynamic responses of the bridge is labeled by levels of dam-
ing at a constant velocity on the centerline of the bridge age or location of damaged cables. The number of samples
deck. During movement, the distance between these three is uniformly distributed among classes, i.e., approximately
loads (4.3 m and 9.0 m) remains constant. In the Abaqus 80 samples (10000/128) for damage localization, and 2500
software, the subroutine DLoad is a convenient tool to (10000/4) for damage severity.
simulate such moving loads. The velocity of trucks is Figure 20 presents the validation accuracy and training
selected from the range [10, 15, 20, 25, 30] m/s, the number loss retrieved during training processes for two tasks: dam-
of trucks varies from 1 to 5, and the distance between two age severity and damage localization. The accuracy on the
consecutive vehicles is fixed at 100 m. The structural valid dataset is increased as a function of the number of
response is virtually measured at 30 different points (15 epochs, and the training loss decreases at the same time.
points each side) along the deck, as highlighted in Once the valid accuracy becomes stable, and the training
Figure 18(b). process could be terminated. It can be seen that for damage
Once a detailed 3D FEM of the bridge is constructed, severity tasks, 2DCNN outperforms the other algorithms
various damage scenarios are introduced into the FEM to with the converged accuracy of around 94%, followed by
generate a synthetic database of structure response. LSTM, 1DCNN, and MLP with accuracy of 90%, 85%, and
Following the experimental program conducted by 80%, respectively. It is also noteworthy that the learning
Nazarian, Ansari, Zhang, and Taylor (2016), methods using curve of 2DCNN becomes stable after only 20 epochs, while
the bridge deck’s responses were inefficient for damages of it requires more than 200 epochs for other methods. The
similar patterns are observed for damage localization tasks.
Table 11. Random parameters for Monte Carlo simulations of the stayed- Otherwise, the damage localization tasks receive higher
cable bridge structure.
accuracy than the damage severity tasks. A possible explan-
Parameter Number Velocity Tension Damage
of trucks (m/s) loss (%) location ation is that the influence of each cable on the dynamic
Distribution Uniform Uniform Uniform uniform behavior of the structure is not the same, a minor tension
Value [1–5] [10, 15, 20, 25, 30] [0, 30, 60, 90] [0–128] loss of a critical cable (long stay-cables connected to the
0: No damage. middle of the bridge) could cause damage equivalent to a

Figure 20. Evolution of training loss and validation accuracy in function of the number of epochs in the training process.
 STRUCTURE AND INFRASTRUCTURE ENGINEERING 17

Table 12. Stayed-cable bridge’s damage detection results obtained with different Deep learning algorithms.
Damage severity Damage localization
Task
Model MLP 1DCNN LSTM 2DCNN MLP 1DCNN LSTM 2DCNN
Storage (Gb) 9.2 9.2 9.2 18.9 9.2 9.2 9.2 18.9
Time for 1 epoch (s) 5.1 11.1 24.9 354.0 4.7 9.5 10.6 290
Total traing time (s) 2550 5550 12450 17700 2350 4750 5300 14500
Testing accuracy 78.8% 87.5% 93.0% 98.9% 83.9% 86.0% 95.0 % 99.2%
The results are obtained after training with 500 epochs for MLP, 1DCNN, LSTM, and 50 epochs for 2DCNN.

Figure 21. Evolution of prediction accuracy versus the added-noise level for different Noise-to-Signal ratios.

medium tension loss of another cable (short stay-cables external factors such as device instability, environmental
close to the pylon). effects, systematic errors. Herein, we consider white-noise,
Table 12 summarizes the computed results of four Deep which is a classical yet essential problem for time series ana-
learning algorithms for the damage detection of the stayed- lysis (Z. Li et al., 2019), reflecting unknown ambient excita-
cable bridge. Although 2DCNN requires the minimal num- tion, usually addressed in the SHM literature (de Castro,
ber of epochs during training, the training time for each Baptista, & Ciampa, 2019). Moreover, present methods will
epoch (354 s) is significantly higher than that of other algo- automatically learn relevant patterns and mitigate the effect
rithms. Therefore, the total training time of 2DCNN is 1.4, of noise through their hierarchical architecture rather than
3.2, and 6.9 times higher than that of LSTM (17,700 s versus resorting to a per-noise preprocessing step; thus, it is poten-
12,450 s), 1DCNN (5550s), and MLP (2550s). In addition, tially applicable to other types of perturbations. Herein, we
apart from the original time-series signals, 2DCNN needs to consider white-noise where its amplitude is defined using
store associated images of 2D time-frequency representation root mean square (RMS) value of the vibration data and is
of vibration signals, therefore its required storage is twice as represented as follows:
much. This drawback impedes the practicality of 2DCNN.
On the other hand, LSTM, which achieves significantly xinoise ¼ xi þ a:gi , (19)
higher accuracy in a commensurable running time without
additional storage, is promising solution in real-time SHM where xi is the vibration signal retrieved from sensor it,
applications. 1DCNN also bears potential for wider use due xinoise is added-noise data; gi is the white noise vector with
to its advantages as demonstrated in these case studies: zero mean and unit variance, a is the noise amplitude
higher accurate than MLP, very rapid, and no need for depended on the RMS of original data.
extended storage. In the Machine Learning literature, to increase the
robustness of models when dealing with noisy data (Yun,
Yi, & Bahng, 2001) suggested to train models with low-level
3.5. Damage identification with noisy data
added noise dataset instead of virgin datasets, referred as
In the following part, the robustness of the DL is investi- the noise injection learning method. Thereby, the models
gated with respect to time-history sensory data contami- are retrained with training data augmented by noise-to-sig-
nated by noise. Noise can be attributed to various sources: nal ratios (NSR) 3%, 5%, and 7% then tested with data cor-
one rooted from internal structures’ properties, other due to rupted by noise with NSR up to 20%.
18 H. V. DANG ET AL.

lower than that of 0% or 3% SNR noise. A possible explan-

ation for this phenomenon is that once trained with 3%
NSR data, the model will perform well with NSR around
3%, but over-noisy data could deteriorate underlying fea-
tures learned by models and also introduce new fictive pat-
terns. Eventually, patterns of noise will become dominating,
thus significantly offsetting the performance of models
trained with free and low-noise data. Another likely reason
is that one opts the same training process setting for all
NSR levels, i.e., the same number of epochs, learning rate,
batch size, which are optimal for noise-free data but not 3%
SNR case.
Remark 4: 2DCNN consistently provides the highest per-
formance; moreover, 2DCNN and LSTM retain better per-
formance when dealing with noisy data. Though all
Figure 22. Detection accuracy on noisy testing data when adding 3% NSR
noise to training data. investigated methods yield satisfying results with noises of
no more than 10% NSR, there exist, in reality, other types
Figures 21 and 22 illustrate the relationship of damage of perturbation with potentially higher levels caused by sto-
detection accuracy with noisy data for investigated methods. chastic vehicle loading, temperature effect, sensor instability
In Figure 21, the first and second rows correspond to the to address. After all, Deep learning methods directly using
damage severity task and damage localization as described time-series are alternative and also complementary SDD
above. In each subfigure, there are four curves represent approaches, providing more insight into struc-
four training dataset augmented with 0%, 3%, 5%, and 7% tural behaviors.
noise, plotted in blue, red, green and black, the y-axis
denotes damage detection accuracy in percentage, the x-axis 4. Conclusions
indicates testing data corrupted by noise up to 20% NSR.
For better observation, Figure 22, directly compares the test- In this article, four prominent deep learning algorithms that
ing performance of four neural network-based methods achieved state-of-the-art results in a spectrum of applica-
trained with 3% augmented training data on damage sever- tions are extended for structural damage detection. In order
ity tasks. to illustrate steps of realization and compare the performan-
In addition, to estimate the uncertainty of testing output, ces of these algorithms, four case studies subjected to vari-
each calculation is repeated 10 times, i.e., investigated noises ous external excitation were conducted. Based on the
are regenerated and reintegrated into the database, then calculation results, some concluding remarks can be given
respective standard deviations are derived. Apparently, the as follows:
errors, as well as the variation of prediction increases as
NSR becomes more pronounced. Among four DL algo-
In general, building a neural network-based model for
rithms, 2DCNN shows the longest plateau regions up to structural damage detection includes the following main
around 10% noise level, which means the algorithm could steps: first, preparation of a database labeled by respect-
maintain its high performance despite added-noise. It is also ive structural states of concerns, which can be obtained
noteworthy that the decreasing rate in the performance of through extensive measurement or Monte Carlo simula-
LSTM with respect to the noise is relatively low compared tions. Second, a neural network architecture is selected
to others. Thus, it is recommended that 10% SNR is the and adapted based on given databases and SHM tasks.
upper bound of noise for algorithms investigated in this Third, the training and validation are proceeded to
study; otherwise, the accuracy decreases to below 80%. determine optimal parameters of the neural network by
Moreover, the influence of added noise on the algorithms’ using an optimization algorithm (Adam) and gradient
performance in damage severity tasks is more pronounced computation algorithm (Backpropagation). The perform-
than those in damage localization, as the performance ance of the model is then estimated through statistics of
curves of the former decrease more significantly with output results such as true positive, false positive, and F1
increasing noise levels. score, etc.
In terms of injection learning, it is noted that DL algo-
In comparison to other approaches, neural network-
rithms trained on low-level added noise database (Figure based models are alternative and complementary meth-
21), i.e., 3% (red curve) initially achieve higher scores com- ods directly using measured vibrational signals without
pared to noiseless data (green curve), but then rapidly decay requiring an additional step to extract structural charac-
with increasing SNR (larger than 10%). On the other hand, teristics such as modal identification. Moreover, it is flex-
training data with 7% SNR noise (black curve) is not benefi- ible to conduct different damage detection tasks with the
cial for the model performance, even when testing with same neural network architecture but the last output
non-perturbated data, accuracies around 70% are observed; layer to be fine-tuned per task. Once the models are
however, it is still informative as the respective decay rate is trained with appropriate datasets and their parameters
 STRUCTURE AND INFRASTRUCTURE ENGINEERING 19

are stably determined, they could deliver monitoring References

assessment in a near real-time fashion due to the fast
AASHTO. (1998). AASHTO LRFD bridge design specifications. USA:
inference time. American Association of State Highway and Transportation

Through various case studies, it is demonstrated that Officials.
present methods are able to perform not only the SHM Abdeljaber, O., Avci, O., Kiranyaz, M. S., Boashash, B., Sodano, H., &
level 1 task, i.e., damage detection but also SHM level 2, Inman, D. J. (2018). 1-d cnns for structural damage detection:
namely damage localization and damage severity with Verification on a structural health monitoring benchmark data.
Neurocomputing, 275, 1308–1317. doi:10.1016/j.neucom.2017.09.069
high accuracy thanks to the feature learning capability of Avci, O., Abdeljaber, O., Kiranyaz, S., Hussein, M., & Inman, D. J.
the neural networks and a proposed flexible modular (2018). Wireless and real-time structural damage detection: A novel
working flow combining patterns from multiple vibration decentralized method for wireless sensor networks. Journal of Sound
signals. In addition, when dealing with a high number of and Vibration, 424, 158–172. doi:10.1016/j.jsv.2018.03.008
structural states, a procedure dividing original states into Awkar, J. C., & Lui, E. M. (1999). Seismic analysis and response of
multistory semirigid frames. Engineering Structures, 21(5), 425–441.
combinations of sub-categories and fine-tuning the net- doi:10.1016/S0141-0296(97)00210-1
work into multi-head, each head for a sub-category, is Brownlee, J. (2019). What is deep learning? Retrieved from https://
advisable to achieve satisfying results. machinelearningmastery.com/what-is-deep-learning.

When comparing four neural networks investigated in Burkov, A. (2019). The hundred-page machine learning book. Quebec
this work, it is noted that 2DCNN method constantly City, Canada: Andriy Burkov.
de Castro, B. A., Baptista, F. G., & Ciampa, F. (2019). New signal proc-
provides the best identification results, while LSTM and essing approach for structural health monitoring in noisy environ-
1DCNN methods balance the performance and the com- ments based on impedance measurements. Measurement, 137,
plexity, i.e., fast in the training process, easy to modify 155–167. doi:10.1016/j.measurement.2019.01.054
for new structural problems, and does not require stor- Cruz, C., & Miranda, E. (2017). Dynamic tests on large cable-stayed
age for images like 2DCNN. And, MLP algorithm has bridge. Engineering Structures, 138, 324–3366. doi:10.1016/j.eng-
struct.2017.02.001
the most straightforward architecture and low computa- Csebfalvi, A. (2007). Optimal design of frame structures with semi-
tional complexity; therefore, it is suitable to serve as a rigid joints. Periodica Polytechnica Civil Engineering, 51(1), 9–15.
baseline model for any new structures before designing a doi:10.3311/pp.ci.2007-1.02
more sophisticated approach. Dassault, S. (2016a). ABAQUS analysis user’s manual. USA: Dassault

A robustness study with noisy data is carried out, show- Systemes.
Dassault, S. (2016b). ABAQUS example problems manual. USA:
ing that the investigated methods can maintain a suffi- Dassault Systemes.
ciently good performance with data corrupted by noise Dassault, S. (2016c). ABAQUS theory guide. USA: Dassault Systemes.
up to 10% NSR. However, with a higher noise level, the Fragiacomo, M., Amadio, C., & Macorini, L. (2004). Seismic response
performance degraded to some extent. of steel frames under repeated earthquake ground motions.
Engineering Structures, 26(13), 2021–2035. doi:10.1016/j.engstruct.
2004.08.005
Therefore, improving the robustness of SHM neural net- François, C. (2017). Deep learning with Python. NY, USA: Manning
work-based methods with respect to noises of considerably Publications Company.
high levels (> 10% NSR) is necessary for future research. It Galloway, G. S., Catterson, V. M., Fay, T., Robb, A., & Love, C. (2016).
would also be interesting to incorporate probabilistic frame- Diagnosis of tidal turbine vibration data through deep neural net-
works into neural networks, for example, Bayesian theory, works. In Proceedings of the Third European Conference of the
Prognostics and Health Management Society.
in order to empower the performance of algorithms, provid- Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning.
ing not only point estimates but also according confidence Cambridge: MIT press.
interval in response to the unpredictability of noisy input. Haddadi, H., Shakal, A., Stephens, C., Savage, W., Huang, M., Leith,
Another research direction is to fuse various types of data W., et al. (2008). Center for engineering strong-motion data
apart from vibration such as strain measurement, tempera- (CESMD). In Proceedings of the Fourteenth World Conference on
Earthquake Engineering.
ture variation, material properties’ changing pattern, etc. for He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for
detecting more range of defects in the structure. image recognition. In Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory.
Disclosure statement Neural Computation, 9(8), 1735–1780. doi:10.1162/neco.1997.9.8.
1735
The authors declare no potential conflict of interests.
Howard, J., & Gugger, S. (2020). Fastai: A layered API for deep learn-
ing. Information, 11(2), 108. doi:10.3390/info11020108
Ince, T. (2019). Seismic response of steel frames under repeated earth-
Funding quake ground motions. Electrical Engineering, 101(2), 599–608. doi:
This work was supported by the Newton Fund Institutional Links 10.1007/s00202-019-00808-7
through the U.K. Department of Business, Energy, and Industrial Janocha, K., & Czarnecki, W. M. (2017). On loss functions for deep
Strategy and managed by the British Council under Grant 429715093. neural networks in classification. Schedae Informaticae, 25, 49. doi:
10.4467/20838476SI.16.004.6185
Jensen, S. K., Pedersen, T. B., & Thomsen, C. (2017). Time series man-
ORCID agement systems: A survey. IEEE Transactions on Knowledge and
Data Engineering, 29(11), 2581–2600. doi:10.1109/TKDE.2017.
Hung V. Dang http://orcid.org/0000-0003-3384-6427 2740932
20 H. V. DANG ET AL.

Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic opti- Nazarian, E., Ansari, F., Zhang, X., & Taylor, T. (2016). Detection of
mization. arXiv Preprint: 1412.6980. tension loss in cables of cable-stayed bridges by distributed monitor-
Kiranyaz, S., Avci, O., Abdeljaber, O., Ince, T., Gabbouj, M., & Inman, ing of bridge deck strains. Journal of Structural Engineering, 142(6),
D. J. (2019). 1D convolutional neural networks and applications: A 4016018. doi:10.1061/(ASCE)ST.1943-541X.0001463
survey. Preprint arXiv:1905.03554. Noguchi, T., & Nemati, K. M. (1995). Relationship between compres-
Kosmatka, S. H., Kerkhoff, B., & Panarese, W. C. (2019). Design and sive strength and modulus of elasticity of high strength concrete.
control of concrete mixtures. Skokie, IL: Portland Cement Journal of Structural and Construction Engineering (Transactions of
Association. AIJ), 60(474), 1–10. doi:10.3130/aijs.60.1_8
Lee, F. F., Karpathy, A., & Johnson, J. (2018). CS231n: Convolutional Olah, C. (2015). Understanding LSTM networks. Retrieved from
neural networks for visual recognition. Retrieved from http://cs231n.
https://colah.github.io/posts/2015-08-Understanding-LSTMs.
stanford.edu.
Olamigoke, O. (2018). Structural response of cable-stayed bridges to
Lei, J., Liu, C., & Jiang, D. (2019). Fault diagnosis of wind turbine
cable loss (Doctoral dissertation). University of Surrey.
based on long short-term memory networks. Renewable Energy, 133,
Pan, S. J., & Yang, Q. (2010). A survey on transfer learning. IEEE
422–432. doi:10.1016/j.renene.2018.10.031
Li, J., Dackermann, U., Xu, Y. L., & Samali, B. (2011). Damage identifi- Transactions on Knowledge and Data Engineering, 22(10),
cation in civil engineering structures utilizing pca-compressed 1345–1359. doi:10.1109/TKDE.2009.191
residual frequency response functions and neural network ensem- Pedro, J., Oliveira, J., & Reis, A. J. (2010). Nonlinear analysis of com-
bles. Structural Control and Health Monitoring, 18(2), 207–226. doi: posite steel–concrete cable-stayed bridges. Engineering Structures,
10.1002/stc.369 32(9), 2702–2716. doi:10.1016/j.engstruct.2010.04.041
Li, Z., Lam, C., Yao, J., & Yao, Q. (2019). On testing for high-dimen- Sekulovic, M., & Salatic, R. (2001). Nonlinear analysis of frames with
sional white noise. The Annals of Statistics, 47(6), 3382–3412. doi: flexible connections. Computers & Structures, 79(11), 1097–1107.
10.1214/18-AOS1782 doi:10.1016/S0045-7949(01)00004-9
Lin, Y. Z., Nie, Z. H., & Ma, H. W. (2017). Structural damage detec- Tang, Z., Chen, Z., Bao, Y., & Li, H. (2019). Convolutional neural net-
tion with automatic feature-extraction through deep learning. work-based data anomaly detection method using multiple informa-
Computer-Aided Civil and Infrastructure Engineering, 32(12), tion for structural health monitoring. Structural Control and Health
1025–1046. doi:10.1111/mice.12313 Monitoring, 26(1), e2296. doi:10.1002/stc.2296
Liu, H., & Zhang, Y. (2020). Bridge condition rating data modeling Vasco de Gama bridge. (2011). Retrieved from https://arab-trip.com/
using deep learning algorithm. Structure and Infrastructure wp-content/uploads/2019/07.
Engineering, 16(10), 1447–1414. doi:10.1080/15732479.2020.1712610 Wang, Y. R. P., & Gao, R. (2017). Virtualization and deep recognition
Los Alamos National Laboratory. (2002). Retrieved from http://www. for system fault classification. Journal of Manufacturing Systems, 44,
lanl.gov/projects/damage_id/index.htm. 310–316. doi:10.1016/j.jmsy.2017.04.012
Lu, C., Wang, Z., & Zhou, B. (2017). Intelligent fault diagnosis of roll- Yuan, M., Wu, Y., & Lin, L. (2016). Fault diagnosis and remaining use-
ing bearing using hierarchical convolutional network based health
ful life estimation of aero engine using lstm neural network. In
state classification. Advanced Engineering Informatics, 32, 139–151.
IEEE International Conference on Aircraft Utility Systems.
doi:10.1016/j.aei.2017.02.005
Yun, C. B., Yi, J. H., & Bahng, E. Y. (2001). Joint damage assessment
Ma, M., Sun, C., & Chen, X. (2018). Deep coupling autoencoder for
fault diagnosis with multimodal sensory data. IEEE Transactions on of framed structures using a neural networks technique. Engineering
Industrial Informatics, 14(3), 1137–1145. doi:10.1109/TII.2018. Structures, 23(5), 425–435. doi:10.1016/S0141-0296(00)00067-5
2793246 Zhang, Y., Miyamori, Y., Mikami, S., & Saito, T. (2019). Vibration-
Maaten, L. V. D., & Hinton, G. (2008). Visualizing data using t-SNE. based structural state identification by a 1-dimensional convolu-
Journal of Machine Learning Research, 9(11), 2579–2605. tional neural network. Computer-Aided Civil and Infrastructure
Montoya, A., Deodatis, G., Betti, R., & Waisman, H. (2015). Physics- Engineering, 34(9), 822–839. doi:10.1111/mice.12447
based stochastic model to determine the failure load of suspension Zhao, R., Wang, J., Yan, R., & Mao, K. (2016). Machine health moni-
bridge main cables. Journal of Computing in Civil Engineering, toring with LSTM networks. In 10th International Conference on
29(4), B4014002. doi:10.1061/(ASCE)CP.1943-5487.0000393 Sensing Technology.