CRANFIELD UNIVERSITY

LAWRENCE COOK

VISUAL INSPECTION RELIABILITY FOR COMPOSITE

AIRCRAFT STRUCTURES

SCHOOL OF APPLIED SCIENCES

PhD THESIS

Supervisors:

Prof. Phil Irving & Dr. Don Harris

October 2009
 CRANFIELD UNIVERSITY

SCHOOL OF APPLIED SCIENCES

PhD THESIS

LAWRENCE COOK

VISUAL INSPECTION RELIABILITY FOR COMPOSITE

AIRCRAFT STRUCTURES

Supervisors:

Prof. Phil Irving & Dr. Don Harris

October 2009

© Cranfield University 2009. All rights reserved. No part of this publication may be
reproduced without the written permission of the copyright owner.
Abstract
This thesis presents a study of the effects of surface colour, surface finish and dent
shape on the visual inspection reliability of 3D surface indentations common in shape
to those produced by impact damage to carbon fibre reinforced epoxy laminates.

Falling weight (2.5kg) apparatus was used to produce impact damage to non-painted,
non-mesh Hexcel AS4/ 8552 carbon fibre reinforced plastic (CFRP) laminates and
painted AS4/ 8552 laminates containing bronze mesh and glass fabric lightning strike
protection layers. Ø20 mm and Ø87 mm hemispherical tip impacts to painted 17ply
and 33ply laminates at varying energy levels typically produced circular shaped,
smoothly contoured, rounded sectional profiles with an absence of surface breaking
cracks.

Sectional profiles through coordinate measuring (CMM) data of the impact dents
were described using a set of geometric variables. Identifying relationships between
impact energy and the geometric variables allowed the typical sectional profile
through impact damage dents from Ø20 mm and Ø87 mm hemispherical tips on 17ply
and 33ply painted CFRP laminates to be calculated for energies between 5J to 80J.

Calculated sectional profiles typical of impact damage dents to CFRP laminates were
reconstructed as simple revolved shapes using 3D computer aided design (CAD)
models. The 3D CAD models were computer numerical control (CNC) machined into
3mm Plexiglas panels to produce facsimiles of hemispherical impact damage dents on
CFRP laminates.

Facsimile specimen sets of sixteen 600 mm x 600 mm panels were produced in gloss
and matt grey, white and blue finishes. Each set contained the same 32 different sized
machined dents representing Ø20 mm and Ø87 mm hemispherical tip impact damage
to 17ply & 33ply painted CFRP laminate. Each facsimile specimen set was combined
with similarly finished unflawed (dent free) panels. 64 panels in each colour/ finish
were presented for 5 seconds in a randomised order to a minimum of 15 novice
participants in a visual inspection task lasting approximately 25 minutes.

I
A set of corresponding visual inspection experiments were performed in which
physical specimens were replaced by digitally projected actual size photorealistic
images of the machining CAD data. Comparisons between the results of the physical
and virtual specimen trials revealed differences in detectability for similarly sized
dents.

The detection results obtained from visual inspection of physical specimens

demonstrated that the detectability of dents similar to those caused by higher (>40J)
energy impacts from a Ø87 mm hemispherical tip was less than that of the dents
caused by lower energy (<20J) impacts from Ø20 mm tips. However, larger
subsurface delamination area was demonstrated by the higher energy Ø87 mm
impacts than lower energy Ø20 mm impacts on 150 mm x 100 mm coupons of the
same thickness laminate. The results of these experiments imply that detectability of
dents caused by larger diameter objects at higher energies cannot be assumed to be
greater than that of lower energy impacts from smaller diameter objects.

The detection results demonstrate that detectability by visual inspection cannot be

assumed the same for an impact dent on different surface colours and finishes. In
general terms, the highest numbers of dents returning >90% detection were observed
on grey specimens and the highest number of dents returning 0% detection were
observed on matt blue specimens. The difference in detection rates for similarly sized
dents on a gloss and matt finish was least on grey coloured specimens and greatest on
blue coloured specimens.

II
Acknowledgements

The author would like to thank the CAA, and everyone at the CAA involved in kindly
sponsoring and conscientiously supporting the work conducted for this thesis.

The author would like to extend personal thanks to the following people, and to all the
people whom were there to help throughout the work of this thesis:

To Rachel, thank you for your love, understanding and support. Thank you for
always being there, for listening, and keeping it all as simple as “dents in planes”.
To Lewis, thank you for being such a wonderful, happy child and for providing
some of the most rewarding distractions from “dents in planes”. To Dad, thank
you for always being encouraging and for taking such keen interest.

To Phil Irving, thank you for being such a patient and approachable supervisor.
Thank you for being willing to listen, for always being involved, and for always
wanting the best. Throughout my time at Cranfield, I could not have wished for
more support and encouragement.

To Don Harris, it seemed you were always able to give just the right input at just
the right time. Thank you for providing the encouragement to explore blind alleys,
whilst all the time finding time for unique and refreshing humour.

To Ted Blacklay and Hazel Courteney, thank you for your unwavering support
throughout the project. Thank you for always being interested in the work, and for
helping out wherever you could.

Thank you to Simon Waite and Larry Ilcewicz for providing the opportunity to
present this work, and for your kind support in Salt Lake City.

To Jim Hurley & Gary Muir – always a joy to walk in the lab, be brought down to
Earth, and be allowed to learn by doing. To Jim, thank you for being a true friend,
an inspiring mentor, and for reminding me to keep things short, know your facts

III
 and not waste time with discussions. To Gary, thank you for the friendship, the
homely welcome in Bedford, and the fatherly understanding.

To Ben Hopper, thank you for all your help, support and input with the trials, for
helping with no end of tedious but invaluable tasks, and above all for being a great
friend.

To Anne Boulic, thank you for conducting some of the most painstaking work, for
learning so quickly in so little time, and for providing us with such valuable
results.

To Trevor Mead & Russell Martin at M&M Patterns and Roy Harris, thank you
for your patience, experience and skill. Without your willingness and enthusiasm
to take on the unusual, this thesis would not have been possible.

Thank you to all at Cranfield University whom kindly gave up their time to
participate in visual inspection trials. Without your help, it would not have been
possible to gather the results presented in this thesis.

To the staff at Kings Norton Library, thank you for consistently providing the
excellent library services on which much of this thesis has depended.

The help and assistance of the following people and organisations is also gratefully
appreciated:

Terry Wynne, (Aero Consultants); Dennis Murphy (DHL Global Logistics); John
Gamblin, Derek Miller & Pete Wetherill and Cranfield Aerospace; Paddy
Rathbone, Stewart Oates, Sue Richardson, Glenda Metcalf & all the team at
Cranfield Students Association; Charles Marshall, Andrew Mills, Tony Scott,
Barry Walker and Cranfield University; Giovanni Marengo & Paul Saunders
(GKN Aerospace); Gavin Creech (Scott Bader); Lt Cassie Williams & the Repair
Management Team (RAF Cottesmore).

IV
For James Thomas Cook…

… January 2010

V
Page left intentionally blank

VI
Contents

Abstract ......................................................................................................................... I
Acknowledgements....................................................................................................III
Notation .................................................................................................................... XII
Chapter 1 – Introduction............................................................................................ 1
Chapter 2 – Literature search.................................................................................... 3
2.1 – Composite materials ..................................................................................................... 3
2.1.1 – Use of composite materials in modern commercial aircraft ............................................... 5
2.1.2 – Typical composite aircraft structures ................................................................................. 7
2.1.3 – Impact damage & failure of composite materials............................................................... 9
2.1.4 – Characterisation of impact damage to composite materials ............................................. 11
2.1.5 – Factors affecting impact damage morphology ................................................................. 11
2.1.6 – Impact testing of composite materials .............................................................................. 15
2.1.7 – Impact damage scenarios for composite aircraft structures.............................................. 18
2.1.8 – Damage tolerance of composite aircraft structures .......................................................... 19
2.1.9 – Non-Destructive Testing/ Inspection (NDT/ NDI) of composite aircraft structures ........ 22
2.2 – Visual Inspection ........................................................................................................ 26
2.3 – Visual perception and visual inspection ..................................................................... 33
2.3.1 – Shadow & Shading........................................................................................................... 33
2.3.2 – Lighting ............................................................................................................................ 36
2.3.3 – Specular Highlights .......................................................................................................... 38
2.3.4 – Specular Reflections......................................................................................................... 39
2.3.5 – Surface Slant and Tilt ....................................................................................................... 42
2.3.6 – Surface colour & surface finish ........................................................................................ 44
2.4 – Aircraft paint finishes ................................................................................................. 49
2.5 – Reliability assessment of NDI/ NDT techniques........................................................ 50
2.5.1 – Participants ....................................................................................................................... 53
2.5.2 – Experimental setup ........................................................................................................... 54
2.5.3 – Specimen Characteristics ................................................................................................. 55
2.5.4 – Flaws ................................................................................................................................ 56
2.5.5 – Specimen numbers ........................................................................................................... 58
2.5.6 – Other issues pertaining to specimens & procedures ......................................................... 60
2.5.7 – Experimental Variables .................................................................................................... 61
2.5.8 – POD Analysis methods .................................................................................................... 65
2.5.9 – Single variate POD models .............................................................................................. 66
2.5.10 – Multi-variate POD.......................................................................................................... 67
2.6 – Overall summary of literature..................................................................................... 68

VII
Chapter 3 – Experimental Design............................................................................ 70
3.1 – Experimental aims & objectives................................................................................. 70
3.2 – Examples of impact damage to CFRP laminates........................................................ 70
3.3 – Initial visual inspection trials with virtual specimens................................................. 71
3.4 – Reproduction of impact dents as facsimile specimens ............................................... 73
3.5 – Visual inspection trials with physical specimens ....................................................... 75
3.6 – Second series of trials with virtual specimens............................................................ 76
Chapter 4 – Experimental Methods ........................................................................ 78
4.1 – Production of fully finished CFRP laminate samples ................................................ 78
4.2 – Ultrasonic C-Scanning................................................................................................ 79
4.3 - Falling weight impact damage .................................................................................... 79
4.4 – Measurement of delamination .................................................................................... 81
4.5 – Measurements of impact damage surface topography................................................ 81
4.5.1 – Width measurement with Vernier callipers ...................................................................... 81
4.5.2 – Depth measurements with a depth gauge ......................................................................... 82
4.5.3 – Depth gauge profiles ........................................................................................................ 82
4.5.4 – CMM (Coordinate Measuring Machine) coupon surface digitisation .............................. 84
4.5.5 – Width measurements from depth gauge & CMM data..................................................... 84
4.5.6 – Metallographic sectioning technique................................................................................ 86
4.6 – Reconstruction of surface flaws using geometric lines & arcs................................... 86
4.7 – Reproduction of impact flaws as 3D CAD data ......................................................... 88
4.8 – Production of virtual facsimile specimens.................................................................. 89
4.9 – Production of physical facsimile specimens............................................................... 90
4.9.1 – Application of matt finish to physical specimens............................................................. 91
4.9.2 – CMM checking of physical specimens............................................................................. 92
4.9.3 – Gloss measurements of physical specimens ..................................................................... 93
4.9.4 – Paint thickness measurement of physical specimens........................................................ 93
4.10 – Flaw and specimen characteristics ........................................................................... 94
4.10.1 – Specimens for virtual trial 1 (series 1)............................................................................ 94
4.10.2 – Specimens for virtual trial 2 (series 1)............................................................................ 95
4.10.3 – Specimens for virtual trial 3 (series 1)............................................................................ 96
4.10.4 – Specimens for virtual trial 4 (series 1)............................................................................ 97
4.10.5 – Specimens for virtual trial 5 (series 2) and trials with facsimile specimens................... 98
4.11 – Display of virtual specimens to participants........................................................... 105
4.12 – Display of physical facsimile specimens to participants ........................................ 106
4.13 – Participant briefing for visual inspection trials....................................................... 109
4.14 – Experimental task for visual inspection trials participants ..................................... 110
4.15 – Recording participant responses – series 1 virtual specimen trials ........................ 110

VIII
 4.16 – Recording participant responses– physical specimen & series 2 virtual specimen
trials ................................................................................................................................... 110
4.17 – Data collection method for visual inspection trials ................................................ 111
4.18 – Participants – Series 1 trials (with virtual specimens)............................................ 113
4.19 – Participants – Series 2 trials (physical & virtual specimens) ................................. 113
Chapter 5 – Experimental Results......................................................................... 114
5.1 – Impact damage to CFRP laminates .......................................................................... 114
5.1.1 – Basic measurements of impact flaw morphology for fully finished CRFP laminates .... 115
5.1.2 – Relationship between flaw and subsurface damage sizes............................................... 116
5.1.3 – Comparisons between impact damage on painted & unfinished laminates.................... 119
5.1.4 – Characterisation of surface flaws from hemi-spherical objects...................................... 126
5.1.5 – Quantitative measurements of surface flaw topographies .............................................. 128
5.1.6 – Relationships between impact flaw geometry variables................................................. 129
5.2 – CMM checking of physical facsimile specimens ..................................................... 134
5.3 – Gloss measurements of physical facsimile specimens ............................................. 135
5.4 – Results of visual inspection trials with virtual specimens – series 1........................ 136
5.4.1 – Results of trial 1 ............................................................................................................. 136
5.4.2 – Results of trials 2 and 3 .................................................................................................. 138
5.4.3 – Results of trial 4 ............................................................................................................. 143
5.4.5 – Summary of results from series 1 of virtual trials .......................................................... 146
5.5 – Results of visual inspection trials – series 2 ............................................................. 146
5.5.1 – Results data for series 2 visual inspection trials ............................................................. 147
5.5.2 – General Summary of results from series 2 trials ............................................................ 156
5.5.3 – Detailed summaries of results from series 2 trials.......................................................... 156
Chapter 6 – Discussion............................................................................................ 162
6.1 – Experimental issues; impact damage........................................................................ 164
6.1.1 – Flaw width measurements on impact damaged CFRP coupons ..................................... 164
6.2 – Experimental issues – visual inspection trials .......................................................... 165
6.2.1 – Inspection time ............................................................................................................... 165
6.2.2 – Use of novice inspectors in visual inspection trials........................................................ 166
6.2.3 – Lighting .......................................................................................................................... 166
6.2.4 – Design of facsimile specimen flaws ............................................................................... 167
6.2.5 – Size range of physical trial results.................................................................................. 169
6.2.6 – significance of participant responses.............................................................................. 169
6.2.7 – Generic applicability of physical trial results ................................................................. 169
6.2.8 – Existing experimental paradigms and protocols............................................................. 171
6.3 – Visual perception cues and visual inspection of impact damage ............................. 174
6.3.1 – Effects of surface colour & surface finish on modularity of visual perception cues ...... 175
6.3.2 – Effect of surface colour & finish on specular reflection cues......................................... 176

IX
 6.3.3 – Effects of flaw shape on visual perception cues ............................................................. 177
6.2.4 – Role of head movements in visual perception/ inspection ............................................. 179
6.4 – Implications for aviation safety ................................................................................ 180
6.4.1 – Lighting for visual inspection tasks on composite structures ......................................... 180
6.4.2 – Implications for designers of composite material structures .......................................... 181
6.4.3 – Implications for aviation personnel ................................................................................ 183
6.5 – Recommendations to industry & regulatory bodies ................................................. 184
6.5.1 – Damage size assumptions............................................................................................... 184
6.5.2 - Damage reporting............................................................................................................ 185
6.5.3 – Reporting culture............................................................................................................ 185
6.5.4 – Effects of surface colour, finish & lighting .................................................................... 185
6.5.5 – Flaw significance/ perceived severity............................................................................. 186
6.5.6 – Reliability assessment protocols..................................................................................... 186
Chapter 7 – Future Work....................................................................................... 187
7.1 – Expansion of flaw size range.................................................................................... 187
7.2 – Experimental protocols/ guidelines .......................................................................... 187
7.3 – Alternative Visual Inspection Trials Methodology .................................................. 188
7.4 – Paint Patterns ............................................................................................................ 191
7.5 – Curved specimen panels ........................................................................................... 191
7.6 – State of knowledge of pilots, maintenance & ground crew personnel ..................... 192
Chapter 8 – Conclusions ......................................................................................... 193
8.1 – Impact damage to fully finished CFRP laminates .................................................... 193
8.2 – Visual inspection reliability of impact damage dents............................................... 193
8.3 - Effects of surface finish and colour on visual inspection reliability ......................... 194
8.4 – Visual inspection reliability for composite aircraft structures.................................. 195
References ................................................................................................................ 196
Appendix A .............................................................................................................. 208
CAD system rendering settings for virtual specimens ...................................................... 208
Lighting settings......................................................................................................................... 208
Materials settings........................................................................................................................ 209
Appendix B .............................................................................................................. 210
Specimen & Flaw specifications for series 1 visual inspection trials................................ 210
Trial 1 specimen/ flaw characteristics ........................................................................................ 210
Trial 2 Specimen/ flaw characteristics ....................................................................................... 211
Trial 3 specimen/ flaw characteristics ........................................................................................ 213
Trial 4 specimen/ flaw characteristics ........................................................................................ 215
Appendix C .............................................................................................................. 217
Drawings of specimen panels for series 2 visual inspection trials .................................... 217

X
Appendix D .............................................................................................................. 225
Examples of impact damage to AS4/8552 CFRP Coupons............................................... 225
Appendix E .............................................................................................................. 256
Hit/Miss results data from Trial 1 ..................................................................................... 256
Hit/Miss results data from Trial 2 ..................................................................................... 257
Hit/Miss results data from Trial 3 ..................................................................................... 258
Hit/Miss results data from Trial 4 ..................................................................................... 259
Appendix F............................................................................................................... 260
POD curve & RIM calculations ........................................................................................ 260
POD curve calculations for Trial 2............................................................................................. 262
POD curve calculations for Trial 3............................................................................................. 265
POD curve calculations for Trial 4............................................................................................. 266
Appendix G .............................................................................................................. 270
Hit/Miss data for series 2 visual inspection trials.............................................................. 270
Hit/Miss data for trials with virtual grey specimens................................................................... 270
Hit/Miss data for trials with gloss grey specimens..................................................................... 271
Hit/Miss data for trials with matt grey specimens ...................................................................... 272
Hit/Miss data for trials with gloss white specimens ................................................................... 273
Hit/Miss data for trials with matt white specimens .................................................................... 274
Hit/Miss data for trials with gloss blue specimens ..................................................................... 275
Hit/Miss data for trials with matt blue specimens ...................................................................... 276
Appendix H .............................................................................................................. 277
Detection plots of results from series 2 visual inspection trials ........................................ 277

XI
Notation
AC Advisory Circular
AGARD Advisory Group for Aerospace Research and Development
AMM Aircraft Maintenance Manual
BVID Barely Visible Impact Damage
BSS Boeing Support Specification
CAA Civil Aviation Authority (United Kingdom)
CAD Computer Aided Design
CAI Compression after Impact
CAP Civil Aviation Publication (CAA)
CATT Computer Aided Tap Test
CFRP Carbon Fibre Reinforced Plastic/ Polymer
CNC Computer Numerical Control
CMM Coordinate Measuring Machine
DIN Deutsches Institut für Normung
DLL Design Limit Load
DLR Deutsches Zentrum für Luft und Raumfahrt
DUL Design Ultimate Load
DVI Detailed Visual Inspection
EASA European Aviation safety Agency
EN European (Technical) Standard
FAA Federal Aviation Administration (United States of America)
FOD Foreign Object Damage
GFRP Glass Fibre Reinforced Plastic/ Polymer
GLARE Glass Reinforced (Fibre/ Metal Laminate)
GU Gloss Units
GVI General Visual Inspection
IGES International Graphics Exchange Specification
ISO International Organisation for Standardisation
LSP Lightning Strike Protection
MAUS Mobile Automated Ultrasonic Scanner
MIL-HDBK Military Handbook (United States of America)
MLE Maximum Likelihood Estimators

XII
MRO Maintenance Repair and Overhaul
NASA National Aeronautics and Space Administration
NTSB National Transportation Safety Board
PMMA Poly (methyl methacrylate)
POD Probability of Detection
POFA Probability of False Alarm
NC Numerical Control
NDE Non Destructive Evaluation
NDI Non Destructive Inspection
NDT Non Destructive Testing
NVD Non Visible Damage
RFI Resin Film Infusion
RIM Range Interval Method
ROC Receiver Operating Characteristic
RTM Resin Transfer Moulding
SAM Scanning Acoustic Microscopy
SLAM Scanning Laser Microscopy
SRM Structural Repair Manual
UD Unidirectional
US United States (of America)
USAF United states Air Force
US DoD United States Department of Defence
2D Two Dimensional
3D Three Dimensional

Ø Diameter
º Degrees
ºC Degrees Celsius
exp Exponential
g Gram
GPa Gigapascal
In Inches
J Joules
Kg Kilogram

XIII
kN Kilonewton
ksi Kilo-pound-force per square inch
log Logarithm
m Metre
mm Millimetre
µm Micrometre
MHz Megahertz
MPa Megapascal
msi Mega-pound-force per square inch
mph Miles per hour
ln Natural logarithm
s Seconds

XIV
Chapter 1 – Introduction
The aviation industry is familiar with inspection of metallic aircraft structures, and
there is a wealth of knowledge and experience amongst trained personnel regarding
damage manifestation in metallic structures. Maintenance crew and pilots perform
regular checks in order to ensure that aircraft are undamaged. Airport ground
handlers are trained to be aware that aircraft are easily damaged, and are instructed to
report suspected damage, such as a bag dropped onto an aircraft skin, or a bump from
a vehicle. Much of the day-to-day damage to aircraft on the ground, i.e. ramp rash, is
caused by impacts from objects.

Most aviation personnel would be capable of identifying a dent or tear in a metallic

structure, and reporting it as suspected damage. However, after a heavy impact, a
fibre reinforced laminate (composite) structure could appear undamaged if no surface
marks or flaws are present. Where flaws are present, they may go unreported due to
having similar appearance to small dents on a metallic structure (which ground crew
may be inclined not to report, and maintenance crew would not need to repair on a
metallic structure). Whilst aviation personnel may be highly experienced at
recognising impact damage to a metallic structure, there is concern that maintenance
technicians, pilots, and ground crew may be less experienced at recognising impact
damage on a composite aircraft structure. This could lead to a situation where damage
to composite aircraft structures is not reported due to personnel not recognising
damage. Failure to report impact damage, and subsequent non-repair of impact-
damaged composite aircraft structures has far greater airworthiness implications for
primary structures (such as pressurised fuselage skins) than for secondary structures
(such as fairings and non-pressurised skins).

Furthermore, where surface flaws are present due to impact damage, personnel must
firstly be able to see the flaw, i.e. detect it. For metallic aircraft structures, the relation
between probability of detection for cracks and flaws and their size is well established
for many non-destructive testing (NDT) techniques, including visual inspection. The
size at which cracks can be detected with 90% probability is well specified. In
contrast, the reliability of detection of surface flaws in composite structures is not well

1
established; surface flaws resulting from impact damage may not be detected,
meaning significant damage may not be reported.

This study was performed in order to identify the visual appearance of flaws arising
from impact to an aircraft structure, and then to determine the relation between
detection reliability and the size and shape of surface flaws for visual inspection
procedures. The potential influences of surface colour and finish on visual detection
reliability were also investigated.

2
Chapter 2 – Literature search
2.1 – Composite materials
Composite materials constitute a combination of two or more materials that are
chemically distinct [Kalpakjian 1995]. Based on such a description, the oldest
composite materials can be dated back to 4000 B.C. when straw was added to clay for
construction of buildings [Kalpakjian 1995]. Today the term ‘composite’ material
generally refers to fibre-reinforced plastics (FRP). The first application of reinforced
plastics was in 1907, when asbestos fibres were used to reinforce a phenolic resin
[Kalpakjian 1995].

Typical modern fibre reinforced plastics (FRP) may contain glass (GFRP), aramid
(Kevlar) (AFRP), boron, carbon or graphite fibres (CFRP) [Kalpakjian 1995; Bunsell
& Renard 2005]. Other materials such as nylon, silicon carbide, silicon nitride, steel,
tungsten, molybdenum and polyethylene (Spectra) can be used as fibre
reinforcements [Kalpakjian 1995; Bunsell & Renard 2005]. Reinforcing fibres have
high tensile strength and high elastic modulus, as described in table 2.01. Carbon
fibres are less dense (1800 Kg/m³) and less costly to produce than boron, making
them a common choice for aerospace structures where higher specific strength and
specific module are desired [Kalpakjian 1995; Campbell 2004]. The reinforcing fibres
themselves are brittle and their strength is effective only in the fibre direction.

Strain to
Tensile Strength Elastic Modulus Density
Fibre Type Failure
(ksi/ MPa) (msi / GPa) (Kg/m³)
(%)
Boron 520 / 3600 11 / 380 2600 0.9
Carbon (AS4) 530 / 3700 33 / 228 1800 1.5
Carbon (IM-7) 730 / 5000 41 / 283 1770 1.8
Kevlar (aramid) 49 550 / 3800 19 / 131 1440 2.8
E-Glass 500 / 3400 11 / 76 2490 4.8
S-Glass 650 / 4500 12.6 / 87 2540 5.6
Spectra 1000 450 / 3100 25 / 172 970 0.7
Table 2.01 – Fibre properties, from Campbell 2004, pp 6

3
The plastic or resin matrix can be produced from a range of materials, although
polyester, polyamide, polypropylene, phenolic and epoxy are the most commonly
used [Bunsell & Renard 2005]. The service temperature resistance of epoxies is
150ºC, their flexural strength can be between 80-140 MPa, and their flexural modulus
is typically 3 GPa [Bunsell & Renard 2005]. However, in order to allow safety
margins, continuous service temperatures of epoxy composite aircraft structures are
limited to 100°C [US DoD 2002, MIL-HDBK-17 & Rouchon 2010]. Thermoplastic
resins can have higher toughness, and thus greater resistance to impact damage, than
thermoset resins, but their temperature resistance can be as low as 100ºC [Kalpakjian
1995]. Polyester resins account for 53% of total matrix material sales, although
advanced (i.e. aerospace) applications favour the low shrinkage, high tensile strength
(90 MPa vs. 35~80 MPa), higher temperature resistance (150ºC vs. 85~125ºC) and
superior chemical resistance offered by epoxy matrices [Kalpakjian 1995; Bunsell &
Renard 2005].

Despite earlier suggestions of the potential for RTM and RFI processes in aerospace
applications [Halpin 1994], the autoclave curing process, with manual lay-up of pre-
preg material continues to be a predominant choice for complex structures in US (and
other) aerospace applications [dti 2006].

A high strength carbon fibre reinforced epoxy laminate offers superior specific
strength (strength/ density) and specific modulus (strength/ modulus) to traditional
materials such as titanium, steel or aluminium, as seen in figure 2.01, hence the
popularity of CFRP materials within the aircraft industry.

However, a unidirectional laminate will only have excellent tensile strength and
compression strength in the fibre, or ply, direction, i.e. the laminate will exhibit
excellent in-plane load performance, but poor out-of-plane performance. To reduce
the anisotropy of a fibre-reinforced laminate, and provide strength in multiple
directions, the fibres are generally laid up in varying configurations of 0º, 90º and +/-
45º orientations [see Matthews & Rawlings 1994; Campbell 2004; Bunsell & Renard
2005; Rouchon 2010].

4
 Figure 2.01 – Material strength & modulus, from Campbell 2004, pp 27

2.1.1 – Use of composite materials in modern commercial aircraft

Including other uses such as automotive, cycling, boating and consumer products
[Campbell 2004; Cuffaro 2006], composite materials have seen increasing use in
aviation applications, for both military and commercial aircraft [Kalpakjian 1995].
The 1970s saw fairing panel, spoiler and pylon skin components produced from
composite materials enter service on commercial aircraft [Harris & Shuart 2001]. Use
of composite materials has grown through the 1970s, 80s & 90s, from structures on
the aircraft that had little structural importance (secondary structures), to primary
structures such as horizontal & vertical stabilisers and ailerons [Harris & Shuart
2001]. An excellent review of the earlier period of the development of composite
materials in aerospace applications is provided by Harris & Shuart, 2001, and a more
recent review is given by Rouchon, 2009.

1980~1990 – The Boeing 757 aircraft entered service in 1983 utilising composite
materials for elevator, aileron, spoiler and rudder components, although the structural
components of the fuselage, wing and stabilisers remained of metallic construction
[Boeing 757-200 2009, Kalpakjian 1995]. In 1985, the Airbus A310 aircraft became
the first commercial airliner to utilise composite materials for construction of the
vertical stabiliser and rudder structures (primary structures) [Harris & Shuart 2001;

5
Rouchon 2009]. Airbus ventured further with composites in 1988 with the A320
family aircraft, employing composite materials for control surfaces, vertical and
horizontal stabilisers (empennage) [Airbus 2009, Robois 2005; Rouchon 2009]. Much
of the composite structures used during this period are of sandwich type construction
(see US DoD MIL-HDBK-17 Vol 1 2002, Campbell 2004 for details on sandwich
construction), although glass and aramid fibres began to be replaced by carbon or
graphite fibres [Kalpakjian 1995, Robois 2005, Rouchon 2010]. The period leading up
to the 1990’s also saw the introduction of monolithic fibre reinforced composite
materials for horizontal and vertical stabiliser structures [Rouchon 2010].

1990~2000 – The Boeing 777 entered commercial service in 1995 containing 9% of

composite materials by weight [Boeing 777 2009; Kalpakjian 1995]. From the same
era, the Airbus A330/A340 family also contain around 10~11% composites by weight
[Robois 2005]. The 777 and A330/A340 aircraft represent the highest utilisation of
carbon-fibre reinforced composite materials on wide-body aircraft currently in
widespread service. The 777 and A330/A340 aircraft are similar in several respects,
and utilise carbon fibre reinforced monolithic and sandwich structures for primary and
secondary structures including floor beams, floor panels, fairings, keel beams,
spoilers, control surfaces, horizontal stabilisers and vertical stabilisers [Kalpajian
1995, Robois 2005; Brosius 2007]. However, the fuselage of both aircraft are
constructed from a traditional metal frame with a metal skin. Furthermore, whist
empennage structures on the 777 are produced from composite materials, their
construction is termed as having a ‘black metal’ approach i.e. the composite panels
are attached to composite stringers and ribs as opposed to the use of unitary
construction methods [Webborn 2006]. Smaller business type aircraft such as the
Beech (Raytheon) Premier and Horizon have employed greater use of unitary
composite structures, albeit sandwich type, for their fuselages [Mills 2006].

2000 ~ – The Airbus A380 entered service in 2007, and contains approximately 25%
composites by weight, which is currently the highest ‘in-service’ level of composite
materials usage on large commercial aircraft. Sections of the Airbus A380 fuselage
are produced from GLARE, a metal/ composite sandwich material [Robois 2005].
Many of the primary structures including the empennage and control surfaces are also
produced from carbon and glass fibre reinforced composite materials [Robois 2005].

6
However, the A380 fleet is operated by only a small number of airlines, and cannot be
considered to be in ‘widespread’ service.

Currently, no commercial aircraft in airline service have monolithic carbon fibre

reinforced plastic (CFRP) fuselage structures. The next generation of wide-body
commercial aircraft will use monolithic carbon fibre reinforced composite laminate
structures for the fuselage skins, centre wing boxes and wing skins. The Boeing 787
utilises composite materials in the construction of its fuselage, wings and horizontal &
vertical stabilisers [Brosius 2007; Boeing July & November 2008]. Airbus has also
designed the A350, a new wide-body aircraft with composite materials used for
fuselage and wing construction. However, the main difference between the A350 and
the 787 is that the A350 employs a ‘black metal’ approach, utilising the traditional
method of fastening fuselage skin panels to a CFRP structural frame [Airbus A350
‘Fuselage Production’ 2009], whereas the 787 fuselage is constructed from three one-
piece barrel sections that are joined together to construct the whole fuselage [Boeing
2007]. The barrel sections of the 787 are produced by placing fibres, pre-impregnated
with Toray 3900 series epoxy resin, onto a rotating mandrel before autoclave curing
[Boeing 2005; dti 2006; Brosius 2007]. Filament winding and fibre placement onto a
rotating mandrel are commonly used methods for the production of cylindrical tubes
and pressure vessels [Bunsell & Renard 2005]. Entry into service for the 787 is
expected in 2010 [Boeing 787 2009] and the A350 is expected to enter service by
2013 [Airbus January 2009], thus large commercial aircraft containing high
percentages of composite structures, including fuselage skins, will soon be
commonplace.

2.1.2 – Typical composite aircraft structures

Both the Boeing 777 and Airbus A330/A340 aircraft use monolithic carbon fibre
reinforced epoxy laminates in the construction of their horizontal stabiliser skins
[Airbus AMM 2002; Airbus SRM 2003; Ishibashi 2006; dti 2006; Brosius 2007]. The
Boeing 777 is understood to use 180ºC cure pre-preg materials containing Toray’s
T800 [Gros et al. 1998; Toray 2001; Ishibashi 2006; Brosius 2007 ] carbon fibres, and
the Airbus A340 is understood to use 180ºC cure Hexcel AS4/8552 [Hexcel AS4]
carbon fibre pre-preg material [see Airbus SRM 2002]. Hexcel AS4/8552 also

7
features heavily on the Boeing 787, although in uses such as window frames and
engine nacelles [Brosius 2007].

The horizontal stabiliser (see figures 2.02 & 2.03) is open to the elements and
subjected to high-speed airflow during flight. The skin structures are painted in order
to protect the composite laminate from ultra-violet degradation, weathering and
erosion during flight. Composite aircraft structures also need to be protected against
the damaging effect of lightning strike [US DoD MIL-HDBK-17 2002], which can be
achieved in several ways, including the use of metal meshes and metal tows within
fibre lay-ups [Gardiner 2006; Welch 2007].

Figure 2.02 – Location of Horizontal

Figure 2.03 – Detail of Horizontal Stabiliser
Stabiliser
Courtesy of Airbus
Courtesy of Airbus

The horizontal stabiliser of the Airbus A340 aircraft is an excellent example of such a
structure [Airbus AMM 2002, Airbus SRM 2003]. A typical cross section through a
horizontal stabiliser skin (figure 2.04), such as that on the A340, would consist of a
carbon fibre reinforced epoxy laminate (CFRP), with the outside facing surfaces
protected against lightning strike by a layer of bronze mesh. Unidirectional carbon
fibre/ epoxy pre-preg tape is laid in variations of 0º, 90º & 45º orientations, with
thickness varying between 17 and 33 plies.

Between the outermost (outside facing) ply of the CFRP and the bronze mesh, a thin
layer of glass fibre fabric is employed to facilitate handling and production operations
relating to the bronze mesh [Rouchon 2010]. The outer surface of the skin would
normally be painted with primer coats and topcoat.

8
 Figure 2.04 – Typical advanced composite aircraft skin structure

When viewing the painted exterior surfaces of an aircraft, one will have few (if any)
visual cues as to what materials were used to construct the various structures.

2.1.3 – Impact damage & failure of composite materials

Aircraft structures may be subjected to excessive loading, environmental damage, and
impact damage threats. Such occurrences may lead to structural degradation, which
can in turn reduce structural performance [US DoD MIL-HDBK-17 2002].

CFRP laminates are susceptible to matrix cracking and delamination failure due to
impact events [Matthews & Rawlings 1994, Campbell 2004]. Dissipation of impact
energy into the laminate causes matrix cracking. During impact events, the laminate
flexes under the force of the impacting object, causing inter-laminar shear forces,
which cause ply separation or delamination [Bunnel & Renard 2005]. Failure of the
matrix material close to the surface leaves the reinforcing fibres unsupported, which
renders them unable to support the transverse force imparted by the impact object,
resulting in fibre breakage. Failure and crushing of the upper surface or plies results in
a dent like appearance [Shyr & Pan 2003; Aktas et al. 2008]. As seen in figure 2.05
and in specimens presented by Lance & Nettles 1990, Shyr & Pan 2003 and Aktas
2008, impact damage from an object to a CFRP material will manifest itself in the
form of subsurface delamination and a three-dimensional (3D) surface indentation
[see also Abdallah et al. 2009]. The subsurface delamination and matrix cracking is
not visible from the painted side of the laminate. Only the surface flaw is visible on
the painted side.

9
 Figure 2.05 – Cross section through 30J, Ø20 mm impact to a bronze mesh
incorporated, painted, 17ply AS4/8552 CFRP laminate [from Boulic 2007]

The impact indentations can be relatively small, compared to the delamination size
beneath, and may exhibit little visual evidence of damage on the impacted surface
[Campbell 2004; Cartié & Irving 2003; Brookes 2004, Shyr & Pan 2003; Aktas et al.
2008]. Delamination widths of between 0 and 60 mm have been observed from Ø4
mm to Ø20 mm indentations (i.e. delamination 3 times greater than indentation width)
on a 7 mm laminate [Brookes 2004].

Abrate 1998 and Shyr & Pan 2003 demonstrate that for thick laminates, the
propagation path of the cracking in a laminate takes the form of a triangular shape,
with the upwards point at the centre of the impact site, as seen in figure 2.05. For thin
laminates, the path is inverted [Abrate 1998; Shyr et al. 2003]. Examples published by
Aktas illustrate that the planar shape of the delamination area depends on the fibre
orientations of the plies, that the delamination is not always circular, and that the
delamination extends furthest in the 0º fibre direction [Aktas 2008]. Damaged matrix
material is unable support the reinforcing fibres, which subsequently buckle and fail
when subjected to compressive loads in the fibre direction. Delamination and matrix
cracking due to impact damage reduces the compressive strength of composite
materials [Dorey 1989; Papanicolaou & Stavropoulos 1995; Mitrovic et al. 1999;
Cartié & Irving 2002; Brookes 2004]. 40 mm of delamination in a 7 mm thick
laminate is sufficient to cause a 50% reduction in compression after impact (CAI)
strength [Brookes 2004].

10
2.1.4 – Characterisation of impact damage to composite materials
Much of the available literature on impact behaviour of composite materials refers to
impact damage in terms of its delamination size. Part of the reason is likely to be the
correlation between delamination size and CAI strength. Some authors such as Shyr
& Pan 2003 and Mitrevski et al. 2006 have presented sectional photographs of impact
damage, which provide only qualitative information. Where surface dent morphology
is presented quantitatively, such as by Mitrevski et al., authors tend to have used dent
depth as the sole metric for describing surface damage, and dent width measurements
such as those given by Brookes 2004 are rare. However, recently published literature
and conferences have begun to question the use of a single metric such as dent depth
to describe impact damage to composite materials [CMH-17 2009, Rouchon 2009].

When observing figure 2.05, one can clearly see that the traced line of the surface
indentation, or impact dent, has a complex curved shape. The dent has not only width
and depth, but also a sectional profile, which requires further information in order to
be fully described. To this effect, Aktas et al. 2008 appear to be the first to have
published a realistic diagrammatic sectional representation of impact damage, as seen
in figure 2.06, although no quantitative delamination or dent sizes are given in their
results.

Figure 2.06 - Typical cross sectional profile of impact damage to

composite laminate, From Aktas et al. 2008

2.1.5 – Factors affecting impact damage morphology

Many factors are reported to affect the impact damage extent of composite laminates.
However, more relevant to this thesis are factors likely to affect the shape, or
morphology of the impact dent.

11
Impact force – Instrumented impact testing of a 32ply [(45/0-45/90)4]s carbon fibre
laminate by Hu et al. 2008 demonstrates predictions and demonstrations of typical
force/ time curves during 7.2J impact events. The first laminate appears to fail, or
crack, at between 4kN and 5kN. Subsequent laminate, or shear failures continue to
occur up to forces of around 6kN, after which flexural failure of the complete
specimen occurs. Impacts below a given energy threshold are unable to generate
sufficient force in order to initiate first ply cracking and cause detectable damage, or
dents, to the laminate. Results from Cartié & Irving, 2004 demonstrate that the energy
level required to achieve the threshold force can vary from 5J to 10J depending on the
laminate properties. A model is given by Davies & Zhang, whereby the square of the
force required to produce matrix cracking in the first ply is “proportional to the cube
of the laminate thickness” [Davies & Zhang 1995, in Shyr & Pan 2003]. Figure 2.07,
illustrates typical damage threshold load/ laminate thickness plots for three resin/ fibre
systems, in which it can be seen that higher impact forces are required in order to
reach the threshold force at which damage initiation begins.

Figure 2.07 – Damage threshold vs. thickness, from Schoeppner & Abrate 2000

Impact energy – it is generally accepted that higher impact energies produce larger
damage under the same conditions [see Robinson & Davies 1992; Papanicolaou and
Stavropoulos 1995; Hounslow 2000; Cartié & Irving 2002]. Given that impact energy

12
is given by ½mv², impactor mass and velocity can affect damage size. Additionally,
high velocity impacts do not allow sufficient time for the laminate to flex before
generating high impact forces capable of breaking and crushing the plies, thus high
velocity impacts are more likely to result in penetration than surface dents [Matthews
& Rawlings 1994]. However, variations of impactor mass or velocity at the same
impact energy have been demonstrated to have no effect on damage size for low
velocity impacts [Robinson & Davies 1992]. Davies et al. [1994] demonstrated
delamination sizes of 500~900 mm² for 10J impacts and 700~1300 mm² for 30J
impacts (see figure 2.08). Studies by Brookes 2004 and Cartié & Irving 2004 have
also demonstrated greater delamination at higher impact energies. Dent depth is also
affected by impact energy [Brookes 2004; US DoD MIL-HDBK-17 Vol. 1 2002], and
Brookes demonstrated dent depths ranging from below 1 mm to greater than 6 mm
across and energy range of 0~60J, with variations in dent depth for the same impact
energy depending on the impactor size and the laminate thickness (see figure 2.09).

Figure 2.08 – Impact Energy/ Delamination Area plot from Davies et al. 1994

13
 Figure 2.09 – Impact Energy/ Dent Depth plot from Brookes 2004

Laminate thickness – it is accepted that thin and thick laminates consisting of the
same matrix & reinforcement materials have different responses to impact damage
[US DoD MIL-HDBK-17 2002]. Results published in MIL-HDBK-17 demonstrate
that the same impact produces delamination three times larger on a 16ply laminate
than on a 10ply laminate. A long span, thin laminate (2 mm) flexes under impact load,
and is susceptible to flexural and penetration failure. A short span, thick laminate is
more susceptible to delamination failure [Robinson & Davies 1992; Matthews &
Rawlings 1994]. Aerospace applications generally use thick (15, 30 or 60 ply)
laminates [Airbus SRM 2002], and are thus more prone to delamination failures
[Matthews & Rawlings 1994; Campbell 2004]. Results from Brookes 2004 show Ø20
mm impacts of over 100J producing dents around 2.5 mm deep on a 7 mm laminate,
whereas less than 20J were required for the Ø20 mm impactor to produce the same
dent depth on a 1.89 mm laminate. It can therefore be said that thick laminates will
exhibit less indentation for the same impact conditions, although the significant
delamination damage (that is not visible to the human eye) may exist beneath the
indented surface.

Ply orientations – altering the ply orientations within a laminate can alter dent depth
for the same impact conditions. Using a target dent depth of 1 mm, Lopes et al. 2009
demonstrated that for a [+/-45/90/0/45/04/-45/02]s (baseline) laminate, the impact
energy required is 24.4J versus 26.4J for [+/-45/0/70/-70/0/15/10/-10/-15/15/-15]s and

14
[+/-45/80/5/20/-20/10/-80/-10/-5/15-15]s (alternative) laminates. At 40J, the impact
indentation was reported as 2.6 mm for the baseline laminate and 3.0~3.2 mm for the
alternative laminates. Lopes et al. report that for a given energy, the delamination
sizes for the alternative laminate, containing 70º-orientated fibres, are smaller than the
baseline (45º orientations) laminate, although the delamination sizes for the 80º
orientation laminate were larger than baseline.

Impactor shape – work by Brookes 2004 and Mitrevski et al. demonstrated that
impactor size and shape could alter the size of impact damage. Mitrevski et al.
October 2006 studied the delamination induced by differently shaped impactors and
the results show that a hemispherical object produced greater (10~15%) delamination
than a conical object at the same energy. However, a later study by Mitrevski et al.
November 2006 demonstrated that that a conical shape produces impact dents
between five and eight times deeper than a hemispherical shape at the same energy.
The work of Mitrevski et al. demonstrates that a sharp, pointed object is likely to
produce a deeper indentation than a blunt shaped object at the same impact energy.

2.1.6 – Impact testing of composite materials

Mechanical testing of composite materials, especially for impact damage and
compression after impact testing is a well-researched topic. It is apparent from recent
conferences on composite materials that academic researchers and manufacturers are
attempting to model the impact damage behaviour, crack propagation behaviour and
compression after impact behaviour of advanced composite materials [CMH 17
2009]. However, when reading literature that concerns either mechanical testing,
design or repair of composite materials, other fundamental issues become apparent.

Specimen size – Robinson & Davies 1992 demonstrated that for 6J impact energy,
the impact dent produced on a Ø102 mm specimen was around 5 mm larger (wider)
than on a Ø337 mm specimen of the same laminate. Although demonstrations of
impact testing on structural elements, such as those of Davies et al. 1994, are
available, published examples of impact damage available to an academic researcher
are generally limited to small-scale test specimens [see Cartié & Irving 2002, Shyr &
Pan 2003, Brookes 2004, Mitrevski et al. 2006, Aktas et al. 2008 and Hu et al. 2008].
Whilst such specimens provide basic information regarding impact damage behaviour

15
& characteristics for composite materials, the results from Robinson & Davies 1992
make it clear that the damage characteristics reported in much of the available
literature are only applicable to small size test specimens.

Furthermore, an issue raised by Ilcewicz in 2009 is that publically available, detailed

information on mechanical testing of composite materials is generally limited to
testing of coupons or structural elements. Figure 2.10 illustrates how the level of
information regarding testing of composite materials can be considered as a pyramid.
The lower level, performed on small coupons, spans a wide range of published
material, and is widely available. Mechanical testing performed on larger structural
elements and complete assemblies such as the wing box tests run by NASA in 1996 &
2000 [Raju 2006] is less frequently performed, and although (sometimes) available on
request to regulatory bodies such as the FAA, EASA or CAA, the results remain
proprietary information and are unavailable in the public domain.

Figure 2.10 – Testing levels for composite materials,

from Rouchon 1990, in US DoD MIL-HDBK-17 Vol 1 2002

Laminate & surface finishes – published works such as those of Robinson & Davies
1992, Davies et al., Sohn et al. 2000, Cartié & Irving 2002, Shyr & Pan 2003,
Brookes 2004 and Mitrevski et al. 2006 and Aktas et al. 2008 all conducted tests on

16
laminates that were prepared without lightning strike protection or paint finishes. The
top layers of such laminates will be visible to the eye, and matrix or fibre cracking (as
seen in the work of Robisnon & Davies, Mitrevski or Aktas) will be clearly visible.
Such laminates are not representative of those that an aircraft inspector is likely to
encounter on an actual aircraft structure, and differences in appearance of impact
damage on finished laminates has already been reported [Gantt 2003]. Furthermore,
the phosphor bronze threads (5.5~7.0 Sn, 0.01~0.4 P, Rest Copper to ISO 9044 [Bopp
2006]) used for lightning strike protection are more ductile than carbon fibres
laminates. Glass fibres have a strain to failure value of between 4.8% & 5.6% (see
table 01), and phosphor bronze can exhibit up to 60% elongation over 50 mm under
strain [Kalpakjian 1995 pp. 184, Table 6.6]. The lightning strike protection materials
are thus more capable of deforming or stretching under impact load than carbon
fibres, which have lower strain to failure values of between 1.5% and 1.8% (see table
01). The higher ductility of the materials used for lightning strike protection means
that they are capable of deforming under impact as opposed to cracking, which may
result in a different surface appearance to a typical impact on a non-finished laminate.

Impactor geometry – specifications for impact test specimens, particularly for

compression after impact tests, require the use of hemispherical shaped impactors
[Boeing BSS7260 1988, DIN EN 6038 1996]. However, the work of Mitrevski et al.
provides a clear indication of the fact that impactor shape can alter the dent
morphology of impact damage to composite materials. Aside from Mitrevski et al. Jul
& Aug 2006, it is apparent that published material on the impact behaviour of
composite materials is limited to impacts from hemispherical objects between Ø10
mm and Ø25 mm in size. Work currently in progress by Kim [see Kim, 2007] and
EASA/ DLR [Waite, 2009] aims to move towards using larger impactors such as the
Ø320 mm example reported by DLR [Waite, 2009]. The study by Kim aims to
demonstrate the damage behaviour full scale composite aircraft structures subjected to
vehicle damage, which are represented low velocity (0.447m/s / 1mph), high mass
(500kg), blunt edge (127 mm radii) impacts. The results of the work by Kim and
DLR, when published, will provide greater understanding of composite material
impact-damage behaviour, under realistic damage scenarios on large, full-scale
specimens. Given that larger sized impact objects are reported to produce indentations
that differ to smaller sized impacts, with subsequent increased likelihood of greater

17
delamination with reduced indentation size (i.e. little visual evidence of significant
damage), work for this thesis should consider impacts that represent the largest
recommended object sizes for impact testing of laminates, as opposed to the smallest.

2.1.7 – Impact damage scenarios for composite aircraft structures

Damage to aircraft can occur from a variety of sources. Impact damage is a common
occurrence with aircraft structures, and around 80% of damage to composite aircraft
structures is caused by impact strike [CAA 2005]. Aircraft can encounter impact
damage from a variety of sources, as identified by Kolesnikov and Herbeck and
illustrated in figure 2.11 [Kolesnikov & Herbeck 2005].

Figure 2.11 – Impact damage scenarios for aircraft in service

(based on information from Kolesnikov & Herbeck 2005)

Bird strike, hail damage and foreign object debris (FOD) damage are everyday
occurrences in airline operations. Bird strikes to aircraft are often noticed by the flight
crew, and reported to maintenance personnel for inspection at a suitable time
[Psymouli et al. 2005]. Evidence of a bird strike often presents itself in the form of
bird remains. Hailstorms are known events, and once again, inspectors can be notified
that an aircraft has been subjected to a hailstorm and may have been damaged
[Psymouli 2005]. FOD damage, such as flying stones and tyre shreds, generally
occurs in specific areas, such as around undercarriages and engine exhausts. Aircraft
inspectors are known to pay particular attention to these areas [Psymouli et al. 2005].
Impacts events such as hail, bird strike and FOD are all expected to happen. Impact

18
damage from ground equipment, whilst the possibility of it occurring is
acknowledged, is not expected to happen, although it is widely known within the
airline industry (as experienced personally by the author) that ground equipment and
aircraft are strangely attracted to one another.

To explain the likelihood of damage from ground equipment and aircraft, one only
has to view figure 2.12, in which the ground equipment arrangements for servicing of
a 787 aircraft are shown. Aircraft are often surrounded by ground handling
equipment, all of which in close proximity to the aircraft, and having the potential to
collide with the aircraft and cause impact damage.

Figure 2.12 - Boeing 787 – Airplane servicing arrangement during typical

turnaround [Boeing 787 Airplane Characteristics, September 2007]

2.1.8 – Damage tolerance of composite aircraft structures

Damage detectability is split into four categories: Undetectable, Detectable, Readily
Detectable and Immediately Obvious, as described in figure 2.13 [US DoD MIL-
HDBK-17 Vol. 3 2002]. The size of damage that falls just between undetectable and
detectable is known as ‘barely visible’. Barely visible impact damage (BVID) is an
important concept in the design of composite aircraft structures, as the BVID size is

19
used to determine design ultimate load (DUL) capability [see US DoD MIL-HDBK-
17 2002, Transport Canada AC 500-009]. As previously described, damage to the
matrix material will reduce the compressive load performance of a fibre-reinforced
laminate. For damage below the detection threshold or BVID size, the structures must
demonstrate capability to sustain design ultimate loads in both tension and
compression “without failure or excessive structural deformation” [EASA 2008] for
an indefinite period or until the structure is replaced due to cyclic or time limits [as
per EASA CS-25, FAA AC-20-107A & Transport Canada AC 500-009].

Figure 2.13 – Damage detection levels, from US DoD MIL-HDBK-17 2002

With regard to published BVID sizes, US regulatory materials such as MIL-HDBK-

17 state an energy value of 140J as highest level of impact damage that is likely to
occur to a composite aircraft structure before the damage mode progresses from above
the barely visible threshold (2.54.mm deep) into readily detectable (RDD) damage.
[US DoD MIL-HDBK-17 2002; Rouchon 2009]. Data presented in MIL-HDBK-17
(see figure 2.13a) demonstrate that higher energy impacts are required to BVID
damage on thicker laminates, and the 140J limit is intended to represent the level of
energy required to produce BVID dents on the thickest of laminates. However, the
impact energy at which the transition from BVID to RDD actually occurs will depend

20
on the impact conditions and the laminate thickness and type [see chapter 2.1.5 &
MIL-HDBK-17 3F, 7.5.1.1, 7-38].

Figure 2.13a – test results demonstrating the higher energies required to produce the
same dent depth, from MIL-HDBK-17 3F, 7-38 [US DoD 2002].

European material is generally limited to using a dent depth metric for defining BVID
[see Rouchon 2009]. Published values for BVID vary from 2.5 mm deep (USAF),
1.25 mm deep (US Navy), 0.3 mm deep (Aerospatiale) [Forsyth et al. 1998] and 0.254
mm [Boeing 2006, in Rouchon 2009]. In order to set the size of damage tolerance size
for a structure, one is required to demonstrate that the chosen damage size can be
reliably detected by the chosen inspection procedure [EASA CS-25 2008; US DoD
MIL-HDBK-17 2002; FAA AC-200-107A, AC 500-009].

If cyclic loading of the structure causes damage growth, inspection intervals must be
set such that damage growth beyond BVID size, and thus past DUL tolerance limits,
can be found before design limit load is reached with possible subsequent structural
failure [US DoD MIL-HDBK-17 Vol.3 2002; EASA CS-25; FAA AC-20-107A;
Transport Canada AC 500-009].

The design philosophy for composite aircraft structures is that they should
demonstrate ‘no-growth’ for BVID damage up to DUL [MIL-HDBK-17 Vol. 3, FAA
AC-20-107A, EASA CS-25]. Experiments by Krutop 2007 demonstrated that for a

21
15J impact, a threefold increase in delamination area occurs after 15000 cycles at 75%
CAI strength for a 32ply CFRP laminate. However, delamination is not visible during
visual inspection. Consider that a 15J impact produces a dent below BVID size. If the
visual inspection reliability is based on surface dent visibility, and the dent does not
grow along with delamination, the dent visibility will remain below detectability
limits whilst the delamination may have grown beyond structural tolerance limits. To
compound the visual detectability issue, composite materials are known to exhibit
surface relaxation after impact – i.e. dent depth reduces by up to 45% with time and
cyclic loading [Komorowski et al. 1993, Donckels et al. 2000]. Furthermore,
increased humidity and subsequent absorption of moisture can lead to dent surface
relaxation [Rouchon 2010]. Hence, other inspection methods capable of
characterising sub-surface damage are available.

2.1.9 – Non-Destructive Testing/ Inspection (NDT/ NDI) of composite

aircraft structures
Several methods of non-destructive testing or inspection techniques are available for
detection of damage in composite materials, and a detailed review is given by Harris,
B. 2003.

Human visual inspection is widely used for inspection of composite aircraft

structures, and there are different levels of visual inspection. A general visual
inspection (GVI) is performed within a distance of 1.2m [CAA CAP562 2005]. GVI
inspections are non-directed inspections, in which the inspector will inspect all areas
of the aircraft structure for signs of damage. A detailed visual inspection (DVI) is
performed within 0.3m [CAA CAP562 2005]. The purpose of a DVI is to search a
specific area on a structure for signs of damage. There also exists a special detailed
inspection (SDI), which may be a visual inspection, in which inspectors are directed
to inspect a specific structure for a specific fault or a specific type of damage.
Published figures for the capability of visual inspection claim 90% probability of
detection (POD), with 95% confidence for dents between 0.3 mm and 0.5 mm deep
[Aerospatiale 1991, in Rouchon 2009]. Aerospatiale results cited by Forsythe et al.
1998 claimed 50% POD at 0.3 mm dent depth under ‘close’ visual inspection, i.e.
within 0.3m. However, the details of the experiments performed in order to arrive at
such values remain unpublished.

22
Non-destructive inspection (NDI) of composites can also be performed by several
other mechanical methods, including tap-testing, computer aided tap testing (CATT),
ultrasonic testing (C-scan), shearography, thermography and scanning acoustic
microscopy (SAM). The probabilities of detection for each method were investigated
by Roach in 2007, and the results are presented in figure 2.14.

Figure 2.14 – NDT/NDI capabilities, from Roach 2007

NDT methods were also investigated by Gros et al. in 1998, and the results are
presented in Figure 2.15. Gros et al. choose to describe minimum size detectable in
terms of the impact energy that could be detected, as opposed to a description of the
delamination size. Gros et al. did present a plot of surface flaw size (length, mm) vs.
impact energy, allowing the minimum detectable impact energy and minimum
detectable surface flaw size to be shown together in figure 2.15. The results of Gros
et al. demonstrate that visual inspection is highly efficient at detecting small sizes of
impact-damage surface flaws. However, it is important to remember that the other
methods would be capable of detecting delamination damage regardless of whether a
surface flaw is indentation, whereas visual inspection is reliant on the presence of a
surface indentation.

23
 Figure 2.15 – NDT capabilities, from Gros et al. 1998
Tap testing can be used to identify areas of delamination in monolithic structures &
areas of disbonded laminate in sandwich structures, is a well-known technique and
can be performed using either specialist equipment or a simple round coin. Damaged
laminate emits a lower frequency sound than undamaged laminate when tapped
[Campbell 2004]. Inspectors are said to be accurate (within ½ inch) at estimating the
size of delamination or disbonds using tap testing [RAF 2006]. The capability of tap-
testing was demonstrated by Roach in 2007, and based on flaw width, the 50%
probability of detection (POD) size was approximately Ø36 mm and the 90% POD
size was approximately Ø70 mm. Computer aided tap testing, or CATT, is a
development of tap testing, which uses a computer to analyse the frequency of sound
produced by the tap, and has been demonstrated to 50% POD for Ø30.5 mm wide
flaws and 90% POD for Ø61 mm flaws [Roach 2007]. Other acoustic testing
procedures, such as Scanning Laser Acoustic Microscopy (SLAM) and Scanning
Acoustic Microscopy (SAM) use ultrasonic waves to detect delamination and matrix
cracking [see Gao & Kim 1999]. SAM has been demonstrated to 50% at a flaw size of
Ø31 mm and although the system was unable to demonstrate above 80% POD (Ø3in /
76 mm) [Roach 2007].

Ultrasonic (C-Scan) scanning of composite components, using 0.5~50MHz probes is

a widely employed means of characterising delamination damage to composite
materials, and can determine delamination size to within 0.4~0.7 mm [see Harris, B.
2003]. Traditional C-scan techniques require a liquid coupling between the probe and

24
the structure being examined. Water immersion is impractical for structures that are
inconvenient to remove from an aircraft, an issue that can be been mitigated by using
gel coupled scanning probes. Mobile ultrasonic equipment such as the Boeing
MAUS-V system [Boeing MAUS-V 2009] have been developed, and Roach 2007
demonstrated a capability of 50% POD at a dent width of Ø12.7 mm and 90% POD
at Ø33 mm. Recent air-coupled C-Scan equipment designs have been developed,
which eliminates the need for a liquid coupling, giving greater freedom of operation
[Imielinska et al. 2004].

Double pass retroflection, or D-Sight, is an optical damage detection system that has
been demonstrated as being capable of detecting impact damage between 0.152 mm
and 0.256 mm deep on composite materials with between 85% and 100% POD
[Forsyth et al. 1998]. The system detects surface flaws as opposed to sub-surface
delamination, and requires a specular (glossy) surface, meaning that matt paint
finishes must be wetted before inspection can be performed [see Forsyth et al. 1998].
Along similar lines, i.e. using analysis of the visual appearance of the laminate to
identify flaws, double-pass retroflection, moiré inferometry and laser shearography
systems are also capable of visualising damage to composite structures [see Harris, B.
2003]. Roach 2007 demonstrated shearography to have 50% POD at a flaw width of
Ø13 mm and 90% POD at a flaw width of Ø51 mm.

Theromography or vibrothermography [see Harris 2003] identifies damaged laminate

by detection of changes in heat emission from the laminate being tested.
Thermography was demonstrated to 50% POD at Ø6.35 mm and Ø90% POD at
Ø16.51 mm [Roach 2007].

Whilst techniques such as tap test, C-Scan, D-Sight, moiré inferometry, thermography
and vibrometry are all available for aircraft inspection, and undoubtedly highly
effective, they all require the use of specialist equipment, which must be set up and
operated by qualified personnel. The aforementioned techniques are all useful for
characterising damage, but to use them to scan an entire aircraft in order to detect
damage would be time consuming due to the relatively small working area offered by
these techniques. To reduce aircraft downtime, the use of NDI/NDT may be
rationalised to inspections of structures suspected of being damaged, or to inspection

25
of structures as directed by maintenance manuals, service bulletins or airworthiness
directives. Visual inspection can be performed relatively quickly over large areas,
thus lending itself to scanning larger sections of aircraft structure for suspected areas
of damage. The results of Gros et al. show that visual inspection is also capable of
detecting small (4 mm/ 4J) impact damage flaws, although minimum the size that can
ever be detected, and the size of damage that can be reliably detected are not
necessarily one and the same. Once damage is suspected, the other available
NDT/NDI techniques can then be implemented. Visual inspection acts as guidance for
effective and time-efficient deployment of more detailed NDT/ NDI procedures.

2.2 – Visual Inspection

Visual inspection accounts for between 80% and 99% of aircraft inspection activities
[CAA CAP716 2003], and is the first line of damage detection for composite
materials [Armstrong et al. 2005; Psymouli et al. 2005].

Instructions for performing a visual inspection are generally found in proprietary

training documents, such as an aircraft maintenance manual [see Airbus A340 SRM;
Airbus A340 AMM], or within regulatory guidance material [see CAA CAP716
2003]. The CAA, in Appendix O of CAP 716, cite the FAA’s description of visual
inspection as being “the process of using the eye, alone in conjunction with various
aids, as the sensing mechanism from which judgements may be made about the
condition of a unit to be inspected” [CAA CAP716 Appendix O p1.].

The statement “using the eye alone” is of key importance. The only ‘equipment’ used
in visual inspection is the inspector’s eyes. Reliability of visual inspection cannot be
considered in the same manner as other NDT techniques, such as tap testing, C-Scan,
Thermography or Inferometry, which require the use of probes and display screens in
order to make the flaws visible to the inspector. The effects of procedural deviations
such as probe lift off, or failure to use probe guides, as demonstrated by Murgatroyd
et al. 1993 are not applicable factors in visual inspection.

Experiments by Spencer, 1996a & Spencer, 1996b indicated that the visual search
method of the inspector, i.e. how the inspector’s eye scans for defects, varies from
inspector to inspector. The results of Spencer’s study also imply that experience in an

26
aviation inspection environment and expectation of where to find defects are
positively correlated to visual inspection performance. However, Spencer does also
report that inspectors using their expectancy to search specific areas tended to look
only in the area defects were expected, rather than searching, or scanning the entire
structure.

Following on from the work of Spencer in 1996, Asada et al. 1998 published the
findings of a study into in-service visual inspection of metallic structures. Asada et al.
also found that directed inspections, i.e. inspecting with expectancy to find flaws,
resulted in 75% of cracks shorter than 1 inch (25.4 mm) being found compared to less
than 50% of cracks less than 1 inch in length being found by undirected inspections.
Asasda et al. state that for around 90% of the 1054 cracks detected, the inspectors had
prior information concerning the crack characteristics, thus highlighting how visual
inspection reliability can be improved by inspectors knowing what to look for. The
probability-of-detection (POD) curves presented by Asada et al. also indicate that
visual inspection reliability is affected by:
• Surface condition – dirty surfaces resulted in lower POD than clean surfaces
• Inspection distance – inspection from greater than 0.5m gave lower POD than
inspection from within 0.5m
• Surface finish – for cracks between 0.5 inch (13 mm) and 3.5 inch (90 mm),
the POD is lower on unfinished structures than on primer coated or topcoat
finished structures

Whilst the results of Spencer and Asada et al. provide valuable information on factors
affecting the visual inspection process, (Spencer’s experiment also includes inspection
of tie clips on wiring looms), the studies were limited to inspection on ‘metallic’
aircraft. The results of studies of visual inspection performed on sections of composite
material aircraft structures have been published [Erhart et al. 2004; Gant 2007].

Erhart et al. 2004 performed a visual inspection experiment using sections of

composite structure provided by an airline from an in-service aircraft. The results
from Erhart et al. identified that flaws below 0.025 inches (0.64 mm) deep were
visible to less than 80% of participants. For flaws below 0.025inches (0.64 mm) deep,

27
detections were increased for the specimen illuminated by a lamp, compared to the
specimen illuminated only by ambient daylight. However, for flaw sizes greater than
0.035inch (0.89 mm) the effect of illumination on detection performance was less
marked. The results show differences of up to 57% between detections on a top
surface and a bottom surface. No differences in detection rates were reported for clean
or dirty areas. The results of Erhart et al. demonstrate that detection of flaws can be
affected by the available lighting, which may depend on the presence of illumination
devices or the position of the inspection surface with regard to the available light.
Gant, 2007 performed an assessment of visual inspection using GFRP/ foam sandwich
structures. The specimens represented wing sections of a Cirrus SR22 aircraft. The
specimens were impacted with a hemispherical tipped Ø25.4 mm steel tube, in
common with aforementioned impact testing procedures. The damage ranged from
20J to 30J, resulting in flaws between 0.18 mm & 7.54 mm deep and 232 mm² &
6087 mm² in planar area. The results published by Gant show that greater numbers of
detections were obtained from filler painted surfaces than top-coat painted surfaces.
This was likely due to surface breaking cracks in the filler paint at the impact site
providing greater visual cues as to the presence of impact damage. Similar to the
Erhart study, Gant’s results also show more flaws detected on upper wing skins than
on lower wing skins, again demonstrating a negative effect on detection rates due to
inspection in an under-wing or under-belly scenario. The results obtained by Gant
using the upper wing specimens, for all surface finishes are plotted below in figures
2.16 and 2.17:

Detection rate vs. flaw depth

100%

80%
Detection rate

60%

40%
20%

0%
0 1 2 3 4 5 6 7 8

Flaw depth (mm)

Figure 2.16 – Plot of detection rate vs. flaw depths from Gant 2007

28
 Detection rate vs. flaw area

100%

80%

Detection rate
60%

40%

20%

0%
0

50

10

15

20

25

30

35

40

45

50

55

60

65
0

00

00

00

00

00

00

00

00

00

00

00

00
Flaw area (mm²)

Figure 2.17 – Plot of detection rate vs. flaw areas from Gant 2007

It is important to note that the data presented in figures 2.16 and 2.17 were obtained
by Gant from visual inspection of impact damage from several 20J & 30J impacts on
laminates with differing ply orientations, stacking sequences, and surface finishes.
However, despite the variance in flaw/ specimen characteristics, increases in detection
rates accompany increases in flaw depths, with detection rates increasing from ~20%
up to 100% for flaw depths below 3 mm. The relationship between flaw area and
detection rates is less well defined, and there is considerable variation in the detection
rates obtained for flaws of areal size between 1000 mm² and 2500 mm². It is also
evident that the flaw with the widest areal size (~6000 mm²) gave lower detection
rates than flaws with lower areal sizes (3000 mm²~4500 mm²). It is thus clear from
the results of Gant that whilst detection rates may increase with increasing flaw depth,
the relationship between detection rates and areal size, and hence flaw width, is less
distinct.

The works of Erhart et al. and Gant were rudimentary investigations into visual
inspection of composite materials. Many of the recommendations concerning
specimen numbers, inspection process control and protocols for NDT reliability
assessment experiments, which will be presented later, were not implemented by
either author. Erhart et al. allowed participants to touch the panels during inspection,
meaning that the inspection was no longer a ‘visual’ inspection and had in fact
become a ‘tactile’ inspection. The study by Gant used very limited flaw numbers, and
a small range of flaw sizes. Furthermore, the flaws were spread across various
different laminates and finishes, making it impossible to compare detectability of

29
similarly sized flaws on different finishes and laminates. However, between them,
Erhart et al. and Gant demonstrated that:
• poor luminance levels had a negative effect on detectability of impact damage
on composite materials
• some inspectors found a flashlight beneficial whilst others did not
• viewing angle affected detectability
• surface finish affected visual inspection reliability
• impact damage dent detectability increased with increasing dent depth
• detectability did not necessarily increase for larger values of dent width (or
dent area)
In 1979, Megaw published a comprehensive list of factors liable to affect the
reliability, or accuracy of the visual inspection process, and listed published
experiments that had been performed to demonstrate them [Megaw, 1979]. Megaw
placed the factors into 4 categories – subject, physical & environmental, task, and
organisational factors. From Megaw’s paper, one concludes that a large proportion of
the research into visual inspection prior to 1979 was concerned with actually
‘detecting’ flaws or defects. This continues after 1979 with publications on eye
movements [Megaw & Richardson 1979], scanning/ search strategies [Megaw &
Richardson 1979], task complexity [Gallwey & Drury, 1986] and lighting [Cayless
1983].

Megaw & Richardson 1979 found that there was little correlation between task
complexity and inspection time for visual inspection. Eye fixations were found to be
shorter for tasks involving inspection of objects that had no defined fixation points,
such as large sheets of material. Megaw & Richardson also proposed that peripheral
vision is an important aspect of visual inspection of large sheet surfaces. Gallwey &
Drury, 1986 found that adding more ‘types’ of fault, or flaw, to a visual inspection
task reduced visual search performance. The results obtained from faults placed in set
regions and results obtained from faults placed indiscriminately all over the inspection
object revealed little effect on visual inspection performance due to flaw location.

Coupled with NTSB findings in 1989 that human factors contributed to a mid-air
decompression of an airliner [NTSB/AAR-89/03], aircraft inspection research efforts

30
of the 1990’s shifts towards research on factors that affect the ‘human’ performing the
inspection. In 1989, Drury described the visual inspection task for airframe
inspection, splitting it into 7 ‘tasks’; Initiate, Access, Search, Decision Making,
Respond, Repair and Buy-back Inspect [Drury 1989]. Drury classes the act of actually
‘searching’ the unit, or structure, for defects as the third task within the sequence of
events [Drury 1989]. Although he was not the first to break down the inspection task
in such a way [see Colquhoun 1964, in Megaw 1979], published works on visual
inspection prior to Drury’s generic task description of visual inspection appear to
have concentrated mainly on the visual ‘search’.

Research on aircraft inspection reliability conducted after Drury’s generic task

description in 1989 begins to study the whole process, from being directed to, and
gaining access to the task, through to factors affecting judgement on whether a
defective structure requires repair or further NDT/ NDI to better determine the
structure’s condition (as per CAP 716 Appendix O p1.). Drury and Lock studied
aircraft inspection reliability in both the United States and the United Kingdom, and
studied not only the inspection procedures themselves, but also how the inspectors
were trained, inspection personnel/ supervisory relationships and inspection
environments [Drury & Lock 1995]. Other examples are works by Murgatroyd et al.
and Lock, which, following on from the work of Megaw in 1979 describe how the
performance of aircraft inspectors can be reduced by a variety of factors, including
poor accessibility, lack of procedural compliance [Murgatroyd et al. 1995], noise and
lighting [Lock 1998]. Training and prior knowledge/ experience were identified as
factors that affect an inspector’s visual search strategy during visual inspection [see
Drury 1993], and training was shown to improve visual inspection search strategy by
Gramopadhye et al., 1997. In the same study by Gramopadhye, training and feedback
were also shown to be beneficial to the ‘decision’ task that follows the ‘search’ task
[Gramopadhye et al. 1997]. However, the person performing a visual inspection and
the person performing further, more detailed NDT/ NDI or repair may not be one and
the same [Drury & Lock 1995]. In such instances, the visual inspector may need to
request further NDT/NDI or repair by more qualified personnel. It is proven that
inspectors can be trained to recognise defects [Gramopadhye et al. 1997], and
inspectors may well call out all suspected defects for further inspection or repair.

31
It is important to remember at this point, that if the visual ‘search’ fails to find defects,
because the inspector fails to detect them, the visual inspection sequence cannot
progress to a ‘decision’. Inspectors cannot be encouraged to recognise, or call out a
defect as requiring further NDT or repair if they cannot actually see it in the first
place. This helps further explain the difference between works on aircraft inspection
published prior to Megaw’s paper and those published after Drury’s 1989 paper. After
the 1970s, authors of visual inspection literature switch from assuming that all cracks
can be made visible and recommending improvements that will make cracks more
visible, to acknowledging that not all cracks will be detected, and recommending how
the number of missed cracks can be reduced. This observation is also made by
Hagemaier 1998 & Georgiou 2006 & Georgiou 2007, whom both acknowledge that
early experiments (pre 1970) had previously asked the question “how small a crack
can be detected?” and later experiments (post 1970) ask “how large a crack can be
missed?” [Hagemaier 1998; Georgiou 2006 & Georgiou 2007] (Reliability
assessments are discussed separately in this chapter).

The distinction between detection and decision is of paramount to this thesis. Signal
detection theory (SDT) [see Wickens 2000] is commonly associated with aviation
inspection reliability. Many of the publications referred to in previous paragraphs
concern factors affecting inspector bias towards calling out flaw ‘signals’ as defects
(hit), rejecting detected flaw ‘signals’ as not being defects or simply failing to call out
detected ‘signals’ (miss) or incorrectly calling out the presence of ‘signals’ or defects
when no flaw is actually present (false call). However, signal detection theory relates
to the ‘decision’ step of inspection, i.e. the next step after the eye actually detects a
‘signal’. SDT is particularly relevant to flaws similar in size to surrounding noise
items not requiring call outs. In visual inspection for impact damage dents, the
decision outcome depends on the inspector’s ability to distinguish detected signals as
being from a dent, rather than something similar in appearance such as a paint chip or
dirt. Signal detection theory is used to determine whether inspectors are simply
calling out every detected signal, whether from dents, paint chips or dirt, as defects, or
judging very few of the ‘signals’ to actually be dents, and only calling out some
signals as dents. For this thesis, the ability of observers to detect dent signals, not their
decision on whether or not the signal comes from a dent, is under investigation;
therefore, signal detection theory shall not be pursued further.

32
A fundamental issue is apparent in all but the most recent of studies on visual
inspection [Psymouli et al. 2005, Erhart et al. 2004; Gant 2007]. Within literature
concerning visual inspection in aviation circles, it is apparent that the word ‘defect’ or
‘flaw’ was synonymous with the word ‘crack’, likely due to most of the published
work having been performed during an era when the aircraft in service were
constructed predominantly from metallic structures. This thesis is concerned with the
visual inspection of composite aircraft structures, which, as previously discussed, do
not necessarily exhibit surface breaking cracks when impacted. Impact damage to a
composite material is three-dimensional in shape, whereas cracks in metallic
structures are effectively two-dimensional. Hence, much of the work on actually
detecting ‘defects’ or ‘flaws’ in aerospace structures using visual inspection is
relevant only to the inspection for cracks in metallic structures. The detection of 3D
surface flaws is influenced by far more complex visual perception issues.

2.3 – Visual perception and visual inspection

When observing an object, as in visual inspection, the human eye sees nothing more
than patterns of light [Todd 2004]. The patterns of light, or depth cues, produced by
3D objects or surfaces are projected onto the retina within the human eye [Gibson
1950a]. Three-dimensional information about an object is reconstructed from a two-
dimensional retinal image [Gibson 1950a; Tsutsui et al. 2005]. Theories on visual
perception, and how the human brain develops a three-dimensional representation
from two-dimensional images, began to emerge in the 19th century (see Gordon 1989
for a review of significant theories), and research on visual perception continues into
the 21st century [Sakata et al. 2003; Todd 2004; Tsutsui et a.l 2005; Wexler & van
Boxtel 2005; Kingdom 2008]. Gibson, 1950a & Gibson, 1979 discusses depth cues in
great detail, and it is also convenient at this point to quote Kingdom, who advises that
patterns of light, or depth cues, can be created by areas of shadow and shading, light
sources, reflections of light sources (highlights) or reflections of the surrounding
environment (specular reflections) [Kingdom, 2008].

2.3.1 – Shadow & Shading

Before continuing, shading must first be distinguished from shadow: “A shadow
results from occlusion of a light source, while shading…results from a change in

33
angle of the surface normal with respect to the direction of illumination” [Kingdom
2008]. Both, however, are dependent on the lighting conditions.

Figure 2.18 – Shading cues Figure 2.19 – Shading and shadow cues
Figure 2.18 illustrates a sphere depicted by shading alone. A point made clear by
Gibson [Gibson 1950a] is that the observer of figure 2.18 will be aware that the shape
is spherical, but the absence of shadow could lead to a perception of either a convex
or a concave spherical shape. Figure 2.19, on the other hand, depicts a shape that
exhibits shadow both on its own surface and on the base on which it sits (lower right
corner). In both images, the shape provides a circular occlusion, leading us to assume
that the shape is spherical [see Gibson 1950a]. In figure 2.19, the sphere blocks out
the light passing over it, resulting in a darkened side, and casts a shadow across the
ground. Without the shadow cue, one could not be sure that the shape in the image is
definitely a sphere sitting on a surface. Both figure 2.18 and 2.19 are of the same
scene, the only difference being that in figure 2.18 the lighting is directed directly
downwards from the centre of the page, and in figure 2.19 the lighting is directed
from 45º above, and 45º to the left side of the object. The effect of lighting direction
on shading and shadow cues is thus shown.

Gibson [Gibson 1950a pp. 91-100] points out that in conjunction with contour and
silhouette information, the shading on an object’s surface makes it appear “object-
like”. In the same section, Gibson provides a useful explanation of gradients of
shading by stating that where two shaded regions are observed, “If the transition from
light to shade is gradual the shape is a curve; if the transition is sudden, the shape
will be a corner” [Gibson 1950a pp96]. Gradients in shading caused by shadow may
also be referred to as ‘penumbra’; “the partially shaded outer region of the shadow

34
cast by an opaque object”, whilst “the fully shaded inner region of a shadow cast by
an opaque object” may be referred to as ‘umbra’ [Oxford Dictionary 2005].

From Gibson’s work, it is clear that shading, penumbra and shadows (umbra) provide
both depth information about a scene or object, and provides the eye with information
about the actual shape of an object. This is represented to good effect in figure 2.20,
which illustrates a set of shapes that exhibit different shading gradients, from which a
perception of their shape can be made. The gentle change from light to dark across the
sphere is distinguishable as a large diameter sphere, compared to the changes in
shading observed along the smaller radius edge running around the top of the
hexagon, which take place within a shorter distance.

Figure 2.20 – Shading (penumbra) and shadow

(umbra) cues from differently shaped objects.
Experiments have shown that humans can extract depth information, and perceive
depth solely from shading cues [Erens et al 1993; Norman et al. 1995, 2004 & 2006].
However, it is not easy to do so [Erens et al. 1993], and experiments have shown that
there is variation in different observer’s judgments of depth when presented with
shading cues alone [Norman et al. 2004]. From the texts available, it is clear that
whilst shading is a powerful cue to 3D perception, it can only provide limited 3D
information.

The shading of an object will change as the direction of the lighting on the object
changes [Todd et al 1996]. The human perception of shading can be influenced by
lighting direction [Erens et al. 1993], as experimentally determined by Todd et al. in

35
1996. Thus, an aircraft inspector may perceive a structure differently when it is
illuminated from different lighting directions.

2.3.2 – Lighting
Megaw 1979 cites lighting (an environmental factor) as a key factor in visual
inspection reliability. Megaw reviewed research on lighting for visual inspection
purposes and identified that previous studies tended to concentrate on lighting levels
and that whilst works such as Dekoker & Frier 1969 had investigated the qualitative
aspects of lighting; further research was required on lighting quality [Megaw 1979].

Reading Dekoker & Frier 1969, Faulkner & Murphy 1973, De Boer 1977 and Cayless
& Marsden 1983, one can conclude that the optimum design for visual inspection
lighting is task dependent. However, whilst some of the studies investigate the
inspection of three-dimensional surface flaws, such as sheet metal flaws [Dekoker &
Frier 1969], defects in woven cloths or scratches in plastic sheets [Faulkner &
Murphy 1973] none of the works considers the inspection of 3D dents on a surface.

In his 1998 paper, Lloyd refers to another work [Lloyd & He 1998] in which it was
determined that defect topography of dents of flaws in painted automobile class-A
surfaces can be characterised quite simply by their “specular highlight area and
modulation”. Drury in 2000 helps define Lloyd’s ‘modulation’ as the contrast
between the defect and the background surface [Drury 2000, p50]. Lloyd found that
lighting effects that would enhance the visibility of specular highlights and increase
defect-background contrast are generally considered as ‘glare’ in other types of visual
inspection task, but are actually highly beneficial to visual inspection of surface
defects on painted surfaces [Lloyd 1998]. Although he does not give any examples of
how to achieve the desired effect, Lloyd recommends that for visual inspection of
painted surfaces, the “illuminance levels should be minimised” and the lighting
environment should be “rich in light source edges” [Lloyd 1998]. Given that the task
studied by Lloyd is similar in nature to visual inspection of an aircraft structure,
particularly fuselage or wing skins, it follows that Lloyd’s recommendations are also
valid for inspection of aircraft structures.

36
The issue of specular highlights and lighting in visual inspection receives further
interest in 2002 and again 2006. Brombach et al. 2006 found that whilst specular
reflections aid the detection of 3D flaws, the lighting conditions that produce such
reflections (directional lighting) can also cause glare, and thus recommended that both
diffuse (non-directional lighting) and directional lighting be employed for visual
inspection tasks [Brombach et al. 2006]. A significant finding of the study by
Brombach et al. is the fact that different lighting conditions suited different surface
finishes. Visual inspection of a brushed finish was found to give better results with a
directional light source, whereas the same lighting conditions produced glare, and
thus a detrimental effect on visual inspection of a vibratory polished finish [Brombach
et al. 2006]. The findings of Brombach et al. are undoubtedly beneficial to visual
inspection of metallic finishing operations. However, these recommendations are
specific to a task that has different requirements to visual inspection of large surfaces
for dents. Whilst they may seem contradictory to the findings of Brombach et al., the
lighting scenarios suggested by Lloyd 1998 to enhance perception of specular
reflections, and changes in them, are thus more likely to aid perception of 3D dents on
aircraft wing or fuselage skins.

Aluze et al. 2002 developed a vision system that analysed deformations or distortions
in specular reflections and surface contour information obtained from specular
highlights in order to identify flaws in perfume bottle tops, which have a highly
reflective, or specular, finish. The equipment developed by Aluze et al. used a lighting
system consisting of a set of backlit, sliding slats. As the slats slid across each other, a
series of illuminated bands were projected onto the inspection specimen. The
movement of the slats caused the width of the illuminated bands to change. The use of
a moving light source has links to theories of human visual perception and motion
parallax, which follow later in this chapter.

The effects of changing lighting direction on the visual perception process, and the
increase in shading and shadow cue saliency afforded by directional lighting shining
across a surface are demonstrated by Todd et al, 1996. Furthermore, whilst inspection
of other subjects may have attempted to avoid such conditions, visual inspection, and
detection performance, of 3D defects may benefit from lighting conditions that
produce, or enhance the visibility of specular highlights & specular reflections.

37
2.3.3 – Specular Highlights
Specular highlights are best described as the regions of high light level intensity
[Norman et al. 2004], usually observed on shiny surfaces. Glossy surfaces exhibit
small, intense specular highlights, whereas less glossy surfaces exhibit less intense,
diffuse highlights [see Norman et al. 2004]. As an example of specular highlights,
figure 2.21 illustrates a shape that has been given a matt, diffuse finish and figure 2.22
illustrates the same shape in a glossy, specular finish.

Figure 2.21 – Matt, diffuse finish Figure 2.22 – Gloss, specular finish
It will be noted that the shading exhibited by the object in figure 2.21 is the same as
that in figure 2.22. However, the presence of specular highlights in figure 2.22 leads
to a perception of a shinier surface, and help identify rounded edges, such as those on
the edge of the hexagon shape. Specular highlights such as these are understood to be
effective, salient cues for the perception of 3D shape [see Norman et al. 2004, Todd
2004 and Kingdom 2008].

The subject of visual perception of specular highlights has received specific scientific
interest, notably from Norman and Todd. It is noteworthy that much of their work was
published in the late 1990s and researchers continue attempts to develop models of
human visual perception for use in machine vision systems. In experiments on shape
perception, Norman et al. found that depth discrimination was at its best when the
participants were presented with specimens that exhibited specular highlights
[Norman et al. 2004], thus re-iterating their importance to visual perception. Todd, in
2004 discusses the deformations in specular highlights that occur as an object is
viewed in motion, i.e. due to head movements when viewing. As late as 2004, Todd

38
feels the need to state that, contrary to wide held belief, the presence of and
deformations in specular highlights should not be considered as detrimental glare, but
as being beneficial to perception of 3D shape [Todd 2004]. Furthermore, Lloyd has
identified, and demonstrated, a positive correlation between the saliency of specular
highlights and the area (size) of the highlights [Lloyd 1998].

From the literature available, one is led to conclude that specular highlights are
compelling and salient cues for accurate visual perception of 3D shape, although
some previous visual inspection researchers may have regarded them as ‘glare’ and
thus recommended the use of lighting that diminishes the saliency of specular
highlights, rather than increasing their saliency.

2.3.4 – Specular Reflections

Specular reflections are the mirror like reflections observed on the surface of objects,
particularly those with shiny or glossy surfaces. Glossy (specular) surfaces reflect
light with little distortion, and so produce clear, mirror like reflections of their
surrounding environments. Matt (diffuse) surfaces spread the reflections of light from
surrounding environment. The clarity of specular reflections is dependent on the
surface finish. Clean, glossy surfaces will exhibit specular, almost mirror like
reflections, whereas reflections on dusty, satin or matt finish (diffuse) surfaces will
vary from faint & blurry to none [Norman et al. 2004].

The use of deformations in specular reflections to identify surface dents was

successfully demonstrated in 1981 [Lippincott & Stark 1981]. By using a video
system to detect deformations of a projected grid pattern, Lippincott & Stark were
able to detect 0.053 mm deep flaws, within 1% of the viewed field on a specular metal
surface [Lippincott & Stark 1981]. The use of a grid, or geometrical, pattern in this
experiment that is the key factor; both machine and human can easily recognise
deformations of a shape with known, and simple geometry such as a grid. The
difference being that the machine is limited to using a grid of known dimensions,
whereas the human eye can determine shape changes from specular reflections of a
far greater range of objects, the size of which may not be known.

39
In figure 2.23, a highly specular sphere is illustrated in a scene surrounded by a brick
wall. The lines of the brick wall appear as specular reflections on the surface of the
sphere. A small indentation on the surface of the sphere is visible, and in this region,
the mortar lines of the brick wall become distorted. Figure 2.24 illustrates how the
sphere would appear without a surface dent, with the brick and mortar lines appearing
undistorted. Figure 2.25 illustrates the distortions that take place due to the presence
of the surface dent. The altering surface slant and tilt in the dented region causes light
to be reflected in different directions, thus distorting the reflections of the mortar
lines.

Figure 2.23 – Specular reflections of a brick wall on a

spherical object with a dented surface

Figure 2.24 – Dent free sphere with Figure 2.25 – Dented sphere with
undistorted reflections of mortar lines in distorted reflections of mortar lines in
brick wall brick wall

40
Humans, when viewing figure 2.23, are able to perceive the deformations in specular
reflections as a surface dent because one is aware that the grid shape of the mortar
lines has been distorted. However, a human observing figure 2.23 may come to three
possible perceptions:

1. The sphere is dented, and the wall is built only from rectangular bricks.
2. The sphere is perfectly spherical and the wall is built from oddly shaped bricks
one of which has a deep dimple in it.
3. The sphere is perfectly spherical but has a surface colour pattern that is similar
to the surrounding wall but for a small region that looks similar to a dent.

Perception, i.e. deciding what is being seen, is understood to be an active process that
builds on previous knowledge and experience, whereas the brain’s processing of light
information from the eye (i.e. ‘seeing’ or sight) is a passive sensory process. It is the
use of prior knowledge and heuristics, and their weighting during visual perception
that allows human to separate specular reflections from object shape and discount
option 3 [see Gibson 1950a, 1979, Todd et al. 1996, Kingdom 2008]. Furthermore,
most humans are likely to have seen a brick wall, and therefore when presented with
specular reflections, and other cues such as shading and shadow, such as those in
figure 2.23, will automatically weight against the possibility of option 2, and default
to perceiving reflections of a perfect brick wall on a dented sphere. However, if the
specular reflections on the sphere were of an unfamiliar, non-geometrical scene or
object, or if the observer had never seen a brick wall, the question arises of whether
the observer would perceive a dent in the surface or a reflection of strangely shaped
object.

Considering an aircraft inspector, it is likely that many of the objects, and possibly
parts of the environment in which they work will be geometric in shape. Furthermore,
geometric or not, the shape of the objects within an inspectors environment will be
familiar to them. A tree, an office block, and an aircraft, for example, are the shape of
a tree, an office block and an aircraft. Whether an inspector recognises what they are,
or their names, is of no concern; their shape is familiar to the inspector. Therefore,
bearing in mind the aforementioned literature of Gibson, Norman, Todd and
Kingdom, it can be assumed that distortions in reflections of objects and the

41
surrounding environment are likely to be perceived as surface shape changes rather
than strangely shaped, unfamiliar objects.

The actual saliency, or visibility of distortions to the inspector, i.e. whether the
distortions are severe enough to be detected, is another matter. Reading of literature,
particularly by Gibson, Norman and Todd leads to the understanding that saliency of
specular reflections will depend on the magnitude and rate of change in surface slant
and tilt.

2.3.5 – Surface Slant and Tilt

The effects of surface slant and tilt on visual perception, and their effects on the
specular highlights, specular reflections and shading that occur on an objects surface
are well researched [See Gibson 1950a, 1950b, & 1979, Norman et al. 2006 & Todd
et al. 1996].

Slant is well described by Gibson as the “angular inclination to the line of regard”
[Gibson 1950b]. When observing objects, surfaces with different slants appear to have
different shading and may exhibit different specular reflections to neighbouring
surfaces of different slant. Considering a dent on a surface, the slant of the surface in
the dented region is different to that of the original, non-dented surface. A dent that
one might term as ‘acute’ in shape will exhibit significant changes in slant and the
changes in slant will occur across short distances, i.e. have a higher rate of change in
slant. However, a less acute dent, or ‘shallow’ dent will be comprised of surfaces that
exhibit smaller changes in surface slant, across greater distances, i.e. a smaller rate of
change in slant.

Small rates of changes in slant will produce subtle changes in shading (‘soft’ shading
gradients), whereas higher rates of change in slant will produce abrupt changes in
shading (‘hard’ shading gradients). Subtle changes in shading, or soft gradients, are
difficult for humans to perceive (i.e. camouflaged) if they occur across large
distances, especially if the surface exhibits other features such as a patterned surface
colour [see Kingdom 2008].

42
Specular reflections are also affected by surface slant, (see Norman et al. 2004. for a
detailed description). Subtle changes in surface slant will cause less salient distortions
to specular reflections than abrupt changes in surface slant. It thus follows that if a
dent in a surface has a smooth, shallow profile, it will produce less salient distortions
in specular reflections than an equally sized dent with a less smooth, profile.

The effect of surface slant on specular highlights is complex [Norman et al. 2004],
and this section represents an understanding of the available literature with a bearing
on visual inspection of aircraft structures. First, consider a small radius on the edge of
two perpendicular surfaces, similar in profile to that shown in figure 2.26. Secondly,
consider a step between two parallel surfaces, which is joined by a very shallow
double curve as illustrated in figure 2.27, placed in the same position and lit from the
same light.

Figure 2.26 – Surfaces oriented in a Figure 2.27 –Surfaces oriented through a

wide range of slants exhibiting specular smaller range of slants failing to exhibit
highlight specular highlights

Although the surface of the smaller radius edge may produce a thin specular highlight,
the surface covers many slant angles, which allows the surface to reflect light from a
wide angular field, i.e. there is a wide angle from which the light source producing the
specular highlights could shine. Consider on the other hand, the shallow double
curvature shape; under the same lighting conditions, the slant of these surfaces is not
conducive to specular highlights being produced. The surfaces of the object in figure
2.27 have not captured a wide enough field of light in order to return or specular
highlight to the observer’s eye. In view of these considerations, one must bear in mind
that whilst larger specular highlights may be more visible to an observer than smaller

43
ones, specular highlights can only occur if the surface slant is oriented with the
observer’s line of regard in such a way that it produces a reflection of a light source.

If the shape of a dent in a surface is such that a wider range of slant is present, there
exists a greater chance that specular highlights are exhibited from a given viewing
angle, which as previously discussed, increases the chance that the shape will be
correctly perceived [Norman et al. 2004]. Further to the issue of specular highlights,
or lack of, the shape and surface slant(s) of a dent may be such that the saliency of
distortions of specular reflections is reduced to a point where an observer is unable to
perceive a change in 3D shape. Couple this with the fact that the subtle changes in
surface slant may also result in poor shading saliency [see Todd et al 1996], and one
is able to see how an observer may have difficulty perceiving the available cues as a
surface dent. Thus, a small acute dent, providing greater variations across a wider
range of surface slant angles may be more visible than a wider, shallow dent that
offers only a small range of slant angles, with very subtle changes in slant.

2.3.6 – Surface colour & surface finish

The visual perception of colour and research into links between surface colour and
surface finish continues to receive scientific interest. It is apparent that the studies by
Todd, Norman & co-researchers cited in this thesis have led way to yet more recent
publications on the specific issues of surface colour and gloss [Xiao & Brainard 2006
& 2008].

Whilst a matt finish object and a gloss finish object may have the same base, or
‘body’ colour, the actual colours reflected by a glossy object will vary significantly
more than on a matt object [Xiao & Brainard 2006]. Consider two identical shapes,
one reproduced in a matt yellow (figure 2.28) and one in a gloss yellow finish (figure
2.29).

44
 Figure 2.28 – Object with matt yellow Figure 2.29 – Object with gloss yellow
finish finish

It can be seen that whilst the colours seen on the matt yellow objects vary from dark
to grey to light yellow, the presence of specular highlights on the gloss object mean
that the colours reflected by the gloss object vary from dark grey to white. Differences
in perception of colour due to variations in surface finish have been experimentally
demonstrated [Xiao & Brainard 2006 & 2008], and it is likely that the colour of the
objects in figures 2.28 and 2.29, although the same, may be perceived a very slightly
different. The fact that surface finish affects perceived colour is fundamental to
understanding the link between surface colour, surface finish and visual perception of
an object’s 3D shape. This understanding is further aided by the work of Kingdom in
2008.

Firstly, the effect of surface finish on visual perception can be explained. The shape of
the objects in figures 2.28 and 2.29 is clearly and unambiguously perceptible to the
observer. The same object has been re-coloured in matt blue and gloss blue, and is
presented again in figures 2.30 & 2.31.

45
 Figure 2.31 – Object with gloss blue
Figure 2.30 – Object with matt blue finish
finish

In figure 2.30, the shading on the matt blue object has poor modularity with the base
colour and it is difficult to distinguish the regions that are darkly shaded from base
colour. There exist regions on the sphere that are so dark that it becomes difficult to
discern 3D topographical features, i.e. the shape is difficult to perceive. However, the
introduction of a gloss finish in figure 2.31 increases the range of colours that are
reflected by the objects surfaces [Xiao & Brainard 2006], and introduces the
appearance of specular highlights [which enhances perception of shape [Norman et al.
2004]. The shape of the object in figure 2.31 is thus slightly more perceptible than
that in figure 2.30, due to the change in surface finish.

In order to visualise how surface colour and surface finish interact, a series of dented
coloured spheres are presented in matt and gloss finishes in different scenes. Consider
first the grey sphere in figure 2.32. Perception of the dent on the side of the sphere is
facilitated by shading cues and distortions in the specular reflections of the
environment. If the same sphere is given a white, glossy finish, as in figure 2.33, the
dent still produces salient cues that allow perception of the same dent. However, on
the same sphere with a glossy blue finish as shown in figure 2.34, the modularity of
the shading cues and the specular reflection cues is poor, resulting in reduced
saliency, i.e. the dent is difficult to see. The effect of surface colour on shading and
specular reflection saliency is thus demonstrated. Furthermore, it will be noted that
the range of colours reflected by the white sphere in figure 2.33 is far greater than the
range of colours reflected by the blue sphere.

46
Figure 2.32 – Gloss grey sphere with dent Figure 2.33 – Gloss white sphere with
in side dent in side

Figure 2.34 – Gloss blue sphere with

dent in side

Surface colour can affect the saliency of not only the shading and specular reflection
cues, but also the saliency of specular highlights. In figures 2.35 to 2.37, the same
grey, white and blue spheres are illustrated, but in a position whereby the dent is at the
top of the sphere. In this position, the dent is perceptible on grey sphere due to the
small deformations in the specular reflection of the brick wall, and a small area of
darker shading. The large specular highlight just below the dent does not affect the
saliency of these cues. On the white sphere, however, the specular reflection and the
white base colour fuse together, thus eliminating the specular reflection cue and the
shading cue. The specular reflections on the blue sphere remain visible at the top of
the sphere, and the distortions in the reflection of the brick wall caused by the dented

47
region are just perceptible. However, whilst the specular highlight on the sphere does
not interfere with the cues in the dented region, the shading cue has poor modularity
with the base colour and reflected colour, making it imperceptible.

Figure 2.35 – Gloss grey sphere with dent Figure 2.36 – Gloss white sphere with
in top dent in top

Figure 2.37 – Gloss blue sphere with

dent in top

The effects of surface colour and surface gloss combine to affect the visibility of 3D
flaws in complex ways. The grey, white and dark blue colours and matt & gloss
surface finishes illustrated in this section are commonly found in aviation applications
(as seen in figure 2.38). Thus, the visual perceptual issues presented in this section are
all likely to be observed during visual inspection of aircraft structures for 3D impact
damage dents.

48
 Figure 2.38 – common colours used in airline liveries, from www.airliners.net 2009

2.4 – Aircraft paint finishes

The topcoat paints used by airlines to paint liveries are usually applied as a glossy
colour coat. Over a period of several months, environmental effects can lead to
dulling of the paint surface, resulting in a less glossy surface finish. A piece of black
coloured glass would be expected to give the maximum reading of 100 gloss units
(GU). Gloss levels for freshly applied aircraft paint usually exceed 90gloss units (GU)
[PRC-DeSoto 2003; Akzo Nobel 2006 & 2008]. However, experimental data from
Guseva et al. 2003 (see figure 2.39) show a reduction in paint gloss to 60GU after 40
months of service, and a reduction to 40GU after 60 months of service.

49
 Figure 2.39 – Plot detailing aircraft paint gloss degradation with
time, from Guseva et al. 2003

Thus, whilst aircraft may have highly glossy (>90GU) paint finishes for the first 3
years of their lives, older aircraft are likely to have less glossy, semi-gloss (<60GU)
paint finishes. In extreme cases, the gloss level may be so low as to be described as
matt finish. Dust and dirt may also combine to reduce the glossiness, or specularity of
the surface, resulting in a more diffuse surface finish. Given the previously described
effects of surface specularity on visual perception, it thus follows that the condition of
an aircraft’s paint may affect the visibility of 3D surface flaws.

2.5 – Reliability assessment of NDI/ NDT techniques

Reliability assessments provide a quantitative means of determining the detection
threshold of a non-destructive testing (NDT) or inspection (NDI) procedure, or
probability of detection for flaws of a certain size using said procedure. Reliability
assessments typically involve asking participants to inspect specimens containing
flaws of known sizes, and identify the presence of said flaws [See Singh 2000;
Georgiou 2006 & Georgiou 2007]. As previously mentioned, most reliability
assessments are no longer performed to determine the smallest size of defect that can
be detected, and since the 1960’s and 70’s, the has question changed to being ‘what is
the largest size of defect that will be missed’ [Hagemaier 1998; Georgiou 2007]. The
effect of factors that may cause defects to be missed can be quantified by the use of
reliability assessments.

50
An excellent review of NDI reliability research is given by Karta Technologies [Singh
2000], and the capabilities of several aerospace NDT/NDI methods have been
quantified and published as a collection of probability of detection (POD) curves
[Rummel & Matzkanin 1997]. However, it is important to note that much of the
published work studied by Singh, and within the NDE capabilities book concern the
inspection of metallic aerospace structures, and the word ‘flaw’ refers exclusively to
cracks in metallic structures. In most published NDT/NDI literature, crack length is
considered adequate for describing the size of flaws or defects. For inspection of
metallic structures, it is thus convenient to give POD as a function of crack length.
The use of a single variable to describe flaw or defect size has consequently led to
published mathematical methods that model POD solely as a function of a single
variable [see Berens & Hovey 1982; Bullock et al. 1994]. The issue of crack width
has been acknowledged [Bullock et al. 1994], but in favour of less complex, single
variable mathematical models, crack length has remained the single flaw size variable
on which POD has been modelled. Current published POD models are thus reliant on
the fundamental assumption that a longer crack will always give a higher POD than a
shorter one. This assumption is so deep rooted into POD literature that inspection
procedures are considered incorrectly designed if POD reduces with increases in flaw
size [Georgiou 2007].

As previously described, impact damage to a composite laminate is three dimensional,

in shape. The damage does not occur solely in the form of simple surface breaking
cracks. Impact damage to composite materials must be characterised by its width,
depth and sectional profile in order for its shape and size to be fully described. As
previously described, it is feasible that a wider dent will be less visible than a dent
with smaller width and equal depth. Currently published POD models cannot
simultaneously calculate POD as a function of multiple parameters and cannot
accommodate a situation where POD may reduce as the size of a parameter increases.
Consequently, published POD models are unsuitable for modelling the reliability of
visual inspection for impact damage on composite materials. The issues surrounding
POD are presented here in due course.

Of all the available literature, three documents stand out as being the widely accepted,
and adopted recommended guidelines for conducting reliability assessments of NDI/

51
NDT procedures. These are Agard Lecture Series 190 [AGARD 1993], Spencer’s
Generic Protocol [Spencer et al. 1993] and Department of Defence Handbook 1823
[US DoD 2007]. MIL-HDBK-1823 appears to be a culmination of many works,
including Spencer’s protocol and AGARD LS190. Unsurprisingly, Spencer and
contributors to LS-190 (Annis, Rummel & Vukelich) are acknowledged as having
considerable input into the MIL Handbook. As a result, MIL-HDBK-1823, represents
the definitive ‘how to guide’ for reliability assessments.

Within MIL-HDBK-1823, there also exists a grading scheme for reliability

assessments, whereby the thoroughness and statistical validity of the experiment is
graded against certain criteria and minimum requirements [Appendix I.5, US DOD
2007]. It is evident that the grading criteria were written for NDT and POD of
metallic structures, although there are recommendations within that remain valid for
reliability assessment of inspection of composite materials. These recommendations,
along with those from other documents will be presented in due course.

As can be deduced from published examples [Murgatroyd 1993; Forsyth & Fahr
1998; Georgiou 2007], the three aforementioned guideline documents and the Karta
review [Singh 2000], published NDT/NDI reliability assessments follow a generic
paradigm, a description of which follows:
• Several participants inspect a series of specimens
• The specimens contain a known range of flaw or defect sizes, in known
locations
• The participants use the specified inspection procedure to search for defects
• The participants call out, or mark down the presence of a defect
• The results are recorded in terms of hits, misses and false calls
• The hit/ miss data are analysed as probability of detection (POD) curves and
the false call data may be analysed in terms of probability of false alarms
(POFA) and may presented as receiver operating characteristic (ROC) curves
[see Spencer et al. 1993 & Singh 2000]
• The 90% POD/ 95% confidence size, (the size of flaw that 90% of participants
can find 95% of the time), is determined, pending sufficient data/ specimens

52
The personnel implementing the experiment, be it one or several people, are usually
referred to as the ‘moderator’ [Spencer et al. 1993]. The persons performing the
inspection may be referred to as ‘inspectors’ [Spencer et al. 1993], although the term
implies some form of formal training or qualification. To avoid such confusion, the
term ‘inspector’ will be substituted with ‘participant’.

In order to define the experimental plan, issues relating to participants, experimental

setup, specimens, flaws and experimental procedures must be addressed.

2.5.1 – Participants
Participant numbers vary in reliability assessments, often just a handful of participants
are used [see Murgatroyd 1993; Spencer 1996, Bruce 1998, Mullis 1998], and there is
no specific requirement for the number of participants in a reliability assessment.
There are only requirements on the number of successful detections required to
demonstrate 90%/ 95% or 90%/ 50% POD/ confidence levels [Hagemaier 1998].
However, after reviewing previous experiments [Murgatroyd 1993; Spencer 1996;
Mullis 1998], it can be considered that 15 individual people is a large number of
participants, and can be considered adequate for obtaining representative, and
statistically valid results.

Part of the problem in obtaining sufficient participants for NDT/NDE reliability

assessments is the reluctance of airlines and MRO facilities to provide inspectors
during their duty hours. It should be noted that most published NDT reliability
experiments have benefited from the support of military, regulatory or academic
establishments such as USAF, NASA, FAA or universities, not commercial airlines
[see Lewis 1978; Rummel & Matzkanin 1997; Gramopadhye et al. 1998; Singh 2000
& Erhart et al. 2004].

In order to obtain greater numbers of participants, novice participants without formal

training in the inspection task may be used. Studies that utilised novice participants
have achieved participant numbers well over 40 [Gallwey & Drury, 1986]. Novice
participants can be used instead of trained aircraft inspectors, without significant
differences in performance [Gallwey & Drury, 1986; Mullis, 1998]. The condition for
doing so being that the experimental procedure is a simulation of the inspection task,

53
not an inspection of an actual aircraft. This eliminates inspector prior knowledge or
expectancy, which can aid inspection reliability [see Spencer 1996], as an
experimental variable or factor. Use has been made of novice inspectors or
participants in several published NDT/ NDI studies [Forsyth et al. 1998; Mullis 1998;
Latorella et al. 1992; Erhart et al. 2004; Vora et al 2002 & Sadasivan 2007].

2.5.2 – Experimental setup

NDT/NDI reliability experiments seldom use an entire aircraft in the experimental
setup, due to cost and logistical considerations [Spencer et al. 1993; Singh 2000].
Experiments commonly represent the section of aircraft structure to which the
inspection procedure would normally be applied [Murgatroyd 1993; Spencer et al.
1993; Mullis 1998; Singh 2000; Erhart et al. 2004 & Gant 2007]. Participant buy-in is
enhanced by an experimental setup that closely replicates the actual inspection task,
and it is recommended that experiments attempt to replicate the environmental
conditions, such as lighting, accessibility and noise that would be experienced in the
field [Spencer et al. 1993; AGARD 1993; US DOD 2007]. Assessment of an
inspection task performed on a difficult to access, poorly lit structure must replicate
such conditions [Spencer et al. 1993]. However, NDT/NDI reliability assessments
need not be performed exclusively in a hangar or aircraft maintenance facility
[Spencer 1998]. The work of Murgatroyd 1993 is an excellent example of how
accessibility conditions of an aircraft were replicated by using a raised platform and
barriers, without actually needing to perform the experiment high off the ground on an
actual aircraft. Murgatroyd’s experiment also utilised a specimen support that
represented an aircraft fuselage belly and the access issues associated with having to
bend or kneel to perform the inspection. Some experiments have asked participants to
travel to the experiment, [Murgatroyd 1993; Spencer 1996; Erhart 2004 & Gantt
2007] whilst others transported specimens to the participants for inspection at
different facilities, as typified by Lewis’ renowned 1978 experiment, which is more
widely known as the “have cracks, will travel” program [Lewis et al. 1978; Singh
2000]. Transporting an experiment to facilities or participants may help furnish the
experiment with greater participant numbers, and indeed Lewis et al. obtained data
from approximately 300 inspectors in the four years that their experiment ran.
However, this option is unfavourable when wishing to avoid uncontrollable facility-

54
to-facility variations in lighting or environment, and logistical issues such as damage
to specimens during transit [see Spencer et al. 1993; AGARD 1993].

2.5.3 – Specimen Characteristics

It is convenient to refer to the Karta Technologies review for an excellent review of
previous NDT/NDI reliability assessment specimens [Singh 2000]. However, it is
important to note that specimens for reliability assessments generally use artificially
produced flaws i.e. flaws or cracks produced by machining as opposed to grown by
cyclic loading. Previous experiments have also made use of facsimile specimens.

The key advantages offered by using artificially produced flaws are that the
specimens all have the same condition, and the flaw locations and sizes can be
specified by experimental design. Facsimile specimens are particularly useful when
the cost of producing the required specimen numbers and logistical considerations
such as weight or production time prohibit the use of original materials [Spencer et al.
1993]. An excellent example of the use of facsimile specimens is the study by
Murgatroyd et al. on the influence of various human factors on inspection reliability,
in which an electronic tablet provided the facsimile of a row of rivets [Murgatroyd
1993].

When designing specimens, it is recommended that specimens are sized such that they
are representative of the original structure [Spencer et al. 1993], and the surface finish
& condition of the specimen should be similarly authentic [AGARD 1993; Spencer et
al. 1993]. With regard to surface finish, the available recommendations pay reference
to metallic finishing operations such as turning, blasting, burnishing or polishing. The
term condition usually refers to the condition, i.e. cleanliness of the component during
inspection, and is in fact an experimental variable, which must be controlled, as
described later in this section.

As previously mentioned, exterior facing composite aircraft structures such as wing,

fuselage and empennage skins are painted. The surface finish is ultimately determined
by paint finish. Thus, the recommendations for surface finish are best interpreted as
meaning that composite specimens should have a surface finish representative of a

55
painted aircraft structure. Inclusion of details such as panel joints, fasteners or access
panels to enhance specimen authenticity [Spencer et al. 1993; AGARD 1993; US
DOD 2007] is only required if inspector buy-in and use of their expectancy of damage
location is sought.

Flaw locations are also to be authentic to those seen in the field. Flaws that would
normally be found on, or originate from component edges or holes, and flaws that
would normally break out on surfaces should be replicated in the same manner on the
specimens [Spencer et al. 1993; AGARD 1993; US DOD 2007].

2.5.4 – Flaws
The term ‘flaw’ in a reliability assessment is used to describe intentional defects and
can cover a variety of defects such as cracks, disbonds, delamination or dents. For
metallic structures, crack and flaw are synonymous. For the purposes of applying the
relevant NDT/NDI reliability literature to impact damage on a composite laminate,
the term flaw must be considered as the dent produced on the laminate surface by the
impacting object.

The flaws used in reliability assessment specimens must be authentic to those seen in
the field [Spencer et al. 1993; AGARD 1993; US DOD 2007]. For published
assessments, this has meant using cracks that are authentic in size, shape and type to
those experienced on service components. For impact damage on a composite
laminate, the requirement translates to meaning that the reliability assessment
specimens must contain impact damage, or representations of impact damage, that are
authentic to the damage an inspector might see on an actual aircraft.

Hovey et al. 1989 suggested a minimum of 30 flaws across a range of sizes covering
the 10% to 90% POD range, i.e. region of interest, are sufficient to estimate
inspection reliability for a process that results in a hit/miss response. However, if the
correct region of interest is unknown, Hovey et al. recommend an extended size range
covered by 60 flaws. These recommendations are widely accepted and feature in all
protocol guidelines [see Spencer et al. 1993; AGARD 1993; US DOD 2007]. With
regard to flaw sizes, one is recommended against using flaw sizes large enough to
guarantee 100% detectability and flaws so small they are undetectable [Spencer et al.

56
1993; AGARD 1993; US DOD 2007]. The contributors to AGARD LS-190 noted that
Previous NDI/ NDT reliability assessments have suffered a “tendency to include too
many large flaws” [AGARD 1993 pp.6]. However, all available reliability assessment
literature acknowledges that the sizes likely to produce the required region of interest
will be unknown, and therefore the flaw sizes for this region can only be estimated
using ‘engineering judgement’ [Hovey at al. 1989]. It is therefore apparent that one
must have some idea of the inspection process capabilities in order to set the flaw size
range for the specimens.

The recommendations for flaw numbers hold true only for each type of flaw within
the experiment [see Spencer et al. 1993; AGARD 1993; US DOD 2007; Georgiou
2007]. For composite materials, if detection of flaws such as impact damage from a
rounded object and impact damage from a pointed object is to be assessed, the
recommendations state that a range of 30 different sizes of flaws for both the rounded
and the pointed object be used, i.e. 60 flaws in total. If one adopts the more extensive
recommendation of 60 flaws per type, the specimens must contain 120 flaws in total.
In view of such recommendations, it becomes clear as to why reliability assessments
require large numbers of specimens.

Having established that the specimens must contain a minimum of 30 flaws for each
type of flaw, one must then distribute the flaws a across a range of sizes. The
unanimous recommendation is to use an even interval between the logs of the flaw
size values [Spencer et al. 1993; AGARD 1993; US DOD 2007]. For metallic
structures, the chosen variable has been crack length, so cracks would be distributed
across a size range, with even intervals between the log values of the crack lengths.
For composite structures, and impact damage dents that require more than one
variable to describe their topography (as previously described), one must choose
which size variable (width, depth or other) will be used for flaw size distribution.

Once produced, the specimens must be re-characterised in order to ensure that they
conform to the original size specifications and are in the correct locations on the
specimens [Spencer et al. 1993; AGARD 1993]. If different to the specification, the
results must be reported using the sizes and locations of the flaws produced on the
specimen as opposed to the specified values.

57
2.5.5 – Specimen numbers
Boredom or fatigue due to repetitive tasks has been identified as a human factor likely
to cause errors in aircraft inspection, and thus reduced reliability [Rummel 1984 &
Dupont 1997 in Hagemaier 1998]. It is recommended that specimens for reliability
assessments also contain unflawed sites [Spencer et al. 1993; AGARD 1993]. This
helps introduce the boredom factor to the experiment by requiring the participants to
keep searching for flaws, even though their inspections may often result in no flaws
being found. Another factor introduced by inspecting unflawed specimens is the
propensity of inspectors to make false calls. False call information can signify
whether an inspector is being conservative in calling out flaws, or simply calling out
flaws even though they are not entirely sure that a flaw is present [see Bullock et al.
1994; Spencer et al. 1993]. Quantification of the probability of false calls or alarms
(POFA) is recommended for reliability assessments [Spencer et al. 1993; AGARD
1993; US DOD 2007] although the mathematical models for POFA are limited by the
same constraints as those previously mentioned for POD [see Bullock et al. 1994]. In
order to follow established experimental paradigms, specimen numbers must therefore
allow for the inclusion of unflawed sites.

The decision on specimen numbers will also be driven by flaw density (number of
flaws per specimen) considerations. Flaw density can either be set by experimental
design, or allowed to mimic the natural flaw densities that would be observed on in-
service aircraft, although a mix of the two is also feasible [Spencer et al. 1993].
Where a representative flaw density would result in too large a number of specimens,
or too great an inspection time, it is recommended that the specimens contain three
unflawed inspection sites for every flawed inspection site [Spencer et al. 1993;
AGARD 1993; US DOD 2007]. The 3:1 unflawed/ flawed ratio is one best adopted,
in view of the lack of a published flaw density value for impact damage on actual
aircraft, i.e. there are no published records of how many ‘dents’ a composite structure
on an average aircraft is likely to have sustained during a given period of service.

It is clear, however, that the 3:1 unflawed/ flawed recommendation has only ever been
considered with regard to inspection of metallic structures for cracks. The key issue is
in the use of the word ‘inspection site’ as opposed to specimen. Consider, for example
an assessment of rivet or fastener hole inspection for cracking. An individual hole

58
could be considered an ‘inspection site’. One thus interprets the 3:1 ratio as meaning
that the specimen set for the assessment should contain three unflawed holes for every
cracked hole. Most published NDT reliability assessments concern inspection of
metallic aircraft structures such as fuselage skins or wing spars, and in particular,
inspection of rivet or fastener holes and bonded or welded joints. Each row of holes or
joints, of which there can be several on any one aircraft, can be considered an
‘inspection site’, thus the 3:1 unflawed/ flawed ratio can be conveniently adopted.
This appears to be the interpretation made by Murgatroyd et al. 1993, whom exceeded
the recommendations and used an approximate ratio of 70 unflawed fasteners for
every cracked fastener. Furthermore, Murgatroyd et al. presented approximately 7
unflawed rows of fasteners for every row that contained cracks. However, the
discrepancy between ‘inspection site’ and specimen is evident in Bode et al. 2006, in
which it is said that NDT reliability data is generally obtained “from sites containing
multiple flaws”. The issue of concern for Bode et al. is whether a ‘site’ containing
multiple flaws has more chance of being detected than one containing a single flaw.
In the paper by Bode, it is clear the individual rivet hole is considered a site, rather
than a whole row of holes.

Furthermore, there are no published precedents for the implementation of the 3:1 ratio
in visual inspection reliability assessments concerning impact damage to composite
structures. Whilst the term dent is easily translated into ‘flaw’, one cannot consider a
dent as being ‘unflawed’ or having ‘multiple flaws’, i.e. a dent cannot be considered
an ‘inspection site’ and one cannot have three ‘unflawed’ dents for every ‘flawed’
dent. Instead, the structure itself, i.e. entire sections of laminate must be considered as
flawed or unflawed. It would be unfeasible (financially & logistically), to propose a
reliability assessment using four entire wing, fuselage or empennage structures of
which only one contained damaged laminate. Instead, the 3:1 ratio is better interpreted
by considering structures such as a wings, fuselages or stabiliser skins as a collection
of individual inspection areas or zones, of which only one zone in every four contains
flaws. The individual zones then become discrete ‘inspection sites’.

The issue herein is whether one presents individual ‘inspection sites’ i.e. single
specimens) one at a time to the participants using the 3:1 unflawed/ flawed ratio for
complete specimens, or whether one arranges several individual specimens in order to

59
represent a larger structure such as a stabiliser, a wing skin or a large section of either.
Alternatively, and similar to the latter option, one may choose to section a large,
single specimen such as a stabiliser or wing skin into individual zones, or ‘inspection
sites’, using tape or marked lines. The disadvantage to this method being that the flaw
sites cannot be moved at any time during the inspection, meaning that if, for example,
a flaw were in a poorly lit location compared to others, one would be unsure whether
it was the flaw size or the lighting that affected inspection reliability. Providing all the
specimens have the same shape and size, the option of using individual specimens
arranged to represent a larger structure would allow one to move the specimens
around in order to asses the effect of flaw size and lighting together. If, however, the
effects of other factors, aside from those characteristic to the specimen (such as flaw
size, flaw shape or surface finish), are to be eliminated, the only option is to present
individual specimens for inspection, one at a time, under the same conditions. In this
case, the 3:1 recommendation is interpreted such that a single specimen constitutes an
‘inspection site’, and for every specimen in the set that contains intentional damage or
flaws, there will be three more containing no intentional damage.

2.5.6 – Other issues pertaining to specimens & procedures

In many NDT/NDI reliability assessments, the specimen sets may need to be reused to
perform repeat inspections, perhaps under the same conditions to provide
demonstration of repeatability or under different conditions to investigate the effect of
various factors [see Spencer et al. 1993 or AGARD 1993]. The participants must not
become familiar with the specimens used in the reliability experiment [Spencer et al.
1993]. If repeat inspections are performed, the presence of dirt marks, unintentional
damage or identification markings may allow participants to recognise a previously
inspected specimen, and memorise the position of previously detected flaws, which
may lead to artificially high response rates [AGARD 1993]. Careful cleaning and
storage of specimens is recommended in order to avoid such occurrences [Spencer et
al. 1993; AGARD 1993]. Similarly, any identification numbers used on the specimens
must be such that no information as to the flawed/ unflawed status of the specimen
can be deduced by the participants [Spencer et al. 1993].

It is recommended that the participants in the reliability assessment be unaware of the

results or outcome of their inspections. Allowing participants to learn where they have

60
made correct calls, or missed flaws would lean to familiarity with the specimens,
which in turn rules out the possibility of repeat inspections with the same specimen
set. [See Spencer et al. 1993; AGARD 1993]

2.5.7 – Experimental Variables

Variables within the experiment are either controllable or uncontrollable [Spencer et
al. 1993]. Controllable variables are those for which the value or condition is
specified by experimental design for each inspection. The value or conditions of
uncontrollable variables are adopted at the inspection facility and are not specified in
the experimental design [Spencer et al. 1993]. The value or conditions for
uncontrollable variables, whilst not specified in the experimental design, should be
recorded at the time of inspection in order to allow identification of possible effects
on the results [Spencer et al. 1993].

Hovey et al. 1989 proposed a model of the factors, which aside from flaw
characteristics, could affect the inspection process. The factors were categorised as
human related factors, inspection procedure related factors and equipment related
factors. Whilst equipment and procedural factors can be controlled, Hovey et al.
imply that the human factors within an experiment will be uncontrollable.

Spencer et al. 1993 assessed the variables likely to affect NDI/ NDT procedures, and
in conjunction with similar publications [Megaw 1979; Hovey et al. 1989; AGARD
1993; Hagemaier 1998 & US DOD 2007] it is possible to place the variables into
general categories, which are described forthwith. Each of the variables within the
categories should be considered, and methods for either controlling the variables or
recording the values/ conditions must be included in the experimental plan and
protocols

Facility characteristics – Lighting, noise and temperature are all characteristics of

the inspection facility. Lighting is likely to vary unless a single facility is used or
standardised lighting that travels with the experiment is employed. ‘White noise’ or
general background has been demonstrated to have little effect on inspection
reliability [Murgatroyd 1993], although unfamiliar loud noises may prove more
distracting. Temperature will be entirely facility dependant, and the experimental plan

61
must consider whether equipment will be required in order to maintain a specified
temperature range during inspection. Outdoor inspections will be open to factors such
as weather (clouds, rain, cold/ heat) and sunlight, and increased uncontrollability of
variables such as lighting and temperature. The experimental plan must consider how
environmental, or facility dependent factors such lighting, noise and temperature will
be addressed.

Inspector – The consensus amongst the NDT/NDE community is that the greatest
source of variability in reliability assessments is actually the human performing the
inspection [see Hovey et al. 1989; Hagemaier 1998; Spencer 1998]. Training levels,
the date of last training and the level of experience of the task are factors specific to
each individual inspector [Hovey et al. 1989]. Age and eyesight are specific to
individual inspectors and have been identified as variables that may produce variance,
albeit minimal, in inspection reliability [Lewis et al. 1978]. Fatigue has also been
demonstrated to affect inspection reliability [Murgatroyd 1993], and therefore the
experimental plan must demonstrate how the factors of fatigue, training, age and
eyesight will be handled.

Equipment – Whilst variables concerning the inspection equipment must be

considered in traditional NDI/ NDT reliability experiments, the equipment for human
visual inspection is the inspector and their eyes. Thus variables such as probe
specifications, display specifications and calibration specifications [Hovey et al. 1989,
Spencer et al. 1993, AGARD 1993, US DOD 2007] need not be considered. The only
possible ‘equipment’ variable to be considered is the eyesight quality of the
participants.

Part processing – The condition of the specimens during the reliability assessment is
not necessarily the same as the condition in which the component would be found in
during service [See AGARD 1993]. The most relevant example to visual inspection of
composite structures is cleaning of the inspection area prior to inspection. The
experimental plan must consider whether inspectors would always find the component
in a clean condition, whether an inspector is expected to clean the component prior to
inspection, or whether inspectors perform the inspection on a dirty component.

62
Inspection process and procedural variations – The experimental plan must
consider the inspection process and the inspection procedure and identify potential
variations. Where the possibility for variation exists, the experimental plan must
address how the variation will be avoided, controlled or recorded if uncontrolled.
[AGARD 1993; Spencer et al. 1993; US DOD 2007] Foreseeable variations in the
visual inspection process include head movements, viewing angle changes and visual
scanning techniques.

Experimental Protocols
The design of a reliability assessment experiment must include the production of a set
of experimental protocols [AGARD 1993; Spencer et al. 1993; US DOD 2007]. The
functions of the protocols are to ensure that:
• The experiment is conducted in a consistent manner
• The data are gathered in a consistent manner
• The participants receive the correct information
• Deviations from the experimental plan are correctly dealt with
• Other parties can perform the same experiment and obtain comparable results

The protocols cover both the actions of the experiment moderator and the participants,
and must provide instructions for the following listed items, which are derived from
the protocol areas suggested by Spencer et al. 1993:

Operating test equipment – the protocols should provide instructions that ensure the
moderators and/ or participants are consistent in their operation of any equipment
used during the inspection process.

Briefing the participants – standardised briefing material should be included in the

protocols in order to ensure that the moderators give each participant the same
briefing containing the same information.

Answering on-the-spot questions – participants may ask questions relating to

specimens and the inspection procedure, to which standardised responses, as defined
in the protocols, must be given by the moderators in order to avoid biasing the

63
inspection results. The key issues to avoid are divulging flaw information to the
participants and allowing the participants to view documentation that may provide
them with links to specimen and flaw characteristics.

Inspection procedures – the protocols should specify the correct inspection

procedure, and provide guidance to the moderator on how to deal with possible
deviations from the specified procedures.

Interruptions – provision must be made, and guidance given in the protocols for
dealing with foreseeable interruptions such as requiring the toilet, participants
requesting help during inspection, unauthorised persons entering the inspection
facility and telephone calls.

Collection of results – the protocols must ensure that the results are collected in a
consistent format, and are correctly labelled or identified by methods such as the use
of serial numbers.

Results
It is recommended that inspection results are recorded using permanent media, and
then transferred to an electronic database at a more suitable time [Spencer et al. 1993].
Results should be collected in the form of hit/miss data for each flaw, and false call
data [Spencer et al. 1993; AGARD 1993; US DOD 2007]. Some inspection
procedures may result in a perceived flaw size, such as a flaw seen on a monitor
screen or display screen, in which the perceived flaw size must be recorded. However,
visual inspection is performed directly on the flaws, and appears as actual size, thus
meaning actual and perceived size are the same. Another technique that may be
employed is to ask the participants to state how certain they are of their response, i.e.
give a percentage of how certain they are of having actually seen a flaw [Spencer et
al. 1993].

It is recommended that the experimental results be presented as a description of the

experimental design, the individual results from each inspection, and a summary of
the test results [AGARD 1993; US DOD 2007]. The results should also identify any
failures during the inspections or known anomalies [US DOD 2007]. Once collected,

64
the recommendations state that the hit/ miss results should be analysed to produce a
set of POD curves, and once the POD curve has been determined, it is recommended
that the false call data be used to determine POFA and produce received operator
characteristic (ROC) curves [Spencer et al. 1993]. The information from ROC curves
can be used to adjust the POD curves for varying levels of inspector conservatism.

2.5.8 – POD Analysis methods

For a review of the POD analysis methods employed by previous reliability
assessment experiment, it is convenient to refer to Singh 2000 and Georgiou 2006 &
2007. Probability of detection for NDT/NDI procedures is normally presented as a
function of flaw size. It is a widely accepted that POD as a function of flaw size is
best modelled using either a ‘log-logistic’ (‘log-odds’) or ‘log-normal’ model [see
Bullock et al. 1994 and Georgiou 2007].

In view of the work of Berens & Hovey 1982, ‘log-odds’ became the accepted model
for POD of hit/miss data [AGARD-LS190; Spencer et al. 1993]. The ‘log-normal’
model was recommended only for POD of an inspection procedure that resulted in a
perceived or apparent flaw size [AGARD-LS190; Spencer et al. 1993]. However, in a
comparison between the two models, Fahr et al. 1993 determined that whilst the log-
odds model was easy to implement, it gave more conservative POD estimates than the
‘log-normal’ model and therefore recommended the use of the ‘log-normal’
distribution for modelling of hit/miss data. Fahr et al. did acknowledge that the ‘log-
normal’ model is mathematically more complex and slower to calculate than the ‘log-
odds’ model. Despite their findings in 1993, a later publication in 1998 by Forsyth &
Fahr does not make the same distinction and merely presents results that identify
differences in the POD curves produced from the same data using the ‘log-odds’ and
‘log-normal’ models. In 1997, Rummel & Matzkanin presented the ‘log-odds’ method
for modelling of hit/miss data, combined with maximum likelihood estimates to
determine parameters within the model, as earlier proposed by Berens and Hovey.
Later publications also recommend the use of the log-odds model for POD from hit/
miss data [Georgiou 2007]. The following section will present the log-odds model for
POD, and explain its limitations.

65
2.5.9 – Single variate POD models
Berens & Hovey 1982 presented the log-logistic, or log-odds model as being the most
suitable distribution for POD as a function of crack length, the formula for which is
given below:

exp (α + βln(ai)
Pi = (1)
1 + exp (α + βln(ai)

Where Pi is the probability of detection of flaw i, ai is the flaw size and α & β are
constant parameters. In order to find α & β, the range interval method (RIM) or
maximum likelihood estimator (MLE) method can be used. Whilst both RIM and
MLE can be used with the log-odds model, only MLE is recommended for use with
the ‘log-normal’ model (as presented by Bullock et al. 1994, and in US DoD 2007).

RIM may also be known as regression analysis [see Bullock et al. 1994]. Bullock et al
describe RIM to good effect, and to summarise, the range interval method places the
flaw sizes into intervals of equal length. Consider the hit/ miss data for cracks of
similar lengths, but possibly varying widths. At this point, all crack lengths within a
length interval end up being grouped together. For each interval, the probability is
calculated as the ratio of detected flaws to total possible flaw detections. A plot using
equation (2) is produced, where pi is the proportion of flaws detected, ai is the flaw
size and i is the interval size (either the mid-pint value or endpoint value of the
interval).

pi
Yi = ln( ) , Xi = ln(ai) (2)
1 – pi

The data points on the graph are fitted with a linear regression line constructed using
equation (3).

Y = α + βX (3)

The values of α + β from regression line can then be substituted into equation (1), and
the POD curve produced. However, the POD of a given flaw size is in fact an
estimation based on the detection data from cracks of similar length but possibly
varying widths – the curve is therefore suggesting that for cracks within a length
range or interval, the POD will not be affected by crack width

66
The limitations of POD models can now be illustrated without delving further into the
literature surrounding MLE methods, the ‘log-normal’ model [Bullock et al. 1994;,
Fahr et al. 1993; US DOD 2007; Georgiou 2007], or statistical treatments for 95%
confidence bounds [Berens & Hovey 1982; Spencer et al. 1993]. There are two
reasons for not presenting the mathematical details of MLE or 95% confidence
bounds. The first being that the MLE is only used to estimate α + β, with equation (1)
still used to construct the POD curve. The second reason being that it is adequate to
state that, similar to equation (1), the ‘log-normal’ model and the mathematical
formulae for the 95% confidence calculations are all based on ai, i.e. ‘flaw’ size [see
Bullock et al. 1994]. For currently published POD analysis methods, ‘flaw’ size has
never needed to be more descriptive than ‘crack length’, and in fact many publications
actually refer solely to ‘crack length’ [see Bullock et al. 1994 for an excellent
example]. Crack width, i.e. a second variable, is not represented in any of the
mathematical formulae – all the currently available models calculate POD as a
function of only one variable. The POD model of equation (1) is therefore inherently
single variable.

Furthermore, both the ‘log-odds’ and ‘log-normal’ models, due to their logarithmic
basis, can only ever produce a positive or negative correlation between POD and flaw
size. Whilst different methods can be used to find α + β, the POD functions
recommended for analysis of NDT reliability data [Spencer et al. 1993; AGARD-
LS190; US DOD 2007], are unable to model an inspection procedure in which flaw
detectability may be demonstrated to both increase and decrease with increasing
values of a given flaw size variable.

2.5.10 – Multi-variate POD

Volker et al. published a three dimensional POD surface that was calculated using
both the width and depths of the flaws [Volker et al. 2004]. Whilst representing a
departure from typical two-dimensional POD curves, it is also noticeable that despite
assessing ultrasonic NDT of metallic pipes, Volker et al. refer to pitting corrosion in
terms of ‘defect’ width and ‘defect’ depth rather than using the traditional reference to
‘crack length’. The POD data presented by Volker et al. were calculated using Annis’
GLM or generalised linear model [Annis 2008]. However, GLM has the same single-

67
variate limitation as aforementioned POD methods, and. Volker et al. were only able
to produce a multivariate plot because they calculated POD for different defect depths
at constant defect widths and POD for different widths at constant depths. The results
presented by Volker et al. also demonstrate only positive correlations with flaw size
and POD.

Whilst the method employed by Volker et al is technically feasible, it requires vast

numbers of flaws in order to estimate POD for a large range of widths and depths,
which Volker et al. were able to replicate using a simulated specimens. However, an
inspection process that cannot be replicated by simulated or virtual specimens would
require actual flaws to be produced. In order for a POD model to be truly multi-
variate, it must be able to give POD as a function of more than one flaw size
parameter, and to date, no such model has been published.

2.6 – Overall summary of literature

New aircraft designs are set to enter service containing 50% composite materials by
weight and utilising monolithic carbon fibre reinforced plastic (CFRP) laminates for
fuselage skins. CFRP laminates are particularly susceptible to reduction in
compression after impact (CAI) strength due to impact damage, and the environment
in which aircraft operate means that impact damage can occur from a multitude of
sources. Examples of the surface appearance of impact damage to actual composite
aircraft structures remain proprietary information. The available literature concerning
impact damage to composite materials has a strong focus on impact testing using
hemispherical impactors, usually Ø15 mm, Ø20 mm or Ø25 mm in size, and provides
only limited information, due to the lack of metal mesh & paint layers on test
specimens. There are no published studies of impact indentation1 topography for
monolithic, fully finished CFRP laminates.

A number of non-destructive testing/ inspection (NDT/ NDI) methods are available

for detection and characterisation of impact damage to CFRP laminates. However,
despite the availability of advanced NDT/ NDI methods, human visual inspection

1
The term ‘flaw’ is used consistently throughout reliability assessment, NDT/ NDI & POD literature
to refer to any form of damage or defect, be it a crack, a dent or delamination. Therefore, to provide
consistency with literary terms, when describing the 3D surface topography of impact damage to
composite laminates, the terms ‘indentation’ or ‘dent’ will hereafter be substituted by the term ‘flaw’.

68
remains an efficient method for scanning large areas of structure, and is thus likely to
remain first line of damage detection for composite materials. There is, however, a
lack of statistically valid information on reliability of visual inspection for impact
damage on CRFP aircraft structures in the public domain.

Damage tolerance principles for commercial aircraft can be summarised as requiring

that aircraft structures demonstrate continuous airworthiness for damage below barely
visible impact damage (BVID) sizes, or inspection procedures be implemented at
appropriate intervals to detect damage before it compromises airworthiness. There is
little published evidence to suggest that the size of impact damage that can be reliably
detected by visual inspection on an actual CFRP aircraft structure is the same as the
BVID sizes that are used to calculate damage tolerance sizes for CFRP aircraft
structures. A focal issue for the Civil Aviation Authority (CAA) is whether reliability
of visual inspection of composite aircraft structures is adversely affected by the
surface colours and surface finishes used on aircraft in airline service.

Reliability studies for NDT processes generally culminate in a set of probability of

detection (POD) curves, which quantitatively model the process reliability as a
function of a single flaw size variable. Impact damage dents on a composite laminate
are 3D in shape, and require multiple variables to describe their topography, all of
which may affect their detectability, i.e. POD for 3D flaws is unlikely be accurately
modelled solely as a function of a single variable. The term ‘flaw size’ in POD
literature has become invariably synonymous with ‘crack length’. The issue of crack
width has been conveniently allayed, and a ‘bigger is better’ mindset prevails in NDT
& POD literature. Furthermore, widely accepted models for POD, due to their
logarithmic basis, cannot model an inspection method whereby detectability
increases, then perhaps reduces with an increase in flaw size.

Surface flaw topography, the surface colour, the surface finish of a structure and the
available lighting can all affect the saliency of visual perception cues such as shading,
specular reflections and specular highlights. Saliency of visual cues is likely to affect
the visual detectability of 3D impact damage dents, thus affecting visual inspection
reliability for composite aircraft structures. The available literature on visual
perception contradicts a ‘bigger is better’ assumption for 3D dent detectability.

69
Chapter 3 – Experimental Design
3.1 – Experimental aims & objectives
The experimental aim was to make measurements of the reliability of visual
inspection for impact damage on CFRP aircraft structures. The experimental
objectives were to explore the effects flaw size, surface colour and surface finish on
visual inspection reliability.

3.2 – Examples of impact damage to CFRP laminates

The literature search identified a lack of examples of the visual appearance and
surface topography of impact damage to composite aircraft structures. A series of
experiments were performed in order to produce examples of different shapes and
energies of impact damage on two thicknesses of fully finished and unfinished CFRP
laminate. Different shaped tips were used in order to provide information on the
impact dent topographies produced by different shapes of striking object. Alongside
the traditional Ø20 mm hemispherical impact tip, a larger radii hemispherical tip, a
flat cylindrical tip, and a wedge or ‘tool’ shape tip were also used. The experimental
methods for the production of the impact dents are reported in chapter 4.3. Ultrasonic
C-scans, as described in chapter 4.2 & 4.4 were made of all test coupons in order to
characterise the delamination damage.

After observing the dents produced by the impact experiments, it was unclear which
flaw characteristic would affect the detectability of such impact dents, i.e. depth,
width or other. It was also noted that the circular dents from the hemispherical tips
were different in shape to the indentations produce by the other impactor tip shapes.
The possible range of impact object shapes likely to be encountered by an aircraft is
vast, and a decision was therefore taken to limit further study to impact dents from
hemispherical tips similar to those used in standard impact tests on fibre reinforced
materials.

A significant issue when describing the impact dents was from hemispherical objects
was that neither impact dent depth and dent width could provide a quantitative
description of dent shape. Metallographic sectioning methods, as described in chapter

70
4.5.6 were employed in order to obtain cross sectional views through a selection of
impact dents. These sections revealed that the impact dents had a common sectional
profile shape, as described in chapter 5.1.4. A geometric model was developed, as
described in chapter 4.6 in order to describe this sectional profile in quantitative
terms.

In order to quantify the surface topography of the hemispherical impact dents, the
values used to construct the geometric model of the dents were derived from touch-
trigger probe scans of the actual impact dents, as described in chapters 4.5.5 & 4.6.

Under visual observation, the impact dents from 20 mm tips appeared to produce
crisper, more discernable shadows, sharper shading gradients and more discernable
highlight distortions than those from the 87 mm tips. After studying available theories
of visual perception, it was suggested that the size of the shadow cast by the impact
dents might be linked to their detectability. Furthermore, it was suggested that dent
detectability might be affected by the “severity” of the flaw shape, i.e. whether the
dents were “hard-edged” (sectional profiles with small radii) or “blended edged”
(sectional profiles with large radii).

3.3 – Initial visual inspection trials with virtual specimens

To test the suggestions regarding dent shape and detectability using actual dents on
physical specimens would have proven time and cost prohibitive. As a means of
quickly testing the above suggestions and providing a means of assessing the possible
detectability of different combinations of dent widths and depths without large
financial outlay, a series of experiments using projected images of computer produced
virtual specimens was devised, as described in chapter 4.11. The series of four virtual
trials was performed back-to-back in sequence using a single group of novice
participants whom were given adequate briefing as described in chapter 4.13.
Inspection time was controlled, as described in chapter 4.11, by means of an
automated display sequence. Viewing angle, by nature of the specimens being two-
dimensional (as described in chapter 4.10), was controlled although head movements
were not controlled. Participant related factors i.e. search/ eye scanning method,
eyesight, contrast sensitivity, training/ experience, fatigue level were neither

71
controlled nor measured. Environmental factors i.e. temperature and noise were
uncontrolled and not measured as it was felt that their levels were within reasonable
limits hence their effect on detection results would be negligible.

The first virtual specimen trial was designed to assess the detectability of different
widths of flaw of the same depth. As described in chapter 4.10, the width and depth of
each flaw was controlled, and by replicating the same simulated lighting conditions,
the computer rendering of each flaw produced shadows that varied in areal size in
direct relation to flaw width. Analysis of the detectability results obtained from this
experiment, as described in chapter 5.4, suggested that the shadow cue produced by
flaws greater than 6.5 mm wide was sufficient in size for at least 50% of participants
to detect.

The second and third virtual specimen trials used specimens containing images of the
shadow produced by three widths of flaws of differing depths, chosen as the 50%,
70% and 90% detectable widths from trial 1, as described in chapter 5.4. The shape of
the flaws was altered to a hemispherical shape, in order to be more representative of
the impact dents seen on CFRP coupons, as opposed to simple cylindrical cut-out
shape flaws as used in trial 1.

Trial 2, as described in chapter 4.10 was used to provide preliminary information on

how changes in dent depth and resultant changes in shadow cue size might affect dent
detectability. The results of the second virtual trial suggested that the shadow cue
produced by hemispherical shaped flaws 0.15 mm deep were detectable by at least
50% of participants.

By adding a radius to the edge of the flaws used in trial 2, the specimens in trial 3 (see
chapter 4.10) were used to test for discernable differences between the detectability of
similarly sized hard-edged and blended-edged flaws. The results of the third trial
suggested that a blended edge could reduce the detectability of hemispherical dents
for similar depths and widths, although the reduction could not be quantified.

In order to quantify possible differences between detectability of hard-edged and

blended-edged dents, a fourth trial was designed in which both hard and blended edge

72
flaws of the same size were inspected within the same trial. The purpose of trial 4 was
to determine whether, for the same participant under the same conditions, a blended
edge affected the detectability of dents of the same width and depth. The dents
represented in trial 4 varied in both width and depth, as described in chapter 4.10,
providing a range of sizes for assessment of the effect of edge blending. The results of
trial 4 demonstrated that reductions in dent detectability due to a blended dent profile
versus a hard-edged profile could be expected for larger width (>15 mm) flaws at a
constant depth.

The first series of virtual trials provided information from which “engineering
judgments” were made concerning the possible range of sizes to be replicated in
future visual inspection trials for CFRP impact damage dents. The general summary
was that a minimum dent width of 5 mm should be used, and that the blended edge
observed on actual impact dents had highly significant detectability implications,
particularly for wide (>15 mm) and shallow (<0.25 mm) dents, and needed to be
authentically replicated in future trial specimens.

An issue of concern following the completion of the virtual trials was whether the
detection results obtained from virtual specimens were comparable to inspection of an
actual physical specimen.

3.4 – Reproduction of impact dents as facsimile specimens

The initial series of virtual trials identified that replication of the blended dent shape
typical of hemispherical impacts to painted CFRP on test specimens would be
essential to obtaining valid visual inspection results. However, facsimile specimens
were the only viable option for this study. A method was developed by the author and
implemented by M&M patterns of Wavendon, Milton Keynes, whereby circular dents
with specified geometries were NC machined & polished into Plexiglas panels in
order to produce representations of impact dents to painted CFRP laminate. The NC
data for the sectional profile of the machined circular dents was produced as 3D CAD
data by revolving a section line that was produced using the same geometric variables
as used to describe the impact dents on CFRP laminates. Hence, the same blended
edges that were observed on actual dents, and understood to be important to visual

73
inspection reliability, could be reproduced in the same proportions on facsimile
specimens.

Reproducing accurate sectional profiles was a key requirement for the facsimile
specimens, in particular retaining the proportions, or relationships between geometry
variables so as to accurately represent either an Ø20 mm or Ø87 mm impact to a 17 or
33 ply laminate. Whilst it would have been possible to reproduce CMM data of
impact dents described in section 4.2 & 5.1, the controlled factor in producing these
dents was impact energy. Flaw sizes and geometries in these sets of dents were not
controlled and as a result there was uncontrolled distribution between the values of
the geometric variables within the dent shapes. To allow the effects of dent depth and
width on flaw detectability to be investigated, it was preferable to distribute the flaw
sizes by either width or depth. However, it was unfeasible to attempt to obtain flaw
geometry data for specified dent depths or widths due to the difficulty in guaranteeing
that falling weight impacts would produce exact dent sizes and shapes, across a
specified size range.

As a solution to the problem of obtaining dent geometry data for given dent sizes, the
geometry data obtained from CMM data of the examples of actual dents produced by
Ø20 mm and Ø87 mm impacts to 17 & 33 ply painted CFRP were plotted, as
described in chapter 5.1.6. Using these plots, relationships were derived to interlink
impact energy to the geometric variables used to describe dent sectional profiles. For
each impact tip and laminate combination, discrete relationships were identified.
These relationships facilitated the calculation impact energy and each of the
geometric variables required for reconstruction of the sectional profile of a dent,
based on a single variable, be it depth, width or impact energy. Thus, for Ø20 & Ø87
mm hemispherical impacts to 17 & 33ply painted CFRP laminates, it became possible
to choose a range of dent sizes, and reconstruct the sectional profile of each size of
dent.

The sectional profile reconstructions were used to build 3D CAD models representing
a range of sizes of impact dents, which were then forwarded to M&M Patterns for NC
machining. Matching the values of the geometric variables in the 3D CAD model to
those calculated using the relationships from data of actual impact dents ensured the

74
sectional profiles of the facsimile dents were accurate representations of impact
damage to a CFRP laminate.

3.5 – Visual inspection trials with physical specimens

Having developed a repeatable method for producing representations of impact
damage to CFRP laminates, facsimile specimen panels were produced for use in
visual inspection trials with physical specimens, as described in chapter 4.9 & 4.10.5.

In order to investigate the effects of colour, it was decided that examples of impact
damage should be replicated on three different colours of specimens. Grey, dark blue
and white were chosen as colours typical to those seen in aerospace applications. It
was also foreseen that the prominence, or saliency of shadow and shading cues for
visual perception (and hence detectability of flaws) could differ across these colours
due to white being a “light” base colour, grey a “medium” base colour and blue a
“dark” base colour. In order to investigate the effect of differences in surface finish on
flaw detectability, it was decided that examples of impact damage (flaws) should be
replicated on both gloss and matt finishes in each colour.

The experimental method for physical specimen trials, as described in chapter 4.12,
was a development of that used for virtual specimen trials. A 20-second interval
between each specimen display period was introduced in order to allow panel
changeover, although the display time of 5 seconds was retained. A trial duration limit
of 20~30 minutes was imposed to avoid participant fatigue, leading to 64 specimens
being chosen as a convenient number of specimens that allowed incorporation of a 3:1
unflawed to flawed specimen panel ratio within the time limit. Thus, 16 of the 64
specimen panels contained between one and three of 32 machined impact dent
facsimiles. The variation in numbers of dents per panel ensured that participants did
not always expect to find a given number of dents on a flawed specimen. The
distribution of flaw sizes across the flawed panels was randomised and the flaw
locations on each panel were also randomised, as described in 4.12. To reduce the
possibility of participants learning where and when flaws might be present, the
display sequence of flawed and unflawed specimens was randomised. To further
guard against specimen familiarisation for participants undertaking more than one

75
trial, the randomised specimen display sequence was alternated between forward and
reverse.

Novice, paid, participants were used and participant briefing (4.13) included examples
of the dents to be detected on the specimen panels. Some of the participants had
previously undertaken the first series of trials with virtual specimens. Several
participants undertook trials with more than one colour/ finish specimens, although
there was no particular sequence due to trials being performed on different days in
different weeks, as per facility, personnel and participant availability. Personal data
such as gender, age, eyesight ability, colour-blindness, previous experience or current
level of fitness/ well being was not recorded. The participants were seated in a chair
placed at a marked distance of 1.2 from the screen. Head movements were allowed
during the visual inspection task, which meant that some participants would have
obtained views of the specimen panels from different visual angles. It was accepted
that some advantage may have been gained by this practice, and it was felt that
physically constraining participant head movements may have deterred participation.
Standing up or reaching forward to touch the specimens was not permitted, as doing
so would have altered the mode of inspection from visual to tactile.

The visual inspection trials with physical specimens were performed in a meeting
room, with ambient lighting provided by external windows and overhead fluorescent
luminaires. Light levels were assumed to be constant and were neither measured nor
controlled. Temperature and humidity were not recorded and were not controlled.

The results of the trials with physical specimens are presented in chapter 5.5.1.

3.6 – Second series of trials with virtual specimens

The first series of trials with virtual specimens proved to be a valuable tool for
generating visual detection data on a cost and time efficient scale, with minimal
logistical requirements compared to use of physical specimens. However, due to
differences in the dent geometries used and variations in the experimental method, a
like for like comparison between the virtual and physical specimen data was not
possible. A second series of virtual trials was implemented in order to identify

76
whether detection results obtained with virtual specimens were comparable to those
obtained with physical specimens. Grey specimens were used in order to provide
comparison with the grey physical specimens. The ability of a grey surface to exhibit
both shadow and shine also helped mitigate possible bias due to increased or reduced
shadow or highlight cue saliency on lighter or darker colours. The methodology for
these trials is described in 4.12. The same display timings and sequences as the
physical specimen trials were retained, the only difference between the trials being the
lack of physical specimens and the reduction of ambient lighting in order to improve
display clarity.

Some of the participants in the second series of virtual trials undertook physical
specimen trials, although the exact sequence in which individual participants
undertook trials was not recorded. Display brightness was not recorded, although the
display size of the images was checked using a rule in order to ensure they were being
displayed at actual 600 mm x 600 mm size.

The detection results from the virtual specimen trials differed to the detection results
obtained with physical specimens, suggesting that visual inspection reliability data
obtained with virtual specimen was not directly comparable to inspection of physical
specimens, although similar detection vs. dent size trends were identified within both
sets of results.

77
Chapter 4 – Experimental Methods
4.1 – Production of fully finished CFRP laminate samples
In order to replicate a typical CFRP aircraft structure, laminates were produced at
Cranfield University from similar materials to those specified in the Structural Repair
Manual for an Airbus A340 horizontal stabiliser skin [Airbus SRM 2003]. The
materials were as follows:

• Hexcel AS4/8552 pre-preg Carbon Fibre Epoxy Pre-Preg Carbon Tape (UD)
• Hexcel P0108 - 8552/42%/120 (Pre-Preg Glass Fibre 120 Style, 120g/m², 4H
Satin)
• Aeroconsultants Bronze Mesh, 166 threads/in [See Bopp 2006]

Four 750 mm x 1000 mm panels of carbon fibre (CFRP) laminate, two 17ply and two
33ply, were produced at Cranfield University by laying up 300 mm wide Hexcel
AS4/8552 pre-preg tape as follows:

17ply - [+45,-45,0,-45,+45,90,-45,+45,0,+45,-45,90,+45,-45,0,-45,+45]

33ply - [+45,-45,0,-45,+45,90,-45,+45,0,+45,-45,0,-45,+45,+45,-45,90,
-45,+45,+45,-45,0,-45,+45,0,+45,-45,90,+45,-45,0,-45,+45]

The lay-up was de-bulked every fifth ply. The lay-up for one of the 17ply and one of
the 33ply panels included a single layer of glass fabric and a single layer of bronze
mesh on one side of the panel before curing. The CFRP lay-up, glass fabric and
bronze mesh were co-cured at 180ºC according to the manufacturers recommended
cure cycle for the 8552 resin [Hexcel 2003]. For the panels containing the lightning
strike protection (LSP) layers, the mesh incorporating side of the panels were painted
by GKN Aerospace with primer and paint according to Airbus specifications [Airbus
Oct 2005, Dec 2005 & Mar 2006]. The completed painted and unpainted panels were cut
into 100 mm x 150 mm coupons for impact testing. The coupons were labelled (on the
unpainted side for fully finished laminate) with a self-adhesive label, which was
numbered with a permanent marker pen. Water immersion ultrasonic C-Scans of each
coupon were made in order to identify, and remove from the specimen set, any
coupons containing voids within the laminate. Samples of undamaged, painted LSP
laminate from a panel of each thickness were prepared using metallographic
sectioning methods, and were used to confirm that the ply-lay up sequence was

78
correct. Figure 4.01 shows a section through the laminate, which is free of significant
voids or resin rich areas.

Figure 4.01 – Section through 33ply laminate with painted surface to the
top

4.2 – Ultrasonic C-Scanning

The equipment used for C-Scans was a Structural Diagnostics Inc. water immersion
ultrasonic scan system coupled with an ultrasonic flaw detector [Baugh & Weedon]
and a 386 personal computer running Windows 3.11. All scans were made using the
SDI SCAN-4 program [Richards Computer Products 1991] with a scanning pitch of
0.25 mm x 0.25 mm thus achieving an acceptable compromise between scan
resolution and scan time.

4.3 - Falling weight impact damage

A Rosand Type 5 falling weight impact tester with interchangeable tips was used to
create impact damage on the coupons. For impact testing, the coupons were placed
into an open aperture support fixture conforming to Boeing BSS7260 [Boeing 1988].
The test conditions for each of the coupons are given in table 4.01. The impact energy
for each impacted coupon was written onto the coupon identification label.

As detailed in Table 4.01, four different types of stainless steel impact tips were used.
The Ø20 mm hemispherical impact tip is typical of that used in published examples of
impact damage and CAI testing of composite materials. The Ø87 mm impact tip is a
larger variant of the hemispherical shape. The flat cylindrical tip was included in
order to provide information on the impact flaws produced by such shapes in
comparison to the hemispherical shaped tips. Likewise, the wedge (or tool) shape

79
impactor was included in order to assess the damage morphology that impacts from
items such as tools might cause. As reported by Cartié & Irving, 2002, impact
energies below 5J are unlikely to produce subsurface delamination, thus the 5J lower
limits of impact energy. Table 4.01 provides details of sample numbers, impact
energies and impact tip type for each of the tested coupons.
Impactor Type

Ø70 mm Flat Cylinder

Ø20 mm Hemispherical Ø87 mm Hemispherical 13 mm x 4 mm

Wedge/ Chisel
Carriage Mass – 2.030kg Carriage Mass – 2.030kg Carriage Mass – 2.030kg Carriage Mass – 2.030kg
Total Mass – 2.172kg Total Mass - 2.431kg Total Mass - 2.960kg Total Mass - 2.054kg
17ply LSP & Painted CFRP coupons
Coupon # Impact Coupon # Impact Coupon # Impact Coupon # Impact
Energy Energy Energy Energy
27 5J 95 5J 36 10J 34 5J
26 10J 94 10J 37 15J 33 10J
25 15J 93 15J 38 20J 32 15J
24 20J 29 20J 40 30J 35 17.5J
23 30J 92 30J 39 40J 31 20J
30 40J
90 50J
17ply non finished CFRP Coupons
Coupon # Impact Coupon # Impact Coupon # Impact Coupon # Impact
Energy Energy Energy Energy
U13 5J U18 5J U25 10J U30 5J
U14 10J U19 10J U26 15J U31 10J
U15 15J U20 15J U27 20J U32 15J
U16 20J U21 20J U28 30J U34 17.5J
U17 30J U22 30J U29 40J U35 20J
U23 40J
U24 50J
33ply LSP & Painted CFRP Coupons
Coupon # Impact Coupon # Impact Coupon # Impact Coupon # Impact
Energy Energy Energy Energy
2 5J 9 20J 18 10J 14 20J
3 10J 8 30J 19 20J 15 30J
4 15J 10 40J 20 30J 16 40J
5 20J 11 50J 21 40J 17 50J
6 30J 12 60J 1 50J
7 50J 13 70J 70 60J
33ply non finished CFRP Coupons
Coupon # Impact Coupon # Impact Coupon # Impact Coupon # Impact
Energy Energy Energy Energy
U38 5J U44 20J U50 10J U56 20J
U39 10J U45 30J U51 20J U57 30J
U40 15J U46 40J U52 30J U58 40J
U41 20J U47 50J U53 40J U59 50J
U42 30J U48 60J U54 50J
U43 50J U49 70J U55 60J
Table 4.01 – Testing matrix for impact testing of CFRP coupons

80
4.4 – Measurement of delamination
Each coupon was C-Scanned after impact, in order to measure the sub-surface
delamination damage at the impact site. Images of the C-Scan data were produced as
bitmap format screen captures from the ANALYSIS program [Richards Computer
Products 1991]. The bitmap images were fitted with a 10 mm grid using suitable
image processing software, and used to measure to the width of the sub-surface
damage (delamination) to the nearest 1 mm. By using a square count method, or
where possible, an area calculation based on measurements taken from the image, the
area of the delamination can be measured. Figure 4.02 illustrates a typical C-Scan
image of an impact-damaged coupon.

Figure 4.02 – C-Scan of 40J impact damage from Ø87 mm on

33ply painted, laminate with bronze mesh

4.5 – Measurements of impact damage surface topography

4.5.1 – Width measurement with Vernier callipers
The width of the surface flaws on all of the coupons was first measured approximately
using a vernier calliper device, to the nearest 0.02 mm.

81
4.5.2 – Depth measurements with a depth gauge
For all coupons, the surface flaw depth was measured through the centre of each flaw,
with the coupon placed atop an open cylinder. The open cylinder allowed the coupon
to sit flat despite the presence of back-face fibre break out. The depth measurements
were made using a mechanical dial gauge (0.01 mm accuracy) mounted to a stand, as
illustrated in figure 4.04. An arbitrary point on an unaffected area of surface, just
outside of the area affected by the surface flaw, was used as the origin.

Figure 4.04 – Dial gauge measurement of surface flaw

depth

4.5.3 – Depth gauge profiles

Depth measurements were made across the 100 mm width of the surface of
hemispherical impact damaged, fully finished coupons, through the centre of the
surface flaw, using a dial gauge (0.01 mm accuracy) and a sliding track (see figure
4.05). Depth measurements were taken at 5 mm intervals outside of the surface flaw
region (as determined from approximate vernier measurements), and at 1 mm
intervals within the flawed area. The coupons were mounted to the slider by pressing
them down onto lumps of putty, thus affording a stable mount and clearance room for
back-face fibre breakouts. Plots of depth vs. track distance gave representations of the
sectional profiles of the surface flaws. Figure 4.06 illustrates an example of a 30J,
Ø20 mm impact on a 17ply painted laminate. The advantage of this method of data

82
representation is the ability to stretch the Y-Scale in order to better observe the
surface flaw shape.

Dial Gauge
0.01 mm Accuracy

Test Coupon

Sliding Track
1 mm Markings

Figure 4.05 – Dial gauge setup for measuring vertical depth vs.
horizontal distance.

1.9

1.4
Depth (mm)

0.9

0.4

-0.1 0 10 20 30 40 50 60 70 80 90 100
x (mm)

Figure 4.06 – example of graph representing section through surface

flaw, produced from depth gauge measurements.

83
4.5.4 – CMM (Coordinate Measuring Machine) coupon surface digitisation
A Renishaw Cyclone Series 2 touch-trigger probe CMM machine was used to
produce digitised traces of hemispherical impact flawed, fully finished coupon
surfaces using three dimensional data points with X, Y and Z coordinates (3D points
cloud). The machine was set to scan with a pitch of 1 mm by 1 mm in the X & Y
axes, with ±1µm accuracy in the Z-axis. The scanning area was 80 mm x 80 mm in
size, centred approximately on the surface flaw on each coupon. The point cloud data
was saved as IGES data, and imported into a computer aided design (CAD) file using
CATIA. The CATIA system was used to produce section lines through the points
cloud in a Z/Y plane through the transverse (100 mm) section of each coupon.

4.5.5 – Width measurements from depth gauge & CMM data

In order to measure the surface flaws caused by hemispherical tip impacts, using the
CMM data, the series of points representing the surface flaw centre, and a series of
points on the coupon surface close to the flaw area were identified within the CMM
point cloud data (see figure 4.07) by producing section lines through the points cloud.
Once these two sections were isolated within the CMM points cloud, the other section
lines were removed leaving only two section lines as seen in figure 4.07 & 4.08.

Figure 4.07 – Section line positions on CMM data points cloud

Vertical (Z-axis) depth measurements were made every 1 mm along the line (figure
4.08). The depth measurements were made using a second section line, made through
the coupon surface outside of the flaw area, as the origin.

84
 Figure 4.08 – section lines obtained from 2.11 mm deep surface flaw
using CMM machine, vertical data lines are 1 mm spaced

The depth measurements from the CMM data were also plotted using Excel, and by
placing them alongside the depth gauge measurements and subsurface damage width
measurement from the C-Scans of each coupon, a clear picture of the damage
morphology could be obtained (See Boulic, 2007). An example is given in figure
4.09.

Figure 4.09 – typical graphical representation of depth gauge, CMM and C-Scan
measurements of impact damage and surface flaw on a coupon.

The approximate flaw width measurements made using Vernier callipers were used to
mark the estimated flaw endpoint regions on the surface flaw profiles. In these
regions, the unflawed surface line was drawn by extending a line of best fit through
data points that were known to represent an unflawed region. Using the vernier
measurement regions as a guide, the first of three points, running towards the flaw

85
centre, in the marked region, that were consecutively lower than the unflawed surface
line were identified. With two of such points identified on each side of the centre of
the flaw, the flaw endpoints were marked. Flaw width was thus obtained by taking the
distance between the two endpoints.

4.5.6 – Metallographic sectioning technique

Selected fully finished, impact damaged coupons were cut into a 30 mm square, with
the centre of the square placed at the centre of each surface flaw. The squares were set
into resin, ground back and polished so that the section ran through the centre of the
surface flaw.

30 mm

Figure 4.10 – Section through 50J, Ø20 mm impact on 33ply painted, mesh
covered laminate

A microscope fitted with a digital camera was used to produce multiple small digital
images of the finished sections. The 3 mm x 3 mm images were then combined, and
arranged in the correct sequence to produce a composite image (figure 4.10) which
provides a high-resolution photographic view through the 30 mm section of damaged
laminate.

4.6 – Reconstruction of surface flaws using geometric lines & arcs

The sectional profiles of hemispherical surface flaws were recreated in a CATIA
CAD system, by overlaying a set of geometric lines onto the section lines produced
by the CATIA system through the CMM points cloud for the surface flaws. The
section lines created by the CATIA system are splines, and are non-parametric, i.e.
the shape of the section line is simply a collection of short, 2D lines running from
point to point much like a dot-to-dot picture. New geometric lines with lengths and
radii were overlaid to match the shape of the non-parametric splines, as seen in figure
4.11.

86
 Figure 4.11 – Construction of geometric lines to match CMM data section lines

In order to ensure a smooth transition between the arcs used to reproduce the surface
flaw section and the line representing original flat surface, without a visible join line,
the tangency at the end of arc rE where it meets the planar surface was set to be
horizontal (i.e. the same as the surface). To ensure symmetry, and not create a V
shape at the centre of the flaw, the end of arc tangency of rI at the flaw centre line was
constrained to be horizontal. A final tangency constraint was placed between the join
of arcs rE and rI, in order to ensure a smooth, step-less transition between the arcs.
These tangencies can be seen in figure 4.11.

The geometric variables of the lines were adjusted until the geometric line fitted as
best as possible to the CATIA section splines through the CMM points data (see
figure 4.11). From the new geometric fitted lines and arcs, it was possible to derive
accurate values for the geometric variables used to describe the flaw shapes. The
values that could be derived from this method are illustrated in figure 4.12.

Whilst actual impact dents may have had varying sectional profiles across the width
of the dent, the characterisation data represent only the deepest section line, i.e. the
central section line though each impact dent.

87
 θc = Cone angle of complete R = Surface flaw radius on surface
surface flaw
d = Surface flaw depth

rI = Internal radius of surface flaw,

θt = Angle between vertical and
rE = Edge radius of surface flaw
tangent of internal & edge radii
Xt = X coordinate of tangent point
between internal & edge radii

Yt = Y coordinate of tangent point

between internal & edge radii

Figure 4.12 – Geometric variables used to reconstruct a surface flaw section

4.7 – Reproduction of impact flaws as 3D CAD data

Representations of flaws were created using the Solid Works CAD program
[SolidWorks 2000] by revolving geometric section lines around a vertical centre axis
as illustrated in figure 4.13. The 3D revolved geometry was cut away from top surface
of a 3D model of a flat panel, leaving a circular surface flaw on the panel surface.

88
 Figure 4.13 – Revolution of surface flaw section line to
produce 3D flaw surface

4.8 – Production of virtual facsimile specimens

Sets of virtual specimens were produced by photo-realistically rendering 3D Solid
Works [SolidWorks 2000] CAD models of panels containing 3D surface flaws. The
2500 pixel x 2500 pixel rendered images were saved as bitmap files. The photo-
realistic images of each specimen represented an observer’s view normal to the
inspection surface of the panel. A gloss grey finish was replicated, and the simulated
lighting direction was at 30 º from the vertical plane and 30 º from the plane normal to
the surface (i.e. facing across, and down on to the panel, from the top right hand
corner to the bottom left). The rendering parameters can be found in appendix A.
Figure 4.14 illustrates an example of a virtual surface flaw that represents impact
damage to a fully finished CFRP laminate, as used in the second series of virtual
trials.

89
 30 mm

Figure 4.14 – Virtual specimen flaw, representing Ø20

mm/33ply 46J impact damage (Ø29.96 mm / 0.57 mm deep)

4.9 – Production of physical facsimile specimens

The 3D CAD geometry of each surface flaw was NC machined, in specified locations,
into 3 mm thick Plexiglas panels, and the surface flaw area was polished to return the
flaw site to its original surface finish quality. Figure 4.15 illustrates a test piece that
contains 12 surface flaws produced in this way, and can be compared with figure 4.16
which shows a photograph of an actual impact damage surface flaw on a
painted/mesh laminate. The lighter area observed across the image in figure 4.15 and
in the upper right quarter of figure 4.16 is the reflection of a lighting luminaire. Three
specimen sets were produced by machining surface flaws into grey, white and blue
Plexiglas panels, thus providing different coloured sets of panels without the need for
painting.

90
 300 mm

Figure 4.15 – Test piece containing machined surface flaws

150 mm

Figure 4.16 – Impact damage surface flaw on a

painted/ bronze mesh CFRP laminate

4.9.1 – Application of matt finish to physical specimens

Upon completion of visual inspection trials with the gloss grey, white and blue panels,
the flawed specimens received a thin coat of matt finish paint of the same colour. This
allowed their use in reliability assessments of the effect of surface finish on visual
inspection, without the need for new machining work. In order to reduce painting
time, and costs, complete sets of unflawed panels were not produced. Instead, a
smaller number of unflawed panels of each colour were prepared with matt finish, and
were recycled into the specimen set during visual inspection trials in order to provide
the required number of unflawed specimens.

91
4.9.2 – CMM checking of physical specimens
Five selected machined flaws on each colour of specimen underwent CMM
digitisation to check for conformity to the original specification. An example of a
comparison between the CMM data, and the original flaw specification can be seen in
figure 5.23. The CMM data taken from the machined flaw are barely distinguishable
from the specified geometry, meaning that the machined flaw is the correct depth,
correct width and correct, specified flaw profile / shape.

Flaw width measurements are difficult to obtain due to the difficulty in determining
the ‘edge’ of the flaw surface and the exact centre point of the flaw. Instead, three Y-
axis, or surface height, measurements of the CMM data were made using CATIA at
the endpoint, the Xt point, and the centre point of the flaw, as shown in Figure 4.18.
The discrepancy between the actual surface height and the specified surface height at
the designated points was recorded in order to identify deviations from the specified
flaw shape. Measuring surface height deviation at the endpoint of the flaw reveals
whether the flaw has been machined to the correct width. A flaw of the correct width
shows no surface height deviation, i.e. a measured value of 0 at the endpoint
measurement. Measuring the flaw depth against the specified flaw depth reveals
whether the flaw has been machined to the correct specified depth. Flaws that have
been machined to the correct depth show a depth measurement deviation of 0. The
surface height at Xt point of the flaw surface was measured in order to determine that
the flaws had been machined to within acceptable shape tolerances. A surface height
deviation of 0 at the Xt point shows that the transition between the internal and
external radius of the sectional profile of the flaw is in the correct location, and thus
the flaw shape has been correctly machined as specified.

Figure 4.17 - CMM data (pink solid line) and specified geometry (white dashed
line) of Ø22.2 mm/ 0.57 mm deep NC machined surface flaw

92
 Figure 4.18 – Surface height checking points for CMM
data vs. original specification

4.9.3 – Gloss measurements of physical specimens

Novo Gloss dual-angle (20º & 60º) gloss meter measurements of the gloss level of
each specimen colour & finish provided quantitative values for surface glossiness, and
allowed comparisons with an actual aircraft surface finish. A gloss meter measures
the intensity of light reflected by a surface from a light source of known intensity, and
quantifies the amount of reflected light in terms of gloss units. Gloss measurements
range from 0 to 100, with 100 being very glossy, i.e. the surface reflects light with
100% of original intensity. A matt finish surface will diffuse the light reflection,
causing less light intensity to be detected, resulting in a lower gloss reading.

4.9.4 – Paint thickness measurement of physical specimens

For the matt finish facsimile specimens, the thickness of the paint was measured. This
was accomplished by metallographic sectioning of samples of a painted Plexiglas test
piece. Figure 4.19 details the image obtained using a microscope. The measurement
of the paint thickness was made using the microscope’s image processing software.
The thickness was measured in several different places, and was found to be
consistently within 12µm to 20µm.

93
 Figure 4.19 – Section of painted Plexiglas material used
for thickness measurements

4.10 – Flaw and specimen characteristics

4.10.1 – Specimens for virtual trial 1 (series 1)

Fig 4.20
Figure 4.21
The aim of the first trial was to determine the effect of defect width on detectability.
For this, circular flat-bottomed flaws were used, as illustrated in Figure 4.20 & 4.21.

The flaws were sized with widths (R) between Ø1 mm and Ø30 mm in intervals of 1
mm. The depth (d) of all the flaws was 1 mm. The flaws were positioned onto single
750 mm x 750 mm panels, using random coordinates. The coordinates used are
detailed in appendix B. Figure 4.21 illustrates a typical flaw as used in this trial.

The display sequence of the panels was randomised. The 30 panels containing flaws
were intermixed with a further 90 unflawed panels, giving 120 panels in the specimen

94
set. Each panel was displayed onto a white screen for 5 seconds, with an interval of 2
seconds between each panel. The total trial time was approximately 14 minutes.

4.10.2 – Specimens for virtual trial 2 (series 1)

Fig 4.22
Figure 4.23

The aim of the second trial was to determine the effect of flaw depth on detectability.
For this, circular semi-spherical flaws were used, as illustrated in Fig 4.22. The flaw
width (R) and depth (d) were set according to design. The flaw radius (Rd) cannot be
controlled, as it is dependent on R and d. The values were simply recorded. Figure
4.23 details an example of a flaw used in Trial 2.

The flaws were positioned onto 750 mm x 750 mm panels, using random coordinates
produced using the method previously described. A margin of 20 mm from each edge
of the panel, leaving an area of 710 mm x 710 mm for flaw positioning. The
coordinates used are detailed in appendix B.

Three flaw widths were used for Trials 2 & 3. These were chosen as the 50% POD
width, the 70% POD width and the 90% POD width from Trial 1. The reason for
including three flaw widths was to investigate whether wider flaws might in fact be
less visible due to the increases in the flaw radii (Rd). The three sets of flaw widths
were thus Ø4.5 mm, Ø6.5 mm and Ø12 mm. The flaw depths were separated by
intervals of 0.1 mm. For the Ø4.5 mm wide flaws, the depths spanned a range from
0.1 mm deep to 1.5 mm. For the Ø6.5 mm wide flaws, the depths ranged from 0.1 mm
to 1.3 mm deep. The Ø12 mm wide flaws spanned a depth range of 0.1 mm to 1.1
mm. The flaw depth ranges being investigated required a greater number of flaws, and
the trial was subsequently designed to include 39 panels containing flaws, with a
further 117 unflawed panels being included to bring the total number of panels to 156.

95
The specimen and flaw characteristics are detailed in appendix B. The specimen
display order was randomised, as detailed in appendix B. Each panel was displayed
for 5 seconds with 2 seconds interval. The total trial length was approximately 18½
minutes.

4.10.3 – Specimens for virtual trial 3 (series 1)

Figure 4.24
Figure 4.25

The aim of the third trial was to determine whether softening the flaw profile had any
effect on detectability. For this, circular semi-spherical flaws with blended edges were
used, as illustrated in figure 4.24. The flaw width (R) and depth (d) were set according
to design. The flaw radius (Rd) cannot be controlled, as it is dependent on R and d.
The values were simply recorded. The edge radius was set at half that of the flaw
radius, i.e. Rd/2. This provided a constant ratio between the edge blend and the size of
the flaw and avoided introducing further variables. Figure 4.25 details an example of
a flaw used in Trial 3.

The flaws were positioned onto 750 mm x 750 mm panels, using random coordinates
as described in appendix B. The same flaw depths as those used in Trial 2 were used
in Trial 3. The flaw and panel characteristics are detailed in appendix B. The
specimen display order was randomised as detailed in appendix B, with 39 panels
containing flaws out of the total 156. Each panel was displayed for 5 seconds with 2
seconds interval. The total trial length was approximately 18½ minutes.

96
4.10.4 – Specimens for virtual trial 4 (series 1)

Figure 4.26 Figure 4.27

The aim of the fourth trial was to determine the effect of depth and the effect of flaw
shape on detectability. This helped provide information on whether small, acute flaws
gave different detectability to wider, shallower flaws.

For this trial, circular semi-spherical flaws were used, with both a hard edge and a
blended edge, as illustrated in figures 4.26 & 4.27. Using both types of contour
allowed a direct comparison between the detectability of each contour type.

The flaw width (R) and depth (d) were set according to design. The results of Trial 2
indicated that a flaw depth (d) 0.155 mm would give approximately 50% POD.
Therefore, the flaws for Trial 4 were all designed with 0.155 mm depth and their
widths varied from 3 mm to 30 mm. The flaw radius (Rd) cannot be controlled, as it is
dependent on R and d. These values were simply recorded. The edge radius for the
blended edge flaws was set at half that of the flaw radius, i.e. Rd/2. The flaws were
positioned onto 750 mm x 750 mm panels, using random coordinates produced using
the method previously described. The flaw and panel characteristics are detailed in
appendix B.

The specimen set for Trial 4 contained 44 flawed panels and 132 unflawed panels,
giving 176 panels in total. Within the 44 flawed panels, 4 flaw sizes were duplicated
i.e. two panels contained the same flaw size (in different locations). This allowed
checking of the consistency of the participant’s responses. The panels were displayed
for 5 seconds with a 2 seconds interval. The inspection time was approximately 21
minutes.

97
4.10.5 – Specimens for virtual trial 5 (series 2) and trials with facsimile
specimens
A total number of 32 surface flaws conforming to the geometry relationships for the
Ø20 mm and Ø87 mm impactors on 17ply & 33ply laminates, as given in the results
chapter, were designed, thus giving 8 sizes of flaw for each “type” of flaw geometry.
Facsimile specimens of 3 mm thick, 600 mm x 600 mm panels containing the
aforementioned surface flaws were designed as 3D CAD data. Virtual specimens were
created by photo-realistic rendering of the CAD data. Physical specimens were
reproduced by NC machining the surface flaws into Plexiglas (PMMA) [Degussa, 2006]
material.

A single set of virtual specimens was produced in a grey colour. Six sets of facsimile
specimen panels containing NC machined surface flaws were produced for the visual
inspection trials:

• Gloss grey specimens (Plexiglas Grey 812)

• Gloss white specimens (Plexiglas White 003)
• Gloss blue specimens (Plexiglas Blue 601)
• Matt painted grey specimens (grey primer)
• Matt painted white specimens (white primer)
• Matt painted blue specimens (RAL 5002 Ultramarine Blue)
The facsimile specimen panels were labelled on the reverse (non-inspection) side, in
order to identify their panel number, flawed/ unflawed status, surface colour and
surface finish.

In order to set the flaw size range, based on realistic values of impact energy, upper
values of impact energy for each flaw type were set as the maximum energy observed
in coupon testing before the impact flaw exhibited surface cracking or severe
distortion of the entire coupon surface occurs. The lower value of the impact energy
range was set at that, which, using the relationships listed in the results chapter, gave a
flaw depth of 0.1 mm, or a positive value for Xt. Using these impact energy ranges
and the relationship for flaw width as a function of impact energy, a series of 8 flaw
widths, spaced with even width intervals was produced, as shown in table 4.02.

98
 Ø20 mm/17ply Ø20 mm/33ply Ø87 mm/17ply Ø87 mm/33ply

Energy Width Energy Width Energy Width Energy Width

(J) (mm) (J) (mm) (J) (mm) (J) (mm)
7.0 9.35 5.0 7.48 17.5 10.50 16.8 14.00
9.7 11.99 11.4 10.97 27.1 15.46 21.5 18.49
12.4 14.63 17.9 14.46 36.8 20.43 26.1 22.99
15.1 17.26 24.3 17.95 46.4 25.39 30.7 27.49
17.9 19.90 30.7 21.43 56.1 30.36 35.4 31.98
20.6 22.54 37.1 24.92 65.7 35.32 40.0 36.48
23.3 25.18 43.6 28.41 75.4 40.28 44.7 40.98
26.0 27.82 50.0 31.90 85.0 45.25 49.3 45.47
Table 4.02 – Width/ Energy ranges for realistic flaw sizes

Using the width & energy values in table 4.02 and the energy/ depth model described
in the results chapter to calculate flaw depths, a plot of flaw width vs. flaw depth was
produced, as seen in figure 4.28. As can be seen in figure 4.28, using an even width
interval gives 9 flaws greater than 30 mm wide and 6 flaws deeper than 0.6 mm.

Figure 4.28 – Plot of flaw widths vs. flaw depths with even distribution
between flaw widths

As stated in published guidelines on reliability assessment design, “specimens

containing flaws so large that they are always found, and flaws so small that they are

99
always missed”, are not useful to the experiment [AGARD 1993]. Advice on flaw sizes
also cites the “tendency to include too many "large" flaws” in reliability assessments
[AGARD 1993]. In order to reduce the number of very wide or very deep flaws within
the experiment, the size distribution was modified in order to include a greater
number of small to mid width & depth flaws in the specimen set without increasing
the overall number of flaws. The natural log of each of the flaw widths in table 4.02
was calculated, as seen in table 4.03. An interval that would space the natural log
values across a similar range to the original log values was chosen. I.e. if the original
range was 2.235 to 3.326, an interval of 0.15 was used to space new log values across
a range of 2.200 to 3.250. Using the exponential of the evenly spaced log intervals, a
new value was obtained for each flaw width. This resulted in a new set of flaw widths,
which are given in table 4.03.

100
 Ø20 mm/17ply Ø20 mm/33ply
Original New Original New
Width New Width Width New Width
(mm) ln Width Value (mm) (mm) ln Width Value (mm)
9.350 2.235 2.200 9.025 7.480 2.012 2.000 7.389
11.988 2.484 2.350 10.486 10.969 2.395 2.200 9.025
14.626 2.683 2.500 12.182 14.457 2.671 2.400 11.023
17.264 2.849 2.650 14.154 17.946 2.887 2.600 13.464
19.902 2.991 2.800 16.445 21.434 3.065 2.800 16.445
22.540 3.115 2.950 19.106 24.923 3.216 3.000 20.086
25.178 3.226 3.100 22.198 28.411 3.347 3.200 24.533
27.816 3.326 3.250 25.790 31.900 3.463 3.400 29.964
Ø87 mm/17ply Ø87 mm/33ply
Original New Original New
Width New Width Width New Width
(mm) ln Width Value (mm) (mm) ln Width Value (mm)
10.499 2.351 2.400 11.023 13.995 2.639 2.650 14.154
15.463 2.738 2.600 13.464 18.492 2.917 2.800 16.445
20.427 3.017 2.800 16.445 22.989 3.135 2.950 19.106
25.391 3.234 3.000 20.086 27.486 3.314 3.100 22.198
30.356 3.413 3.200 24.533 31.983 3.465 3.250 25.790
35.320 3.564 3.400 29.964 36.480 3.597 3.400 29.964
40.284 3.696 3.600 36.598 40.977 3.713 3.550 34.813
45.248 3.812 3.800 44.701 45.474 3.817 3.700 40.447
Table 4.03 – Values for flaw widths with even interval between natural log of
width

The flaw widths in table 4.03 and the model for energy as a function of flaw width
were used to calculate new energy values, which are listed in table 4.04. A second
plot of flaw width vs. flaw depth, as seen in figure 4.29, was produced using the new
flaw widths and energies in table 4.04, and again the energy/ depth models described
in the results section. On this plot, it can be seen that the modified distribution
includes only 4 flaws wider than 30 mm and 1 flaw greater than 0.6 mm deep.

101
 Ø20 mm/17ply Ø20 mm/33ply Ø87 mm/17ply Ø87 mm/33ply

Energy Width Energy Width Energy Width Energy Width

(J) (mm) (J) (mm) (J) (mm) (J) (mm)
11.9 9.03 4.8 7.39 18.5 11.02 11.7 14.15
14.3 10.49 7.8 9.03 23.3 13.46 13.2 16.44
17.0 12.18 11.5 11.02 29.1 16.44 14.9 19.11
20.2 14.15 16.0 13.46 36.1 20.09 17.0 22.20
23.9 16.44 21.5 16.44 44.8 24.53 19.3 25.79
28.2 19.11 28.2 20.09 55.3 29.96 22.1 29.96
33.2 22.20 36.4 24.53 68.2 36.60 25.3 34.81
39.0 25.79 46.4 29.96 83.9 44.70 29.0 40.45
Table 4.04 – Revised surface flaw widths & energies

Figure 4.29 – Graph of flaw widths vs. flaw depths with even distribution between
lognormal of flaw widths

By using the flaw widths in table 4.04 and the geometry variable models derived in
the results chapter to calculate the flaw depths, Xt values and Yt values as a function
of flaw energy, the flaw geometry variables for 32 surface flaws were calculated.
Table 4.05 lists the variable sizes required to reproduce the flaws in a CAD system.
The geometric variables in table 4.05 were used to construct the 2D section lines for
each of the 32 flaws. It should be noted that the values of rE and rI are driven by the

102
position of Xt and Yt due to the tangency constraints between the curves and the
planar surface (see figure 4.11). Their calculations were thus not required as Xt and
Yt were specified.

Each of the 32 flaws were placed into random locations on 16 flawed specimen
panels, using randomised X and Y coordinates. A margin of 50 mm from the panel
edges was used in order to avoid flaws reaching the panel edges. The placement order
of the panels was also randomised, such that each panel contained up to 3 flaws of
different sizes and types. The specifications for each of the flawed panels are given in
table 4.06. The production drawings for the specimens are given in appendix C.

Ø20 mm/17ply Ø20 mm/33ply

Width (2R)

Width (2R)
Energy (E)

Energy (E)
Depth (d)

Xt (mm)

Yt (mm)

Xt (mm)

Yt (mm)
Depth
(mm)

(mm)

(mm)

(mm)
(d)
(J)

(J)

6.7 9.03 0.09 1.352 0.056 4.8 7.39 0.10 0.303 0.089
8.2 10.49 0.11 1.723 0.071 7.8 9.03 0.11 0.351 0.102
9.9 12.18 0.14 2.154 0.093 11.5 11.02 0.13 0.410 0.119
11.9 14.15 0.19 2.656 0.127 16.0 13.46 0.16 0.481 0.145
14.3 16.44 0.26 3.238 0.183 21.5 16.44 0.20 0.568 0.183
17.0 19.11 0.37 3.915 0.280 28.2 20.09 0.26 0.675 0.245
20.2 22.20 0.57 4.701 0.460 36.4 24.53 0.37 0.805 0.348
23.9 25.79 0.94 5.614 0.816 46.4 29.96 0.57 0.965 0.535

Ø87 mm/17ply Ø87 mm/33ply

Width (2R)

Width (2R)
Energy (E)

Energy (E)
Xt (mm)

Yt (mm)

Xt (mm)

Yt (mm)
Depth

Depth
(mm)

(mm)

(mm)

(mm)
(d)

(d)
(J)

(J)

18.5 11.02 0.10 1.681 0.077 17.0 14.15 0.16 0.184 0.136
23.3 13.46 0.12 2.072 0.082 19.3 16.44 0.17 0.749 0.139
29.1 16.44 0.13 2.549 0.089 22.1 19.11 0.18 1.407 0.141
36.1 20.09 0.16 3.132 0.098 25.3 22.20 0.19 2.171 0.145
44.8 24.53 0.19 3.844 0.110 29.0 25.79 0.20 3.058 0.149
55.3 29.96 0.25 4.713 0.128 33.3 29.96 0.21 4.090 0.154
68.2 36.60 0.33 5.775 0.152 38.3 34.81 0.23 5.288 0.160
83.9 44.70 0.49 7.072 0.189 44.1 40.45 0.25 6.679 0.167

Table 4.05 – Geometric variables used to define size of specimen flaws

103
Table 4.06 – Specimen panel details

104
4.11 – Display of virtual specimens to participants
The virtual specimens were displayed onto a large, clean, white display screen using a
Sanyo PLC-XW20A colour projector connected to a laptop computer, as illustrated in
figure 4.30. The need for ‘keystone adjustment’ was eliminated by placing the
projector on a 1.1m high stand. This ensured that the display image was projected
without shape distortion or stretching. The display size was checked and adjusted to
ensure the virtual specimen panels were displayed at the correct size. Blinds were
drawn across all windows to maximise projection clarity, and a desk lamp was
provided to allow participants to see their answer sheets. The lamp was placed such
that its beam did not interfere with the projected display. A single operator was
required to run and moderate the projected specimen trials.

The participants were seated slightly offset to the centre of the display, and within
1.2m from it, as seen in figure 4.30. The slight offset, which is exaggerated in the
diagram, was required to stop the heads of taller participants interrupting the
projection beam.

Figure 4.30 – Virtual trials setup

The virtual specimen bitmaps were arranged in a predetermined random order in a

Microsoft PowerPoint presentation file. The operator started the PowerPoint
presentation once the participants were ready, and each panel was displayed for 5

105
seconds, with an interval between each panel. For the first series of trials, specimens
were displayed in rapid succession with a short (2 seconds) blank screen interval
between each specimen. For the second series of virtual trials, a longer interval of 20
seconds was used to simulate the time required to change specimen panels over when
physical facsimile specimens were used. A countdown timer and three bleeps alerted
the participants of when the next panel would appear. For all trials, the panel numbers
were displayed with the panels and for the second series of trials, the number of the
next panel was displayed during the “changeover” sequence.

4.12 – Display of physical facsimile specimens to participants

A pivoted display stand was used to display the physical specimens to the participants
(see figures 4.31 & 4.32). The design of the stand allowed the participants to inspect
the panel, after which the operators swung the panel round and replaced it with the
next specimen, without the participants seeing the changeover.

Figure 4.31 – Specimen display stand for visual inspection trials

106
 Figure 4.32 – Specimen display stand setup in meeting room

Figures 4.31 and 4.32 show the inspection trials experiment as it was set up. The
display stand was table mounted, and curtains attached to it in order to provide a
screen behind which the personnel performed the panel changeover. A small,
detachable display curtain hid the panels from view whilst they were swung into the
inspection position.

The lighting of the inspection task was provided by downwards reflecting fluorescent
tube ceiling luminaires of the type commonly found in office buildings. The lights
were located above, and in line with the front of the inspection side of the display
stand. A computer with a display screen and speakers was used to run a Microsoft
PowerPoint presentation, which provided the audible and visual timing signals for the
display curtain operations, and changeover of the panels. The display screen was
hidden from the participants view in order to avoid alerting them as to whether a
flawed or unflawed specimen would be shown next. The participants were given a
clipboard and paper answer sheet, on which they marked their answers with a pen.

The participants were seated in a chair, viewing the panels normal to the inspection
surface plane, in a position where the participant’s eyes were 1.2m from the specimen

107
panels. The specimen panels were displayed individually, in a pre-determined,
random order, for 5 seconds each. A regular interval of 20 seconds between each
panel display was allowed for panel changeover. The complete task duration was 27
minutes.

Two operators were required to run the visual inspection trials, operator 1 to perform
the panel changeover, and operator 2 to remove and replace the small display curtain
and act as moderator.

After starting the PowerPoint timing presentation, operator 2 removed the display
curtain on hearing a single beep from the computer, moved & held it clear of the
display stand for a 5 seconds, and replaced it as instructed by a second beep from the
computer. The display stand was swung round, and operator 1 removed the specimen
panel, and replaced it with the next panel in the specimen display sequence. The
computer display behind the display screen gave information on which panel to place
into the display stand. An audible signal of three consecutive beeps sounded after 17
seconds, alerting the operators that the display stand & new specimen should now be
ready in display position. After 20 seconds, a single beep sounded, thus repeating the
process until the 64th panel was displayed.

For trials with gloss finish specimens, 48 unflawed specimen panels were available.
This made it possible to stack the complete set of 64 (16 flawed/ 48 unflawed)
specimens in the correct display sequence prior to beginning the first trial, ready for
display to participants as instructed by the PowerPoint timing presentation. During the
trial, the changeover operator re-stacked the specimens in reverse order, when
removing them from the display stand. For the next trial, the specimens were
displayed in reverse sequence. Two different timing presentations were used, one for
forwards order trials and one for reverse order trials. A record was kept of which
sequence the participants saw the specimens in by adding FO (forwards order) or RO
(reverse order) into the serial number on each participants answer sheet.

For the inspection trials with matt finish specimens, only 16 unflawed specimens were
available. In order to display the 48 required unflawed specimens, the set of 16
unflawed specimens were displayed cyclically. The specimen sequence was displayed

108
on the panel change screen (see figure 35.5) for these trials, and the flawed and
unflawed specimens were kept in separate piles. The changeover operator chose the
correct flawed specimen, or an unflawed specimen, as instructed by the panel change
screen. Prior to the first trial with a matt finish specimen set, the flawed specimens
were stacked in the correct order in which the display screen would request their
display for inspection. Upon removing a flawed specimen from the display stand,
operator 2 restacked the flawed panels in the reverse order ready for the next trial, in
which the display sequence reversed, as described for the gloss specimens.

In any situation where a pause in the trial was required, the moderator stopped the
inspection trial timer at the end of the display time for the current specimen, and re-
started when convenient in order to avoid the possibility of distractions affecting
participant performance during the trial.

4.13 – Participant briefing for visual inspection trials

All participants were given a 10-minute briefing before undertaking a visual
inspection trial. The briefing explained the experimental task, provided examples of
the surface flaws that the participants would be asked to find, and gave instructions on
how to complete the answer sheet.

Participants were advised of the trial duration and how many panels they would view.
Where applicable, the participants were told whether panels would contain either
single flaw or contain more than one flaw, and that some panels would be unflawed.
The participants were advised that the panel display order had been randomised, and
were told not to be anxious if they experienced a succession of panels on which they
did not see flaws. The participants were not told how many flaws were in the
specimen set, or how many flawed and unflawed specimen panels there were in the
experiment. Participants were not told of the size range of flaws they would be
expected to see. The participants were advised that their payment was not
performance related, i.e. payment would not be increased for finding greater numbers
of flaws, and deductions would not be made for missed flaws.

The participants were asked if they would normally wear corrective eyewear for
medium to long-range vision, (i.e. to see presentation screens during lectures or for

109
driving), and if so, that they wear them during the trial. The participants were told that
they could move their heads to a reasonable extent during the visual inspection trials,
but persons moving excessively would be asked to reduce the amount by which they
moved their viewing position. When undertaking trials with physical specimens, the
participants were instructed not to touch the panels. After being briefed, the
participants were given a clipboard, pen, and answer sheet. The answer sheet was
marked with a serial number that identified the type of trial, the colour & finish of the
panels (for physical specimens), the date of the trial, and the participant number.

4.14 – Experimental task for visual inspection trials participants

The experimental task for the visual inspection trial experiments was for the
participants to view each specimen panel and search for the presence of impact
damage flaws. The participants were asked to signal the presence of suspected flaws
by marking their responses on an answer sheet.

4.15 – Recording participant responses – series 1 virtual specimen trials

The participants were asked to mark their responses down on a response sheet. The
response sheet contained a box for each panel number. Within each box was a space
for a mark against ‘yes’ or ‘no’. Participants were requested to use a tick, a cross or a
line through the ‘yes/ no’ boxes in order to mark their response to each panel. Most
complied, with only one individual choosing to ring the ‘yes/ no’ text instead of using
the boxes provided. The hit/ miss results were obtained by overlaying an answer sheet
onto the response sheets, and marking the response sheets as either a correctly
identified flaw, a missed flaw or a false call. The results data was input into a
Microsoft Excel spreadsheet and stored electronically.

4.16 – Recording participant responses– physical specimen & series 2

virtual specimen trials
For the first series of trials, the flawed specimens contained only a single impact flaw,
which allowed the use of a simple ‘yes’ or ‘no’ response from participants when
identifying the presence of flaws.

For the second series of trials, the specimens contained up to 3 impact flaws on a
single specimen panel, which negated the use of a yes/ no response sheet. Instead, the
participants were asked to mark the location of suspected flaws on a diagrammatic
representation of each individual specimen panel.

110
The diagrammatic answer sheets contained a diagram of each of the 64 specimen
panels within the specimen sets. The diagrams, as seen in figures 4.33 & 4.34
illustrated the edge of the specimen panels, and a nine square guide grid to help the
participants place their answers in the correct location. The participants were asked to
mark the location of surface flaws with a cross (figure 4.33), and to put a line through
the entire panel for any specimens that the participant deemed free of surface flaws
(figure 4.34)

Figure 4.33 – Surface flaw Figure 4.34 – No surface flaws

Detected detected

4.17 – Data collection method for visual inspection trials

Hit/ miss data, and false call data were extracted from the answer sheets by comparing
the answer sheets with marking sheets. For the first series of trials, correctly identified
flaws, or hits, were denoted by ‘yes’ answers marked against panels known to contain
flaws. Correct answers for unflawed panels were denoted by ‘no’ answers against
those panels. Undetected flaws, i.e. misses, were denoted by ‘no’ answers against
panels that were known to contain flaws. False calls were identified by ‘yes’ answers
against panels that were known to be free of flaws.

For the second series of trials, in which the participants had to denote the location of
suspected impact flaws, or flaws, the answer sheets were compared with a marking
sheet that contained the actual locations of the 32 flaws within the specimen set. A
‘hit’ was made when the participant viewed a flawed specimen panel ,detected a flaw
or flaws, and marked down the presence of each flaw in the correct position on the

111
diagram. A miss occurred when the participant did not detect a flaw, i.e. failed to
make a mark in the correct position on the panel diagram or crossed the panel as
unflawed despite having viewed a panel that actually contained flaws. False calls
were classed as occurrences of participants marking the presence of a flaw on a panel,
or an area of a panel, that was denoted on the marking sheet as being free of flaws.
The moderator’s criteria for successful detections, or hits, were as follows:

• For multiple surface flaws, the participant answer marks were expected to be in
similar positions, or follow the same pattern as the flaws shown on the marking
sheet.

• For the remaining flaw answer marks on a multiple flaw specimen, where the
participant has missed one or more flaws, the above criteria applied, and
participant answer marks were expected to be within a distance equal to the size
of one grid square of the position shown on the marking sheet

• For single flaws, or for judging a borderline hit/miss for individual flaws on a
multiple flaw specimen, the participant answer was expected to be within a
distance equal to the size of one grid square of the position shown on the
marking sheet.

The following additional criteria were set out before marking began:
• Any answer mark made by the participant not corresponding to a surface flaw on a
specimen was to be deemed a false call. The number of false calls made by each
participant was to be recorded, in case of the data being required for further
analysis in other work.
• Some participants may believe they saw a surface flaw and marked it down, but
then decided that what they saw may not have been a surface flaw, and crossed
out their answer. In such cases, marking was to ignore crossed out marks that did
not correspond to a surface flaw, as such marks were deemed the result of
participants realising that they had made a false call and correctly crossing out
their mistake.
• Where a participant correctly detected, and marked down a surface flaw, but
subsequently crossed the answer out, the mark was recorded as a hit, as the
participant must have had some cause to make the original mark.

112
The reason for adopting the described approach to marking was to include all
occurrences of ‘detections’ (i.e. the participants eye detecting a target) in the results.
The implications for the detection results are that for a very limited number of results,
what the participant may have felt was a false call might actually be recorded as a hit.
However, the implications of this treatment are insignificant given the very limited
numbers of crossed out calls that were in flaw locations (<2) and the fact that the
false call data were not to be used for determining operator bias as per signal
detection theory (which, as described in chapter 2, is outside of the scope of this
thesis).

The hit/ miss & false call data for each of the visual inspection trials were transferred
into electronic format by recording them as Microsoft Excel data tables, one for each
set of specimens used. The tables list each of the flaws within the specimen set, and
each of the participant’s responses. A hit is recorded as 1 and a miss as 0.

4.18 – Participants – Series 1 trials (with virtual specimens)

The trials were undertaken by participants from Cranfield University. Each participant
was given £5.00 for a single trial. As an incentive to encourage participation & good
attendance, participants attending all the trials in the series were given an extra £5.00.
There were 18 participants in total; 4 of whom attended just one trial. 2 participants
attended two of the trials, and 12 participants attended all three trials.

All participants were able to see the example flaws, and were thus deemed to have
suitable eyesight for continuing with the trials. The ages of the participants were not
recorded. However, the typical demographic of Cranfield University, and thus the
participant ages, can be considered as generally being between 20 and 35 years old.

4.19 – Participants – Series 2 trials (physical & virtual specimens)

As per series 1 trials, the participants for the second series of visual inspection trials
were volunteers from Cranfield University, and were paid £5 for participating in a
single trial. Actual aircraft inspectors were not required, as the trials constitute a
reliability assessment that simulates the inspection task [see Gallwey & Drury, 1986].

The daily schedule of 6 trials per day allowed the operators sufficient time to absorb
late participant arrivals and minor delays during trials. Delays mostly arose in early

113
trials due to participants keen to ask numerous questions during the briefing. To
accommodate participant availability for 6 inspection trials, two participants
performed the inspection trial sat side-by-side. The possible advantage of being seated
at a slight angle to the specimen panel was negligible, especially given that other
participants were free to move their heads during inspection. The operators checked,
and found that the view of a panel, and surface flaw, at a slight angle was
indiscernible to the view from exactly 90º to the panel. The results from these trials
showed no difference to trials with a single participant.

Participant 12 experienced some eyestrain during an inspection trial with matt white
specimens, and experienced difficulty deciding between being better able to inspect
with or without their glasses. The operators offered to pause, or stop the trial, but the
participant expressed a wish to continue, and inspected without glasses thereafter. The
results for this trial did not appear significantly different to any other participants, and
were thus included in the experimental results. Apart from this isolated case, there
were no other indications from participants that they were becoming bored, or
suffering from fatigue.

Chapter 5 – Experimental Results

5.1 – Impact damage to CFRP laminates
Examples of impact damage to painted, mesh incorporating and unfinished CFRP
laminate coupons were produced using falling weight apparatus at a range of impact
energies. The impact events resulted in surface indentations (flaws), subsurface
delaminations and back face fibre breakout, in a manner consistent with the typical
impact damage behaviour of CFRP laminate materials. The visual appearance of the
impact damage to the unfinished coupons was typical of that reported in other studies,
i.e. a surface flaw with surface breaking cracks appearing in the upper plies of the
laminate. Impacts to the painted, mesh incorporated2 coupons also resulted in surface
flaws, although their visual appearance was more ‘rounded’ than on the unfinished
coupons and surface breaking cracks did not appear on the painted coupons unless the

2
The term ‘painted’ shall be used forthwith to refer to CFRP laminate containing a single layer of
bronze lightning strike mesh and having a primer & topcoat paint finish. The term ‘unfinished’ shall be
used to refer to CFRP laminate without mesh layers or painted finishes.

114
impact tip penetrated or buckled the laminate (which occurred at varying energy
levels depending on the laminate thickness and impactor geometry). The results of the
impact damage experiments, which identify differences in impact damage
morphology between painted and unfinished laminates, are described forthwith.

5.1.1 – Basic measurements of impact flaw morphology for fully finished

CRFP laminates
The surface area and depth of the surface flaws on each of the impacted coupons was
measured as per 4.5.1 & 4.5.2. The associated delamination size was obtained using
ultrasonic C-Scan methods as per 4.4. The measurements obtained by these methods
are given in table 5.01. Some of the impacts by the flat cylindrical impactor produced
more than one damage zone, in which cases the maximum indentation depth is given,
along with the total damaged surface area and delamination area.

115
 Table 5.01 – Measurements of impact damage on CFRP coupons

5.1.2 – Relationship between flaw and subsurface damage sizes

The images obtained from the C-Scanning of the coupons provide useful comparisons
between the subsurface damage size and the visible surface damage size. Examples of
photographs of impact damage to painted CFRP coupons and the associated C-Scan
image are provided in figures 5.01 to 5.03. These examples indicated that the
subsurface damage caused by impact to a fully finished laminate extends further than
the visible surface flaw regions.

116
Figure 5.01 – Photograph and C-Scan of 10J / Ø20 mm impact on
17ply painted CFRP laminate

Figure 5.02 – Photograph and C-Scan of 70J / Ø87 mm impact on

33ply painted CFRP laminate

117
 Figure 5.03 – Photograph and C-Scan of 50J / Wedge shaped impact on
33ply painted CFRP laminate

Figure 5.04 details a plot of delamination area vs. flaw area for all of the impact
indentations on the fully finished CFRP coupons. From the plot, it can be seen that the
subsurface delamination size varied depending on the size of the impactor and the
impact energy. The general tend was for delamination area to increase in size as the
flaw area increased. The disparity between flaw size and delamination size is also
evident in figure 5.04. The delamination area produced by the 15J/ Ø87 mm impact
on 17ply laminate was approximately 1.5 times the flaw area size, whilst the
delamination produced by the 50J/ Ø87 mm impact on 33ply laminate was
approximately 73 times the size of the surface flaw area.

Also of note is that the large flat cylindrical impact tip produced relatively small
delamination areas (<2000 mm²) for very large surface flaw areas (>1500 mm²). In
comparison, the smaller tool shaped impact tip produced large delamination sizes (up
to 5500 mm²) with small surface flaw areas (<500 mm²). The same comparison can be
made for the larger (Ø87 mm) and smaller (Ø20 mm) hemispherical impact tips.

118
 Dent Area vs. Delamination Area
17ply/ Ø20mm
10000
17ply/ Ø87mm
9000
17ply/ Tool Shaped
Delamination Area (mm²)

8000
17ply/ Ø70mm Flat Cyl
7000
33ply/ Ø20mm
6000
33ply/ Ø87mm
5000
33ply/ Tool Shaped
4000
33ply/ Ø70mm Flat Cyl
3000
2000

1000

0
0 500 1000 1500 2000 2500 3000 3500 4000 4500
Dent Area (mm²)

Figure 5.04 – Plot of surface flaw area vs. delamination area for impacts on fully-
finished laminate

5.1.3 – Comparisons between impact damage on painted & unfinished

laminates
Figures 5.05 – 5.18 illustrate a selection of photographs of impact-damage surface
flaws that were created on painted and unfinished laminates. A photograph of each
damaged coupon is given in appendix D. Comparisons between surface flaw
appearance on the protected and unprotected laminates are also given.

Ø20 mm Impactor

100 mm
100 mm

Figure 5.05 – rounded, circular surface Figure 5.06 – jagged edged, circular
flaw produced by 20J / Ø20 mm impact surface flaw produced by 20J / Ø20 mm
on 17ply painted laminate impact on 17ply unfinished laminate

119
The indentation of Ø20 mm hemispherical impact-damage surface flaws on painted
laminates (figure 5.05), and unfinished laminates (figure 5.06), remained similar in
shape. However, on unfinished laminates, the impact caused the upper plies to split,
with small surface cracks becoming evident as seen in figure 5.08. The cracks were
not seen on the painted laminate, as seen in figure 5.07.

Figure 5.07 – 15J / Ø20 mm impact on Figure 5.08 – 15J / Ø20 mm impact on
17ply painted laminate 17ply unfinished laminate

As seen in figures 5.05 to 5.08, the typical visual appearance of Ø20 mm impactor
surface flaws was a circular, rounded depression. As a general trend, increasing
impact energy resulted in deeper surface flaws, and greater flaw areas. Surface flaw
sizes reached a maximum of 1.85 mm deep / 255 mm² on the painted coupons and
2.36 mm deep / 346 mm² on the unfinished coupons. Figure 5.09 illustrates a plot of
surface flaw depth vs. impact energy, produced using the values in table 5.01. The
plot in figure 5.09 shows that the flaws on unfinished coupons vary from being 0.1 to
2.0 mm deeper than for the same impact conditions on the painted coupons. The plot
of impact energies vs. flaw areas in figure 5.10 illustrates how flaw areas are between
4 mm² and 300 mm² larger on the unfinished coupons than for the same impact
conditions on the painted coupons.

120
 Impact Energy vs. Dent Depth - Ø20mm impactor
2.5

2
Dent Depth (mm)

1.5

17ply / Painted
0.5 33ply / Painted
17ply / Unfinished
33ply / Unfinished
0
0 10 20 30 40 50 60
Impact Energy (J)

Figure 5.09 – Impact energy vs. Flaw depth plot for Ø20mm impactor

Impact Energy vs. Dent Area - Ø20mm impactor

700

600

500
Dent Area (mm²)

400

300

200 17ply / Painted

33ply / Painted
100 17ply / Unfinished
33ply / Unfinished

0
0 10 20 30 40 50 60
Impact Energy (J)

Figure 5.10 – Impact energy vs. Flaw area plot for Ø20mm impactor

121
Ø87 mm Impactor

100mm
100mm

Figure 5.11 - surface flaw produced by Figure 5.12 –surface flaw produced by
30J / Ø87 mm impact on 17ply painted 30J / Ø87 mm impact on 17ply
laminate unfinished laminate

The appearance of Ø87 mm hemispherical impact-damage surface flaws on painted

laminates (figure 5.11), and unfinished laminates (figure 5.12), was very similar.
Surface breaking cracks did not appear on either the painted of unfinished coupons
when impacted by the Ø87 mm impactor. Typical visual appearance of surface flaws
was a rounded, circular indentation type flaw. As seen in the plot in figure 5.13, flaw
depths remained below 0.5mm despite energy levels of up to 70J and the flaws, for
the same impact conditions, were deeper on unfinished coupons than on painted
coupons. However, as seen in the impact energy vs. flaw area plot in figure 5.14, the
flaw areas produced by the Ø87 mm impactor were smaller on the painted 33ply
coupons than on the unfinished 33ply coupons.

Also noted in figure 5.14 are the significantly lower flaw areas measured on the
unpainted coupons compared to the painted coupons impacted by the 87 mm
impactor. This apparent discrepancy is evident when figure 5.11 and figure 5.12 are
compared. The visibly affected flaw area on the 33ply painted coupon (figure 5.11)
appears larger in the photograph than the flawed area exhibited in a similar
photograph of the same impact (30J / Ø87mm) on a 33ply unpainted laminate. The
reason for the discrepancy in flaw width measurements is thus attributed to the impact
causing deformation to the bronze mesh on the painted laminate, which spreads across
a greater area than the simple impact dent produced on the unpainted laminate. Thus
phenomena is does not occur with the Ø20 mm impacts due to the impact causing
more of a puncture into the laminate surface as opposed to a simple depression.

122
 Impact Energy vs. Dent Depth - Ø87mm impactor
0.5

0.4
Dent Depth (mm)

0.3

0.2

17ply / Painted
0.1 33ply / Painted
17ply / Unfinished
33ply / Unfinished
0
0 20 40 60 80
Impact Energy (J)

Figure 5.13 – Impact energy vs. Flaw depth plot for Ø87mm impactor

Impact Energy vs. Dent Area - Ø87mm impactor

2000

1750

1500
Dent Area (mm²)

1250

1000

750

17ply / Painted
500
33ply / Painted
250 17ply / Unfinished
33ply / Unfinished
0
0 10 20 30 40 50 60 70 80
Impact Energy (J)

Figure 5.14 – Impact energy vs. Flaw area plot for Ø87mm impactor

123
Ø70 mm Flat Cylindrical Impactor

100mm
100mm
Figure 5.15 - 30J / Ø70 mm flat cylinder Figure 5.16 - 30J / Ø70 mm flat
impact on 17ply painted laminate cylinder impact on 17ply unpainted
produced two distinct surface flaw areas laminate produced two surface flaw
with circular shaped depressions (0.2 areas with deeper (0.24 mm), less
mm deep) complete circular shaped depressions

The typical visual appearance of surface flaws was a circular impression on the
surface, which became more pronounced as impact energy increased. Some impacts
from the Ø70 mm cylindrical impactor resulted in two surface flaw sites that appeared
as sections of a circular depression. These can be seen clearly in figure 5.17, which
illustrates a C-Scan of a painted coupon impacted by the Ø70 mm cylindrical
impactor.

Figure 5.17 – C-Scan of 30J / 70 mm cylindrical impact

on painted 17ply coupon

124
On the painted laminates (figure 5.15), the impact from the flat cylinder appeared to
leave a less deep depression in the surface than the other types of impactor, but
affected a greater area, giving the visual appearance of a far more serious surface flaw
than was actually present. The depressions generally formed more of a complete
circle on the painted coupons than on the unfinished coupons (figure 5.16).
Wedge Shaped Impactor

100mm 100mm

Figure 5.18 - 15J / tool shaped impact Figure 5.19 - 15J / tool shaped impact
on 17ply painted laminate produced a on 17ply unfinished laminate
small depression in surface produced a small depression and
surface cracking

The appearance of surface flaws on painted laminates (figure 5.18), and unfinished
laminates (figure 5.19), remained similar in shape. However, on unfinished laminates,
the impact caused the upper plies to split, with surface cracks becoming evident as
seen in figure 5.19. Typical visual appearance of surface flaws is a small, rectangular
impression, which at higher impact energies became surrounded by a shallow
depression. The delamination sizes caused by the tool shape impacts are particularly
large compared to the size of the surface indentation, as illustrated in figure 5.03, for
both painted and unfinished laminates.

From observing the impact damage flaws on both fully finished and unfinished
laminates, the following summary was made:

• There were differences in surface flaw appearance depending on the laminate

finish. Surface breaking cracks occurred in flawed regions on the unfinished
laminates even at low (<15J) impact energies, whereas such cracks only began
to appear on the fully finished laminates once the impactor had caused laminate
penetration. The energy level required for penetration of the painted laminates

125
 varied from 30J with the Ø20mm / 17ply impacts to 50J for the Ø20mm / 33ply
impacts. The hemispherical impact flaws on the unfinished laminates appeared
to have jagged edges, whereas the flaws on the fully finished coupons were best
described as having a ‘smooth, rounded’ flaw shape.

• The impactor shape affected the flaw topography. The hemispherical shaped
impactors produced rounded, soft ‘edged’ flaws, whereas the tool shaped and
flat cylindrical impactors produced flaws, or flaws with defined, i.e. ‘hard’
edges.

• Impact energy and severity of the surface flaw profile were not considered
positively correlated. Higher energy impacts (>30J) from the Ø87 mm impactor
appeared to affect large surface areas (>300 mm²), and produce flaws with low
depths (<0.5mm) and very smooth sectional profiles. These ‘shallow’ surface
profiles were felt to be far less discernable to the observer than the lower energy
(<30J), smaller area (<300 mm²) and greater depth (up to 1.85 mm) Ø20mm
impacts which were said to have more ‘acute’ surface indentation profiles.

5.1.4 – Characterisation of surface flaws from hemi-spherical objects

The images from the metallographic sectioning performed by Boulic made it possible
to visualise the impact damage morphology in sectional profile, and identify a
geometric shape within the surface flaw that could be used to describe hemi-
spherically shaped impact damage surface flaws. Figures 5.20 & 5.21 illustrate
metallographic sections through hemispherical tip impact flaws on fully finished
CFRP coupons. Above the section photograph, figures 5.20 & 5.21 illustrate a non-
geometric line, which was overlaid using Photoshop image editing software, onto the
upper ply of the laminate in order to show the sectional profile of the surface
indentation. Each of the surface flaws in Boulic’s work had similar basic geometric
shapes i.e. an internal and external radius (see ‘geometric shape’ in figure 5.20 &
5.21). It was thus possible to describe and recreate the sectional profile of the
indentations using these geometric lines by constructing the lines in a CAD system
(see chapter 4.6 & 4.7) using a common set of geometric variables, as described in
figure 5.22, in order to define the lines. These geometric variables were adopted for
quantitatively describing the sectional profile of each of the impact-damage flaws
produced by hemispherical tips on the painted CFRP coupons.

126
Figure 5.20 – Ø20 mm/50J Surface flaw on a 33ply coupon

Figure 5.21 – Ø20 mm/ 30J Surface flaw on a 17ply Coupon

127
 θc = Cone angle of complete surface R = Surface flaw radius on surface
flaw d = Surface flaw depth
rI = Internal radius of surface flaw,
rE = Edge radius of surface flaw
θt = Angle between vertical and Xt = X coordinate of tangent point
tangent of internal & edge radii between internal & edge radii
Yt = Y coordinate of tangent point
between internal & edge radii

Figure 5.22 – Geometric variables used to reconstruct a surface flaw section

5.1.5 – Quantitative measurements of surface flaw topographies

CMM data of a set of 26 surface flaws on impact damaged, painted CFRP coupons
was used to obtain depth measurements, width measurements and values for the
geometric variables listed in figure 5.22, as per the method desribed in chapter 4.5 &
4.6. The measurements obtained are given in table 5.03.

128
 Energy (J)
Thickness
Impactor

Xt (mm)

Yt (mm)
rE (mm)
rI (mm)
R [w/2]

d (mm)
(mm)

θc (º)
θt (º)
Size

5 2.5 0.11 11.78 16.68 1.03 0.06 84.96 87.48

10 6 0.07 85.73 171.45 2 0.05 88.66 89.33
17ply 15 10.75 0.32 58.84 121.89 3.5 0.22 86.59 88.29
20 11 0.67 8.22 82.15 1 0.61 83.01 86.5
Ø20 mm

30 15 2.11 5.44 48.94 1.5 1.9 73.99 81.99

15 6 0.17 6.9 96.63 0.4 0.16 86.68 88.34
20 7.5 0.2 9.57 134.02 0.5 0.18 87.01 88.5
20 8 0.19 12.64 155.88 0.6 0.18 87.28 88.64
33ply
30 11.5 0.19 24.09 322.21 0.8 0.18 88.1 89.05
50 15 0.86 8.75 122.49 1 0.8 83.44 86.72
50 16.5 0.61 13.54 209.92 1 0.57 85.77 87.88
10 2.5 0.12 10.42 84.5 1 0.12 84.5 87.25
15 6 0.11 41.68 125.04 1.5 0.08 87.94 88.97
20 5 0.08 46.89 109.4 1.5 0.06 88.17 89.08
17ply 20 6.5 0.18 36.14 81.31 2 0.12 86.83 88.41
30 8 0.02 666.67 1466.67 2.5 0.01 89.79 89.89
40 11 0.21 91.7 196.5 3.5 0.14 87.81 88.91
Ø87 mm

40 11.25 0.45 43.82 97.03 3.5 0.31 85.42 87.71

20 6.5 0.14 78.69 204.6 2.5 0.1 88.18 89.09
30 10.5 0.18 101.55 203.1 3.5 0.12 88.02 89.01
40 14 0.3 104.01 219.57 4.5 0.21 87.52 88.76
40 30 0.26 286.48 1442.42 5 0.26 89.01 89.5
33ply
50 23 0.37 203.73 517.16 6.5 0.26 88.17 89.09
60 20 0.28 286.79 430.19 8 0.17 88.4 89.2
60 35 0.29 521 1893.82 13 0.13 88.57 89.53
70 31 0.34 659.17 750.09 14.5 0.18 88.74 89.37
Table 5.03 – Table of geometric variable data obtained from surface flaws on
painted, mesh incorporated CFRP laminate coupons

5.1.6 – Relationships between impact flaw geometry variables

The data in table 5.03 were plotted using Excel scatter-plots in order to identify
relationships between the geometric variables. The data points were assessed in terms
of the general trend, or data shape, and suitable best-fit lines were added. Plots of
surface flaw width vs. impact energy, surface flaw depth vs. energy, surface flaw
width vs. Xt value, and surface flaw depth vs. Yt value were produced, and are given
in figures 5.23 to 5.26.

The data on the plot (figure 5.23) of impact energy and surface flaw radius R, i.e. ½
the flaw width, were fitted with linear best-fit lines. The fit equation, as shown on the

129
plot, for each of the impactor/ laminate thickness combinations allow the surface flaw
width (2R) to be modelled as a function of impact energy and vice versa. Variations in
the relationships were observed for each type of impactor on the two laminate
thicknesses, resulting in 4 different relationships.

Impact energy vs Surface Flaw Radius (R)

40

35 y = 0.4843x - 1.1478
Surface Flaw Radius (R) mm

30

25

20
y = 0.4858x + 1.277 y = 0.2713x + 2.384 Ø20mm 17ply
15
Ø20mm 33ply
10 Ø87mm 17ply
y = 0.2574x + 0.7447 Ø87mm 33ply
5

0
0 10 20 30 40 50 60 70 80
Impact Energy (J)

Figure 5.23 – Plot of impact energy vs. surface flaw radius with linear lines
of best fit

The data on the plot of surface flaw depth vs. impact energy (figure 5.24) was best
fitted by an exponential model. The data for the Ø20 mm/ 17ply flaws exhibited a
comparatively high rate of increase in flaw depth as impact energy increased from 10J
to 30J. The remaining impactor type / laminate thickness combinations exhibited
lower rates of increase in depth with increasing impact energy. Each impactor /
thickness combination resulted in a different relationship between impact energy and
flaw depth.

130
 Im pact energy vs Surface Flaw Depth (d)

2.5
Ø20mm 17ply
2.25 y = 0.0367e 0.1357x
Surface Flaw Depth (d) mm Ø20mm 33ply
2
Ø87mm 17ply
1.75 Ø87mm 33ply
1.5

1.25
1
0.75 y = 0.078e 0.043x

0.5 y = 0.1242e 0.0159x

0.25
y = 0.0664e 0.0237x
0
0 10 20 30 40 50 60 70 80
Im pact Energy (J)

Figure 5.24 – Plot of impact energy vs. surface flaw depth with exponential
lines of best fit

The data plotted in figure 5.25 were fitted with linear lines to represent the four
different relationships between R (½ flaw width) and the Xt point variable (see figure
5.22). The rate of increase in Xt with increasing values of R for the Ø20 mm / 33ply
flaws was comparatively lower than the other impactor / thickness combinations. The
data plotted in figure 5.26 were also fitted with linear lines in order to represent the
relationships between flaw depth and the Yt geometric variable value. The linear best-
fit lines shown in figure 5.26 exhibit comparatively less variation than the other
relationships modelled in figures 5.23 – 5.25, suggesting that the increase in Yt with
increasing flaw depth was less affected by impactor / thickness variations.

131
 Surface Flaw Radius (R) vs X Coordinate of Tangent Point (Xt)

15

12.5 y = 0.3437x - 0.1163

X Coordinate (Xt) mm

10

7.5

y = 0.3003x + 0.2495
5 (20m m 17ply)
Ø20mm 17ply
y = 0.3084x + 0.0006 Ø20mm 33ply
2.5 (87m m 17ply)
Ø87mm 17ply
y = 0.0587x + 0.0853 Ø87mm 33ply
0
0 5 10 15 20 25 30 35 40
Flaw Radius (R) m m

Figure 5.25 – Plot of surface flaw radius vs.Xt with linear lines of best fit

Surface Flaw Depth (d) vs Tangent Point Y Coordinate (Yt)

2.25

2 y = 0.9186x - 0.0346
(20m m /17ply)
1.75
Y Coordinate (Yt) mm

1.5

1.25

1 y = 0.9305x + 0.0007
(20m m /33ply)
0.75 Ø20mm 17ply

y = 0.6707x + 0.0079 Ø20mm 33ply

0.5
(87m m /17ply) Ø87mm 17ply
0.25 y = 0.5507x + 0.0301 Ø87mm 33ply
(87m m /33ply)
0
0 0.25 0.5 0.75 1 1.25 1.5 1.75 2 2.25
Flaw Depth (d) m m

Figure 5.26 – Plot of surface flaw depth vs.Yt with linear lines of best fit

132
Table 5.04 gives the relationships obtained by the best fit-line method from tested
coupons for two impactor sizes and two laminate thicknesses. Also given is the
energy range for which the relationship remains valid. All the relationships are only
valid for impacts greater than 4J; impacts below this energy level will not produce
subsurface damage, and are thus irrelevant to this study. Impacts above the given limit
will penetrate the laminate, or cause a significant shape change to the laminate, in
which the case the surface flaw characteristics are outside the scope of this study,
hence the use of an upper limit.

17ply/ Ø20 mm 17ply/ Ø87 mm

Energy E = (R-1.277)/0.4858 Energy E = (R-0.7447)/0.2574
Depth d = 0.0367(exp(0.1357E)) Depth d = 0.0664(exp(0.0237E))
X Point Xt = (0.247E)-0.2933 X Point Xt = (0.0824E)+0.1555
Y Point Yt = 0.0199(exp(0.1553E)) Y Point Yt = 0.0598(exp(0.0137E))
Applicable energy range: 4J -30J Applicable energy range: 4J – 50J

33ply/ Ø20 mm 33ply/ Ø87 mm

Energy E = (R-2.384)/0.2713 Energy E = (R+1.1478)/0.4843
Depth d = 0.078(exp(0.043E)) Depth d = 0.1242(exp(0.0159E))
X Point Xt = (0.0159E)+0.2263 X Point Xt = (0.2393E)-3.8805
Y Point Yt = 0.0727(exp(0.043E)) Y Point Yt = 0.1198(exp(0.0075E))
Applicable energy range: 4J – 80J Applicable energy range: 4J – 80J

Table 5.04 – Geometry variable relationships for impact damage surface flaws.

The relationships listed in table 5.04 were used to calculate the flaw energy (E) as a
function of flaw width (2R) and the flaw depth (d), Xt value and Yt value as a
function of impact energy (E) for chosen values flaw widths (2R). Using these values,
the sectional profiles of representations of surface flaws caused by impacts, at
energies within the ranges specified in table 5.04, from Ø20 mm & Ø87 mm tips on
17ply & 33ply fully finished CFRP laminate were able to be defined. By inputting the
depth, R, Xt and Yt values, the geometric lines representing sectional profiles of
impact damage flaws were created and associated 3D models of the flaws were
produced in a 3D CAD system (see chapter 4.7).

133
5.2 – CMM checking of physical facsimile specimens
Five selected flaws on each colour of specimen underwent CMM measurement (as
described in chapter 4.9.2) to check for conformity to the original specification, and
table 5.05 details the findings. One can see in table 5.05 that the surface height
measurements reveal no depth discrepancies, meaning that all the flaws were
machined to the correct depth. Discrepancies between the specified surface height at
the Xt point and the scanned surface height varied from 0.002 (#28 / white) to 0.050
(#7 / white) mm. The average discrepancy across all specimen colours for each
measured flaw was between 1.9% and 8.5% of the specified surface height at the Xt
point. As the average discrepancies were less than 10%, the machined flaws were all
deemed to be within acceptable tolerance of the original specified shape. The surface
height measurements at the flaw endpoints revealed that flaws 7, 9, 14, 20 & 28 had
been machined with endpoint surface height discrepancies of less than 5% of the flaw
depths across the three specimen colours. These flaws were thus deemed to be within
an acceptable width tolerance. The measurements of flaw 9 revealed similar sizes of
surface height discrepancies at the flaw endpoints, although due to flaw 9 being only
0.1mm deep, this translated into an average discrepancy of 15% of the total flaw
depth. Despite this, and given that the surface height discrepancies for the endpoints
of flaw 9 were all relatively small (<0.02mm), each example of flaw #9 was deemed
to be the correct, specified width.

134
 Specified Surface Height Discrepancy Average
Flaw Check
Surface discrepancy across
No Point Grey White Blue
Height specimens
Specimen Specimen Specimen
0.0037mm (0.64%
Endpoint 0 mm 0.044mm -0.010mm -0.023mm
of depth)
7 Xt Point -0.329mm 0.018mm -0.050mm -0.005mm -0.0123mm (3.75%)
Centre (d) -0.57mm 0mm 0mm 0mm 0mm
-0.0150mm (15% of
Endpoint 0mm -0.019mm -0.013mm -0.013mm
depth)
9 Xt Point -0.092mm -0.007mm -0.008mm -0.004mm -0.0063mm (6.88%)
Centre (d) -0.1mm 0mm 0mm 0mm 0mm
-0.0130mm (5% of
Endpoint 0mm -0.011mm -0.017mm -0.011mm
depth)
14 Xt Point -0.243mm 0.000mm 0.007mm 0.007mm 0.0047mm (1.92%)
Centre (d) -0.26mm 0mm 0mm 0mm 0mm
0.0073mm (4.58%
Endpoint 0mm 0.019mm -0.007mm 0.010mm
of depth)
20 Xt Point -0.11mm -0.011mm -0.004mm -0.013mm -0.0093mm (8.48%)
Centre (d) -0.16mm 0mm 0mm 0mm 0mm
0.0093mm (4.91%
Endpoint 0mm 0.027mm -0.011mm 0.012mm
of depth)
28 Xt Point -0.153mm -0.014mm -0.002mm -0.010mm -0.0087mm (5.66%)
Centre (d) -0.19mm 0mm 0mm 0mm 0mm

Table 5.05 – +/- Y-Axis measurements from CMM data of specimen surface
flaws

5.3 – Gloss measurements of physical facsimile specimens

Table 5.06 gives the measurements made from the panels using a Novo Gloss dual-
angle (20º, 60º) gloss meter.

Panel/ Specimen Colour Gloss units (measurement

angle)
Gloss Grey 84.666 (60º)
Gloss White 88.033 (60º)
Gloss Blue 79.833 (60º)
Matt Grey 1.733 (60º)
Matt White 7.766 (60º)
Matt Blue 0.533 (60º)
Table 5.06 – Gloss measurements of specimens

The gloss-finish Plexiglas specimens are no glossier than one would expect a painted
aircraft structure to be [Akzo Nobel 2006 & 2008; PPG 2003; Guseva 2003], and the matt

135
finish grey and blue specimens are below the maximum gloss level that one would
expect from a matt finish aircraft structure [Akzo Nobel 2006 & 2008; PPG 2003]. For the
matt white finish specimens, the gloss reading of 7.766 gloss units is low enough be
considered as a matt finish, despite being slightly higher than that specified by
aircraft paint manufactures for matt finish aircraft paint [Akzo Nobel 2006 & 2008; PPG
2003].

5.4 – Results of visual inspection trials with virtual specimens – series 1

After collecting the hit/miss data from the answer sheets as described in chapter 4, the
number of participants correctly detecting each flaw was divided by the number of
inspections for each flaw to provide the detection rate as a percentage. I.e. for a flaw
inspected by 17 participants, and detected by only 15 of those participants, the
detection rate would be 88%. Presenting detection results as a % value allows
comparison between results from trials that used different numbers of participants.

5.4.1 – Results of trial 1

Trial one was successfully undertaken by 7 participants. The main purpose of trial 1
was to determine a width of flaw that gave approximately 50% detectability. Trial 1
also served as a dress rehearsal for the virtual trials methodology hence the low
number of participants. The hit/ miss data for trial 1 are presented in appendix E.
From the hit/miss data, it was possible to produce the scatter plot seen in figure 5.27,
which gives the percentage of participants that successfully detected each flaw. The
data on the plot in figure 5.27 show a general trend for flaw detectability to reach over
85% for flaws wider than Ø6 mm. The key observations from the scatter plot were
that:
• Flaws below 5 mm wide, at a depth of 1 mm, resulted in 0% detections.
• Detection of flaws greater than 6 mm wide varied between 85% and 90%
The variation in detection between 85% & 100% for flaws over 6 mm wide is
accounted for by the fact that the response of a single participant was sufficient to
impart variation of 15% in the total detectability result for a single flaw. Reviewing
the hit/ miss data in appendix E reveals that the 15% variation in results for flaws >6
mm wide was caused solely by the responses of participant 2. The reason for the
discrepancy between participant 2 and other participant results is unknown.

136
 Trial 1 - Flaw depth1mm
Scatterplot of % Detections against Width
100%

90%

80%

70%
% Detections

60%

50%

40%

30%

20%

10%

0%
-5 0 5 10 15 20 25 30 35
Flaw Width (mm)

Figure 5.27 – Scatter plot of results from Trial 1 with virtual specimens

The results data from trial 1 were analysed using the log-odds/ range interval method
[Bullock et al. 1994] in order to determine the POD curve seen in figure 5.28. The
data and calculations used to generate the POD curve are given in appendix F. Using
the POD curve for trial 1, it was determined that a flaw size of Ø4.5 mm would give
approximately 50% POD, a flaw size of Ø6.5 mm would give approximately 70%
POD and >90% POD could be expected for flaws >Ø12 mm. The data for trial 1 also
indicated that for a flaw depth of 1mm, the detection rates (figure 5.27), or POD
(figure 5.28), increased with increasing flaw width.

137
 Trial 1
(Log-logistic (Log-Odds) Model/ Range Interval Method)

1
0.9
0.8
0.7
0.6
POD

0.5
0.4
0.3
0.2
0.1
0
0 5 10 15 20 25 30 35
Flaw Width (mm)

Figure 5.28 – log-odds/ RIM POD curve produced from trial 1 data

5.4.2 – Results of trials 2 and 3

Trials 2 and 3 also utilised virtual specimens, and were both undertaken by 15
participants. The virtual trials methodology, utilising a projector to display the virtual
specimens was again successfully implemented, and all participants successfully
completed their trials.

As described in chapter 4.10, the virtual specimens for Trial 2 contained hard edged,
hemispherical shaped circular flaws representing hard-edged flaws. Trial 3 contained
the same sizes of surface flaws, with the addition of a radius, or blended edge to the
flaw, thus resulting smoother flaw profiles. The following flaws were included in the
specimen set:

• Ø4.5 mm flaws, 0.1 mm to 1.5 mm deep (50% POD width from trial 1)
• Ø6.5 mm flaws, 0.1 mm to 1.3 mm deep (70% POD width from trial 2)
• Ø12 mm flaws 0.1 mm to 1.1 mm deep (90% POD width from trial 3)

From the hit/miss data presented in appendix E for trials 2 & 3, the percentage of
participants successfully detecting each flaw was calculated. Scatter plots of the
detection percentage of each flaw vs. flaw depth, as seen in figures 5.29 to 5.31, were

138
produced by inputting the % detection data into Statistica [Statsoft Inc. 2008]. Figure
5.29 illustrates the results of both trial 2 &3 using Ø4.5 mm wide flaws, and for flaw
depths of below 0.5 mm, there are inconsistent differences in the detectability of the
hard edge and blended edged flaws. There is considerable scatter in this flaw depth
range, with the detection % for blended edge higher than hard edged for the 0.2, 0.4 &
0.5 mm deep flaws and vice a versa for the 0.1 & 0.3 mm deep flaws. Thus, for the
Ø4.5 mm wide flaws, it is not possible to identify any consistent effect on detection
rate (% detections) due to the different flaw profiles.

Trial 2& 3 - Width=Ø4.5mm

Scatterplot of % Detection for Blended (BE) & Hard Edge (HE) against Depth

100%

90%

80%

70%
% Detections

60%

50%

40%

30%

20%
% Detections (BE)
10%
% Detections (HE)
0%
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6
Depth (mm)

Figure 5.29 – Scatter plot of detection results for Ø4.5 mm flaws on virtual specimens

139
The hit/miss data from the Ø6.5 mm flaws in trial 2 & 3 (see appendix E) are
summarised by the scatter plot given in figure 5.30. The results for hard edged and
blended edged flaws are similar for flaws greater than 0.2 mm deep. However, the
detection rates for the 0.1 mm deep flaw are over 20% greater for the hard-edged
flaws, suggesting that the detectability of this size (0.1 mm deep/ Ø6.5 mm) of flaw
was reduced by the addition of a blended edge. For the flaws, of both types, of depths
<0.5 mm, the detection rate is consistently over 90%, compared to the lower rates for
the same flaw depths for Ø4.5 mm wide flaws. This suggests that the increased flaw
width resulted in increased detectability for < 0.5 mm deep flaws.

Trial 2 & 3 - Width=Ø6.5mm

Scatterplot of % Detection for Blended (BE) & Hard Edge (HE) against Depth

100%

90%

80%

70%
% Detections

60%

50%

40%

30%

20%

10% % Detections (BE)

% Detections (HE)
0%
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6
Depth (mm)

Figure 5.30 – Scatter plot of detection results for Ø6.5 mm flaws on virtual specimens

The hit/miss data (appendix E), and detection rate results for the Ø12 mm wide flaws
are summarised by the scatter plot in figure 5.31.

140
 Trial 2 & 3 - Width=Ø12mm
Scatterplot of % Detection for Blended (BE) & Hard Edge (HE) against Depth

100%

90%

80%

70%
% Detection

60%

50%

40%

30%

20%

10% % Detections (BE)

% Detections (HE)
0%
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6
Depth (mm)

Figure 5.31 – Scatter plot of detection results for Ø12 mm flaws on virtual specimens

For a flaw width of Ø12 mm, and depth >0.2 mm, the flaws were detected by all the
participants. None of the participants were able to detect the 0.1 mm deep, Ø12 mm
hard-edged flaw.

The results from trial 2 with the Ø4.5 mm and Ø6.5 mm hard-edged flaws were
plotted as POD curves, as seen in figure 5.32 (see appendix F for POD curve
calculations). The Ø4.5 mm flaw data indicated 50% POD at 0.15 mm deep and 90%
POD at approximately 0.3 mm deep. The Ø6.5 mm flaws returned a minimum of 65%
POD for the 0.1 mm deep flaws, and the 90% POD size was estimated at 0.25 mm
deep. The data from the Ø12 mm flaws could not be plotted as a POD curve due to
only a single flaw returning less than 100% detections, which results in insufficient
data points to construct the regression & POD curves from.

141
 Trial 2 - 4.5mm & 6.5mm Hard edged flaws
(Log-logistic (Log-Odds) Model/ Range Interval Method)

1
0.9
0.8
0.7
6.5mm Wide Flaws
0.6
POD

0.5 4.5mm Wide Flaws

0.4
0.3
0.2
0.1
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5
Flaw Depth (mm)

Figure 5.32 – POD curve from results of trial 2

As for the results of trial 2, the results from trial 3 were also plotted as POD curves
(see appendix F). However, only the detection data from the Ø4.5 mm flaws could be
plotted, due to the low frequency of <100% detections for the other flaw sizes. The
POD curve for trial 3 is given in figure 5.33, and it can be seen that for the blended
edge, Ø4.5 mm flaws, the 50% POD size is approximately 0.125 mm deep.

Trial 3 - Ø4.5mm Blended edge flaws

(Log-logistic (Log-Odds) Model/ Range Interval Method)

1
0.9
0.8
0.7
0.6
POD

0.5
0.4
0.3
0.2
0.1
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5
Flaw Depth (mm)

Figure 5.33 – POD curve from results of trial 3

142
The observations made from the results of trials 2 and 3 were:
• For flaws between 0.2 mm and 0.6 mm deep, and of widths of Ø4.5 mm, Ø6.5
mm and Ø12 mm, the detectability was greatest for the wider flaws.

• Flaws less than 0.1 mm deep gave less than 50% detectability

• Flaws greater than 0.6 mm deep were detected by all participants, irrespective
of the flaw width.

• 50% POD occurred for Ø4.5 mm flaws at a flaw depth of 0.15 mm deep for
hard-edged flaws, and at a depth of 0.125 mm for blended edged flaws.

• There was insufficient difference in the POD data for the Ø4.5 mm flaws, and
no comparisons available for the Ø6.5 mm & Ø12 mm flaws in order to
quantify the effect of the blended edge on detectability

• Due to trials 2 & 3 using only 3 values of flaw width, the effects of varying
flaw width at lower flaw depths (<0.5 mm) on detectability could not be
demonstrated

5.4.3 – Results of trial 4

The virtual specimens used in trial 4 contained both hard and blended edged flaws, all
of 0.15 mm depth and of widths ranging from 3 to 30 mm. The % detection data for
each flaw, as calculated from the hit/miss data (see appendix E) returned by the 14
participants are summarised in the scatter plot shown in figure 5.34. As seen in figure
5.34, the blended and the hard-edged flaws returned similar results for the flaws up to
Ø15 mm in size, thus providing little evidence that the addition of a blended edge had
any significant effect on detection rates. As seen in figure 5.34, for flaws >Ø15 mm
and with the exception of the Ø28 mm flaw, the detection % for the blended edge
flaws were lower than the hard-edged flaws of the same width. A difference of up to
75% in detection rates was seen for the 22 mm flaw, and detection rates for the
blended edge flaws at 22, 26 and 30 mm wide were less than 10%, compared to
between 85% and 40% for the same size hard-edged flaws. Apparent in figure 5.34 is
that the detection % does not increase with flaw width. Instead, the shape of the data
plots is such that the detection % increases with flaw width until Ø15 mm. For flaws
>Ø15 mm, the detection percentages become scattered, with flaws of similar sizes
such as the Ø20 ~ Ø25 mm range exhibiting considerable variation in detection
percentage (<10% to 50% for blended edge flaws).

143
 Trial 4 - Scatterplot of % Detections vs. Width
Blended (BE) & Hard (HE) Edged flaws

100% % Detections (HE)

% Detections (BE)
90%

80%

70%
% Detections

60%

50%

40%

30%

20%

10%

0%
0 5 10 15 20 25 30 35
Width (mm)

Figure 5.34 – Scatter plot of detection results from trial 4

POD analysis of the detection data from trial 4 was performed (appendix F) in order
to attempt to reduce the scatter seen in figure 5.34 and help improve the distinction of
trends within the data. The detection data from trial 4, as represented in figure 5.34,
presented a novel issue in terms of POD analysis, in that as flaw width increased, the
detectability reached a peak before reducing again, albeit with considerable scatter.
When the probability values for the data were calculated, (as seen in appendix F), the
hard-edged flaws within the 3 mm to 9 mm flaw width returned increasing probability
values as the flaw width interval size increased. For flaws within the 11 mm to 29 mm
width intervals, the probability values reduced with increasing flaw width.

In order to plot a POD curve that reflected both increasing and decreasing POD with
increasing flaw width, two values for α & β were produced using RIM/ regression
analysis [Bullock et al. 1994]. The first α & β values were obtained for the 3 mm to 9
mm flaw width intervals. The second α & β values were obtained for the 11 mm to 29
mm flaw width intervals. To create the POD curve, the POD function was run using
the first set of α & β values with the detection data for the 3 mm to 9 mm flaw width
intervals, and the values recorded. The same was done for the 11 mm to 29 mm flaw

144
size intervals, this time using the second set of values for α & β. The resulting POD
curve (figure 5.35) combines all of the function outputs, and plots them against the
corresponding flaw size interval, hence, the curve having a clear ‘switch’ point as
POD begins to reduce.

The POD curve produced from the detectability data from trial 4 is presented in figure
5.35. The full data tables can be seen in appendix F. For both types of flaws, the 3
mm to 11 mm flaw width intervals gave increasing detectability with increasing flaw
width, and the 13 mm to 29 mm flaw width intervals gave decreasing detectability
with increasing flaw width. As seen in figure 5.35, the POD curves for the blended
and hard edged flaws were similar for flaws up to 10 mm wide. The POD curves also
indicate a greater rate of decrease in POD with increasing flaw width for the blended
edge flaws than that of the hard-edged flaws greater than 11 mm wide.

Trial 4 - Hard & Blended Edge Flaws

(Log-logistic (Log-Odds) Model/ Range Interval Method)

1
0.9
0.8 Hard Edged Flaws
0.7
0.6
POD

0.5
0.4
0.3
Blended Edged Flaws
0.2
0.1
0
0 5 10 15 20 25 30
Flaw Width Ø (a) mm

Figure 5.35 – Comparison of POD vs. flaw width for hard edged and
blended edge flaws, from results of trial 4

As seen in the scatter plot in figure 5.34, for all but one flaw greater than >15 mm
wide, the detectability is lower for the equivalent sized blended edge flaw than the
hard edged flaw. The difference varies from between approximately 10% and 75%
lower detection rate for the blended edge flaws. The POD analysis of the results data
similarly indicate lower detectability for the blended edge flaws greater than >15 mm

145
wide. The results from trial 4 suggest that a hard-edged, 0.15 mm deep, hemispherical
shaped flaw would be detected with greater than 50% POD so long as it is between
3.8 mm and 26 mm wide. However, the results from the blended edge flaws suggest
that a 0.15 mm deep hemispherical flaw will only >50% detectable up to a width of
Ø17 mm.

5.4.5 – Summary of results from series 1 of virtual trials

The results from the first series of virtual trials can be summarised as follows:

• <Ø5 mm flaws with blended edges returned detection rates less than 90% and
approaching 0% for flaw depths <0.5 mm in trial 3.

• For flaws of the same width and >Ø15mm, flaws with blended edge shapes
were generally less detectable than flaws with hard-edged shapes in trial 4,
with the discrepancy in detection percentage varying from <10% up to >75%.

• For a constant flaw depth of 0.15 mm, detection rates for wide flaws (>Ø15
mm) reduced with increasing flaw width for the blended edge flaws, which are
similar in size and shape to impact flaws observed on fully finished CFRP
laminate coupons.

5.5 – Results of visual inspection trials – series 2

The second series of visual inspection trials to assess the reliability of visual
inspection for composite aircraft structures was performed using volunteer
participants and both virtual specimens and physical facsimile specimens, as
described in chapter 4.10. A single trial constituted visual inspections of a set of 600
mm x 600 mm specimen panels, by an individual participant. The trial length was
approximately 30 minutes, of which 10 minutes were set aside for briefing and the
remaining 20 minutes were for performing the inspection. Seven sets of trials were
performed:

1. Visual Inspection Trials with grey virtual specimens – 15 participants

2. Visual Inspection Trials with gloss grey specimens – 15 participants
3. Visual Inspection Trials with gloss white specimens – 16 participants
4. Visual Inspection Trials with gloss blue specimens – 16 participants
5. Visual Inspection Trials with matt grey specimens – 17 participants

146
 6. Visual Inspection Trials with matt white specimens – 18 participants
7. Visual Inspection Trials with matt blue specimens – 20 participants

Each trial was successfully implemented, and all participants successfully completed
their trials. In each trial, the participants marked their answers as detailed in chapter
4.14. After completion of trials with each colour / finish, the answer sheets were
checked against a marking sheet in order to identify correct detections, missed flaws
and false calls, as described in chapter 4.15. The results of each trial are presented
forthwith.

5.5.1 – Results data for series 2 visual inspection trials

The results of each of the visual inspection trials were originally recorded as a set of
simple hit/ miss data as seen in appendix G. The detection % for each surface flaw
was calculated by dividing the total number of successful detections for each flaw by
the total number of participants that inspected each flaw. Using this value instead of
the total number of hits for each flaw allowed the results of trials with different
numbers of participants to be compared. A table of the detection percentage for each
of the flaws in each specimen set is given in table 5.07.

147
 Flaw Size % Detections

Flaw Number
Flaw Type

Gloss Grey

Matt Grey

Gloss Blue

Matt Blue
Projector
Width

Depth

White

White
Gloss
(mm)

(mm)

Matt
1 9.03 0.09 0% 33% 41% 6% 0% 44% 5%
2 10.49 0.11 7% 7% 35% 19% 22% 44% 10%
Ø20 mm/17ply

3 12.18 0.14 0% 87% 71% 75% 39% 75% 10%

4 14.15 0.19 20% 93% 94% 100% 94% 69% 45%
5 16.44 0.26 80% 27% 35% 75% 94% 69% 50%
6 19.11 0.37 100% 100% 94% 100% 100% 69% 95%
7 22.2 0.57 100% 93% 94% 81% 89% 81% 50%
8 25.79 0.94 100% 100% 100% 100% 100% 100% 100%
9 7.39 0.1 7% 13% 12% 6% 6% 25% 0%
10 9.03 0.11 0% 27% 29% 19% 39% 6% 5%
Ø20 mm/33ply

11 11.02 0.13 0% 80% 100% 75% 56% 31% 5%

12 13.46 0.16 0% 20% 18% 13% 0% 50% 0%
13 16.44 0.2 40% 20% 47% 50% 33% 63% 10%
14 20.09 0.26 60% 93% 100% 100% 94% 69% 60%
15 24.53 0.37 100% 93% 94% 88% 100% 56% 95%
16 29.96 0.57 100% 100% 100% 100% 100% 81% 85%
17 11.02 0.1 0% 33% 24% 13% 39% 38% 5%
18 13.46 0.12 0% 33% 29% 38% 44% 6% 5%
Ø87 mm/17ply

19 16.44 0.13 13% 13% 24% 13% 6% 56% 10%

20 20.09 0.16 7% 67% 71% 81% 67% 75% 30%
21 24.53 0.19 33% 100% 88% 81% 61% 38% 50%
22 29.96 0.25 73% 93% 88% 75% 33% 81% 40%
23 36.6 0.33 47% 93% 100% 100% 89% 56% 75%
24 44.7 0.49 67% 100% 88% 94% 78% 69% 85%
25 14.15 0.16 20% 87% 53% 81% 56% 19% 45%
26 16.44 0.17 27% 53% 65% 56% 50% 19% 25%
Ø87 mm/33ply

27 19.11 0.18 7% 80% 41% 50% 17% 38% 25%

28 22.2 0.19 27% 60% 47% 50% 83% 56% 15%
29 25.79 0.2 47% 73% 53% 38% 22% 75% 15%
30 29.96 0.21 27% 40% 35% 19% 17% 50% 10%
31 34.81 0.23 20% 27% 59% 13% 0% 50% 0%
32 40.45 0.25 7% 67% 59% 6% 22% 44% 0%
Table 5.07 - % of detection for each flaw on each colour & surface finish

Whilst it is possible to plot simple graphs of percentage of detections vs. flaw width,
or flaw depth, such graphs are unable to illustrate flaw depth at the same time.
Similarly, simple graphs of detections vs. flaw depth are unable to convey the fact that
two flaws of the depths may have different widths. As flaw visibility is believed to be
dependent on both width and depth, it is of little use to attempt to base conclusions on
graphs that only list one of the variables.

148
To allow display of both the width and depth of a surface flaw at the same time as the
percentage of detections, 2D plots of detection percentage vs. flaw width & flaw
depth were produced. Figures 5.36 to 5.42 illustrate matrices of detection percentages
for surface flaws on the grey virtual specimens, and the gloss & matt grey, white and
blue facsimile specimens. The flaw depth range was split into 18 equal intervals of
0.05 mm depth and the flaw width range was split into 8 equal intervals of 5 mm
width. Each flaw in the specimen set was assigned to the appropriate size interval, and
the detection percentages of all the flaws within each interval range were averaged,
thus providing a single percentage detection value for all of the flaws within a given
depth / width interval.

Figure 5.36 – Matrix of detection percentages arranged by flaw width and flaw depth,
obtained using virtual grey specimens

Figure 5.37 – Matrix of detection percentages arranged by flaw width and flaw depth,
obtained using gloss grey specimens

149
Figure 5.38 – Matrix of detection percentages arranged by flaw width and flaw depth,
obtained using matt grey specimens

Figure 5.39 – Matrix of detection percentages arranged by flaw width and flaw depth,
obtained using gloss white specimens

Figure 5.40 – Matrix of detection percentages arranged by flaw width and flaw depth,
obtained using matt white specimens

150
Figure 5.41 – Matrix of detection percentages arranged by flaw width and flaw depth,
obtained using gloss blue specimens

Figure 5.42 – Matrix of detection percentages arranged by flaw width and flaw depth,
obtained using matt blue specimens

The detection matrices are useful for comparing the detection rates of a particular size
range. For example, it can be seen that for flaws within the 0.1 to 0.149 mm depth and
5.1 to 10 mm width range, the detection rate never exceeds 25% (Gloss Blue).
Likewise, it is also possible to identify that the flaw size range 0.9 to 0.949 mm deep
and 25.1 to 30 mm gives 100% detection on the virtual specimens and all variations of
the facsimile specimens. However, the effects of width, depth, colour and surface
finish on detection rates are difficult to discern using matrices.

A second method of presenting the results was to use Statistica statistical data analysis
software [Statsoft Inc. 2008] to produce scatter graphs, with the percentage of
participants detecting each flaw plotted against the corresponding flaw depth and flaw
width of each flaw (see figures 5.43-5.49, and appendix H). A natural log scale is
used for the X and Y-axis, allowing greater separation of the individual data points at

151
lower values of width and depth. The scatter plots allowed the detection percentage
for each flaw to be seen, and compared, in terms of depth and width, against the other
flaws in the specimen set. For example, in figure 5.44 it can be seen that on the gloss
grey specimens, there is no clear correlation between flaw width and detection %.
However, all the flaws over 0.3 mm deep were detected by more than 90% of
participants. On gloss grey specimens, there were instances of greater than 90%
detection when flaws were below 0.3 mm. On gloss blue specimens, however, there
were no instances of greater than 90% detection when flaw depth was below 0.3 mm.

The scatter plots were also useful for comparing the detectability of individual flaws
across the range of surface colours and finishes. For example, it can be seen that the
detection of the 44.7 mm/ 0.49 mm deep flaw varied between 69% on gloss blue
specimens and 100% on gloss grey specimens.

Figure 5.43 – Scatter plot of % of participants detecting each flaw vs. flaw depth &
width for virtual grey specimens

152
Figure 5.44 – Scatter plot of % of participants detecting each flaw vs. flaw depth &
width for gloss grey specimens

Figure 5.45 – Scatter plot of % of participants detecting each flaw vs. flaw depth &
width for matt grey specimens

153
Figure 5.46 – Scatter plot of % of participants detecting each flaw vs. flaw depth &
width for gloss white specimens

Figure 5.47 – Scatter plot of % of participants detecting each flaw vs. flaw depth &
width for matt white specimens

154
Figure 5.48 – Scatter plot of % of participants detecting each flaw vs. flaw depth &
width for gloss blue specimens

Figure 5.49 – Scatter plot of % of participants detecting each flaw vs. flaw depth &
width for matt blue specimens

155
When viewing the scatter plots, it became apparent that the levels of detection
percentage appeared to form bands, or contours, of constant detectability. In order to
explore the possibility of detectability bands further, the wafer plot function of
Statistica was used to fit the scatter plots with contours representing levels of
detection percentage. The wafer plots are shown in figures 5.50 to 5.56, and in
appendix H without annotations. The wafer plots also use a natural log scale on the X
and Y-axis. The contours on the wafer plots are best-fit lines of constant detection
percentage. The effects of flaw width and depth on detection rates, and the differences
in detection rates obtained on different specimen colours and finishes were easily
visualised on the wafer plots.

5.5.2 – General Summary of results from series 2 trials

All of the plots in figs 5.50 to 5.56 have a number of similar features, differing in
detail with changes in sample colour and surface finish. At small values of depth and
width percent detections are low and approach zero. The lower limit of the data is at
flaws of 0.1 mm depth and 10 mm width. The upper limit of data is at flaws
approaching 1 mm deep and 30-40 mm wide. Here detections approached 100%. The
extent of the region of 90-100% detections varied depending on the colour and
surface finish of the samples. The region of high (90 to 100%) detection percentages
generally extended in a ridge from a centralised region at the highest values of depth
towards smaller values of width and depth. The shape of the high detection percentage
region meant that at constant flaw depth the effect of increasing width was first to
increase the detection percent, and then with further increases in width to decrease the
detection probability. All the data sets from the physical facsimile specimens show
some local contour anomalies, which probably reflect variability within the hit/miss
data. However, the contours of the results obtained using virtual specimens exhibit far
better defined contour lines and clear variations in detectability with variations in
width for flaws of the same depths.

5.5.3 – Detailed summaries of results from series 2 trials

The wafer plot produced from the results of the trials with virtual specimens (figure
5.50) is similar in overall shape to the plots obtained from trials with physical
facsimile specimens. However, the region of 90~100% detection is limited to flaws
<Ø35 mm and >0.3 mm deep. Flaws below 0.2 mm deep failed to achieve >60%

156
detection and detection of flaws >Ø35 mm varied, with detection increasing with
depth from a minimum of 10% to a maximum of 90%.

Figure 5.50 – Wafer plot of virtual grey specimen results with comments

In figure 5.51, the wafer plot of gloss grey results shows a large flaw depth and width
region in which flaws were detected by over 90% of participants. The contours
indicate that for flaws of small depth and width, the detection was at its lowest rate. If
a nominal flaw depth of 0.2 mm is chosen, the wafer plot illustrates that flaw
detection rates can vary between <10% and >90% as width varies; i.e. detection can
vary for the same depth of flaw, depending on the flaw width.

Figure 5.52 illustrates the wafer plot of the matt grey results. It can be seen that the
matt finish has little effect on detectability of flaws greater than 0.25 mm deep; the
region of 90-100% detection is similar in size and shape to the same region on the
gloss grey diagram. There is a slight change to the contour shape for flaws of smaller
width and depth, where the <10% detection band is no longer present. As also
observed for the gloss grey specimens, the detection contours vary with width for
constant depths. The overall implication is that there is little difference in detection
rates on gloss and matt grey specimens.

157
 Figure 5.51 – Wafer plot of gloss grey specimen results with comments

Figure 5.52 – Wafer plot of matt grey specimen results with comments

In figure 5.53, the wafer plot of the gloss white specimens illustrates that there are
some differences compared to the results with grey specimens. The large area of 90-
100% detection has changed shape slightly. For flaws between 0.2 mm - 0.3 mm
deep, and >25 mm wide, the detection percentage reduces as width increases.
However, for flaws below 0.2 mm deep, there is little difference to the detection

158
contours compared to the gloss and matt grey results. This implies that are slight
differences in detection rates on gloss white specimens compared to gloss and matt
grey specimens.

Figure 5.53 – Wafer plot of gloss white specimen results with comments

Figure 5.54 – Wafer plot of matt white specimen results with comments

In figure 5.54, which shows the results from matt white specimens, there are notable
differences to the contour shapes on the gloss white results. The large area of >90%
detection has changed shape, and no longer includes flaws wider than 38 mm. The
detection of flaws >38 mm wide now varies from <10% to >90% depending on flaw

159
depth. For the flaws between 0.2 mm & 0.3 mm deep and >25 mm wide, there is
greater variation in detection rates compared to the same region on the gloss white
results. Detection of flaws <0.2 mm deep and <20 mm wide saw reduced rates on the
matt white specimens compared to the gloss grey and white, and matt grey specimens.
The results seen in figure 5.54 imply that the matt finish altered detection rates on the
white specimens.

Figure 5.55 – Wafer plot of gloss blue results with comments

Figure 5.56 – Wafer plot of matt blue results with comments

160
Figure 5.55 shows the results obtained from the gloss blue specimens. In this plot, it
can be seen that there is a significantly different shape to the detection contours
compared to the results from grey and white specimens. The lower limit of the region
of >90% detection has shifted to 0.7 mm deep compared to extending down to 0.25
mm deep for the results from the grey and white specimens. The detection contours
for flaws >0.2 mm vary with flaw width from 60% to 80% detection compared to
variations from <10% to >90% detection on the other plots (fig 5.51 – 5.54 & 5.56),
i.e. flaw detections for the same flaw depth vary less with width on the gloss blues
specimens. For the gloss blue specimens, in particular for flaws >0.2 mm deep, the
detection contours vary less (0 to 30% variation vs. >50% variation) with width for
flaws of the same depths than for the same region on grey & white specimens.

Figure 5.56 illustrates the results from the matt blue specimens. It can be seen in
figure 5.56 that detection percentages for flaws 0.2 mm to 0.5 mm deep vary from
<30% to >90% with width, compared to <50% to <80% on gloss blue specimens . It
can also be seen that detection of flaws with <0.3 mm depth is reduced to below 50%
for matt blue specimens. The results seen in figure 5.56 imply that a matt blue finish
gives different detection results to a gloss blue finish.

161
Chapter 6 – Discussion
The visual inspection trials were performed in order to investigate the detection of 3D
surface flaws that represent impact damage to a painted, mesh incorporated CFRP
laminate. As detailed in chapter 5.1, falling weight impact damage to painted, mesh
incorporated CFRP laminates produced a different surface topography, and different
visual appearance to the same impact conditions on an unpainted, non-mesh laminate
of the same thickness. Many of the examples of impact damage from hemispherical
shaped objects to painted laminates, as presented in chapter 5.1, exhibited a 3D
surface indentation, which was free of surface breaking cracks. However, the lack of
published examples of impact damage to fully finished, painted monolithic CFRP
laminates inhibits comparisons with other works.

The lack of published examples of impact damage to painted laminates also means
that the obtained flaw geometry, or topography data cannot be compared to published
works. However, the data obtained for flaw depth vs. impact energy plots, as seen in
chapter 5.16, were best fitted by an exponential model or line, which is in keeping
with results obtained on unfinished laminates [Brookes 2004]. The fit quality of the
geometry variable relationships is discussed later in this chapter.

In previous published studies on reliability of visual inspection of composite

materials, surface colour and surface finish were identified, but not demonstrated, as
factors having the potential to affect visual inspection reliability [Erhart et al., 2004;
Gant, 2007]. To date, no published study demonstrates the effects of both surface
colour and surface finish on visual inspection reliability for impact damage to
composite materials. The results presented in this thesis successfully demonstrate the
effects of both surface colour and surface finish. Furthermore, the results demonstrate
that colour and finish had interactive effects on visual inspection reliability.

A particularly noteworthy issue arising from the visual inspection trials results is the
low detectability (<70%) of flaws representing impact damage from larger diameter
objects (87 mm), at relatively high (60J) energy impacts on 33ply painted CFRP
laminate. Traditionally, a 10, 15 or 20 mm impactor is used for impact testing and
CAI studies on composite materials, and flaws representing lower energy (50J) impact

162
damage from a 20mm tip returned greater than 80% detection rates. The lack of
published impact testing work using larger diameter (>25 mm) tips has resulted in few
examples of larger size, higher energy impact damage to CFRP laminates. However,
the impact damage examples presented in this thesis demonstrate significant
differences between the flaw depth/impact energy and flaw width/ impact energy
relationships for smaller and larger size hemispherical impactors. Furthermore, the
visual inspection trials results lead to the previously unpublished suggestion that
impact damage at relatively high impact energy from larger sized objects may be less
detectable by visual inspection.

The “bigger is better” assumption for NDT reliability is inherent and deep rooted in
the 20+ years of research into NDT system reliability assessment. However, the visual
inspection reliability data presented in chapter 5.5 pose a significant challenge to the
traditional reliability assessment assumption of increasing probability of detection
(POD) with increasing flaw size. Within the results obtained from visual inspection
trials with facsimile specimens, it is evident that flaw width and visual inspection
reliability are not positively correlated for impact damage to painted CFRP laminates.
In fact, as seen in figures 5.51 to 5.56, for flaw depths < 0.25 mm there is strong
evidence that detection rates will be reduced for wider flaws (>30 mm). Whilst the
results presented in chapter 5.5 demonstrate greater detection rates at greater depths, it
is unfeasible to assume that all types of damage above a given depth threshold will
have greater detectability. Whilst the results may support this assumption for a
particular type of impact damage, i.e. a 20 mm hemispherical object, the results
demonstrate that when the detectability of different types of impact damage are
considered altogether, an increase in flaw depth does not necessarily correspond to an
increase in detectability.

Published experimental paradigms and protocols are written solely with NDT of
metallic structures, and inspection for cracks, in mind. In implementing a reliability
assessment of visual inspection for three-dimensional impact damage to composite
materials, it has become apparent that the established “metallic” experimental
paradigms and protocols require a certain degree of interpretation in order to be
applied to the experimental design. Hence, a rather poignant question; has the same

163
interpretation, if any, been made by each aircraft manufacturer when determining
BVID sizes for composite structures?

Either because of inability to translate elements, or ignorance of existing paradigms &

recommendations the specimens used in published work on visual inspection of
composite aircraft structures [Erhart et al.; 2004, Gant 2007] fail to demonstrate
constancy between experimental variables such as flaw geometries and sizes, lighting
conditions, inspection environment and specimen set-up. The combination of these
issues with variations in laminate type and surface finish treatments leads to an
inability to make meaningful direct comparisons between detection results obtained
by others and those presented in this thesis.

6.1 – Experimental issues; impact damage

6.1.1 – Flaw width measurements on impact damaged CFRP coupons
The examples of impact damage to coupons of painted, mesh incorporated CFRP
laminate were produced according to manufacturer’s specifications, and standard
falling weight impact procedures were employed. However, there are currently no
published standard procedures for characterising the subsequent surface flaw
topographies. Various methods for characterising impact damage morphology are
presented in this thesis, and whilst delamination size and flaw depth measurements
were made with relatively high accuracy, width measurements presented a particular
challenge.

Flaws with low depth and highly blended shapes presented the greatest challenge to
making the distinction between flawed and unflawed surface. Small undulations in
surface angle due to the paint finish caused such significant ‘noise’ in surface angle
measurements. Such noise rendered useless any attempts to use differences in surface
angle as indicators of where flaws finished and returned to having the same surface
angle as the unflawed surface. The method described in chapter 4.5.5 was thus
developed as a constant, workable solution to the width measurement challenge. The
three depth measurement points described in chapter 4.5.5 are spaced at 1 mm
intervals, and it is acknowledged that an error of a single point either side of the three
chosen points was possible in determining three points with consecutively lower
depths. Hence, the accuracy of the flaw width measurements was within +/- 5 mm for

164
each side of the flaw, resulting in an overall width measurement accuracy of +/- 10
mm.

Others [Brookes 2004] have reported the use of digital images and subsequent image
processing to make width measurements. However, this method appears to use
changes in surface angle to distinguish between flawed and unflawed surface. NDT
methods such as moiré reflectology (D-Sight) also rely on changes in surface profile
to quantify impact damage size. Autoclave cured, unpainted CFRP laminates typically
have smooth surfaces, free of paint finish effects such as pinholes or orange peel
which cause noise in profile or angle measurements. Use of peel plies during laminate
processing presents a further source of surface profile noise, which may increase the
difficulty in using a surface angle change threshold to identify flaw endpoints when
measuring flaw widths.

Bearing in mind the surface angle or surface profile noise associated with paint
finishes or peel plies, future work may wish to investigate the accuracy of flaw-width
measurement methods or procedures for impact damage on fully finished CFRP
laminates.

6.2 – Experimental issues – visual inspection trials

6.2.1 – Inspection time
For the series two visual inspection trials, a minimum of 20 seconds were required for
change over of each panel, and a maximum of 30 minutes trial duration was imposed
in order to avoid the onset of fatigue in participants. This resulted in a maximum
display time for each specimen of 5 seconds for each of the 64 specimen panels.
Drury, 1993 demonstrated that a search time of 5 seconds on a complex visual
inspection task was the minimum required in order to identify the correct target
amongst 10 others, although actual eye fixations on targets (i.e. detections of flaws as
opposed to comparison between flaws to identify a particular target) occur in
milliseconds rather than seconds [Megaw & Richardson, 1979]. The visual
participants in the trials presented in this thesis were asked to perform the simpler task
of identifying only the presence of flaws, as opposed to searching for all the flaws and
making a decision between which one should be marked down. The display duration
of 5 seconds was therefore considered more than adequate time to search the

165
specimen surface and detect several flaws. The ability of some participants to identify
all three flaws on facsimile specimen panels during visual inspection trials suggests
that the display of 5 seconds was in fact adequate, and that non-detections were as a
result of flaws being too ‘small’ to detect, as opposed to participants having
insufficient time to search the entire specimen surface. However, without the
employing eye tracking equipment to determine participant visual search patterns, this
can only remain a suggestion and cannot be considered hard evidence.

6.2.2 – Use of novice inspectors in visual inspection trials

The 600 mm x 600 mm panels were intended to represent a small inspection zone on
an aircraft structure, and the surface flaws represented impact damage to a CFRP
aircraft structure. The participants were shown examples of the facsimile flaws prior
to each trial, and the visual inspection task was simply to identify the presence of any
similar flaws on a specimen panel. The participants were not required to make
judgements regarding the severity of each flaw or whether repair, or further NDT was
required, as would normally be the case during a real inspection of an actual aircraft.
Furthermore, the panels did not represent a particular section of aircraft structure, thus
eliminating any expectancy of finding a particular type of damage. Any aviation
experience or experience of composite materials would have had no bearing on the
inspection outcome; hence, the use of novice participants. Furthermore, the work of
Gallwey & Drury, 1986 demonstrated that novice and trained inspectors returned
comparable results for tasks where inspector experience and expectancy were
redundant in the task. The participant’s responses in the visual inspection trials
therefore represent the simplest possible indication of whether a human eye, be it that
of a novice or trained inspector, could or could not detect the presence of each flaw.

6.2.3 – Lighting
The visual inspection trials setup used only ambient lighting without additional
directional lighting or sources of light ‘edges’, which are aids to detection of 3D
surface flaws (see Faulkener & Murphy, 1979 and Lloyd & He, 1998). Gant, 2007
investigated the use of additional directional lighting and/ or flashlights and found
subsequent improvements in detection rates for visual inspection of impact damage to
composite aircraft structures. However, lighting conditions, lighting design and
lighting availability in aircraft hangars will vary from facility to facility, making them
difficult to represent in a single trial. Inspection surfaces may also be poorly lit due to

166
being on the underside of an aircraft or outside the range of available light sources,
leading to reduced detection rates (as demonstrated by Gant, 2007 and Erhart et al.,
2004).

The results of the visual inspection trials presented in section chapter 5.5 represent
inspection under ambient light, which can be considered the basic lighting
requirement for visual inspection (see Brombach et al., 2006). The detection results
therefore represent a ‘baseline’ level of visual inspection reliability. The work of Gant
and Erhart et al. suggests that the detection rates for flaws will be reduced due to
inspection on under wing surfaces, and the work of Gant, 2007 and Lloyd & He, 1998
suggests detection rates would increase with the use of flashlights & lamps, additional
directional light sources and grid or strip luminaires.

6.2.4 – Design of facsimile specimen flaws

The design if the facsimile specimen flaws was based on relationships between impact
energy, flaw width and the other variables (see chapter 5.1.6) that were derived by
fitting lines to scatter plots of the CMM characterisation data from actual impact
damaged painted CFRP laminate coupons (described in chapter 5.1.6). As can be seen
in the plots, particularly in figures 5.23 & 5.25, the quality of fit varied depending on
the impactor / laminate combination. However, the key objective for the
characterisation work was to arrive at a method by which the sectional profile of
impact damage from different impactor/ laminate combinations could be constructed
for any given impact energy or flaw width. In order to avoid using several different
model types (i.e. linear, logarithmic, exponential) for each geometry variable across
the four combinations of impactor size and laminate thickness, the variance in fit
quality was accepted, and only a single model type was adopted for each relationship.
Improvements in fit quality could be obtained by producing further examples of
falling weight impact damage in order to populate the same impact energy range at
reduced impact energy intervals, with repeat examples at the same impact energy.
Due to the inherent variability in flaw size and geometry of falling weight impact
damage, and the approximate nature of the flaw geometry variable relationships, it
was accepted that slight variations might occur between flaw geometries of actual
flaws produced by others on similar CFRP test coupons and the facsimile flaws used

167
in the visual inspection trials. Such variations should not detract from the overall
visual inspection reliability issues that arise from the visual inspection trials results.

The sizes and geometries of the facsimile specimens and machined flaws used in the
visual inspection trials were based on characterisation data from actual examples of
impact damage to CFRP laminate coupons. Close tolerances (<15% of the specified
dimensions) to the original size and geometry specifications resulted in machined
flaws that were the most authentic possible representations of impact damage flaws on
painted CFRP laminates. Their authenticity could only be further improved by using
more sophisticated computer aided machining methods whereby the impact damage
flaws on the CFRP laminate coupons would be reproduced by machining a CMM
digitisation of the damaged laminate surface into the specimen. Such a method might
provide the means by which non-symmetrical, non-geometric flaws could be
reproduced on facsimile specimens.

In reliability assessment experiments, it is traditional to vary only a single flaw size

parameter or variable, which has conveniently only ever needed to be crack length for
metallic NDT/ NDI reliability assessments. However, the sectional profiles of the
differing surface topographies of impact damage to CFRP laminates could only be
quantitatively described by using a series of geometric variables, which, as previously
described, were interlinked by various relationships. A change in size or value for one
variable subsequently resulted in a new value for the others. Varying a single flaw
geometry value and holding the others constant would have broken the relationship to
the other flaw geometry variables, i.e. the flaw geometry would no longer be
representative of a particular impactor geometry/ laminate combination. In order to
ensure authenticity to the Ø20 mm & Ø87 mm impact damage on 17 & 33 ply painted
CFRP laminates whilst covering a broad range of realistically sized and shaped dents,
the specimens for the series 2 trials contained flaws that had varying values of flaw
width, flaw depth and other geometric variables as described in table 4.06. It was thus
difficult to isolate the effect of a particular flaw geometry (flaw size) variable, or
impactor geometry (flaw type) on visual inspection reliability within the results
obtained.

168
6.2.5 – Size range of physical trial results
It is acknowledged that the results presented in chapter 5.5 cover a relatively narrow
band of flaw widths and depths. Consequently, the question of detectability of flaws
outside of this band arises. Accurate predictions of flaw detectability outside of the
reported results would require further visual inspection experiments using an extended
range of flaw sizes in order to populate detection data for the sizes not covered,
allowing the contours produced in figures 5.50 to 5.56 to be extended. However, some
discussion is made in the following sections of the possible implications for
detectability of flaws outside of the range used.

6.2.6 – significance of participant responses

The physical specimen trials were undertaken by between 15 and 20 participants,
hence a single participant’s response could alter the detection rate for a single flaw by
between 5% and 6.7%. Fifty participants would be required to reduce this figure to
2%, and 100 participants would be required for 1%. By reducing the variation in
detectability for a single flaw imparted by a single participants answer, greater
numbers of participants may also serve to improve the distinction of the detectability
bands represented in the contour plots shown in figures 5.50 to 5.56. Increased
participant numbers coupled with an increased flaw size range (as described above),
would help increase the viability for using the contour plots described in this thesis as
guides to impact damage detectability.

6.2.7 – Generic applicability of physical trial results

The visual inspection results obtained in the series 2 trials are considered
representative of the visual inspection reliability of Ø20 mm & Ø87 mm impact
damage to 17ply and 33ply painted CFRP laminates. In addition, the visual inspection
trials results also represent visual inspection reliability of impact damage from any
impactor size/ impact energy/ laminate combinations that produce non-surface
breaking crack, 3D flaws with similar geometries to those used in the visual
inspection experiments. Impact flaw geometries obtained by others may be compared
to those presented in table 4.06, and if similar or matching geometries are obtained,
the associated detectability of such flaws may be approximated from the results
shown in chapter 5.5. It is important to note that the visual perception, and hence
detectability of flaws exhibiting surface breaking cracks is different to that of

169
smoothly shaped 3D surface indentations, therefore such flaws are outside of the
visual inspection reliability results presented in this thesis.

It is also noted in figures 5.50 to 5.56 that for each surface colour and finish
combination, a region of >90% detections occurred. Whilst the range of widths and
depths that fall into this region vary with colour and finish, it can be seen in figures
5.44 to 5.49 that a 25.79 mm wide, 0.94 mm deep facsimile dent representing 39J 20
mm hemispherical impact damage to a 17ply painted CFRP laminate was detected by
all participants. Therefore, it can be said that for impact damage dents similar in size
and shape to the described dent, >90% POD may be expected, regardless of surface
finish or colour. However, the contour plots in figures 5.50 to 5.56 demonstrate that
>90% detectability cannot be assumed for flaws of greater width or greater depth than
the described flaw.

The results presented in chapter 5.1 demonstrate that for the same laminate type, the
surface flaw geometry can vary depending on the impactor geometry and impact
energy. Furthermore, it should be remembered that the surface flaw topographies and
associated sub surface delaminations produced by falling weight impacts were
established in this instance on 100 mm x 150 mm coupons manufactured from
AS4/8552 pre-preg. In large-scale structures, the flaw topography and delamination
size will be influenced by the compliance of the impacted structure as well as by the
toughness and stiffness of the composite resin and fibre system used.

The flaws used in the visual inspection trials, and the detection results therefore
represent only a small selection of the possible impact scenarios that could occur on
aircraft in service. Thus, although the detection results obtained using various sizes
and shapes of flaw on various colours and surface finishes will have generic
applicability across all materials and structures where similar surface flaws occur,
flaw topography, level of sub surface damage and subsequent flaw detectability needs
to be determined by experiment for each application.

A specific question was posed to the author of this thesis regarding the applicability of
the results data to existing damage detectability thresholds such as the 1.2 to 1.3 mm
dent depth detectability threshold for GVI used by Airbus and the Boeing threshold

170
dent depth of 0.254 to 0.508 mm [Rouchon 2010]. The detection results reported in
this thesis do not extend to the dent depths used by Airbus, but do cover the lower
threshold value used by Boeing. At first glance, therefore, it would appear that the
results imply that the 0.5 mm depth upper BVID threshold employed by Boeing is
less conservative than the Airbus approach. Flaws within the 0.254 to 0.508 mm depth
range were demonstrated to be less than 50% detectable, depending on colour and
finish whereas detectability approached 100% for flaws approaching 1 mm in depth.
However, if the detectability contours plotted in figures 5.50 to 5.56 were to be
extended or extrapolated to cover a greater range of flaw depths and widths, it is
plausible that the issue of reduced detectability for very wide flaws even at higher
depths may present itself even above the dent depth threshold employed by Airbus.
The implications of this possibility are discussed in later sections.

6.2.8 – Existing experimental paradigms and protocols

Extensive research was conducted in order to ensure that the visual inspection
experiments carried out for this thesis conformed to existing experimental paradigms
and followed guidelines & recommendations for NDT/ NDI reliability assessments.
From this research, it is apparent that NDT/ NDI reliability for aircraft structures has
predominantly been concerned with inspection for cracks in metallic structures. As a
result, recommendations and guidelines for experimental design of NDT/ NDI
reliability assessment, such as Spencer’s generic protocol [Spencer et al. 1993]
AGARD LS-190 [AGARD 1993] or MIL-HDBK-1823 [US DoD 2007] are written
exclusively with inspection of metallic aircraft structures in mind.

Translation of recommendations from ‘metallic’ to ‘composite’ context

For ‘metallic’ inspection, it is relatively simple to implement or translate elements of
the existing experimental paradigms and recommendations such as use of specimens
that faithfully represent the aircraft structure under inspection, and use of flawed and
unflawed specimens, with a 1:3 flawed to unflawed ratio. However, there is currently
no established experimental paradigm or published recommendations to aid
translation of these elements into reliability assessment of visual inspection of
composite materials.

171
Whilst various barely visible damage (BVID) sizes have been published (Forsyth et
al. 1998 & Rouchon 2009) for impact damage on composite materials, the details of
how such sizes were determined are not available in the public domain. Damage
tolerance for composite aircraft structures is intrinsically linked to BVID sizes, and
despite the development of, and adherence to recommendations for metallic NDT/
NDI reliability assessment protocols, there is little evidence available as to how such
recommendations have been interpreted into protocols for determining BVID sizes for
composite structures.

For the visual inspection trials presented in this thesis, the 3:1 recommendation
[Spencer et al. 1993; AGARD 1993; US DoD 2007] was interpreted such that a single
specimen constituted an ‘inspection site’, and for every specimen in the set that
contained intentional damage or flaws, there were three more containing no
intentional damage. When attempting to translate the 3:1 ratio recommendation from
a ‘metallic’ to a ‘composites’ context, an issue of concern was that similar work such
as Gant 2007 and Erhart et al. 2004 did not implement the recommendation. There
was thus no published experimental precedent of how the recommendation should be
translated or implemented.

Without an available precedent of conformity to experimental recommendations, the

actual setup and size of specimens was also open to interpretation. Gant, 2007 used
specimens that were pieced together to represent a composite wing structure, and
presented it ‘on-aircraft’, thus representing inspection of an actual aircraft. The study
by Erhart et al., 2004, however, used a single panel specimen from an actual aircraft,
representing inspection of a single piece of damaged structure, presented in an
isolated environment to an actual aircraft. Major airliner manufacturers have not
published experimental procedures used for determining BVID sizes in the public
domain, although there is evidence to suggest that large panels comprising many
small size (100 x 100 mm) impact damaged coupons have been employed (see figure
6.01) [Fualdes 2006].

The methodology used in obtaining results for this thesis is believed be novel in
presenting individual, large size (600 mm x 600 mm) faithful facsimiles of impact

172
damaged composite aircraft structure, whilst using a 3:1 unflawed specimen panel to
flawed specimen panel ratio.

Figure 6.01 – Manufacturer’s setup for visual inspection reliability assessment

from Fualdes, 2006

Limitations of reliability assessments based on a single flaw size variable

The word ‘flaw’ itself is a contentious issue, as for impact damage to a composite
material, the resultant damage morphology is generally a surface indentation, which
may or may not exhibit surface breaking cracks, beneath which lies subsurface
delamination or cracking. Such damage cannot be simply described as a ‘crack’.
Whilst the use of the word ‘dent’ is more appropriate, dents are rarely described in
published literature by more than their depth and/ or width. These measurements
alone provide insufficient description of the surface flaw or dent topography, and as
such, are not suitable flaw size variables on which models of flaw detectability can be
solely based. Some description of flaw shape is required in order to be able to
determine its detectability. The visual inspection trials results demonstrate that

173
detection rates did not always increase with increasing flaw depth or increasing flaw
width. Furthermore, different shaped flaws, although similar in size returned different
detection rates (such as flaw number 22, as demonstrated in table 5.07).

Current log based probability of detection (POD) models are unable to accommodate
the above issues. Bi-variate plots of POD go some way to resolving the issue, by
allowing POD to be calculated as a function of two variables. However, the accepted
models and analysis methods for POD are logarithmic based, and unable to
accommodate a situation in which detectability increases then decreases with
increasing values of a given flaw size variable. As such, existing treatments for POD,
which are recommended for use in NDT/ NDI reliability assessment, are unsuitable
for use in assessment of the reliability of visual inspection for impact damage to
composite aircraft structures.

6.3 – Visual perception cues and visual inspection of impact damage

The visual inspection trials results identify effects on detectability of flaws,
representing impact damaged CFRP, due to surface colour and surface finish. These
results were obtained using ambient, non-directional light, leaving the specimen panel
surfaces free of reflections light sources, i.e. specular highlights. The area behind the
participants was kept free of objects, thus any specular reflections on the panel
surfaces were of a flat, light coloured wall.

Norman et al., 2004 identified that specular highlights aided 3D shape perception and
Lloyd & He, 1998 identified positive correlation between ‘severity’ or ‘saliency’ of
defects and their specular highlight area. Hence, if specular highlights were the
influential factor affecting flaw detectability, greater differences would be expected
between the contours plotted for detection of flaws on the highly specular, gloss
specimens and detection of flaws on the diffuse, matt grey finish specimens.
However, the fact that the contour plots of matt and gloss results (see figures 5.51 to
5.56) for each colour of specimens have similar features provides evidence that
specular reflections and highlights had minimal role in the perception of the 3D
surface flaws. Such evidence leads to the conclusion that much of the variation in
detectability of surface flaws came from variations in shading cue saliency.

174
This is not to say, however, that the flaws on the grey (and other coloured) specimens
failed to exhibit ‘lighter’ areas. Shading, and perceived colour as described in chapter
2.3, is a discrete visual cue, separate from specular highlights. Although the object
shown in figure 2.28 has a matt finish, it still exhibits ‘lighter’ areas of shading. As
deduced from Xiao & Brainard, 2008, and Norman et al., 2006, lighter areas appear
on a shape when the surface slant is such that the colour shade reflected back to the
observer is perceived as being lighter than neighbouring surfaces. A change in surface
slant, such as that of an indentation on a flat surface, may result in a ‘lighter’ colour or
‘darker’ colour being observed, hence, the appearance of lighter and darker shaded
regions where surface flaws are present (see again Xiao & Brainard 2008 and Norman
et al. 2006).

Combining the modularity between shading cues and original the surface colour, for
surface indentations, with the link between modularity & defect ‘severity’ raised by
Lloyd & He provides an explanation for the results obtained from the visual
inspection trials with physical specimens. Poor modularity between shading cues and
the base colour will lead to the two being perceived as the same colour, making the
shading imperceptible [Xiao & Brainard 2008], thus making the flaw difficult to
detect.

6.3.1 – Effects of surface colour & surface finish on modularity of visual

perception cues
Considering small width (<15 mm), low depth (<0.25 mm) flaws and larger,
shallower shaped (>Ø35 mm/ <0.3 mm) flaws it is reasonable to suggest that the
lighter and darker shading cues exhibited by such flaws occupy small surface areas,
and/ or their modularity with the surface colour is low, resulting in poor saliency.
Hence, the trend for lower detection rates for such flaws demonstrated on all colours
of specimen (see figures 5.44 to 5.49). For the grey specimens, both lighter (tending
towards white under white lighting) and darker areas (tending towards black under
white lighting) of shading had modularity with the grey base colour, thus both may be
perceived by the observer, if large enough. For the white coloured specimens, the
darker shaded areas had increased modularity (being dark grey/ black against white),
and greater saliency against the background colour. However, white is a ‘light’
colour, hence, surfaces whose slant causes them to reflect more light, and thus appear
lighter than their neighbours are indistinguishable due to the lack of contrast i.e. low

175
modularity with the light base colour. The lack of improvement in detection rates for
the gloss white over matt white specimens is thus explained by the following:
• Specular highlights, whilst they would also have poor modularity, were not
present to aid perception [see Norman et al. 2004].
• The areas of lighter shading on both finishes had poor modularity with the
base colour making them difficult to distinguish [see Xiao & Brainard 2008],
thus making the flaws difficult to detect.

Conversely, for the blue specimens, the darker areas of shading had poor modularity
with the dark base colour, reducing their saliency [Xiao & Brainard 2008] and thus
explaining why detection contours for the blue specimens are somewhat different to
those of the grey and white specimens. However, areas of lighter coloured shading
had greater saliency, due to their increased modularity with the dark base colour,
which implies that for flaws with higher detection rates, i.e. >0.5 mm deep, on the
blue specimens, it is the lighter areas of shading that made them detectable. As
described in chapter 2.3, specular, glossy surfaces produce small, precise highlights
whereas a matt surface provides diffuse, less intense, but larger size lighter shaded
areas. The reason for higher detection rates for flaws 0.3 to 0.4 mm deep on matt blue
specimens, as seen in figure 5.56, is believed to be that the flaws exhibited larger,
albeit less intense regions of lighter shading than the same flaws on the gloss blue
specimens under the visual inspection trial lighting conditions.

The general trend for higher detection rates on grey specimens than on white and blue
specimens is explained by the fact that both darker and lighter areas of shading had
modularity with the grey base colour, i.e. the observer had the chance of detecting
either or both cues, thus an increased chance of detecting the flaw.

6.3.2 – Effect of surface colour & finish on specular reflection cues

It is believed that due to the predominance of shading and shadow cues on the grey
and white specimens, distortions in specular reflections of the light coloured wall
behind the participants offered little advantage over the matt finish specimens, as
supported by the relative similarity between the contour plots for gloss & matt
finishes on grey and white specimens. Furthermore, the specular reflections of the
light coloured wall on the light coloured specimens would have been faint, most likely

176
indiscernible, to the participants. However, for the gloss blue specimens it is
conceivable that the contrast between the light reflected (specular) colour and the dark
surface colour would have increased the saliency of specular reflection distortions, or
increased the saliency of darker areas within the specular reflection caused by changes
in surface slant at flaw sites.

6.3.3 – Effects of flaw shape on visual perception cues

The issue of shadow cues provides explanation for the scatter in detection rates, and
thus irregularly shaped detection contours (as seen in figures 5.51 to 5.56). Although
the lighting used was not directional, for some of the flaws on the specimens, the
inclination of their internal surfaces may have been at sufficiently great an angle from
the unflawed surface as to result in occlusion of the available light, as illustrated by
figures 6.02 & 6.03.

Figure 6.02 – Section through ‘shallow’ surface flaw with low value of Øt and no
occlusion shadow present

Figure 6.03 – Section through surface flaw with higher value of Øt and subsequent
occlusion shadow

177
For relatively shallow (i.e. >25 mm/ <0.3 mm deep) flaws such as those produced by
larger radii (Ø87 mm) impactors even at high impact energies (>40J), the transition
between the large sized internal and external radii occurs over relatively shallow
angles, thus resulting in a ‘soft’ profile, and no occlusion shadow. The angle Øt
between the curve endpoint tangency line (see figure 6.02) and the unflawed surface
is relatively low for such flaws. However, for a ‘hard’ profile flaw, the sectional
profile is such that the internal and / or external radii are small, and their transition
occurs over a short distance, resulting in a large angle Øt between the tangency line
(i.e. flaw surface) and the unflawed surface as seen in figure 6.03. For these ‘hard’
profiled flaws, such as flaw number 8 (see table 4.06), which typically represent high
energy (>30J), smaller size (Ø20 mm) impactor damage to a CFRP laminate, an
occlusion shadow has a greater likelihood of occurring, thus providing a further cue to
aid visual detection.

Furthermore, changes in sectional profile shape translate into changes in surface slant,
which, as the work of Norman et al., 2004 demonstrates, affect surface shape
perceptibility. ‘Hard’ profile flaws will provide more significant changes in surface
slant, than ‘soft’ profiled flaws (see figures 6.02 & 6.03), resulting in more salient,
and thus more detectable changes in surface shading and colour.

For impact damage dents on CFRP laminates, the geometric variables that are driven
by the dent shape profile are those that relate to the internal curvature (see figure
4.12). During the experimental design for the visual inspection trials with physical
specimens described in this thesis, a decision was taken to distribute the flaw sizes by
widths and depths (see chapter 4.10.5). The values for the internal curvatures were
driven by maintenance of relationships to width and depth that were determined for 4
types of impact damage. Within the specimen set, few flaws had both similar widths
and depths, meaning that for a given flaw width or depth, the effect of differences in
internal geometry on dent detectability could not be isolated. Plotting detection results
as a function of internal geometry variables would be meaningless without first
isolating their effect from those of dent width and depth, i.e. holding width and depth
constant for variations in internal flaw geometry. Isolation would be required in order
to provide conclusive evidence as to the effect of internal geometry variables on
detection rates.

178
However, holding a constant flaw depth and flaw width whilst varying internal
geometry variables would mean that the resultant flaw shapes would not be
representative of a single ‘type’ of impact damage across a range of impact energies
on the same laminate.

6.2.4 – Role of head movements in visual perception/ inspection

Whilst shading and shadow cues remain constant regardless of viewing angle under a
constant lighting condition (see Gibson 1950a, & Kingdom 2008), the position of
specular highlights and the appearance of specular reflections will alter with changes
in viewing angle (see Norman et al. 2004) i.e. with head movements during visual
inspection. Changes in specular reflection have been proven to enable identification of
surface flaws (see Norman et al. 2004). Furthermore, by altering their viewing angle,
participants may have been able to observe small specular highlights that were not
present in the normal viewing angle. The virtual specimens used in the series 1 and
series 2 trials were rendered such that the shading cues of each flaw were replicated
for a constant viewing angle and constant lighting condition, and without specular
highlights or reflections. When viewing the specimens, participants saw the same
image regardless of whether they moved their heads, i.e. the cues appeared the same
from every viewing angle. Comparing the detection contours obtained from virtual
grey specimens with those obtained from physical gloss grey specimens reveals lower
detection rates (40% ~ >90% vs. 80% ~ >90%) for >Ø30 mm / >0.3 mm deep flaws
on virtual specimens. It is apparent that the detection contours are far more clearly
defined, with less scatter for the <0.3 mm deep flaws on virtual specimens. Also
apparent is the fact that flaws <Ø15 mm/ <0.2mm deep (small size) flaws achieve
only up to 60% detection on virtual specimens compared to up to 100% on physical
specimens. The only significant difference between each set of results is that the
participants viewing the physical specimens were able to obtain a new, albeit perhaps
very similar, view of each flaw by moving their head, unlike the fixed, single view
offered by the virtual specimens. The results therefore demonstrate that head
movement during visual inspection, and subsequent detection of (possibly very small)
specular highlights & reflections (or changes to them) is advantageous to visual
inspection for 3D surface flaws, such as those caused by impact damage to a CFRP
laminate. This is consistent with the findings of Norman et al., 2004 & Todd et al.

179
1996, on the role of changes in specular highlights in shape perception. Therefore,
there is sufficient evidence to state that, in order to provide accurate estimations of
visual inspection reliability, virtual specimens must replicate the changes in visual
cues that occur as the observer moves their head. Such specimens require a high-
resolution display, capable of displaying real time photo-realistic renderings of the
specimen surface that change in coordination with the participants head movements.

6.4 – Implications for aviation safety

6.4.1 – Lighting for visual inspection tasks on composite structures
For the visual inspection trials, the ceiling mounted diffuse lighting provided a
luminance level such that the base colours of the grey, white and blue specimens
appeared as those colours. The panels were viewed normal to their surface, thus
eliminating reflections of light sources and specular highlights. Shadows were also
minimised by the use of diffuse lighting. The lighting conditions thus provided an
optimum balance between producing shadow, shading, specular reflection and
specular highlight cues. However, lighting conditions for inspection of 3D surface
flaws can be adjusted to increase the modularity, and thus saliency of specific cues in
order to increase the likelihood of flaw detection.

The work of Faulkener & Murphy, 1979 and Brombach et al., 2006 provides evidence
to suggest that improvements in detection rates on all specimens would be made by
adding a directional light source, transverse to the specimen panel surface, whilst
maintaining diffuse lighting, as used in the experimental setup. If transverse lighting
were used, shadows would be cast by all surface flaws on all finishes, and specular
highlights would become visible on gloss specimens, both of which undoubtedly
enhancing flaw visibility, especially for the wider, less deep flaws (>Ø35mm/ <0.25
mm). Such conditions represent the use of a flashlight or lamp held transverse to the
inspection surface, and future work could evaluate the benefits offered by such
equipment to inspection of light, mid, and dark coloured surfaces in either gloss or
matt finishes.

For conditions where low luminance levels are present, surface colours may appear
darker than their actual colour, resulting in reduced modularity between cast shadows
or areas of darker shading and base colour. In such conditions, such as undersides of

180
aircraft structures, specular reflections, highlights and areas of lighter shading will
have increased modularity against the background colour. Under lower levels of
luminance, white surfaces may appear darker, thus lighter shading or specular
highlights & reflections will have greater modularity against the surface colour whilst
dark shading and shadows will retain reasonable modularity. For lighter coloured
surfaces, reduced luminance levels will therefore be less detrimental to flaw visibility
wherever sources of specular reflections or highlights (such as reflections of floor
markings & lamps or highlights from lamps) are present.

Dark colours, on the other hand, will appear yet darker, and offer poor dark shading
and shadow modularity under reduced luminance levels. The detection of surface
flaws on a dark surface is believed to be predominantly based on detection of lighter
shaded and highlight areas, due to the lack of modularity between dark shading or
shadowed areas and dark base colours. Modularity between specular reflections and
specular highlights is at its greatest on dark base colours. The work of Lloyd & He
1998 provides evidence that flaws or indentations may be detected simply by
detection of specular highlights, and highlight modularity & size are good indicators
of flaw detectability. It is therefore suggested that the detection of flaws on dark
coloured glossy surfaces can be improved by providing lighting sources that produce
specular highlights or specular reflections on the inspection surface. Lighting sources
such as strip lights or grid lights produce linear specular reflections in which
distortions are highly salient, and easily identified. The work presented in Aluze et al.,
2002 and Lloyd & He 1998, provides evidence that such lighting will enhance the
detectability of surface flaws, especially on darker colours. This leads to the
suggestion that detection of impact damage flaws, and thus visual inspection
reliability of composite aircraft structures would benefit from the provision of
luminaires fitted with grids or strip type lighting.

6.4.2 – Implications for designers of composite material structures

Aircraft maintenance documents such as structural repair manuals (SRM) commonly
give repair criteria for composite structures based on the depth of the surface flaws,
and delamination size. Whilst flaw depth may be acceptable as sole criteria for repair
assessment, aircraft & aircraft structure designers cannot simply assume that all
damage over a certain depth or width will be found. The results of the visual

181
inspection trials (as seen in chapter 5.5) demonstrate that flaws of small depth &
width are difficult to detect, and that for a given flaw depth, wider flaws are also more
difficult to detect. This trend is also apparent in results presented by Gant, 2007, as
seen in figure 2.17. When the detection percentages obtained by Gant are plotted
against flaw width, it is apparent that whilst greater detections are achieved for wider
flaws, the widest flaw in the specimen set returns less than 90% detection. The visual
inspection trials results, and the results presented by Gant provide evidence that a
blanket ‘bigger is better’ approach is invalid for width of impact damage and
associated detectability on composite aircraft structures.

Given the points discussed in chapter 6.1.1 regarding impact damage dent width
measurements, it is acknowledged that such measurements are difficult to obtain
compared to dent depth measurements. However, the results of the visual inspection
trials presented in chapter 5 demonstrate that the probability of detection of a 3D
surface flaw has both a width and depth component. Therefore, when determining the
damage tolerance size, i.e. size of impact damage that can be found (or missed) on
composite structures, the limit of visual inspection reliability should be considered in
terms of both the surface flaw depth and the flaw width, as both are required in order
to determine a surface flaw’s visibility.

Aircraft designers and regulatory bodies should not assume that visual inspection
reliability of a composite aircraft structure increases for larger energy impacts, or
impacts from larger objects. Table 6.01 gives the flaw characteristics of actual impact
damage on a painted, mesh incorporated CFRP laminate coupon, produced during the
flaw characterisation work described earlier in this report. Alongside, the
characteristics of a similar sized flaw used in the visual inspection trials, and its
detectability are given.

182
Actual surface flaws Facsimile specimen flaws
% Detections obtained

Delamination
Surface Flaw

Surface Flaw

Surface Flaw

Surface Flaw
Width (mm)

Width (mm)

Width (mm)

Gloss White
Depth (mm)

Depth (mm)

Matt White
Gloss Grey

Gloss Blue
conditions

Matt Grey

Matt Blue
Impact

Ø20 mm 17ply/ 5J 5 0.11 10 9.03 0.11 33 41 6 0 44 5

Ø20 mm 17ply/ 15J 21.5 0.32 26 19.11 0.37 100 94 100 100 69 95
Ø20 mm 33ply/ 20J 15 0.2 47 16.44 0.2 20 47 50 33 63 10
Ø20 mm 33ply/ 50J 30 0.86 79 29.96 0.57 100 100 100 100 81 85
Ø87 mm 17ply/ 20J 13 0.18 67 13.46 0.12 33 29 38 44 6 5
Ø87 mm 17ply/ 40J 22 0.21 100 24.53 0.19 100 88 81 61 38 40
Ø87 mm 33ply/ 20J 13 0.14 12 14.15 0.16 87 53 81 56 19 45
Ø87 mm 33ply/ 60J 40 0.28 83 40.45 0.25 67 59 6 22 44 0
Table 6.01 – Links between detectability and delamination sizes

From table 6.01, it is evident that a Ø24.53 mm/ 0.19 mm deep surface flaw, that
could have 100mm wide impact delamination beneath, whilst being 100% detectable
on a gloss grey specimen, was only detected by 38% of participants on a gloss blue
specimen. Furthermore, a 0.25mm, 40.45mm wide surface flaw that could have 83mm
of delamination beneath was undetectable on a matt blue surface, yet 67% detectable
on a gloss grey surface. Of particular concern, is that the surface flaw that represents
impact damage with 83mm wide delamination was no more than 67% detectable. The
visual inspection trials results show that surface flaws, occurring from impacts that
cause relatively large delamination widths, may be undetectable, or difficult to detect,
depending on the surface colour and finish of the structure. For example, a flaw
representing a 44J impact from an Ø87 mm impactor (or 60 mm of delamination) was
not detected by any participants on a matt blue specimen, but was detected by 67% of
participants on a gloss grey specimen. Whilst one may argue that NDT techniques
such as tap-test and C-Scan should easily identify large areas of delaminated
structure, one must remember that because the surface flaw of such damage could be
undetectable, the NDT may never actually be requested.

6.4.3 – Implications for aviation personnel

Aviation personnel may drop tools or objects onto an aircraft, or may cause impact
damage to aircraft with ground handling equipment. The impact testing work

183
performed on mesh incorporated, painted composite laminate, as seen in chapter 5.1,
provided evidence that impact damage can have a surface appearance that is more
typical of a small dent on a metallic structure. The testing work also demonstrated that
impact damage to a composite aircraft structure does not always produce a visible
crack. Aviation personnel need to be made aware that damage with an appearance
they may pass off as being an insignificant dent on a metallic structure could in fact
be quite significant impact damage to a composite structure, and should be reported or
further NDT requested.

The visual inspection trials show that for a 40.45 mm wide, 0.25mm deep surface
flaw, which would represent significant impact damage to a 33ply laminate, there is
between 30% and 100% chance of not detecting the damage, depending on whether
the surface is glossy grey or matt blue. Aviation personnel must be made aware that
they could produce impact damage that is impossible, or difficult to see, and should
be encouraged to report any impact incident on a composite aircraft structure,
regardless of whether or not they can see a surface flaw. If the person causing the
impact is unable to see a surface flaw, the inspector performing a requested inspection
or NDT may also be unable to locate the impact site. It may be beneficial to request
that aviation personnel mark, or record the location of any known impact occurrence
immediately, in order to avoid a requested NDT or further inspection being conducted
in the wrong place. Due to the poor modularity and subsequent saliency of shading
and shadow cues associated with dark coloured, matt finish surfaces, visual inspection
personnel should be encouraged to be particularly vigilant when inspecting them.

6.5 – Recommendations to industry & regulatory bodies

6.5.1 – Damage size assumptions
Impact damage from large objects may cause wide, shallow surface dents, coupled
with significant delamination. Despite being associated with significant hidden
delaminations, wide, shallow surface flaws can be difficult to detect by visual
inspection. Persons involved in the design and certification of composite aircraft
structures are recommended against assuming that impact damage dents associated
with large sub-surface delamination will be more reliably detected than dents with
smaller sub-surface damage. A “bigger is more detectable” metric should not be
applied to visual inspection reliability for impact damage to composite aircraft

184
structures. If such a metric is adopted, it is conceivable that significant delamination
damage may be left un-repaired due to the associated surface indentation being non-
detectable.

6.5.2 - Damage reporting

The results of the impact experiments on coupons of CFRP laminate demonstrate that
a measurement of surface flaw depth alone cannot be used to determine delamination
size for different impact dent scenarios. A measurement of surface flaw width and
surface flaw depth may provide an improved estimate of delamination size, although
for any given values of width and depth delamination size was seen to vary depending
on impact conditions. Given that structural performance of composite aircraft
structures is affected by delamination size, and that different delamination sizes can
occur for the same surface flaw width and depths, it is recommended that only
delamination size be used for repair criteria assessment. Consequently, it is
recommended that unless sufficient data can be generated to categorise damage
scenarios and resultant delamination size into surface flaw width and depth bands, all
impact damage suspected during visual inspection is assessed using further NDT
before a decision on repair or airworthiness is made.

6.5.3 – Reporting culture

If aviation personnel are to be encouraged to report all impact occurrences to
composite aircraft, the aviation industry must be prepared for inspection or NDT by
qualified personnel to be requested more frequently than with metallic aircraft
structures. Aviation personnel should not be reprimanded for calling out impact
occurrences that result in insignificant damage, or no damage being found, for fear of
creating a non-reporting culture, which could lead to significant impact damage going
unreported, undetected and un-repaired. Although further NDT of all flaws suspected
during visual inspection implies an increased frequency of flaw detections and NDT
operations, this does not imply that the frequency of impact events to aircraft will
increase.

6.5.4 – Effects of surface colour, finish & lighting

Detection rates of 3D surface flaws produced by impact damage were affected by
flaw depth and flaw width, surface colour and finish. The lighting conditions available
during inspection could also affect the reliability of visual inspection of 3D surface
flaws. When assessing the reliability of visual inspection of composite structures,

185
attention must be paid to colours and finish of the inspection area, what the available
lighting conditions will be, and the size and shape of the damage that is actually likely
to occur. For example, inspection reliability data for relatively deep, flaws of small
width on gloss grey structures that are brightly lit by grid lights could be used for
calculation of damage tolerance sizes. In a worse case scenario, if the structure when
in service is actually painted gloss blue, is located on a poorly lit fuselage underside,
and suffers impact damage causing significant subsurface damage and a wide, shallow
flaw, the flaw will have much reduced probability of detection than originally. If the
flaw is not detected, the repair will not be made, and the structure may continue to fly
with an airworthiness hazard. Therefore, it is recommended that if highly conservative
estimates are sought for determining the base values on which visual inspection
reliability for impact damage to composite aircraft structures can only ever improve,
dark coloured matt finishes should be replicated and lit solely by low luminance level
ambient lighting.

It also recommended that when assessing visual inspection reliability for impact
damage to composite aircraft structures, attention should also be paid to the gloss
levels of the inspection surface. If the gloss level of a structure’s finish degrades after
several months of service, such as reported by Guseva et al. 2003, the visual
inspection reliability for that structure could be negatively affected. Visual inspection
reliability may need to be demonstrated for a worst-case gloss degradation scenario.

6.5.5 – Flaw significance/ perceived severity

Consideration must be paid to the likely interpretation of flaw significance by
personnel whose experience has been with metallic structures. Such personnel may be
accustomed to the presence of shallow, insignificant surface dents on a metallic
structure, which pose no airworthiness issues. However, the same personnel may be
unaware that similarly shaped dents on a composite structure could be an indication of
significant sub-surface damage, and failure to instigate further investigation or repair
could have severe airworthiness implications.

6.5.6 – Reliability assessment protocols

There is currently a wealth of published literature available to assist in the design of
reliability assessment experiments concerning NDT/ NDI procedures for metallic
aircraft structures. However, this literature currently requires significant interpretation

186
before it can be applied to reliability assessment of visual inspection of composite
materials. Regulatory bodies should ensure that a common set of experimental
protocols are being used in determining visual inspection reliability and subsequent
BVID sizes for composite aircraft structures, and consider publishing a recommended
protocol or experimental design guidelines/ recommendations.

Chapter 7 – Future Work

The following sections describe areas where the work presented in this thesis has
identified the possibility or requirement for future work.

7.1 – Expansion of flaw size range

A better understanding of width/depth effects on detection rates may be obtained by
expanding the flaw size range in future experiments. However, the values of the flaw
geometry variables for such flaws may extend to width/ depth combinations that fall
outside the relationships for Ø20 mm & Ø87 mm hemispherical impacts on 17ply &
33 ply painted CFRP laminates. In such instance, it would be beneficial to vary only a
single flaw geometry variable at a time, whilst using constant relationships to the
other flaw geometry variables, in order to isolate the effects of each in visual
inspection reliability. For example, if flaw widths were varied at constant flaw depths,
and vice a versa, with constant relationships between the other flaw geometry
variables, a 3D surface plot of detectability as function of both flaw depth and flaw
width could easily be produced. However, as the visual inspection trials results show
(see chapter 5.5), the detectability may not always be greater for wider flaws, whilst
still being low for the smallest width flaws at a constant flaw depth. As widely
accepted logarithmic based POD functions are unable to model detectability that both
increases and decreases with increasing flaw size, the application of traditional POD
models to the data will remain unfeasible.

7.2 – Experimental protocols/ guidelines

The current publications concerning design of NDT/NDI reliability assessments such
as LS190 [AGARD 1993], MIL HDBK 1823 [US DoD 2007] and the Generic
Protocol produced by Spencer et al., 1993 were written in the context of inspection of
metallic aircraft structures. However, many of the recommendations within these
documents such as specimen characteristics & numbers, flaw size ranges, flaw

187
densities and flawed/ unflawed ratios are valid regardless of context, although they
require interpretation depending on the process under assessment.

However, despite the many published BVID sizes, there are currently no published
protocols or experimental recommendations for assessing visual inspection reliability,
and thus BVID sizes, for composite aircraft structures. Future work should consider
the valuable recommendations and protocols that have been published for design of
NDT/NDI reliability assessment experiments, and apply them in the context of visual
inspection of composite aircraft structures. The aim for such work should be to arrive
at a set of protocols or recommendations that ensure consistent experimental practices
are employed in determining visual inspection reliability and BVID sizes for
composite aircraft structures under realistic conditions. The objectives would be to
standardise the design of flaws, specimens, experimental setup, lighting and data
collection methods used in determining visual inspection reliability and BVID sizes
for composite aircraft structures.

7.3 – Alternative Visual Inspection Trials Methodology

The visual inspection trials described in this thesis (chapter 4.12) were performed
using flat specimen panels presented vertically upright, with the inspection surface
normal to the participants. It is acknowledged that this represents only a single
inspection condition, and that visual inspection of an actual aircraft will involve the
inspection of surfaces at various angles, such as fuselage skin, wings skins, and
horizontal & vertical stabiliser skins. The work of Erhart et al. 2004 demonstrates that
different viewing angles and under-wing inspections conditions are likely to produce
different inspection results. The results presented in this thesis also demonstrate that
an inspector’s ability to move around the inspection area must not be discounted as an
aid to visual inspection. The work of Aluze et al. 2002 and Lloyd & He, 1998
demonstrates that grid or strip lighting will also benefit visual inspection performance.

Furthermore, the surface flaws used in this experiment represent only 8 sizes of 4 flaw
types. Whilst the experiment has successfully investigated the effects of surface
colour and surface finish on visual inspection reliability, it would be desirable to
expand the number different sizes of each flaw type, in order to produce more
information on how the flaw geometry affects visual inspection reliability. However,

188
using the current experimental methodology would require that the total inspection
time is increased to over 30 minutes, which may lead to participant fatigue or
boredom.

A solution to providing a more realistic inspection scenario, and a solution for

increasing the number of specimens inspected in a single visual inspection trial,
would be to arrange square specimen panels containing realistically shaped flaws into
a mock up aircraft shape, as seen in figure 7.01.

Figure 7.01 –Specimen panels arranged into an aircraft shape

A single participant, or several participants, would inspect the entire “aircraft”, and a
maximum inspection time could be imposed if desired. The participants would be
asked to search for surface flaws on each of panels, and mark their location on a
diagram. The specimen panels would be a mix of flawed and unflawed panels, and
with this arrangement, it is possible to increase the total number of specimens, thus
increasing the number of surface flaws that can be included in the experiment, without
increasing total inspection time. A further advantage to this arrangement is that it
allows panels to be placed in different positions, in order to simulate horizontal and
vertical topside and underside surfaces. As seen in figure 7.02, to inspect the
underside surfaces, the inspector must view the underside panels from below, against

189
a bright background. Any effects on visual inspection performance due to this lighting
condition and accessibility constraint would be revealed in the inspection results.

Figure 7.02 – Inspection of the panels representing the underside of an aircraft

If the specimens were laid out to represent an aircraft, or section of aircraft structure,
the participants would expect to walk around the specimens and inspect from different
angles. Due their ability to view each panel from a variety of angles, the participants
would also observe specular reflections and specular highlights on glossy specimen
panels laid out as in figure 7.01. The introduction of specular reflection and specular
highlight visual perception cues may reveal greater differences in visual inspection
performance on matt and gloss finishes than those demonstrated in the results
obtained for this thesis. Because arranging the specimens as illustrated in figure 7.01
would inherently introduce specular reflection cues into the experiment, the effects on
visual inspection performance due to different lighting designs such as grid luminaires
or strip lights could also be investigated by incorporating them into the setup.

Arranging the panels into an aircraft shape would also factor inspector expectation
into the experiment. For example, an aircraft inspector may expect to see damage aft
of a wheel well, where stone & foreign object damage (FOD) is common. However,
the inspector would not expect to see the same damage on the topside of a wing. An
impact damage flaw may be detected in a wheel well, or under wing location, yet
missed on the top surface of the wing. This would raise the question of whether the

190
inspector missed the damage or flaw on the topside of the wing because it was too
small for them to see, or because they did not expect to see damage in that location. In
a scenario where an aircraft is being simulated, it is expected that an inspector will use
their experience to determine where they would expect to find damage, and
concentrate their visual inspection on that area [see Gallwey & Drury 1986]. An
untrained participant, however, would be more likely to look everywhere for the
damage or flaws. Therefore, it is foreseeable that the value of prior experience or
damage location expectancy could be quantified by conducting trials with trained
inspectors and trials with novice participants.

7.4 – Paint Patterns

By using either the original visual inspection trials methodology (see chapter 4.12), or
the aforementioned alternative setup, it would be advantageous to investigate the
effect of a paint pattern on visual inspection reliability. There is some concern that a
pattern could disguise the shadow and shine cues provided by a surface flaw to an
extent that the flaw detectability reduces, or the flaw becomes undetectable. In fact,
this forms the basis of how camouflage works. Furthermore, the camouflaging effect
of a paint pattern would be exacerbated by the fact that impact damage on an aircraft
structure will not always be geometrically shaped, especially if caused by an
irregularly shaped object. However, in order to produce a pattern on a specimen panel,
a suitable production method must be researched and tested to ensure that the pattern
is consistently the same on each specimen, and that the production process does not
alter the flaw characteristics.

7.5 – Curved specimen panels

The specimen panels used in the visual inspection trials were all flat panels,
representing impact damage to a flat laminate. Aircraft surfaces are generally curved.
The surface flaw due to impact damage may be different in shape & size on a curved
laminate panel to the flaw on a flat laminate, due to internal material stresses &
relaxation at the flaw site. A more realistic assessment of visual inspection reliability
could use curved specimen panels that represent impact damage to a curved shaped
composite aircraft structure.

191
7.6 – State of knowledge of pilots, maintenance & ground crew
personnel
Whilst this study assessed the reliability of visual inspection of surface flaws due to
impact damage on a composite aircraft structure, the participants in the experiments
were all given examples of surface flaws prior to inspecting, i.e. they all knew what to
look for. There is some concern that not all aviation personnel are aware of what a
surface flaw due to impact actually looks like on a composite aircraft structure. In
fact, the early stages of this study required work to address this lack of information.
Therefore, in order to assess whether there is an industry wide knowledge deficit, and
what future training may be required, it would be beneficial to study the current state
of knowledge regarding impact damage to composite aircraft within aviation
personnel groups i.e. pilots, maintenance personnel and ground crew.

192
Chapter 8 – Conclusions
8.1 – Impact damage to fully finished CFRP laminates
1. Impact experiments on CFRP laminate coupons demonstrated that the widely
reported visual appearance of impacts to unfinished laminates is not typical of
impact events on actual aircraft structures.
2. The experimental results demonstrate that impact damage to painted, bronze
lightning strike protection mesh incorporated CFRP produces smoothly
shaped, crack free dents that offer different visual perception cues to the rough
edged, cracked surfaces of impact damage sites on unfinished laminates.
3. For a typical finished aerospace CFRP laminate, higher energy (40J) impact
damage from a larger (87 mm) diameter hemispherical object resulted in
damage with a lower depth (0.21 mm vs. 0.67 mm), and greater delamination
(4900 mm² vs. 1900 mm²) than lower energy (20J) impact damage from a
smaller (20 mm) diameter object.

8.2 – Visual inspection reliability of impact damage dents

4. The detection results obtained using facsimile specimens of hemispherical
impact damage to CFRP laminates in visual inspection trials are consistent
with theories of visual perception of surface slant & shading, occlusion
shadows, specular highlights and specular reflections.
5. Using facsimile specimens in visual inspection trials, increased detection rates
were observed with larger depths & widths of impact damage dents, but the
positive effect of depth on detection rate was mitigated by a negative effect of
width on detection rates for relatively large widths.
6. The visual inspection trials demonstrate that the detectability of all impact
damage producing damage of a given width or depth cannot be considered the
same.
7. The results of the trials with facsimile specimens demonstrate that impact
damage dents associated with large subsurface delamination (as per
conclusion 3) were less reliably detected by visual inspection than impact
damage associated with smaller subsurface delamination (as per conclusion 3).

193
 8. Visual inspection reliability cannot be accurately modelled by probability of
detection estimates based solely on dent width, dent depth, impact energy or
delamination size for impact damage to composite aircraft structures.
9. Visual inspection using 2D projected images of virtual specimens provides a
time and cost efficient tool for developing experimental methodologies, or for
generating detection data on which to base experimental design engineering
judgments for visual inspection assessment experiments.
10. Visual detection results obtained using static 2D display specimens cannot not
considered directly comparable to inspection of a physical specimen due to the
inability to replicate the changes in visual perception cues on a 3D surface that
occur during head movements.

8.3 - Effects of surface finish and colour on visual inspection reliability

11. In the detection results obtained with facsimile specimens of impact damage to
CFRP specimens, a region of 90-100% detection occurred in all specimens at
the largest depths and widths. The size and location of this region varied with
surface finish and colour.
12. The grey surface colour provided the optimum modularity between light and
dark shading cues, resulting in optimum cue saliency and the largest range of
flaw sizes returning >90% detection (0.25 ~ 0.95 mm deep & Ø20 ~ Ø45 mm
wide).
13. Poor modularity of lighter areas of shading reduced the visibility of flaws on
white coloured specimens, reducing the saliency of lighter shaded areas, hence
a reduction in the range of flaw sizes returning >90% detection (0.3 ~ 0.95
mm deep & Ø25 ~ Ø40 mm wide).
14. Increased modularity of lighter areas of shading on blue coloured specimens
was mitigated by poor modularity of darker areas of shading and poor
modularity of shadow cues against the dark background colour, reducing their
saliency, hence the smallest range of flaw sizes returning >90% occurring on
the blue specimens (0.7 ~ 0.95 mm deep & Ø25 ~ Ø30 mm wide).
15. Specular highlights were not visible to the observer, and specular reflections
had insufficient saliency against the grey and white surface colours for
distortions caused by flaws to be perceived, hence, the relative similarity
between the results obtained from matt and gloss specimens in these colours.

194
 16. The gloss blue specimens exhibited salient specular reflections, and distortions
in the reflections caused by the flaws made the flaws visible, hence the lower
number of flaws (12 flaws) returning <50% detection compared to the matt
blue specimens (25 flaws) which were devoid of specular reflections.

8.4 – Visual inspection reliability for composite aircraft structures

17. Visual inspection reliability for composite aircraft structures must be
demonstrated for impact damage to the actual laminate surface finishes seen
by a field inspector and not bare, unfinished laminates.
18. Visual inspection detection thresholds such as 50% probability of detection,
(POD), 90% POD or barely visible (BVID) must not be considered in terms of
either dent depth OR dent width alone. As a minimum, both the indentation
width and the indentation depth are required to determine detectability
thresholds.
19. In order to accurately model visual inspection reliability of impact damage to
composite aircraft structures, a multi-variate POD analysis method must be
developed.
20. Due to the significant effects of surface colour and finish, visual inspection
reliability for composite aircraft structures must be demonstrated on at least
one light, one mid and one dark colour in finishes typical for that structure in
order to determine best and worst case reliability scenarios.

195
References
Abdallah, E. A., Bouvet, C., Rivallant, S., Broll, B. and Barrau, J. (2009), "Experimental
analysis of damage creation and permanent indentation on highly oriented plates",
Composites Science and Technology, vol. 69, no. 7-8, pp. 1238-1245.

Abrate, S. (1998), Impact on Composite Structures, Cambridge University Press, Cambridge,

UK.

Aerospatiale (1991), Substantiation of Composite Parts - Impact Detectability Criteria, 440

225/91.

AGARD (1993), A recommended methodology for quantifying NDE/NDI based on aircraft

engine experience, AGARD-LS-190, Advisory Group for Aerospace Research and
Development, North Atlantic Treaty Organization, 7 rue Ancelle, 92200 Neuilly-sur-
Seine, France.

Airbus (2009), A350 XWB Programme Update January 2009, Airbus Press Office, 31707
Blacgnac, France.

Airbus (2009), Airbus A350 Fuselage Production, available at:

http://www.airbus.com/en/aircraftfamilies/a350/efficiency/by_design/fuselage.html
(accessed 16 June 2009).

Airbus (2003), Airbus A340-200/300 Structural Repair Manual 41st Revision Jan 01/03,
A340 SRM, Airbus Customer Services, Technical Data Support and Services, 31707
Blagnac Cedex, France.

Airbus (2002), Airbus A340 Aircraft Maintenance Manual, A340 AMM, Airbus Customer
Services, Technical Data Support & Services, 1, rond-point Maurice Bellonte, 31707
Blagnac Cedex, France.

Aktaş, M., Atas, C., Đçten, B. M. and Karakuzu, R. (2009), "An experimental investigation of
the impact response of composite laminates", Composite Structures, vol. 87, no. 4, pp.
307-313.

ANAC (2008), Eclipse High Solids Polyurethane Topcoat, Technical Data, June 2008, Akzo
Nobel Aerospace Coatings, Sassenheim, The Netherlands.

ANAC (2006), Aerodur Finish C 21/100 Technical Data September 2006, 30-02, Akzo Nobel
Aerospace Coatings BV, Sassenheim, The Netherlands.

Annis, C. (2008), Generalized Linear Models on a P/C Spreadsheet, available at:

http://www.statisticalengineering.com/glm.htm (accessed 02 July 2009).

Armstrong, K. B., Bevan, L. G. and Cole, W. F. (2005), Care and repair of advanced
composites. 2nd ed. ed, SAE International, Warrendale, Pa.

Asada, H., Sotozaki, T., Endoh, S. and Tomita, T. (1998), "Practical Evaluation of Crack
Detection Capability for Visual Inspection in Japan", Papers presented at the RTO AVT
Workshop on "Airframe Inspection Reliability under Field/ Depot Conditions", Brussels,
Belgium, May 13-14 1998 and published in RTO MP-10, pp. 15-1.

196
Berens, A. P. and Hovey, P. W. (1982), "Characterization of NDE reliability", Review of
progress in quantitative nondestructive evaluation. Volume 1 - Proceedings of the
Eighth U.S. Air Force/Defense Advanced Research Projects Agency Symposium on
Quantitative Nondestructive Evaluation, Boulder, CO; United States; 2-7 Aug. 1981,
New York, Plenum Press.

Bode, M. D., Spencer, F. W. and Piotrowski, D. (2006), "Analytical Issues in Determining

Probability of Detection", 9th Joint FAA/DoD/NASA Aging Aircraft Conference, 6-9
March, Atlanta, GA.

Boeing (2009), Boeing Commercial Airplanes 757-200 Background, available at:

http://www.boeing.com/commercial/757family/pf/pf_200back.html (accessed 16 June
2009).

Boeing (2009), Boeing Commercial Airplanes 777 Background, available at:

http://www.boeing.com/commercial/777family/background.html (accessed 6 May
2009).

Boeing (2009), Boeing Commercial Airplanes 787 Dreamliner Background, available at:
http://www.boeing.com/commercial/787family/background.html (accessed 16 June
2009).

Boeing (2009), Boeing Integrated Defense Systems, Commercial Products, MAUS V System
Overview, available at: http://www.boeing.com/defense-
space/support/maintenance/commercial/maus.html (accessed 16 June 2009).

Boeing (2008), Boeing 787 Dreamliner Structure Test a Success (Rome, July 02), available
at: http://www.boeing.com/commercial/787family/news/2008/q3/080702b_nr.html
(accessed 16 June 2009).

Boeing (2008), Boeing Completes Destructive Testing on 787 Dreamliner Wing Box (Seattle,
November 15), available at:
http://www.boeing.com/commercial/787family/news/2008/q4/081115a_nr.html
(accessed 16 June 2009).

Boeing (2007), Airplane Characteristics for Airport Planning, Revision A September 2007
(Preliminary), D6-58333, Boeing Commercial Airplanes, Seattle, Washington.

Boeing (2007), Final Assembly Begins on First Boeing 787 Dreamliner (Everett, May 21),
available at:
http://www.boeing.com/commercial/787family/news/2007/q2/070521c_nr.html
(accessed 16 June 2009).

Boeing (2005), Boeing Completes First 7E7 Composite Fuselage Section (Seattle, January
11), available at:
http://www.boeing.com/commercial/787family/news/2005/q1/nr_050111g.html
(accessed 16 June 2009).

Boeing (1988), Advanced Composite Compression Tests, Boeing Specification Support

Standard, BSS 7260 Rev. C, The Boeing Company, Seattle, WA.

Bopp (2006), Metal Wire Cloth - Data Sheet/ Certificate for phosphor bronze mesh batch
No.223118, G. Bopp & Co. AG, Zürich, CH.

197
Boulic. (Cranfield University), (2007), Quantitative Morphology of Impact Damage to
Composite Aircraft Structures (unpublished report), Polytech’Orléans, France.

Brombach, J., Hütte, T. and Strasser, H. (2006), "Surface-specific lighting scenarios

enhancing the perceptibility of 3-dimensional flaws", Occupational Ergonomics, vol. 6,
no. 2, pp. 75-83.

Brookes, R. E. (2004), Surface and sub-surface damage in plain weave carbon epoxy
laminates due to low velocity impacts (unpublished M.Sc. thesis), Cranfield University,
United Kingdom.

Brosius, D. (2007), "Boeing 787 Update", High Performance Composites, vol. 2007, no.
May, pp. 56-59.

Bruce, D. A. (1998), "NDT Reliability Estimation From Small Samples and In-Service
Experience", Papers presented at the RTO AVT Workshop on "Airframe Inspection
Reliability under Field/ Depot Conditions", Brussels, Belgium, May 13-14 1998 and
published in RTO MP-10, pp. 3-1.

Bullock, M., Forsyth, D. and Fahr, A. (1994), Statistical Functions and Computational
Procedures for the POD Analysis of Hit/Miss NDI Data, LTR-ST-1964, Institute for
Aerospace Research, National Research Council Canada, Ottawa, Ontario.

Bunsell, A. R. and Renard, J. (2005), Fundamentals of fibre reinforced composite materials,

Institute of Physics Publishing, Bristol % Philadelphia.

CAA (2005), CAP 562: Civil Aircraft Airworthiness Information and Procedures, Issue 2,
Amendment 1, 2005, CAP 562, Civil Aviation Authority, Safety Regulation Group,
United Kingdom.

CAA (2003), CAP 716: Aviation Maintenance Human Factors (EASA Part-145), CAP 716,
Civil Aviation Authority, Safety Regulation Group, United Kingdom.

Campbell, F. C. (2004), Manufacturing processes for advanced composites, Elsevier, New

York.

CMH-17 (2009), "Polymer Matrix Composites Forum", CMH-17 Composites Materials

Handbook, Coordination Meeting, 4th March 2009, Salt Lake City, UT.

Colquhoun, W. P. (1964), "Recent research in the psychology of inspection", Journal of

Textile Institute and Industry, vol. 2, pp. 252-255.

Composites World (2009), New boat uses Zyvex carbon nanotube-enhanced prepreg, 12th
May 2009, available at: http://www.compositesworld.com/news/new-boat-uses-zyvex-
carbon-nanotube-enhanced-prepreg.aspx (accessed 14/05/2009).

Cuffaro, D. (2006), Process, Materials and Measurements, Rockport Publishers, Inc.,

Gloucester, Mass.

Davies, G. A. O., Zhang, X. and Edlund, A. (1994), "Predicting damage in composite aircraft
structures due to low velocity impact", Aerotech 1994, Structures & Materials Seminar,
January 1994, Birmingham, UK.

198
Davies, G. A. O. and Zhang, X. (1995), "Impact damage prediction in carbon composite
structures", International Journal of Impact Engineering, vol. 16, no. 1, pp. 149-170.

de Boer, J. B. (1977), "Performance and comfort in the presence of veiling reflections",

Lighting Research and Technology, vol. 9, no. 4, pp. 169-176.

Dekoker, N. and Frier, J. P. (1969), "Visibility of specular and semispecular tasks in sheet
metal surfaces", Illuminating Engineering, vol. 64, pp. 167-175.

Degussa (2006), Plexiglas GS Product Description November 2006, Degussa Methacrylates,

Röhm GmbH, Darmstadt, Germany.

DIN (1996), Aerospace series - Fibre reinforced plastics - Test method; determination of the
compression strength after impact, DIN EN 6038, Deutsches Institut für Normung e. V.,
Berlin, Germany.

Donckels, Y., Matthews, F. L. and Foreman, A. D. (2000), "Impact dent depth relaxation in
polymer matrix composites", ECCM9, Composites from Fundamentals to Exploitation,
4-7 June 2000, Brighton, UK.

Drury, C. G. (2000), "Human factors and quality: Integration and new directions", Human
Factors and Ergonomics in Manufacturing, vol. 10, no. 1, pp. 45-59.

Drury, C. G. (1993), "Exploring search strategies in aircraft inspection", Visual Search, vol. 2,
pp. 101-112.

Drury, C. G. (1989), "The Information Environment in Inspection", Second FAA Meeting on

Human Factors Issues in Aircraft Maintenance and Inspection, 13-14 December 1989,
Alexandria, Virginia.

Drury, C. G. and Lock, M. W. B. (1995), "Reliability in Aircraft Inspection: UK and USA

Perspectives", DOT/FAA/AM-95/14 Human Factors in Aviation Maintenance - Phase 4,
Progress Report (DTIC: ADA294756), pp. 147-161.

Drury, C. G., Maheswar, G., Das, A. and Helander, M. G. (2001), "Improving visual
inspection using binocular rivalry", International Journal of Production Research, vol.
39, no. 10, pp. 2143-2153.

dti (2006), Hybridmat 3: Advances in the Manufacture of Advanced Structural Composites in

Aerospace : a Mission to the USA (Global Watch Mission report), Department of Trade
and Industry, UK.

Dupont, G. (1997), "The Dirty Dozen Errors in Maintenance", Human Factors Issues in
Aviation Maintenance and Inspection, March 12-13, San Diego, CA, FAA.

EASA (2008), Certification Specifications for Large Aeroplanes, Ammendment 5, September

2008, CS-25, European Aviation Safety Agency, Koeln, Germany.

Erhart, D., Ostrom, L. T. and Wilhelmsen, C. A. (2004), "

Visual Detectability of Dents on a Composite Aircraft Inspection Specimen: An Initial
Study", International Journal of Applied Aviation Studies, vol. 4, no. 2, pp. 111-122.

199
FAA (1984), Advisory Circular 20-107A Composite Aircraft Structure, AC20-107A, U.S.
Department of Transportation, Federal Aviation Administration.

Fahr, A., Forsyth, D. and Bullock, M. (1993), A comparison of Probability Of Detection

(POD) data determined using different statistical methods, LTR-ST-1947, Institute for
Aerospace Research, National Research Council Canada, Ottawa, Ontario.

Faulkner, T. W. and Murphy, T. J. (1973), "Lighting for difficult visual tasks", Human
factors, vol. 15, no. 2, pp. 149-162.

Forsyth, D. S. and Fahr, A. (1998), "An Evaluation of Probability of Detection Statistics",

Papers presented at the RTO AVT Workshop on "Airframe Inspection Reliability under
Field/ Depot Conditions", Brussels, Belgium, May 13-14 1998 and published in RTO
MP-10, pp. 10-1.

Forsyth, D. S., Gould, R. W. and Komorowski, J. P. (1998), "Double pass retroflection versus
visual inspection of impact damage in composites", Papers presented at the RTO AVT
Workshop on "Airframe Inspection Reliability under Field/ Depot Conditions", Brussels,
Belgium, May 13-14 1998 and published in RTO MP-10, pp. 18-1.

Fualdes, C. (2006), "Composites at Airbus, Damage Tolerance Methodology", FAA

Workshop for Composite Damage Tolerance and Maintenance, July 19 - 21, 2006,
Chicago, IL.

Gallwey, T. J. and Drury, C. G. (1986), "Task complexity in visual inspection", Human

factors, vol. 28, no. 5, pp. 595-606.

Gant, S. K. (2007), "Visual Assessment of Impact Damage on Painted Composite Aircraft

Structures", International SAMPE symposium and exhibition proceedings, June 3-7,
Baltimore, MD.

Gao, S. and Kim, J. (1999), "Scanning acoustic microscopy as a tool for quantitative
characterisation of damage in CFRPs", Composites Science and Technology, vol. 59, no.
3, pp. 345-354.

Gardiner, G. (2006), "Lightning Strike Protection for Composite Structures", High

Performance Composites July 2006, [Online], pp. 11/05/09 available at:
http://www.compositesworld.com/articles/lightning-strike-protection-for-composite-
structures.aspx.

Georgiou, G. A. (2007), "PoD Curves, their deivation, applications and limitations",

INSIGHT - Non-Destructive Testing and Condition Monitoring, vol. 49, no. 7, pp. 409-
414.

Georgiou, G. A. (2006), Probability of Detection (PoD) curves - Derivation, applications and

limitations, Research Report 454, Health and Safety Executive, UK.

Gibson, J. J. (1979), The ecological approach to visual perception, Houghton Mifflin, Dallas ;
London.

Gibson, J. J. (1950b), "The perception of visual surfaces", American Journal of Psychology,

vol. 63, no. 3, pp. 367-384.

200
Gibson, J. J. (1950a), The Perception of the Visual World, Riverside Press, Cambridge, Mass.

Gramopadhye, A. K., Bhagwat, S., Kimbler, D. and Greenstein, J. (1998), "The use of
advanced technology for visual inspection training", Applied Ergonomics, vol. 29, no. 5,
pp. 361-375.

Gramopadhye, A. K., Drury, C. G. and Prabhu, P. V. (1997), "Training strategies for visual
inspection", Human Factors and Ergonomics in Manufacturing, vol. 7, no. 3, pp. 171-
196.

Gros, X. E., de Smet, M. A. and Takahashi, K. (1998), "On the efficiency of current NDT
methods for impact damage detection and quantification in thermoplastic toughened
CFRP materials", International Conference on Advanced Composites (ICAC 98), 15-18
Dec. 1998, Hurghada, Egypt, Auburn University, Auburn, AL.

Guseva, O., Brunner, S. and Richner, P. (2003), "Service life prediction for aircraft coatings",
Polymer Degradation and Stability, vol. 82, no. 1, pp. 1-13.

Hagemaier, D. (1998), "Factors Influencing Eddy Current PoD in the Field Environment",
Papers presented at the RTO AVT Workshop on "Airframe Inspection Reliability under
Field/ Depot Conditions", Brussels, Belgium, May 13-14 1998 and published in RTO
MP-10, pp. 2-1.

Halpin, J. C. (1994), "The evolution of design and material criteria for polymeric structural
materials", Composite Structures, vol. 27, no. 1-2, pp. 3-6.

Harris, B. (ed.) (2003), Fatigue in composites: science and technology of the fatigue response
of fibre-reinforced plastics, CRC Press, Boca Raton, Florida.

Harris, C. E. and Shuart, M. J. (2001), An Assessment of the State-of-the-Art in the Design

and Manufacturing of Large Composite Structures for Aerospace Vehicles, NASA
Langley Research Center, USA.

Hexcel (2003), HexPly8552 Product Data, Publication FTA 072a, May 2003, Hexcel
Composites, Cambridge, UK.

Hounslow, L. E. (2000), The influence of Z-pinning on low velocity impact performance of

laminated carbon composites (unpublished M.Sc. thesis), Cranfield University, United
Kingdom.

Hovey, P. W., Sproat, W. H. and Schattle, P. (1989), "The Test Plan for the Next Air Force
NDI Capability and Reliability Assessment Program", in Thompson, D. O. and
Chimenti, D. E. (eds.) Review of Quantitative Nondestructive Evaluation, Vol. 8B ed,
Plenum Press, pp. 2213-2220.

Hu, N., Zemba, Y., Okabe, T., Yan, C., Fukunaga, H. and Elmarakbi, A. M. (2008), "A new
cohesive model for simulating delamination propagation in composite laminates under
transverse loads", Mechanics of Materials, vol. 40, no. 11, pp. 920-935.

Ilcewicz, L. (2009), "FAA Composite Safety & Certification Initiatives", CMH-17 Composite
Materials Handbook PMC Coordination Meeting, 2-5 March 2009, Salt Lake City,
Utah.

201
Imielinska, K., Castaings, M., Wojtyra, R., Haras, J., Le Clezio, E. and Hosten, B. (2004),
"Air-coupled ultrasonic C-scan technique in impact response testing of carbon fibre and
hybrid: glass, carbon and Kevlar/epoxy composites", Journal of Materials Processing
Technology, vol. 157-158, pp. 513-522.

Ishibashi, S., ( 2006), Use of Toray carbon fibre products in aerospace applications;
Telephone communication with S Ishibashi, Technical Manager of SOFICAR (Toray/
Arkema), January 2006, Soficar, France.

Kalpakjian, S. (1995), Manufacturing Engineering and Technology, 3rd ed, Addison-Wesley,

Reading, Mass.; Wokingham.

Kim, H. (2007), "Blunt Impact Formation on Composite Aircraft Structures", Commercial

Aircraft Composite Repair Committee (CACRC) Meeting, 12-16 November 2007,
Wichita, Kansas.

Kingdom, F. A. A. (2008), "Perceiving light versus material", Vision research, vol. 48, no.
20, pp. 2090-2105.

Kolesnikov, B. and Herbeck, L. (2004), "Carbon Fiber Composite Airplane Fuselage:

Concept and Analysis", Merging the Efforts: Russia in European Research Programs on
Aeronautics, 11-12 May 2004, Berlin, Germany, German Aerospace Center (DLR)
http://elib.dlr.de/49781.

Komorowski, J. K., Gould, R. W. and Marincak, A. (1993), "Study of the effect of time and
load on impact visibility", 2nd Canadian International Composites Conference and
Exhibition (CANCOM 93), September 1993, Ottawa, Ontario, Canada, Canadian
Association for Composite Structures and Materials, Montréal, pp. 441-446.

Krutop, B. N. E. (2007), Effect of impact damage and subsequent fatigue damage

development in carbon-fibre polymer composites for aircraft (unpublished M.Sc. thesis),
Cranfield University, United Kingdom.

Latorella, K. A., Gramopadhye, A. K., Prabhu, P. V., Drury, C. G., Smith, M. A. and
Shanahan, D. E. (1992), "Computer-simulated aircraft inspection tasks for off-line
experimentation", Proceedings of the Human Factors Society 36th Annual Meeting. Part
2 (f 2), Vol. 1, 12 October 1992 through 16 October 1992, Atlanta, GA, USA, Publ by
Human Factors Soc Inc, Santa Monica, CA, United States, pp. 92.

Lewis, W. H., Sproat, W. H., Dodd, B. D. and Hamilton, J. M. (1978), Reliability of

Nondestructive Inspections, Final Report, SA-ALC/MME 76-6-38-1, Lockheed-Georgia
Company, Marietta, GA.

Lippincott, H. W. and Stark, H. (1982), "Optical-digital detection of dents and scratches on

specular metal surfaces.", Applied Optics, vol. 21, no. 16, pp. 2875-2881.

Lloyd, C. J. (1998), "Towards the Development of More Effective Surface Inspection

Methods", Automotive Finishing '98, June 9-11, 1998, Detroit, Michigan, Society of
Manufacturing Engineers, Dearborn, MI.

Lloyd, C. and He, Y. (1998), "An Objective Measure of Severity for Small Topographical
Defects in Automotive Paint (SAE 982316)", International Body Engineering
Conference & Exposition September 1998, Detroit, Michigan, USA.

202
Lock, M. W. B. (1998), "The Effect of Aircraft Maintenance on Human Factors", Papers
presented at the RTO AVT Workshop on "Airframe Inspection Reliability under Field/
Depot Conditions", Brussels, Belgium, May 13-14 1998 and published in RTO MP-10,
pp. 14-1.

Lopes, C. S., Seresta, O., Coquet, Y., Gürdal, Z., Camanho, P. P. and Thuis, B. (2009), "Low-
velocity impact damage on dispersed stacking sequence laminates. Part I: Experiments",
Composites Science and Technology, vol. 69, no. 7-8, pp. 926-936.

Mallet, L., Lee, B. C., Staszewski, W. J. and Scarpa, F. (2004), "Structural health monitoring
using scanning laser vibrometry: II. Lamb waves for damage detection", Smart Materials
and Structures, vol. 13, no. 2, pp. 261-269.

Matthews, F. L. and Rawlings, R. D. (1994), Composite materials : engineering and science,

Chapman and Hall, London.

Megaw, E. D. (1979), "Factors affecting visual inspection accuracy", Applied Ergonomics,

vol. 10, no. 1, pp. 27-32.

Megaw, E. D. and Richardson, J. (1979), "Eye movements and industrial inspection", Applied
Ergonomics, vol. 10, no. 3, pp. 145-154.

Mills, A. (2006), "Developments in Resin Infusion Materials and Processes; Structural

Unitisation and Component Integration Techniques", in Hybridmat 3: Advances in the
Manufacture of Advanced Structural Composites in Aerospace, Department of Trade and
Industry, London, UK, pp. 18-20.

Mitrevski, T., Marshall, I. H. and Thomson, R. (2006), "The influence of impactor shape on
the damage to composite laminates", Composite Structures, vol. 76, no. 1-2, pp. 116-
122.

Mitrevski, T., Marshall, I. H., Thomson, R. S. and Jones, R. (2006), "Low-velocity impacts
on preloaded GFRP specimens with various impactor shapes", Composite Structures,
vol. 76, no. 3, pp. 209-217.

Mitrovic, M., Hahn, H. T., Carman, G. P. and Shyprykevich, P. (1999), "Effect of loading
parameters on the fatigue behavior of impact damaged composite laminates",
Composites Science and Technology, vol. 59, no. 14, pp. 2059-2078.

Mullis, R. T. (1998), "C-141 Spanwise Splice Advanced NDI Method (Probability of

Detection Experiment Results)", Papers presented at the RTO AVT Workshop on
"Airframe Inspection Reliability under Field/ Depot Conditions", Brussels, Belgium,
May 13-14 1998 and published in RTO MP-10, pp. 19-1.

Murgatroyd, R. A., Worrall, G. M., Waites, C., AEA Technology and Civil Aviation
Authority (1995), A study of the human factors influencing the reliability of aircraft
inspection, Civil Aviation Authority, London.

Norman, J. F., Todd, J. T., Norman, H. F., Clayton, A. M. and McBride, T. R. (2006), "Visual
discrimination of local surface structure: Slant, tilt, and curvedness", Vision research,
vol. 46, no. 6-7, pp. 1057-1069.

203
Norman, J. F., Todd, J. T. and Orban, G. A. (2004), "Perception of three-dimensional shape
from specular highlights, deformations of shading, and other types of visual
information", Psychological Science, vol. 15, no. 8, pp. 565-570.

Norman, J. F., Todd, J. T. and Phillips, F. (1995), "The perception of surface orientation from
multiple sources of optical information.", Perception and Psychophysics, vol. 57, no. 5,
pp. 629-636.

Papanicolaou, G. C. and Stavropoulos, C. D. (1995), "New approach for residual compressive

strength prediction of impacted CFRP laminates", Composites, vol. 26, no. 7, pp. 517-
523.

PRC-DeSoto (2003), Desothane HS polyurethane topcoats/ CA8000 Series, Technical Data,

July 2003, PRC-DeSoto International, Glendale, California.

Psymouli, A., Harris, D. and Irving, P. (2005), "The Inspection of Composite Aircraft
Structures: A Signal Detection Approach", Human Factors and Aerospace Safety, vol. 5,
no. 2, pp. 91-108.

RAF, ( 2006), Personal communication - Visit/ discussions with Repair Management Team,
Royal Air Force (RAF) Cottesmore, Oakham, Rutland, 2nd May 2006.

Raju, I. S. (2006), "NASA Langley Damage Tolerance Experiences", FAA Symposium on

Composite Damage Tolerance and Maintenance, July 19-21 2006, Chicago.

Roach, D. (2007), "Improving In-Service Inspection of Composite Structures", Commercial

Aircraft Composite Repair Committee (CACRC) Meeting, 12-16 November 2007,
Wichita, Kansas.

Robinson, P. and Davies, G. A. O. (1992), "Impactor mass and specimen geometry effects in
low velocity impact of laminated composites", International Journal of Impact
Engineering, vol. 12, no. 2, pp. 189-207.

Robois, S. (2005), "Composites at Airbus, In-Service Experience", Composites and metallics

in partnership: In service experience, Proceedings of one-day conference, 12th October
2005, Royal Aeronautical Society, London, UK.

Rouchon, J. (1990), "Certification of Large Aircraft Composite Structures, Recent Progress

and New Trends in Compliance Philosophy", 17th International Council of the
Aeronautical Sciences (ICAS), Stockholm, Sweden.

Rouchon, J. (2009), "Fatigue and Damage Tolerance Evaluation of Structures: The Composite
Materials Response, 22nd Plantema Memorial Lecture", 25th Symposium, International
Committee on Aeronautical Fatigue (ICAF), 27 May 2009, Rotterdam, Netherlands,
National Aerospace Laboratory NLR, Amsterdam, Netherlands.

Rouchon, J. (2010) personal communication at Cranfield University on 12th January 2010.

RTO (1998), Airframe Inspection Reliability under Field/Depot Conditions, RTO MP-10,
Research and Technology Organization, North Atlantic Treaty Organization, F-92201
Neuilly-sur-Seine Cedex, France.

204
Rummel, W. D. (1984), "Human factors considerations in the assessment of nondestructive
evaluation (NDE) reliability", Review of progress in quantitative nondestructive
evaluation, Proceedings of the Tenth Annual Review, Vol. 3A, 7-12 Aug. 1983, Santa
Cruz, CA, Plenum Press, New York, pp. 44.

Rummel, W. D. and Matzkanin, G. A. (1997), Nondestructive Evaluation (NDE) Capabilities

Databook, Third Edition, NTIAC: DB-97-02, Nondestructive Testing Information
Analysis Center (NTIAC), TRI/Austin, Inc., 415 Crystal Creek Drive, Austin, TX
78746-4725.

Sadasivan, S., Vembar, D., Washburn, C. and Gramopadhye, A. K. (2007), "Evaluation of

interaction devices for projector based virtual reality aircraft inspection training
environments", 2nd International Conference on Virtual Reality, ICVR 2007, Vol. 4563
LNCS, 22 July 2007 through 27 July 2007, Beijing, pp. 533.

Sakata, H., Tsutsui, K. and Taira, M. (2003), "Representation of the 3D world in art and in the
brain", International Congress Series, vol. 1250, pp. 15-35.

Schoeppner, G. A. and Abrate, S. (2000), "Delamination threshold loads for low velocity
impact on composite laminates", Composites Part A: Applied Science and
Manufacturing, vol. 31, no. 9, pp. 903-915.

Shyr, T. and Pan, Y. (2003), "Impact resistance and damage characteristics of composite
laminates", Composite Structures, vol. 62, no. 2, pp. 193-203.

Singh, R. (2000), Three Decades of NDI Reliability Assessment, Karta-3510-99-01, Karta

Technologies Inc., San Antonio, TX, USA.

Sohn, M. S., Hu, X. Z., Kim, J. K. and Walker, L. (2000), "Impact damage characterisation of
carbon fibre/epoxy composites with multi-layer reinforcement", Composites Part B:
Engineering, vol. 31, no. 8, pp. 681-691.

SolidWorks (2000), Solid Works 2000 Computer Aided Design software, Solid Works
Corporation.

Spencer, F. W. (1998), Identifying Sources of Variation for Reliability Analysis of Field

Inspections. SAND-98-0980C CONF-980552, US Department of Defense, Washington,
DC.

Spencer, F. W. (1996a), "Visual inspection reliability of transport aircraft", Proceedings of

SPIE - The International Society for Optical Engineering, vol. 2945, pp. 160-171.

Spencer, F. W. (1996b), Visual Inspection Research Project Report on Benchmark

Inspections (Final Report), AD-A321199; DOT/FAA/AR-96/65; NIPS-97-17825.

Spencer, F. W., Borgonovi, G., Roach, D., Schurman, D. L., SMITH, R. O. N. and Sandia
National Labs., Albuquerque, NM. (1993), Reliability assessment at airline inspection
facilities. Volume 1: A generic protocol for inspection reliability experiments (Final
Report), DOT/FAA/CT-92/12-VOL-1.

Staszewski, W. J. (2009), "Silence of the lamb-wave", Aerospace Testing International

March 2009, pp. 76-77.

205
Staszewski, W. J., Boller, C. and Tomlinson, G. R. (eds.) (2004), Health Monitoring of
Aerospace Structures - Smart Sensor Technologies and Signal Processing, John Wiley
& Sons, Ltd, Chichester, UK.

Statsoft Inc. (2008) Statistica 8.0 statistical analysis computer software, Statsoft Inc, Tulsa,
OK.

Todd, J. T. (2004), "The visual perception of 3D shape", Trends in cognitive sciences, vol. 8,
no. 3, pp. 115-121.

Todd, J. T., Koenderink, J. J., Van Doorn, A. J. and Kappers, A. M. (1996), "Effects of
Changing Viewing Conditions on the Perceived Structure of Smoothly Curved
Surfaces", Journal of Experimental Psychology: Human Perception and Performance,
vol. 22, no. 3, pp. 695-706.

Toray (2001), 3631/ T800H Technical Data Sheet May 2001, Toray Composites America,
Inc., USA.

Transport Canada (2004), Advisory Circular 500-009 Composite Aircraft Structure, 5009-6-
500, Transport Canada.

Tsutsui, K., Taira, M. and Sakata, H. (2005), "Neural mechanisms of three-dimensional

vision", Neuroscience research, vol. 51, no. 3, pp. 221-229.

US DoD (2007), Nondestructive Evaluation System Reliability Assessment, United States

Department of Defense Handbook, Latest Revision, MIL-HDBK-1823, United States
Department of Defence, Wright-Patterson AFB, OH.

US DoD (2002), Composite Materials Handbook, Volume 1. Polymer Matrix Composites,

Guidelines for Characterisation of Structural Materials, MIL-HDBK-17-1F, United
States Department of Defence.

US DoD (2002), Composite Materials Handbook, Volume 3. Polymer Matrix Composites,

Materials Usage, Design & Analysis, MIL-HDBK-17-3F, United States Department of
Defence.

Volker, A. W. F., Dijkstra, F. H., Terpstra, S., Heerings, H. A. M. and Lont, M. A. (2004),
"Modeling of NDE Reliability: Development of a POD-Generator", 16th World
Conference on Nondestructive Testing, Vol. Vol. 9, No. 11, Aug 30 - Sep 3 2004,
Montreal, Canada, Canadian Institute for NDE, Hamilton, Ontario.

Vora, J., Nair, S., Gramopadhye, A. K., Duchowski, A. T., Melloy, B. J. and Kanki, B.
(2002), "Using virtual reality technology for aircraft visual inspection training: Presence
and comparison studies", Applied Ergonomics, vol. 33, no. 6, pp. 559-570.

Waite, S. (2009), "EASA Perspectives on Safe Composite Damage Tolerance and

Maintenance Practices", Commercial Aircraft Composite Repair Committee (CACRC)
Meeting and Workshop for Composite Damage Tolerance & Maintenance, 1-5 June
2009, Tokyo, Japan.

Webborn, C. (2006), "Lower Cost Manufacturing in Composites", in Hybridmat 3: Advances

in the Manufacture of Advanced Structural Composites in Aerospace, Department of
Trade and Industry, London, UK, pp. 33-38.

206
Welch, J. M. (2007), "Repair, Design, Test and Process Considerations for Lightning Strikes",
CACRC/ MIL-HDBK-17 Conference, May 7-11, 2007, Amsterdam, Netherlands.

Wexler, M. and Van Boxtel, J. J. A. (2005), "Depth perception by the active observer",
Trends in cognitive sciences, vol. 9, no. 9, pp. 431-438.

Wickens, C.D. (2000), Engineering psychology and human performance, 3rd Ed, Prentice-
Hall, New York.

Xiao, B. and Brainard, D. H. (2008), "Surface gloss and color perception of 3D objects",
Visual neuroscience, vol. 25, no. 3, pp. 371-385.

Xiao, B. and Brainard, D. H. (2006), "Color perception of 3D objects: Constancy with respect
to variation of surface gloss", pp. 63.

207
Appendix A
CAD system rendering settings for virtual specimens
Lighting settings
The following settings were used to define the ambient and directional light
parameters for the virtual specimens in SolidWorks 2000.

208
Materials settings
The following materials editor settings were used for the PhotoWorks rendering
application in the SolidWorks 2000 CAD Program to produce the virtual specimens:

Primary colour settings for material editor

Hue: 160 Red: 242
Saturation: 0 Green: 242
Luminosity: 228 Blue: 242

Reflectance settings for material editor

Ambient 1
Diffuse 1
Specular 0.8
Transparency 0
Hue: 160, Sat: 0, Lum: 240
Specular Colour
Red: 255, Green: 255, Blue: 255

209
Appendix B
Specimen & Flaw specifications for series 1 visual inspection trials
Trial 1 specimen/ flaw characteristics
Panel Number/ Flaw Details Panel Number/ Flaw Details Panel Number/ Flaw Details Panel Number/ Flaw Details
1 UNFLAWED 31 UNFLAWED 61 UNFLAWED 91 UNFLAWED

2 UNFLAWED 32 Ø22 62 UNFLAWED 92 UNFLAWED

X = 527mm, Y = 524mm
3 UNFLAWED 33 Ø9 63 UNFLAWED 93 UNFLAWED
X = 671mm, Y = 385mm
4 UNFLAWED 34 UNFLAWED 64 UNFLAWED 94 UNFLAWED

5 UNFLAWED 35 Ø14 65 Ø5 95 UNFLAWED

X = 659mm, Y = 550mm X = 540mm, Y = 240mm
6 UNFLAWED 36 Ø28 66 UNFLAWED 96 UNFLAWED
X = 186mm, Y = 259mm
7 UNFLAWED 37 UNFLAWED 67 X = 437mm, Y = 561mm 97 Ø8
Ø27 X = 321mm, Y = 254mm
8 UNFLAWED 38 Ø18mm 68 UNFLAWED 98 UNFLAWED
X = 368mm, Y = 360mm
9 UNFLAWED 39 UNFLAWED 69 UNFLAWED 99 Ø4
X = 315mm, Y = 134mm
10 Ø26mm 40 Ø30 70 UNFLAWED 100 UNFLAWED
X = 649mm, Y = 85mm X = 74mm, Y = 315mm
11 Ø6mm 41 UNFLAWED 71 UNFLAWED 101 UNFLAWED
X = 142mm, Y = 533mm
12 UNFLAWED 42 UNFLAWED 72 UNFLAWED 102 UNFLAWED

13 UNFLAWED 43 UNFLAWED 73 Ø20 103 UNFLAWED

X = 677mm, Y = 698mm
14 UNFLAWED 44 UNFLAWED 74 UNFLAWED 104 UNFLAWED

15 UNFLAWED 45 UNFLAWED 75 Ø16 105 UNFLAWED

X = 578mm, Y = 526mm
16 UNFLAWED 46 UNFLAWED 76 Ø23 106 UNFLAWED
X = 640mm, Y = 374mm
17 UNFLAWED 47 UNFLAWED 77 Ø17 107 Ø12
X = 5mm, Y = 392mm X = 339mm, Y = 454mm
18 UNFLAWED 48 Ø19 78 UNFLAWED 108 Ø13
X = 370mm, Y = 341mm X = 580mm, Y = 425mm
19 UNFLAWED 49 UNFLAWED 79 UNFLAWED 109 UNFLAWED

20 UNFLAWED 50 UNFLAWED 80 UNFLAWED 110 Ø10

X = 703mm, Y = 94mm
21 UNFLAWED 51 Ø29 81 UNFLAWED 111 Ø25
X = 305mm, Y = 665mm X = 120mm, Y = 664mm
22 UNFLAWED 52 UNFLAWED 82 UNFLAWED 112 UNFLAWED

23 Ø21 53 UNFLAWED 83 Ø7 113 UNFLAWED

X = 165mm, Y = 233mm X = 361mm, Y = 631mm
24 UNFLAWED 54 Ø3 84 Ø2 114 UNFLAWED
X = 535mm, Y = 100mm X = 567mm, Y = 412mm
25 UNFLAWED 55 UNFLAWED 85 UNFLAWED 115 UNFLAWED

26 UNFLAWED 56 UNFLAWED 86 UNFLAWED 116 UNFLAWED

27 Ø24 57 UNFLAWED 87 Ø1 117 UNFLAWED

X = 547mm, Y = 537mm X = 210mm, Y = 456mm
28 UNFLAWED 58 UNFLAWED 88 Ø15 118 UNFLAWED
X = 645mm, Y = 529mm
29 UNFLAWED 59 UNFLAWED 89 UNFLAWED 119 UNFLAWED

30 Ø11 60 UNFLAWED 90 UNFLAWED 120 UNFLAWED

X = 375mm, Y = 486mm

210
Trial 2 Specimen/ flaw characteristics
Panel Numbers & Flaw sizes: Ø## = Flaw Width #.## d = Flaw Depth

1 Unflaw ed 21 Ø12mm 41 Unflaw ed 61 Unflaw ed 81 Unflaw ed 101 Unflaw ed 121 Unflaw ed 141 Ø12mm
0.2mm d 0.9mm d
2 Unflaw ed 22 Unflaw ed 42 Unflaw ed 62 Unflaw ed 82 Ø4.5mm 102 Unflaw ed 122 Unflaw ed 142 Ø4.5mm
1.5mm d 0.1mm d
3 Unflaw ed 23 Unflaw ed 43 Unflaw ed 63 Unflaw ed 83 Unflaw ed 103 Ø12mm 123 Unflaw ed 143 Unflaw ed

0.4mm d
4 Ø6.5mm 24 Unflaw ed 44 Ø6.5mm 64 Unflaw ed 84 Unflaw ed 104 Ø4.5mm 124 Unflaw ed 144 Unflaw ed

0.4mm d 0.3mm d 1.2mm d

5 Unflaw ed 25 Unflaw ed 45 Unflaw ed 65 Unflaw ed 85 Unflaw ed 105 Ø4.5mm 125 Ø4.5mm 145 Ø6.5mm
0.2mm d 0.4mm d 0.5mm d
6 Unflaw ed 26 Unflaw ed 46 Unflaw ed 66 Unflaw ed 86 Unflaw ed 106 Ø4.5mm 126 Unflaw ed 146 Unflaw ed

0.6mm d
7 Unflaw ed 27 Ø4.5mm 47 Unflaw ed 67 Unflaw ed 87 Unflaw ed 107 Unflaw ed 127 Ø12mm 147 Unflaw ed

0.3mm d 0.8mm d
8 Unflaw ed 28 Unflaw ed 48 Unflaw ed 68 Unflaw ed 88 Ø12mm 108 Unflaw ed 128 Ø4.5mm 148 Unflaw ed

0.7mm d 0.5mm d
9 Unflaw ed 29 Unflaw ed 49 Unflaw ed 69 Ø4.5mm 89 Unflaw ed 109 Unflaw ed 129 Unflaw ed 149 Ø4.5mm
1.1mm d 1.0mm d
10 Unflaw ed 30 Unflaw ed 50 Unflaw ed 70 Unflaw ed 90 Unflaw ed 110 Unflaw ed 130 Unflaw ed 150 Unflaw ed

11 Unflaw ed 31 Unflaw ed 51 Unflaw ed 71 Unflaw ed 91 Ø4.5mm 111 Unflaw ed 131 Unflaw ed 151 Unflaw ed

0.8mm d
12 Unflaw ed 32 Ø12mm 52 Ø6.5mm 72 Unflaw ed 92 Ø6.5mm 112 Unflaw ed 132 Unflaw ed 152 Unflaw ed

1.0mm d 1.2mm d 1.3mm d

13 Ø6.5mm 33 Unflaw ed 53 Unflaw ed 73 Unflaw ed 93 Unflaw ed 113 Unflaw ed 133 Unflaw ed 153 Unflaw ed

0.2mm d
14 Unflaw ed 34 Unflaw ed 54 Ø12mm 74 Ø6.5mm 94 Unflaw ed 114 Unflaw ed 134 Unflaw ed 154 Ø6.5mm
0.1mm d 1.1mm d 0.7mm d
15 Unflaw ed 35 Ø4.5mm 55 Ø6.5mm 75 Unflaw ed 95 Unflaw ed 115 Unflaw ed 135 Ø6.5mm 155 Unflaw ed

1.3mm d 0.9mm d 0.6mm d

16 Unflaw ed 36 Unflaw ed 56 Ø12mm 76 Unflaw ed 96 Unflaw ed 116 Unflaw ed 136 Ø4.5mm 156 Unflaw ed

0.3mm d 0.7mm d
17 Unflaw ed 37 Unflaw ed 57 Unflaw ed 77 Unflaw ed 97 Unflaw ed 117 Unflaw ed 137 Ø4.5mm
1.4mm d
18 Unflaw ed 38 Unflaw ed 58 Ø6.5mm 78 Unflaw ed 98 Unflaw ed 118 Ø12mm 138 Unflaw ed

0.1mm d 1.1mm d
19 Unflaw ed 39 Unflaw ed 59 Ø12mm 79 Unflaw ed 99 Unflaw ed 119 Ø6.5mm 139 Unflaw ed

0.5mm d 0.8mm d
20 Ø12mm 40 Unflaw ed 60 Ø4.5mm 80 Ø6.5mm 100 Unflaw ed 120 Unflaw ed 140 Unflaw ed

0.6mm d 0.9mm d 1.0mm d

211
X/Y Panel location coordinates; All in mm
Flaw Width Flaw Depth X Coordinate Y Coordinate Internal Flaw Radius
0.1 38 93 25.363
0.2 686 294 12.756
0.3 152 469 8.588
0.4 13 4 6.528
0.5 677 502 5.313
0.6 64 428 4.519
0.7 303 426 3.996
Ø4.5mm 0.8 600 226 3.564
0.9 8 548 3.263
1 104 616 3.031
1.1 436 394 2.851
1.2 693 94 2.709
1.3 161 123 2.597
1.4 299 477 2.508
1.5 33 492 2.438
0.1 619 302 52.863
0.2 26 439 26.506
0.3 485 305 17.754
0.4 607 624 13.403
0.5 91 3 10.813
0.6 674 602 9.102
Ø6.5mm 0.7 362 604 7.895
0.8 253 645 7.002
0.9 29 643 6.318
1 289 274 5.781
1.1 43 381 5.351
1.2 699 79 5.001
1.3 277 455 4.713
0.1 703 547 180.05
0.2 472 262 90.1
0.3 654 397 60.15
0.4 147 142 45.2
0.5 240 644 36.25
Ø12mm 0.6 10 325 30.3
0.7 60 387 26.064
0.8 356 333 22.9
0.9 174 681 20.45
1 281 286 18.5
1.1 432 381 16.914

212
Trial 3 specimen/ flaw characteristics
Panel Numbers &Flaw sizes: Ø## =Flaw Width #.## d =Flaw Depth

1 Unflaw ed 21 Unflaw ed 41 Unflaw ed 61 Unflaw ed 81 Ø4.5mm 101 Unflaw ed 121 Unflaw ed 141 Ø4.5mm
0.4mm d 0.1mm d
2 Unflaw ed 22 Unflaw ed 42 Unflaw ed 62 Ø12mm 82 Unflaw ed 102 Unflaw ed 122 Unflaw ed 142 Unflaw ed

0.3mm d
3 Unflaw ed 23 Unflaw ed 43 Unflaw ed 63 Ø6.5mm 83 Ø6.5mm 103 Ø6.5mm 123 Unflaw ed 143 Unflaw ed

1.0mm d 0.1mm d 0.2mm d

4 Unflaw ed 24 Ø12mm 44 Unflaw ed 64 Ø6.5mm 84 Unflaw ed 104 Unflaw ed 124 Unflaw ed 144 Unflaw ed

1.1mm d 0.4mm d
5 Unflaw ed 25 Unflaw ed 45 Unflaw ed 65 Unflaw ed 85 Ø12mm 105 Unflaw ed 125 Unflaw ed 145 Unflaw ed

0.6mm d
6 Unflaw ed 26 Unflaw ed 46 Unflaw ed 66 Unflaw ed 86 Unflaw ed 106 Unflaw ed 126 Unflaw ed 146 Ø12mm
0.2mm d
7 Unflaw ed 27 Unflaw ed 47 Unflaw ed 67 Unflaw ed 87 Unflaw ed 107 Unflaw ed 127 Ø4.5mm 147 Unflaw ed

0.9mm d
8 Unflaw ed 28 Ø6.5mm 48 Unflaw ed 68 Ø12mm 88 Unflaw ed 108 Unflaw ed 128 Ø4.5mm 148 Ø4.5mm
0.7mm d 1.0mm d 0.6mm d 1.0mm d
9 Ø4.5mm 29 Unflaw ed 49 Unflaw ed 69 Unflaw ed 89 Unflaw ed 109 Unflaw ed 129 Unflaw ed 149 Unflaw ed

1.2mm d
10 Unflaw ed 30 Unflaw ed 50 Ø12mm 70 Unflaw ed 90 Unflaw ed 110 Ø6.5mm 130 Unflaw ed 150 Ø4.5mm
0.8mm d 0.6mm d 0.2mm d
11 Unflaw ed 31 Ø12mm 51 Unflaw ed 71 Ø6.5mm 91 Unflaw ed 111 Unflaw ed 131 Ø4.5mm 151 Unflaw ed

0.4mm d 0.5mm d 1.5mm d

12 Unflaw ed 32 Unflaw ed 52 Unflaw ed 72 Ø6.5mm 92 Unflaw ed 112 Ø4.5mm 132 Ø12mm 152 Unflaw ed

0.8mm d 1.3mm d 0.5mm d

13 Unflaw ed 33 Unflaw ed 53 Unflaw ed 73 Ø6.5mm 93 Unflaw ed 113 Unflaw ed 133 Unflaw ed 153 Unflaw ed

1.1mm d
14 Unflaw ed 34 Unflaw ed 54 Unflaw ed 74 Unflaw ed 94 Unflaw ed 114 Unflaw ed 134 Unflaw ed 154 Unflaw ed

15 Unflaw ed 35 Unflaw ed 55 Unflaw ed 75 Unflaw ed 95 Unflaw ed 115 Ø12mm 135 Unflaw ed 155 Unflaw ed

0.9mm d
16 Ø6.5mm 36 Unflaw ed 56 Ø12mm 76 Unflaw ed 96 Unflaw ed 116 Ø4.5mm 136 Unflaw ed 156 Unflaw ed

0.9mm d 0.1mm d 0.8mm d

17 Unflaw ed 37 Unflaw ed 57 Ø6.5mm 77 Ø6.5mm 97 Unflaw ed 117 Ø4.5mm 137 Unflaw ed

1.3mm d 1.2mm d 0.7mm d

18 Unflaw ed 38 Unflaw ed 58 Unflaw ed 78 Ø4.5mm 98 Ø4.5mm 118 Unflaw ed 138 Unflaw ed

1.4mm d 1.1mm d
19 Unflaw ed 39 Ø4.5mm 59 Unflaw ed 79 Ø4.5mm 99 Unflaw ed 119 Unflaw ed 139 Unflaw ed

0.5mm d 0.3mm d
20 Unflaw ed 40 Ø12mm 60 Unflaw ed 80 Unflaw ed 100 Unflaw ed 120 Unflaw ed 140 Ø6.5mm
0.7mm d 0.3mm d

213
Flaw details & X/Y Panel location coordinates; all in mm
Flaw Width Flaw Depth X Coordinate Y Coordinate InternalFlaw Radius Edge Blend Radius
0.1 454 315 25.363 12.682
0.2 13 107 12.756 6.378
0.3 339 63 8.588 4.294
0.4 665 425 6.528 3.264
0.5 253 194 5.313 2.657
0.6 400 563 4.519 2.260
0.7 608 70 3.996 1.998
Ø4.5mm 0.8 607 433 3.564 1.782
0.9 522 262 3.263 1.632
1 210 546 3.031 1.516
1.1 32 174 2.851 1.426
1.2 247 30 2.709 1.355
1.3 374 563 2.597 1.299
1.4 446 175 2.508 1.254
1.5 275 218 2.438 1.219
0.1 235 446 52.863 26.432
0.2 81 361 26.506 13.253
0.3 546 227 17.754 8.877
0.4 294 113 13.403 6.702
0.5 45 15 10.813 5.407
0.6 468 299 9.102 4.551
Ø6.5mm 0.7 95 167 7.895 3.948
0.8 201 530 7.002 3.501
0.9 183 52 6.318 3.159
1 186 72 5.781 2.891
1.1 683 183 5.351 2.676
1.2 625 269 5.001 2.501
1.3 510 446 4.713 2.357
0.1 360 275 180.05 90.025
0.2 245 366 90.1 45.050
0.3 389 369 60.15 30.075
0.4 426 640 45.2 22.600
0.5 33 296 36.25 18.125
Ø12mm 0.6 175 270 30.3 15.150
0.7 513 456 26.064 13.032
0.8 664 410 22.9 11.450
0.9 124 342 20.45 10.225
1 246 65 18.5 9.250
1.1 371 228 16.914 8.457

214
Trial 4 specimen/ flaw characteristics
Panel Numbers & Flaw sizes: Ø## = Flaw Width #.## d = Flaw Depth

1 Unlfaw ed 23 Unlfaw ed 45 Ø16mm 67 Unlfaw ed 89 Unlfaw ed 111 Unlfaw ed 133 Unlfaw ed 155 Unlfaw ed

HARD EDGED

2 Unlfaw ed 24 Unlfaw ed 46 Ø10mm 68 Unlfaw ed 90 Ø22mm 112 Unlfaw ed 134 Ø28mm 156 Unlfaw ed

HARD EDGED HARD EDGED BLENDED EDGE

3 Unlfaw ed 25 Unlfaw ed 47 Unlfaw ed 69 Ø10mm 91 Unlfaw ed 113 Unlfaw ed 135 Unlfaw ed 157 Ø16mm
BLENDED EDGE BLENDED EDGE

4 Unlfaw ed 26 Unlfaw ed 48 Unlfaw ed 70 Unlfaw ed 92 Ø14mm 114 Unlfaw ed 136 Unlfaw ed 158 Unlfaw ed

HARD EDGED

5 Unlfaw ed 27 Unlfaw ed 49 Unlfaw ed 71 Unlfaw ed 93 Unlfaw ed 115 Ø24mm 137 Unlfaw ed 159 Ø5mm
HARD EDGED BLENDED EDGE

6 Ø5mm 28 Ø7mm 50 Ø18mm 72 Unlfaw ed 94 Unlfaw ed 116 Unlfaw ed 138 Unlfaw ed 160 Unlfaw ed

HARD EDGED BLENDED EDGE BLENDED EDGE

7 Unlfaw ed 29 Unlfaw ed 51 Unlfaw ed 73 Ø9mm 95 Unlfaw ed 117 Unlfaw ed 139 Ø30mm 161 Ø7mm
HARD EDGED BLENDED EDGE HARD EDGED

8 Unlfaw ed 30 Ø20mm 52 Unlfaw ed 74 Unlfaw ed 96 Ø3mm 118 Unlfaw ed 140 Unlfaw ed 162 Unlfaw ed

HARD EDGED BLENDED EDGE

9 Ø26mm 31 Unlfaw ed 53 Ø30mm 75 Unlfaw ed 97 Ø14mm 119 Ø16mm 141 Unlfaw ed 163 Ø28mm
HARD EDGED HARD EDGED BLENDED EDGE HARD EDGED HARD EDGED

10 Unlfaw ed 32 Ø10mm 54 Unlfaw ed 76 Ø9mm 98 Unlfaw ed 120 Unlfaw ed 142 Ø10mm 164 Unlfaw ed

HARD EDGED BLENDED EDGE BLENDED EDGE

11 Unlfaw ed 33 Unlfaw ed 55 Ø30mm 77 Unlfaw ed 99 Unlfaw ed 121 Unlfaw ed 143 Ø4mm 165 Ø4mm
BLENDED EDGE HARD EDGED BLENDED EDGE

12 Unlfaw ed 34 Ø6mm 56 Unlfaw ed 78 Unlfaw ed 100 Unlfaw ed 122 Unlfaw ed 144 Unlfaw ed 166 Unlfaw ed

BLENDED EDGE

13 Unlfaw ed 35 Unlfaw ed 57 Unlfaw ed 79 Unlfaw ed 101 Unlfaw ed 123 Ø24mm 145 Unlfaw ed 167 Unlfaw ed

BLENDED EDGE

14 Unlfaw ed 36 Unlfaw ed 58 Unlfaw ed 80 Unlfaw ed 102 Ø30mm 124 Unlfaw ed 146 Unlfaw ed 168 Ø8mm
HARD EDGED BLENDED EDGE

15 Unlfaw ed 37 Unlfaw ed 59 Unlfaw ed 81 Unlfaw ed 103 Ø12mm 125 Unlfaw ed 147 Unlfaw ed 169 Unlfaw ed

BLENDED EDGE

16 Unlfaw ed 38 Unlfaw ed 60 Unlfaw ed 82 Unlfaw ed 104 Unlfaw ed 126 Unlfaw ed 148 Ø8mm 170 Unlfaw ed

HARD EDGED

17 Unlfaw ed 39 Unlfaw ed 61 Ø5mm 83 Unlfaw ed 105 Unlfaw ed 127 Unlfaw ed 149 Unlfaw ed 171 Unlfaw ed

BLENDED EDGE

18 Ø26mm 40 Ø16mm 62 Unlfaw ed 84 Unlfaw ed 106 Unlfaw ed 128 Unlfaw ed 150 Unlfaw ed 172 Unlfaw ed

BLENDED EDGE BLENDED EDGE

19 Unlfaw ed 41 Unlfaw ed 63 Unlfaw ed 85 Unlfaw ed 107 Unlfaw ed 129 Unlfaw ed 151 Ø20mm 173 Unlfaw ed

BLENDED EDGE

20 Unlfaw ed 42 Unlfaw ed 64 Unlfaw ed 86 Ø6mm 108 Unlfaw ed 130 Ø5mm 152 Unlfaw ed 174 Unlfaw ed

HARD EDGED HARD EDGED

21 Unlfaw ed 43 Ø12mm 65 Unlfaw ed 87 Unlfaw ed 109 Ø3mm 131 Unlfaw ed 153 Unlfaw ed 175 Unlfaw ed

HARD EDGED HARD EDGED

22 Unlfaw ed 44 Unlfaw ed 66 Ø18mm 88 Unlfaw ed 110 Unlfaw ed 132 Unlfaw ed 154 Unlfaw ed 176 Ø22mm
HARD EDGED BLENDED EDGE

215
Flaw details & X/Y Panel location coordinates; all in mm
Flaw Depth Flaw Width Edge Type X Coordinate Y Coordinate Internal Flaw Radius Edge Blend Radius
3 Blended Edge 410 138 7.575 7.575
3 Hard Edge 643 60 7.575 ~
4 Blended Edge 695 143 13.408 13.408
4 Hard Edge 333 204 13.408 ~
5 Blended Edge 286 501 20.908 20.908
5 Blended Edge 11 543 20.908 20.908
5 Hard Edge 626 72 20.908 ~
5 Hard Edge 236 480 20.908 ~
6 Blended Edge 546 375 30.075 30.075
6 Hard Edge 183 694 30.075 ~
7 Blended Edge 91 318 40.908 40.908
7 Hard Edge 339 379 40.908 ~
8 Blended Edge 177 116 53.408 53.408
8 Hard Edge 701 184 53.408 ~
9 Blended Edge 143 363 67.575 67.575
9 Hard Edge 380 310 67.575 ~
10 Blended Edge 298 66 83.408 83.408
10 Blended Edge 570 594 83.408 83.408
10 Hard Edge 46 270 83.408 ~
10 Hard Edge 522 446 83.408 ~
12 Blended Edge 71 519 120.075 120.075
12 Hard Edge 16 646 120.075 ~
0.15
14 Blended Edge 635 140 163.408 163.408
14 Hard Edge 642 303 163.408 ~
16 Blended Edge 616 485 213.408 213.408
16 Blended Edge 522 663 213.408 213.408
16 Hard Edge 84 325 213.408 ~
16 Hard Edge 15 489 213.408 ~
18 Blended Edge 261 115 270.075 270.075
18 Hard Edge 337 138 270.075 ~
20 Blended Edge 497 65 333.408 333.408
20 Hard Edge 422 635 333.408 ~
22 Blended Edge 453 21 403.408 403.408
22 Hard Edge 343 525 403.408 ~
24 Blended Edge 150 92 480.075 480.075
24 Hard Edge 69 213 480.075 ~
26 Blended Edge 386 522 563.408 563.408
26 Hard Edge 404 318 563.408 ~
28 Blended Edge 351 348 653.408 653.408
28 Hard Edge 679 687 653.408 ~
30 Blended Edge 486 113 750.075 750.075
30 Blended Edge 576 300 750.075 750.075
30 Hard Edge 423 124 750.075 ~
30 Hard Edge 428 168 750.075 ~

216
Appendix C
Drawings of specimen panels for series 2 visual inspection trials

217
218
219
220
221
222
223
224
Appendix D
Examples of impact damage to AS4/8552 CFRP Coupons
The impact damage to coupons of 17ply and 33ply AS4/8552 CFRP laminates are
presented in this appendix in the form of photographs of the damage and
corresponding C-Scan images fitted with a 10 mm x 10 mm grid. Where available,
corresponding photographs from metallographic sectioning of the impact damage
sites from Boulic 2007 are also provided.

Photograph of impact damage coupon #27

C-Scan of impact damage coupon #27
(17ply + mesh painted, Ø20mm/5J)

Metallographic section image of Ø20mm/5J impact (from Boulic, 2007)

Photograph of impact damage coupon #26

C-Scan of impact damage coupon #26
(17ply + mesh painted, Ø20mm/10J)

225
Photograph of impact damage coupon #25
C-Scan of impact damage coupon #25
(17ply + mesh painted, Ø20mm/15J)

Metallographic section image of Ø20mm/15J impact (from Boulic, 2007)

Photograph of impact damage coupon #24

C-Scan of impact damage coupon #24
(17ply + mesh painted, Ø20mm/20J)

226
Photograph of impact damage coupon #23
C-Scan of impact damage coupon #23
(17ply + mesh painted, Ø20mm/30J)

Metallographic section image of Ø20mm/30J impact (from Boulic, 2007)

Photograph of impact damage coupon #U13

C-Scan of impact damage coupon #U13
(17ply unpainted, Ø20mm/5J)

Photograph of impact damage coupon #U14

C-Scan of impact damage coupon #U14
(17ply unpainted, Ø20mm/10J)

227
Photograph of impact damage coupon #U15
C-Scan of impact damage coupon #U15
(17ply unpainted, Ø20mm/15J)

Photograph of impact damage coupon #U16

C-Scan of impact damage coupon #U16
(17ply unpainted, Ø20mm/20J)

Photograph of impact damage coupon #U17

C-Scan of impact damage coupon #U17
(17ply unpainted, Ø20mm/30J)

228
Photograph of impact damage coupon #2
C-Scan of impact damage coupon #2
(33ply + mesh painted, Ø20mm/5J)

Photograph of impact damage coupon #3

C-Scan of impact damage coupon #3
(33ply + mesh painted, Ø20mm/10J)

Metallographic section image of Ø20mm/10J impact (from Boulic, 2007)

Photograph of impact damage coupon #4

C-Scan of impact damage coupon #4
(33ply + mesh painted, Ø20mm/15J)

229
Photograph of impact damage coupon #5
C-Scan of impact damage coupon #5
(33ply + mesh painted, Ø20mm/20J)

Metallographic section image of Ø20mm/20J impact (from Boulic, 2007)

Photograph of impact damage coupon #6

C-Scan of impact damage coupon #6
(33ply + mesh painted, Ø20mm/30J)

230
 Photograph of impact damage coupon #7
C-Scan of impact damage coupon #7
(33ply + mesh painted, Ø20mm/50J)

Metallographic section image of Ø20mm/50J impact (from Boulic, 2007)

Photograph of impact damage coupon #U38

C-Scan of impact damage coupon #U38
(33ply unpainted, Ø20mm/5J)

Photograph of impact damage coupon #U39

C-Scan of impact damage coupon #U39
(33ply unpainted, Ø20mm/10J)

231
Photograph of impact damage coupon #U40
C-Scan of impact damage coupon #U40
(33ply unpainted, Ø20mm/15J)

Photograph of impact damage coupon #U41

C-Scan of impact damage coupon #U41
(33ply unpainted, Ø20mm/20J)

Photograph of impact damage coupon #U42

C-Scan of impact damage coupon #U42
(33ply unpainted, Ø20mm/30J)

232
Photograph of impact damage coupon #U43
C-Scan of impact damage coupon #U43
(33ply unpainted, Ø20mm/50J)

Photograph of impact damage coupon #95

C-Scan of impact damage coupon #95
(17ply + mesh painted, Ø87mm/5J)

Photograph of impact damage coupon #94

C-Scan of impact damage coupon #94
(17ply + mesh painted, Ø87mm/10J)

Metallographic section image of Ø87mm/10J impact (from Boulic, 2007)

233
Photograph of impact damage coupon #93
C-Scan of impact damage coupon #93
(17ply + mesh painted, Ø87mm/15J)

Photograph of impact damage coupon #29

C-Scan of impact damage coupon #29
(17ply + mesh painted, Ø87mm/20J)

Metallographic section image of Ø87mm/20J impact (from Boulic, 2007)

Photograph of impact damage coupon #92

C-Scan of impact damage coupon #92
(17ply + mesh painted, Ø87mm/30J)

234
 Photograph of impact damage coupon #30
C-Scan of impact damage coupon #30
(17ply + mesh painted, Ø87mm/40J)

Metallographic section image of Ø87mm/40J impact (from Boulic, 2007)

Photograph of impact damage coupon #90

C-Scan of impact damage coupon #90
(17ply + mesh painted, Ø87mm/50J)

Photograph of impact damage coupon # U18

C-Scan of impact damage coupon #U18
(17ply unpainted, Ø87mm/5J)

235
Photograph of impact damage coupon # U19
C-Scan of impact damage coupon #U19
(17ply unpainted, Ø87mm/10J)

Photograph of impact damage coupon # U20

C-Scan of impact damage coupon #U20
(17ply unpainted, Ø87mm/15J)

Photograph of impact damage coupon # U21

C-Scan of impact damage coupon #U21
(17ply unpainted, Ø87mm/20J)

236
Photograph of impact damage coupon # U22
C-Scan of impact damage coupon #U22
(17ply unpainted, Ø87mm/30J)

Photograph of impact damage coupon # U23

C-Scan of impact damage coupon #U23
(17ply unpainted, Ø87mm/40J)

Photograph of impact damage coupon # U24

C-Scan of impact damage coupon #U24
(17ply unpainted, Ø87mm/50J)

237
Photograph of impact damage coupon #9
C-Scan of impact damage coupon #9
(33ply + mesh painted, Ø87mm/20J)

Metallographic section image of Ø87mm/20J impact (from Boulic, 2007)

Not available

Photograph of impact damage coupon #8

C-Scan of impact damage coupon #8
(33ply + mesh painted, Ø87mm/30J)

238
Photograph of impact damage coupon #10
C-Scan of impact damage coupon #10
(33ply + mesh painted, Ø87mm/40J)

Metallographic section image of Ø87mm/40J impact (from Boulic, 2007)

Photograph of impact damage coupon #11

C-Scan of impact damage coupon #11
(33ply + mesh painted, Ø87mm/50J)

239
Photograph of impact damage coupon #12
C-Scan of impact damage coupon #12
(33ply + mesh painted, Ø87mm/60J)

Metallographic section image of Ø87mm/60J impact (from Boulic, 2007)

Photograph of impact damage coupon #13

C-Scan of impact damage coupon #13
(33ply + mesh painted, Ø87mm/70J)

Photograph of impact damage coupon #U44

C-Scan of impact damage coupon #U44
(33ply unpainted, Ø87mm/20J)

240
Photograph of impact damage coupon #U45
C-Scan of impact damage coupon #U45
(33ply unpainted, Ø87mm/30J)

Photograph of impact damage coupon #U46

C-Scan of impact damage coupon #U46
(33ply unpainted, Ø87mm/40J)

Photograph of impact damage coupon #U47

C-Scan of impact damage coupon #U47
(33ply unpainted, Ø87mm/50J)

241
Photograph of impact damage coupon #U48
C-Scan of impact damage coupon #U48
(33ply unpainted, Ø87mm/60J)

Photograph of impact damage coupon #U49

C-Scan of impact damage coupon #U49
(33ply unpainted, Ø87mm/70J)

Photograph of impact damage coupon #36

C-Scan of impact damage coupon #36
(17ply + mesh painted, Ø70mm cylinder/10J)

242
 Photograph of impact damage coupon #37
C-Scan of impact damage coupon #37
(17ply + mesh painted, Ø70mm cylinder/15J)

Photograph of impact damage coupon #38

C-Scan of impact damage coupon #38
(17ply + mesh painted, Ø70mm cylinder/20J)

Photograph of impact damage coupon #40

C-Scan of impact damage coupon #40
(17ply + mesh painted, Ø70mm cylinder/30J)

243
 Photograph of impact damage coupon #39
C-Scan of impact damage coupon #39
(17ply + mesh painted, Ø70mm cylinder/40J)

Photograph of impact damage coupon #U25

C-Scan of impact damage coupon #U25
(17ply unpainted, Ø70mm cylinder/10J)

Photograph of impact damage coupon #U26

C-Scan of impact damage coupon #U26
(17ply unpainted, Ø70mm cylinder/15J)

244
Photograph of impact damage coupon #U27
C-Scan of impact damage coupon #U27
(17ply unpainted, Ø70mm cylinder/20J)

Photograph of impact damage coupon #U28

C-Scan of impact damage coupon #U28
(17ply unpainted, Ø70mm cylinder/30J)

Photograph of impact damage coupon #U29

C-Scan of impact damage coupon #U29
(17ply unpainted, Ø70mm cylinder/J40)

245
 Not available

Photograph of impact damage coupon #18

C-Scan of impact damage coupon #18
(33ply + mesh painted, Ø70mm cylinder/10J)

Not available

Photograph of impact damage coupon #19

C-Scan of impact damage coupon #19
(33ply + mesh painted, Ø70mm cylinder/20J)

Not available

Photograph of impact damage coupon #20

C-Scan of impact damage coupon #20
(33ply + mesh painted, Ø70mm cylinder/30J)

Not available

Photograph of impact damage coupon #21

C-Scan of impact damage coupon #21
(33ply + mesh painted, Ø70mm cylinder/40J)

246
 Not available

Photograph of impact damage coupon #1

C-Scan of impact damage coupon #1
(33ply + mesh painted, Ø70mm cylinder/50J)

Not available

Photograph of impact damage coupon #70

C-Scan of impact damage coupon #70
(33ply + mesh painted, Ø70mm cylinder/60J)

Photograph of impact damage coupon #U50

C-Scan of impact damage coupon #U50
(33ply unpainted, Ø70mm cylinder/10J)

247
Photograph of impact damage coupon #U51
C-Scan of impact damage coupon #U51
(33ply unpainted, Ø70mm cylinder/20J)

Photograph of impact damage coupon #U52

C-Scan of impact damage coupon #U52
(33ply unpainted, Ø70mm cylinder/30J)

Photograph of impact damage coupon #U53

C-Scan of impact damage coupon #U53
(33ply unpainted, Ø70mm cylinder/40J)

248
Photograph of impact damage coupon #U54
C-Scan of impact damage coupon #U54
(33ply unpainted, Ø70mm cylinder/50J)

Photograph of impact damage coupon #U55

C-Scan of impact damage coupon #U55
(33ply unpainted, Ø70mm cylinder/60J)

Photograph of impact damage coupon #34

C-Scan of impact damage coupon #34
(17ply + mesh painted, tool shape/5J)

249
Photograph of impact damage coupon #33
C-Scan of impact damage coupon #33
(17ply + mesh painted, tool shape/10J)

Photograph of impact damage coupon #32

C-Scan of impact damage coupon #32
(17ply + mesh painted, tool shape/15J)

Photograph of impact damage coupon #35

C-Scan of impact damage coupon #35
(17ply + mesh painted, tool shape/17.5J)

250
Photograph of impact damage coupon #31
C-Scan of impact damage coupon #31
(17ply + mesh painted, tool shape/20J)

Photograph of impact damage coupon #U30

C-Scan of impact damage coupon #U30
(17ply unpainted, tool shape/5J)

Photograph of impact damage coupon #U31

C-Scan of impact damage coupon #U31
(17ply unpainted, tool shape/10J)

251
Photograph of impact damage coupon #U32
C-Scan of impact damage coupon #U32
(17ply unpainted, tool shape/15J)

Photograph of impact damage coupon #U34

C-Scan of impact damage coupon #U34
(17ply unpainted, tool shape/17.5J)

Photograph of impact damage coupon #U35

C-Scan of impact damage coupon #U35
(17ply unpainted, tool shape/20J)

252
Photograph of impact damage coupon #14
C-Scan of impact damage coupon #14
(33ply + mesh painted, tool shape/20J)

Not available

Photograph of impact damage coupon #15

C-Scan of impact damage coupon #15
(33ply + mesh painted, tool shape/30J)

Not available

Photograph of impact damage coupon #16

C-Scan of impact damage coupon #16
(33ply + mesh painted, tool shape/40J)

Photograph of impact damage coupon #17

C-Scan of impact damage coupon #17
(33ply + mesh painted, tool shape/50J)

253
Photograph of impact damage coupon #U56
C-Scan of impact damage coupon #U56
(33ply unpainted, tool shape/20J)

Photograph of impact damage coupon #U57

C-Scan of impact damage coupon #U57
(33ply unpainted, tool shape/30J)

Photograph of impact damage coupon #U58

C-Scan of impact damage coupon #U58
(33ply unpainted, tool shape/40J)

254
Photograph of impact damage coupon #U59
C-Scan of impact damage coupon #U59
(33ply unpainted, tool shape/50J)

References
Boulic. (Cranfield University), (2007), Quantitative Morphology of Impact Damage to
Composite Aircraft Structures (unpublished report), Polytech’Orléans, France.

255
Appendix E
Hit/Miss results data from Trial 1
Participant Number:

Total Detections
1

7
for Flaw Size (a)
Flaw Size (a)
1 0 0 0 0 0 0 0 0
2 0 0 0 0 0 0 0 0
3 0 0 0 0 0 0 0 0
4 0 0 0 0 0 0 0 0
5 0 0 1 1 1 1 1 5
6 1 1 1 1 1 1 1 7
7 1 0 1 1 1 1 1 6
8 1 1 1 1 1 1 1 7
9 1 0 1 1 1 1 1 6
10 1 1 1 1 1 1 1 7
11 1 0 1 1 1 1 1 6
12 1 1 1 1 1 1 1 7
13 1 0 1 1 1 1 1 6
14 1 1 1 1 1 1 1 7
15 1 1 1 1 1 1 1 7
16 1 0 1 1 1 1 1 6
17 1 1 1 1 1 1 1 7
18 1 1 1 1 1 1 1 7
19 1 1 1 1 1 1 1 7
20 1 0 1 1 1 1 1 6
21 1 1 1 1 1 1 1 7
22 1 1 1 1 1 1 1 7
23 1 1 1 1 1 1 1 7
24 1 0 1 1 1 1 1 6
25 1 0 1 1 1 1 1 6
26 1 1 1 1 1 1 1 7
27 1 1 1 1 1 1 1 7
28 1 1 1 1 1 1 1 7
29 1 1 1 1 1 1 1 7
30 1 0 1 1 1 1 1 6
Total No of
Flaws
25

16

26

26

26

26

26

Detected (per
Inspector)

256
Hit/Miss results data from Trial 2
Participant Number: Total
Detections
for Flaw

10
11
12
13
14
15
Flaw Size (a)

1
2
3
4
5
6
7
8
9
Size (a)
0.1mm d 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 2
0.2mm d 0 0 1 1 1 1 0 0 1 0 1 1 1 1 0 9
0.3mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.4mm d 1 0 1 1 1 1 0 1 1 1 1 1 0 1 0 11
0.5mm d 1 1 1 1 1 1 0 1 1 1 1 1 1 1 1 14
0.6mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
Ø4.5mm

0.7mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.8mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.9mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.0mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.1mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.2mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.3mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.4mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.5mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.1mm d 0 0 0 1 0 1 0 0 0 1 0 1 1 0 0 5
0.2mm d 1 1 1 1 1 1 1 1 1 1 0 1 1 1 1 14
0.3mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.4mm d 1 1 1 1 1 1 1 0 1 1 1 1 1 1 1 14
0.5mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
Ø6.5mm

0.6mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.7mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.8mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.9mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.0mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.1mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.2mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.3mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.1mm d 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.2mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.3mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.4mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
Ø12mm

0.5mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.6mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.7mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.8mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.9mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.0mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.1mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
Total No of Flaws
35
34
36
37
37
37
34
34
36
36
35
37
36
36
34

Detected (per
Inspector)
No of False Calls 0 0 0 0 0 3 2 0 0 0 0 1 0 2 0

257
Hit/Miss results data from Trial 3
Participant Number: Total
Detections
for Flaw

10
11
12
13
14
15
Flaw Size (a)

1
2
3
4
5
6
7
8
9
Size (a)
0.1mm d 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0.2mm d 1 1 1 1 1 1 1 1 1 1 1 1 0 1 0 13
0.3mm d 1 1 1 1 1 1 1 1 1 1 1 1 0 1 0 13
0.4mm d 1 1 1 0 1 1 1 1 1 1 1 1 0 1 0 12
0.5mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.6mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
Ø4.5mm

0.7mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.8mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.9mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.0mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.1mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.2mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.3mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.4mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.5mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.1mm d 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1
0.2mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.3mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.4mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.5mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
Ø6.5mm

0.6mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.7mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.8mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.9mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.0mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.1mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.2mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.3mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.1mm d 1 0 0 1 0 0 0 1 0 0 0 0 0 0 0 3
0.2mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.3mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.4mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
Ø12mm

0.5mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.6mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.7mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.8mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
0.9mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.0mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
1.1mm d 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 15
Total No of Flaws
37
36
36
36
36
36
37
37
36
36
36
36
33
36
33

Detected (per
Inspector)

No of False Calls 0 0 0 1 0 0 0 1 0 0 0 0 2 0 0

258
Hit/Miss results data from Trial 4
Participant number:
Flaw

10

11

12

13

14
Diameter Total

9
(mm) Detections
03 BE 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1
03 HE 0 0 0 0 0 1 1 0 0 0 0 0 0 0 2
04 BE 1 1 1 1 1 1 1 0 1 1 1 1 1 1 13
04 HE 1 1 1 1 0 0 1 1 0 1 1 0 0 0 8
05 BE (a) 1 1 1 1 0 1 1 0 1 1 1 1 0 1 11
05 BE(b) 0 0 0 1 1 1 0 1 0 1 0 1 0 1 7
05 HE(a) 0 0 1 0 0 1 1 0 1 1 1 1 1 1 9
05 HE(b) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 14
06 BE 1 1 1 0 1 1 1 1 1 1 0 1 0 1 11
06 HE 1 1 0 1 1 0 1 0 1 1 1 1 1 1 11
07 BE 1 1 1 0 0 1 1 0 1 1 1 1 0 1 10
07 HE 1 1 1 1 1 1 1 0 1 1 1 1 1 0 12
08 BE 1 1 1 1 1 1 1 0 1 1 1 0 1 1 12
08 HE 0 0 1 0 0 1 0 1 0 0 0 1 1 0 5
09 BE 1 1 1 1 1 1 1 0 0 1 1 1 1 1 12
09 HE 1 1 1 1 1 1 1 0 0 1 1 1 1 1 12
10 BE (a) 1 0 1 1 0 1 1 1 1 1 1 1 1 1 12
10 BE (b) 1 1 1 1 0 1 0 1 1 1 1 1 1 1 12
10 HE (a) 1 1 1 0 1 1 1 1 1 1 1 1 1 1 13
10 HE (b) 1 1 1 1 1 1 1 0 0 1 1 1 1 1 12
12 BE 1 1 1 1 1 1 1 1 1 1 1 0 1 1 13
12 HE 1 1 0 1 1 1 1 1 1 1 1 0 1 1 12
14 BE 1 1 1 1 0 1 1 0 1 1 1 1 1 1 12
14 HE 1 1 1 1 0 1 0 1 1 1 1 0 1 1 11
16 BE (a) 1 1 1 1 1 1 1 0 1 1 1 0 1 1 12
16 BE (b) 1 0 0 0 1 0 0 1 0 0 0 1 1 1 6
16 HE (a) 1 1 1 1 1 1 1 1 1 1 1 1 1 1 14
16 HE (b) 1 1 0 1 1 1 0 0 0 1 1 0 0 1 8
18 BE 0 0 1 0 0 1 1 0 0 1 1 0 0 0 5
18 HE 1 0 1 1 1 1 1 0 1 1 1 0 1 1 11
20 BE 1 0 1 0 0 1 0 0 1 0 1 1 1 0 7
20 HE 0 1 1 1 1 1 1 1 0 1 1 1 0 1 11
22 BE 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1
22 HE 1 1 1 1 0 1 1 1 0 1 1 1 1 1 12
24 BE 0 1 0 1 1 1 0 1 1 0 1 0 0 0 7
24 HE 0 1 1 1 1 1 1 0 1 1 1 1 0 1 11
26 BE 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1
26 HE 0 0 1 1 0 1 0 0 1 1 1 1 0 1 8
28 BE 0 0 1 0 0 1 0 0 1 0 0 0 0 1 4
28 HE 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1
30 BE (a) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
30 BE (b) 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1
30 HE (a) 0 0 0 1 0 1 1 0 1 0 0 0 1 1 6
30 HE (b) 1 0 1 1 1 0 0 1 1 1 0 1 1 0 9
Total
27

25

31

29

22

36

27

17

26

31

30

25

25

31

Detections
1
False Calls 0 1 1 1 0 0 3 0 1 1 1 0 1
8
BE = Blended edge flaw HE = Hard edge flaw (a) & (b) used to denote flaws of same size

259
Appendix F
POD curve & RIM calculations
The data from the series 1 visual inspection trials were plotted using the following
log-odds POD function, as given by Berens & Hovey 1982 in Bullock et al. 1994.

exp (α + βln(ai)
Pi =
1 + exp (α + βln(ai)

Where Pi is the probability of detection of flaw i, ai is the flaw size and α & β are
constant parameters.

In order to determine values for α & β, the range interval method (RIM) was used (see
Bullock et al. 1994). Regression plots were produced by calculating the following
values for each flaw size interval:

pi
Yi = ln( ) , Xi = ln(ai)
1 – pi

Where pi is the proportion of flaws detected, ai is the flaw size and i is the interval
size.

The values of α & β were derived by fitting the regression plot with a linear
regression line constructed using the following equation (as per Bullock et al. 1994)

Y = α + βX

Refeferences

Berens, A. P. and Hovey, P. W. (1982), "Characterization of NDE reliability", Review of

progress in quantitative nondestructive evaluation. Volume 1 - Proceedings of the
Eighth U.S. Air Force/Defense Advanced Research Projects Agency Symposium on
Quantitative Nondestructive Evaluation, Boulder, CO; United States; 2-7 Aug. 1981,
New York, Plenum Press,.

Bullock, M., Forsyth, D. and Fahr, A. (1994), Statistical Functions and Computational
Procedures for the POD Analysis of Hit/Miss NDI Data, LTR-ST-1964, Institute for
Aerospace Research, National Research Council Canada, Ottawa, Ontario.

260
POD curve calculations for Trial 1
Regression plot values:

Y = ln(P/1-P), X = ln(a) RIM values

Flaw Width Interval No. of No.

ln(a) P ln(P/ 1-P)
Interval Value (a) flaws Detected
1mm – 5mm 2.5 0.92 35 5 0.14 -1.79
5mm -10mm 7.5 2.01 35 33 0.94 2.80
10mm – 15mm 12.5 2.53 35 33 0.94 2.80
15mm – 20mm 17.5 2.86 35 33 0.94 2.80
20mm – 25mm 22.5 3.11 35 33 0.94 2.80
25mm – 30mm 27.5 3.31 35 34 0.97 3.53

Regression plot:

ln(a) vs ln(P/ 1-P)

4
y = 1.973x - 2.6913

0
-2 -1 0 1 2 3 4
-1

-2

POD Function values

Values for:
P= EXP (A + Bln(a)
1 + EXP (A + Bln(a))

Interval Size (a) P

2.5 0.29 Alpha (A) (α) Beta (B) (β)
7.5 0.78 -2.6913 1.973
12.5 0.91
17.5 0.95
22.5 0.97
27.5 0.98
30 0.98

261
POD curve calculations for Trial 2
Trial 2 – 4.5 mm wide flaws
Regression plot values:

Y = ln(P/1-P), X = ln(a) RIM values

Flaw Depth Interval No. of No.

ln(a) P ln(P/ 1-P)
Interval Value (a) flaws Detected
0mm - 0.2mm 0.1 -2.30 30 11 0.37 -0.55
0.2mm - 0.4mm 0.3 -1.20 30 26 0.87 1.87
0.4mm - 0.6mm 0.5 -0.69 30 29 0.97 3.37
0.6mm - 0.8mm 0.7 -0.36 30 30 1.00 N/A
0.8mm - 1.0mm 0.9 -0.11 30 30 1.00 N/A
1.0mm - 1.2mm 1.1 0.10 30 30 1.00 N/A
1.2mm - 1.4mm 1.3 0.26 30 30 1.00 N/A
1.4mm - 1.6mm 1.5 0.41 15 15 1.00 N/A

Regression plot:

ln(a) vs ln(P/ 1-P)

4.00

y = 2.3951x + 4.9171
3.00

2.00

1.00

0.00
-6.00 -5.00 -4.00 -3.00 -2.00 -1.00 0.00
-1.00

-2.00

262
POD Function values

Values for:
P= EXP (A + Bln(a)
1 + EXP (A + Bln(a))

Interval Size (a) P

0.1 0.35 Alpha (A) (α) Beta (B) (β)
0.3 0.88 4.9171 2.3951
0.5 0.96
0.7 0.98
0.9 0.99
1.1 0.99
1.3 1.00
1.5 1.00

Trial 2 – 6.5 mm wide flaws

Regression plot values:

Y = ln(P/1-P), X = ln(a) RIM values

Flaw Depth Interval No. of No.

ln(a) P ln(P/ 1-P)
Interval Value (a) flaws Detected

0mm - 0.2mm 0.1 -2.30 30 19 0.63 0.55

0.2mm - 0.4mm 0.3 -1.20 30 29 0.97 3.37
0.4mm - 0.6mm 0.5 -0.69 30 30 1.00 N/A
0.6mm - 0.8mm 0.7 -0.36 30 30 1.00 N/A
0.8mm - 1.0mm 0.9 -0.11 30 30 1.00 N/A
1.0mm - 1.2mm 1.1 0.10 30 30 1.00 N/A
1.2mm - 1.4mm 1.3 0.26 15 15 1.00 N/A

263
Regression plot:

ln(a) vs ln(P/ 1-P)

4.00

y = 2.5676x + 6.4586
3.00

2.00

1.00

0.00
-6.00 -5.00 -4.00 -3.00 -2.00 -1.00 0.00
-1.00

-2.00

POD Function values:

Values for:
P= EXP (A + Bln(a)
1 + EXP (A + Bln(a))

Interval Size (a) P

0.1 0.63 Alpha (A) (α) Beta (B) (β)
0.3 0.97 6.4586 2.5676
0.5 0.99
0.7 1.00
0.9 1.00
1.1 1.00
1.3 1.00

264
POD curve calculations for Trial 3
Trial 3 – 4.5 mm wide flaws
Regression plot values:

Y = ln(P/1-P), X = ln(a) RIM values

Flaw Depth Interval No. of No.

ln(a) P ln(P/ 1-P)
Interval Value (a) flaws Detected

0mm - 0.2mm 0.1 -2.30 30 13 0.43 -0.27

0.2mm - 0.4mm 0.3 -1.20 30 25 0.83 1.61
0.4mm - 0.6mm 0.5 -0.69 30 30 1.00 N/A
0.6mm - 0.8mm 0.7 -0.36 30 30 1.00 N/A
0.8mm - 1.0mm 0.9 -0.11 30 30 1.00 N/A
1.0mm - 1.2mm 1.1 0.10 30 30 1.00 N/A
1.2mm - 1.4mm 1.3 0.26 30 30 1.00 N/A
1.4mm - 1.6mm 1.5 0.41 15 15 1.00 N/A

Regression plot:

ln(a) vs ln(P/ 1-P)

4.00

3.00

2.00
y = 1.7092x + 3.6672

1.00

0.00
-6.00 -5.00 -4.00 -3.00 -2.00 -1.00 0.00
-1.00

-2.00

265
POD Function values

Values for:
P= EXP (A + Bln(a)
1 + EXP (A + Bln(a))

Interval Size (a) P

0.1 0.43 Alpha (A) (α) Beta (B) (β)
0.3 0.83 3.6672 1.7092
0.5 0.92
0.7 0.96
0.9 0.97
1.1 0.98
1.3 0.98
1.5 0.99

POD curve calculations for Trial 4

Trial 4 – blended edge flaws
Regression plot values:

Y = ln(P/1-P), X = ln(a) RIM values

Flaw Depth Interval No. of No.

ln(a) P ln(P/ 1-P)
Interval Value (a) flaws Detected

2mm - 4mm 3 1.10 28 14 0.50 0

4mm - 6mm 5 1.61 42 29 0.69 0.80
6mm - 8mm 7 1.95 28 22 0.79 1.30
8mm - 10mm 9 2.20 42 36 0.86 1.79
10mm - 12mm 11 2.40 14 13 0.93 2.56

12mm - 14mm 13 2.56 14 12 0.86 1.79

14mm - 16mm 15 2.71 28 18 0.64 0.59
16mm - 18mm 17 2.83 14 5 0.36 -0.59
18mm - 20mm 19 2.94 14 7 0.50 0.00
20mm - 22mm 21 3.04 14 1 0.07 -2.56
22mm - 24mm 23 3.14 14 7 0.50 0.00
24mm - 26mm 25 3.22 14 1 0.07 -2.56
26mm - 28mm 27 3.30 14 4 0.29 -0.92
28mm - 30mm 29 3.37 28 1 0.04 -3.30

266
Regression plot:

ln(a) vs ln(P/ 1-P)

y = 1.8626x - 2.1539
2

0
-2.00 -1.00 0.00 1.00 2.00 3.00 4.00
-1

y = -4.9032x + 13.932
-2

-3

POD Function values:

Values for:
P= EXP (A + Bln(a)
1 + EXP (A + Bln(a))

Interval Size (a) P

3 0.47 Alpha (A) (α) Beta (B) (β)
5 0.70 -2.1539 1.8626
7 0.81
9 0.87
11 0.91

13 0.80 Alpha (A) (α) Beta (B) (β)

15 0.66 13.932 -4.9032
17 0.51
19 0.38
21 0.27
23 0.19
25 0.14
27 0.10
29 0.07

267
Trial 4 – hard edge flaws
Regression plot values:

Y = ln(P/1-P), X = ln(a) RIM values

Flaw Depth Interval No. of No.

ln(a) P ln(P/ 1-P)
Interval Value (a) flaws Detected

2mm - 4mm 3 1.10 28 10 0.36 -0.59

4mm - 6mm 5 1.61 42 34 0.81 1.45
6mm - 8mm 7 1.95 28 17 0.61 0.44
8mm - 10mm 9 2.20 42 37 0.88 2.00

10mm - 12mm 11 2.40 14 12 0.86 1.79

12mm - 14mm 13 2.56 14 11 0.79 1.30
14mm - 16mm 15 2.71 28 22 0.79 1.30
16mm - 18mm 17 2.83 14 11 0.79 1.30
18mm - 20mm 19 2.94 14 11 0.79 1.30
20mm - 22mm 21 3.04 14 12 0.86 1.79
22mm - 24mm 23 3.14 14 11 0.79 1.30
24mm - 26mm 25 3.22 14 8 0.57 0.29
26mm - 28mm 27 3.30 14 1 0.07 -2.56
28mm - 30mm 29 3.37 28 15 0.54 0.14

Regression plot:

ln(a) vs ln(P/ 1-P)

3
y = -2.4724x + 8.0908
2

0
-2 -1 0 1 2 3 4
-1

-2
y = 1.8946x - 2.4211
-3

268
POD Function values:

Values for:
P= EXP (A + Bln(a)
1 + EXP (A + Bln(a))

Interval Size (a) P

3 0.42 Alpha (A) (α) Beta (B) (β)
5 0.65 -2.4211 1.8946
7 0.78
9 0.85

11 0.90 Alpha (A) (α) Beta (B) (β)

13 0.85 8.0908 -2.4724
15 0.80
17 0.75
19 0.69
21 0.64
23 0.58
25 0.53
27 0.49
29 0.44

269
Appendix G
Hit/Miss data for series 2 visual inspection trials
Hit/Miss data for trials with virtual grey specimens
Hit/Miss data from trials with virtual grey specimens

270
 271
Hit/Miss data for trials with gloss grey specimens Hit/Miss data from trials with gloss grey facsimile specimens
 272
Hit/Miss data for trials with matt grey specimens Hit/Miss data from trials with matt grey facsimile specimens
 273
Hit/Miss data for trials with gloss white specimens Hit/Miss data from trials with gloss white facsimile specimens
 274
Hit/Miss data for trials with matt white specimens Hit/Miss data from trials with matt white facsimile specimens
 275
Hit/Miss data for trials with gloss blue specimens Hit/Miss data from trials with gloss blue facsimile specimens
 276
Hit/Miss data for trials with matt blue specimens Hit/Miss data from trials with matt blue facsimile specimens
Appendix H
Detection plots of results from series 2 visual inspection trials

Original scatter plot of detection rates obtained with virtual specimens

Original scatter plot of detection rates obtained with gloss grey specimens

277
Original scatter plot of detection rates obtained with matt grey specimens

Original scatter plot of detection rates obtained with gloss white specimens

278
Original scatter plot of detection rates obtained with matt white specimens

Original scatter plot of detection rates obtained with gloss blue specimens

279
Original scatter plot of detection rates obtained with matt blue specimens

Original wafer plot of detection rates obtained with virtual specimens

280
Original wafer plot of detection rates obtained with gloss grey specimens

Original wafer plot of detection rates obtained with matt grey specimens

281
Original wafer plot of detection rates obtained with gloss white specimens

Original wafer plot of detection rates obtained with matt white specimens

282
Original wafer plot of detection rates obtained with gloss blue specimens

Original wafer plot of detection rates obtained with matt blue specimens

283