See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/349467250

AI Explainability. A Bridge Between Machine Vision and Natural Language

Processing

Chapter in Lecture Notes in Computer Science · February 2021

DOI: 10.1007/978-3-030-68796-0_19

CITATIONS READS

13 845

1 author:

Mourad Oussalah
University of Oulu
274 PUBLICATIONS 3,227 CITATIONS

SEE PROFILE

All content following this page was uploaded by Mourad Oussalah on 15 November 2022.

The user has requested enhancement of the downloaded file.

 AI Explainability. A Bridge Between Machine
Vision and Natural Language Processing

Mourad Oussalah(B)

Faculty of Information Technology, University of Oulu, CMVS, 90014 Oulu, Finland

[email protected]

Abstract. This paper attempts to present an appraisal review of explainable Arti-

ficial Intelligence research, with a focus on building a bridge between image pro-
cessing community and natural language processing (NLP) community. The paper
highlights the implicit link between the two disciplines as exemplified from the
emergence of automatic image annotation systems, visual question-answer sys-
tems. Text-To-Image generation and multimedia analytics. Next, the paper iden-
tified a set of natural language processing fields where the visual-based explain-
ability can boost the local NLP task. This includes, sentiment analysis, automatic
text summarization, system argumentation, topical analysis, among others, which
are highly expected to fuel prominent future research in the field.

Keywords: Explainable AI · Machine vision · Natural language processing

1 Introduction
Aided by the advances in computer system computational performances and learning
system theory, the success of machine learning methods in the last decade has been
phenomenal in various fields, especially, computer vision and natural language process-
ing, which enhanced the prediction and automated decision-making capabilities. This
has taken machine intelligence and artificial intelligence (AI) to a new frontier that wit-
nessed the emergence of new industry standard (e.g., industry 4.0) and human-computer
interaction modes where a machine guides medical diagnosis systems, creates recom-
mender systems, makes investment decisions and instructs driverless vehicles. On the
other hand, the state-of-the-art systems in many AI applications use ensembles of deep
neural networks that are even more difficult to interpret, even for skilled programmer
users. This negatively impacts trust. For instance, during the PwC’s 2017 Global CEO
Survey [1], although it is acknowledged the substantial increase of AI market to more
than $15 trillion, 67% of the business leaders believe that this will impact negatively
stakeholder trust levels in their industry in the next five years. This fosters the emer-
gence of explainable AI research that seeks to ensure transparency and interpretability
of machine learning and AI based algorithms. Indeed, many applications have seen a
huge increase in demand for transparency from the various stakeholders involved at
various levels of product pipeline. For instance, in precision-medicine, explanation is
required to support system diagnosis outcome and clinical investigation; in finance and

© Springer Nature Switzerland AG 2021

A. Del Bimbo et al. (Eds.): ICPR 2020 Workshops, LNCS 12663, pp. 257–273, 2021.
https://doi.org/10.1007/978-3-030-68796-0_19
258 M. Oussalah

management, explanation is needed to evaluate various investment scenarios with qual-

itative/quantitative risk evaluations; in autonomous systems, explanation enhances fault
inspection and recovery based strategies. In general stakeholders are reticent to adopt
techniques that are not directly trustworthy, tractable and interpretable [2], especially
given the increasing scope of ethical AI [3].
Beyond academia, since 2017, the European Union’s General Data Protection Reg-
ulation (GDPR) introduced the “right to explanation” which states that a user can ask
for an explanation of an algorithmic decision that was made about them [4].
Strictly speaking, the need for AI explainability was recognized well earlier, and
was an inherent component of many of the first AI diagnostic systems where “IF-Then”
rules and inference engine were widely employed to explain the actions of the underlined
expert system for instance. This was implemented in early MYCIN systems [81] that
formed the basis of many subsequent medical systems; although the exact scope and
nature of these rules can be debatable. In the literature, the concept of explainability is
related to transparency, interpretability, trust, fairness and accountability, among others
[5]. Interpretability, often used as a synonym of explainability as well, is defined by
Doshi and Kim [6] as “the ability to explain or to present in understandable terms to a
human”.
According to Samek et al. [7], the need of explainable systems is rooted in four points:
(a) Verification of the system: Understand the rules governing the decision process in
order to detect possible biases; (b) Improvement of the system: Understand the model
and the dataset to compare different models and to avoid failures; (c) Learning from
the system: “Extract the distilled knowledge from the AI system”; (d) Compliance with
legislation (particularly with the “right to explanation” set by European Union): To find
answers to legal questions and to inform people affected by AI decisions.
Lewis [8] states that “to explain an event is to provide some information about its
causal history. In an act of explaining, someone who is in possession of some information
about the causal history of some event – explanatory information – tries to convey it
to someone else”. Halpern and Pearl [9] define a good explanation as a response to a
Why question, that “(a) provides information that goes beyond the knowledge of the
individual asking the question and (b) be such that the individual can see that it would,
if true, be (or be very likely to be) a cause of”.
Miller [10] extracts four characteristics of explanations: “explanations are con-
trastive” (why this and not that), “explanations are selected in a biased manner (not
everything shall be explained)”, “probabilities don’t matter” and finally “explanations
are social”.
Traditionally, transparency has always been at odds with performance where an
increase in transparency is often translated into a decrease in system performance because
of large number of parameters that require tuning [11], see Fig. 1 for exemplification.
Therefore, a trade-off between the level of transparency and performance required.
Encompassing the broad scope of the explainability and its multi-disciplinary nature,
this paper attempts to reconcile explainable AI research on two complementary fields:
Machine Vision System (MVS) and Natural Language Processing (NLP), trying to sur-
vey the explainable AI in each field and seek complementary aspects in a way to boost
fruitful XAI research in the two fields.
 AI Explainability 259

2 Background
We will adopt in this paper Gunning definition of Explainable Artificial Intelligence
(XAI) [12]: “XAI will create a suite of machine learning techniques that enables human
users to understand, appropriately trust, and effectively manage the emerging generation
of artificially intelligent partners”, see Fig. 1.

Fig. 1. XAI concept as described in Gunning’s [12] and Performance-interpretability trade-off

This definition brings together two concepts; namely, understanding and trust that
need to be addressed. Such concepts are ultimately linked to several other aspects
that overlap with cognitive operations of understanding or comprehension tasks. This
includes for instance causality, transferability, informativeness, fairness and confidence
[13].
Regardless of the type of applications involved or the system inputs, the explanation
differs according to the underlying chosen criterion. Especially, explanation methods
and techniques for ML interpretability can be classified according to different criteria.

• Pre-Model vs. In-Model vs. Post-Model

Interpretability methods can be classified depending whether this is applied before

(pre-model), during (in-model) or after (post-model) the machine-learning model [14].
Pre-model interpretability techniques tackle the data itself regardless of the employed
ML model, focusing on the structure of the inputs and associated features with their visu-
alization. Intuitive features and sparsity are some properties that help to achieve data
interpretability. This includes techniques related to descriptive statistics to data visualiza-
tion methods including Principal Component Analysis (PCA) [15], t-SNE (t-Distributed
Stochastic Neighbor Embedding) [16], and clustering methods, such as MMD-critic [17]
(Maximum Mean Discrepancy) and k-means [18]. Hence, data visualization is critical
for pre-model interpretability.
260 M. Oussalah

In-model interpretability concerns ML models that have inherent interpretability in

it, either with or without constraints, being intrinsically interpretable. Post-model inter-
pretability refers to improving interpretability after building a model or model training
(post hoc). This answers the question: what else can the model tell us?

• Model-Specific vs. Model-Agnostic

Model-specific interpretation restricts the analysis to specific model classes. For

example, the interpretation of weights in a linear model is a model-specific interpreta-
tion [19]. In contrast, model-agnostic methods apply to any ML model after training
phase (post-hoc), so without having access to model inner workings (i.e., weighting).
By default, all post-hoc methods are model-agnostic since they were applied after the
training.
In essence, two approaches can be distinguished to explain ML model prediction
through either a global method that treats the group of predictions of interest as if it was
the whole dataset or by applying local methods on each individual prediction followed
by aggregating and joining these individual explanations afterwards [13, 20].

• Local versus global explanation

According to the scoop of interpretability, one distinguishes global interpretability,

which concerns comprehending the entire model behavior and local interpretability,
which rather focuses on a single prediction, or a single feature behavior. For instance,
Yang et al’s. [21] GIRP (global mode interpretation via recursive partitioning) builds a
global interpretation tree. Nguyen et al. [22] advocate an activation-maximization based
approach for global explanation. One of the most popular local interpretability model is
LIME model [23], which enables approximating ML model in the neighborhood of any
prediction of interest.
Other techniques for local explainaibility models include decomposition models
(omitting some features of the original dataset and attempt to combine the effects of
various features), Shapely explanations [24], sensitivity maps, saliency maps [26].

• Type of explanation

This includes Feature summary (either through visualization or textual input), Model
internals (model specific), Data point (output data points that make the model inter-
pretable), Surrogate intrinsically interpretable model—through approximation of ML
model either locally or globally with an intrinsically interpretable model.

• Simulatability

This refers to comprehending how the model makes decisions, grounded on a holistic
view of the data features and individual components (e.g., weights, parameters) in order
to simulate the overall system.
 AI Explainability 261

• Visualization and interaction

• Popular visualization techniques applied in ML interpretability include partial-
dependence plot [27], surrogate models [28, 29], individual conditional expectation
[30]. Depending on the stage where the visualization techniques is conducted, one
can also distinguish pre-model, in-model or post-hoc based visualization. Similarly,
visualization can be performed for local or global like interpretability purpose (Table
1).

• Approximation versus sensitivity

Using the universal approximator property of neural network system, several inter-
pretable approximation models have been proposed to gain insights into the functioning
of the black-box of the ML model. For instance, rule-based approximation [31] approx-
imates the decision-making process in ML model by utilizing the input and the output of
the ML only. Decompositional approaches look at extracting rules at the level of individ-
ual units within ML model [32]. This includes Orthogonal search-based rule extraction
proposed in [33] for medicine application. Another approximation method cited in the
literature is model distillation [34], which acts as a model compression algorithm from
deep network to shallow networks. DarkSight [35] combines ideas from distillation
model and visualization to explain the black-box functioning.
On the other hand, sensitivity analysis [36] focuses on how black box model is
influenced by its input weight perturbations. Feature importance as in Fisher et al. [37]’s
Model Class Reliance or SFIMP [38] for permutation based shapely feature importance
are other sensitivity like analysis for explanation tackling.

3 Link Between Image and Text in Explainability

Intuitively, the link between image and text is either explicit or implicit from several
standpoints:

Purpose and Outcome Expectation

Both image processing and NLP based XAI system do share the same purpose of using
ML model for classification purpose, and therefore, seeking an explanation of the ML
outcome using the XAI model. They may seek a global / local, model-specific or agnostic,
pre-training, in-training or post-training explanation. Although, the classification task
might be different for NLP and image processing cases.

Universality of Many XAI Tools

Tools like LIME framework that enables observing the explainer function on the cor-
rectly predicted instance can be applied regardless the context of application domain
(e.g., image or text based exploration). Similar reasoning applies to many visualiza-
tion toolkits, e.g., heat-map that visualizes the extent to which each element contributes
to the prediction result, saliency map, feature importance map, among others, that are
independent of the application context.

Structure of Multimedia and Social Media Posts

With the advances in Web 2.0 technology that enabled the users to post various types of
files (text, images, multimedia) and the memory efficiency for handling large scale mul-
timedia files, the need for building a capacity to handle equally image and textual inputs
 262
Table 1. Review of main XAI techniques

Techniques Global Model Pre-model, Approximation Visualization/Interaction Simulatability References

(G)/Local Specific In-mode, (Ap)/Sensitivity
(L) (Sp)/model Post-model (S)
Agnostic (A) (Po)
M. Oussalah

Decision trees G Sp All x [39, 40]

LIME L A Po x Yes Yes [23, 41]
Shapely explanations L A Po x Yes Yes [24]
Rule extraction G/L A Po x Yes Yes [43, 44]
Decomposition L A Po Ap Yes Yes [32]
Activation-maximization G/L A Po x x x [22]
Surrogate models G/L A Po S Yes x [28, 29]
Individual conditional L A Po x Yes x [30]
expectation
Model distillation G A Po x Yes Yes [34]
Feature importance G/L A Po Ap / S Yes x [45]
Saliency map L A Po S Yes x [26]
Sensitivity analysis G/L A Po S Yes x [36]
Counterfactuals L A Po x Yes x [38]
explanations
Tree View G/L A Po x Yes x [47]
Rule set G Sp Po x Yes x [48]
(continued)
 Table 1. (continued)

Techniques Global Model Pre-model, Approximation Visualization/Interaction Simulatability References

(G)/Local Specific In-mode, (Ap)/Sensitivity
(L) (Sp)/model Post-model (S)
Agnostic (A) (Po)
DecText G/L A Po x x x [49]
DeepLift G A Po x Yes x [50]
Layer-wise relevance G/L A Po Ap x x [46]
propagation
Fuzzy inference system G A Po Ap Yes Yes [51]
AI Explainability
263
264 M. Oussalah

on the same setting is growing. This motivates the development of unified frameworks
in XAI to handle both types of inputs as well.

Development of Automatic Annotation Services

Automatic image annotation is the process of automatically creating textual based
description for the different regions of the image highlighting the content of the images.
This is especially important to identify sensitive content on online media platforms. With
the advances in deep learning technology and large-scale image database, several tools
were made available to scientific community for this task. This includes Google Cloud
Vision [52], GoogLeNet [53], a deep learning model trained on the ILSVRC dataset.

Development of Visual Question Answering Services

Visual Question Answering (VQA) is the task of addressing open-ended questions about
images; namely, given an image and a natural language question about that image, the task
is to provide an accurate natural language answer, see Fig. 2 for an example. Typically,
VQA requires visual and linguistic comprehension, language grounding capabilities as
well as a common-sense knowledge. A variety of methods have been developed [54,
55]. In the latter, the vision component of a typical VQA system extracts visual features
using a deep convolutional neural network (CNN), then linguistic components encode
the question into a semantic vector using a recurrent neural network (RNN). An answer
is then generated conditioned on the visual features and the question vector.

Fig. 2. Example of Visual-Question answer systems

Development of Text to Image Generation

Similarly to the previous visual question answering, the problem of explaining textual
content through visual representation is also important for education and learning pur-
poses. This has also been investigated by many computer vision scientists. Indeed, deep
generative models [22] have been proposed to address the task of generating appropriate
images from natural description by inferring a semantic layout, which is then converted
 AI Explainability 265

into image using image generator. Methods based on conditional Generative Adversar-
ial Network (GAN) have been employed in several text-to-image synthesis tasks and
competitions [56, 57] and tested on large scale dataset such as birds [58], flowers [59],
MS-COCO [60]. In this regard, the task of image generation is viewed as a problem
of translating semantic labels into pixels. Nevertheless, the complexity of the reason-
ing cannot be ignored. Especially, learning a direct mapping from text to image is not
straightforward and layout generator requires several constraints to enhance its practi-
cality due to the vast amount of possibilities of potential image candidates that fit a given
textual utterance, see, for instance, the example in Figs. 3, 4.

Fig. 3. Example of Text-to-Image based explanation

Fig. 4. Architecture for image generator conditioned on the text description and semantic layout
generator in [78].

4 Potential Benefits to NLP Community

4.1 Word-Sense Disambiguation
Word-sense disambiguation aims to assign an appropriate sense to the occurrence of a
word in a given context. For instance, the meaning of word “chair” in the sentence “I
have been awarded a chair in Computer Science” is different from that in the sentence
“I bought a chair in the city market today”, where the sense of the target word “chair” is
chosen among the set of senses listed in a given dictionary. Typically, standard Lesk’s
algorithm [61] looks into the number of overlapping words of each sense of the target
word with the underlined context (sentence), so that the sense that yields the highest
number of overlap is used to disambiguate the target word. Other variants of Lesk’s
266 M. Oussalah

algorithm as well as supervised and/or semi-supervised algorithms have been proposed

for word-sense disambiguation tasks [62, 63, 79].
Making use of visual description raised, for instance, by text-to-image mapping can
provide insights into word-sense disambiguation task. This assumes that an overall visual
representation is generated for the whole sentence for each sense of the target word, and
appropriate metrics are constructed to quantify the relevance and commonsense of each
global visual representation.
Visualization techniques issues from XAI can also be accommodated for the word-
sense disambiguation feature. For instance, various senses of target words can play the
role of features and utilize LIME-like-approach to visualize the contributions of the var-
ious senses, and thereby, disambiguate accordingly. Similarly, the emergence of graph
based approaches for word sense disambiguation where, in the same spirit as Navigli
and Lapata [63], the senses are mapped to a graph representation where the graph is
built using various connectivity algorithms such as PageRank, hyperlink induced topic
search, key player problem, various senses can be ranked accordingly, and, thereby,
handle disambiguation task.

4.2 Text Argumentation Theory

With the emergence of Dung’s Argumentation Framework [64], a central approach for
performing reasoning within argumentation in artificial intelligence and natural language
processing become tenable. This opens up opportunities for legal text analysis, medical
science and opinion mining. In this course, arguments are viewed as abstract entities,
so that the use of argumentation semantics, i.e., criteria for finding acceptable sets of
arguments, suffices to reason in an argumentative way for a variety of application sce-
narios. Arguments are supposed to support, contradict, and explain statements, and they
are used to support decision-making. What may constitute an argument is very much
context dependent. In natural language processing, this can be short utterance that fits
a given ontology or might be extracted using text summarization like technique from a
large or a multi-document source file. The possibility to represent both positive and nega-
tive views using the employed argumentation framework provides opportunity to sustain
debate and boost interactions. Recently, abstract argumentation has been suggested to
explain predictions of neural network system and diaelectrically explainable prediction
[65]. This ultimately builds bridge with XAI and offers nice opportunities to use the
abstract argumentation framework as a means to derive explanation and interpretability.

4.3 Sentiment Analysis

Sentiment analysis refers to the use of NLP, text analysis and computational linguistics
techniques to systematically identify, extract and quantify affective states and subjective
information, classifying the polarity of a given text at sentence, document or multi-
document level in order to find out whether the expressed opinion is positive, negative
or neutral. This has been extensively employed in a range of applications ranging from
marketing to customer services to clinical medicine. Key approaches to sentiment anal-
ysis include knowledge-based techniques, which classify text based on affect categories
according to presence of affect words such as happy, sad using affective lexical database
 AI Explainability 267

of lexicon dictionary. Furthermore, supervised and machine-learning like techniques

have also been populated for the same purpose [66]. Several open sources are made
available for the purpose of sentiment analysis. This includes, Python NLTK, TextBlob,
Pattern.en, RapidMiner, Stanford’s CoreNLP, SentiStrength, among others.
Through explainability-based reasoning, sentiment analysis can be boosted a step-
further to provide the reason for the sentiment score. For instance, in [67], the authors
proposed layer-wise relevance propagation model for explaining recurrent network
architecture for sentimentzanalysis.
Zhang et al. [68] proposed an Explicit Factor Model (EFM) based on phrase-level
sentiment analysis to generate explainable recommendations.
Interestingly, the presence of contrastive statements from opinionated documents in
sentiment analysis context opens up the door wide to application of more advanced argu-
mentation system, reinforcement learning or Markov-Chain based reasoning in the same
spirit as [69] inherited from question-answer system analysis. Similarly, the emergence
of multimedia documents in social media platforms provides opportunities to mix-up
image-analysis & text-analysis based reasoning. For instance, face-emotion recognition
in videos can provide useful insights into sentiment polarity of the underlined textual
input.

4.4 Topical Modelling

Since the emergence of Latent Dirichlet Allocation (LDA) [70], the task of automatic
discovery of topics in a textual document has seen a new landmark. In essence, LDA
introduces sparse Dirichlet prior distributions over document-topic and topic-word dis-
tributions, encoding the intuition that documents cover a small number of topics and
that topics often use a small number of words. Topic models are a form of unsupervised
machine learning, in that the topics and mixture parameters are unknown and inferred
solely from the data where each topic is represented by its N most probable words.
Humans can judge whether words of a given topic (cluster) form interpretable concept
(s). Therefore, it is important to seek automatic alternative to measure the interpretabil-
ity of the outputted set of words of each topic. A commonly employed approach is
based on the co-occurrence analysis, stipulating that words that have high frequency
of co-occurrence (either within the document under investigation or in a more wider
corpus) would indicate high coherence and relatedness, as for words caught and fever
for instance [71]. The development of word embedding promoted by Google researchers
has also promoted the so called embedded topic model [72] where each word is mod-
elled as a categorical distribution whose natural parameter is the inner product between
a word embedding and an embedding of its assigned topic. This has shown to discover
interpretable topics even with large vocabulary. On the other hand, the development of
interactive topic modelling [73], where more interaction modes with system output are
enabled, offers a nice setting to apply a range of visualization tools developed in the
context of explainable AI for this purpose.
268 M. Oussalah

4.5 Automatic Textual Summarization

Automatic text summarization has been a hot topic in NLP focusing on how to sum-
marize the main content of the document while preserving the semantic meaning and
key messages conveyed by the original document in a way to reduce redundancy and
maximize the diversity. Typically, two streams have been identified in the literature [74].
Extractive summarization where the summary is constituted of a selected sentences from
original document through some scoring analysis mechanism that takes into account sen-
tence similarity, location, presence of selected keywords, among others, and abstractive
summarization where the summary sentences may be different from that of original
documents. Extractive summarization is by far the most investigated research stream in
automatic summarization. Various graph-based approaches have been put forward for
extracting relevant sentences. Examples include TextRank [75] where the nodes are sen-
tences and the edges the relations (which is context dependent, e.g., semantic similarity
beyond certain threshold) between the sentence, and the importance of a given sentence
is quantified using PageRank like algorithm. Similarly, latent semantic analysis [76, 77],
which provides a lower dimensional representation of words, has also been applied to
summarization purpose [80].
Strictly speaking, explainable research benefits summarization from both directions.
First, summarization can be used as a tool to construct and identify arguments that can be
used to guide the explanation process. Second, the interactive tools, LIME like approach
can also be adapted to boost the sentence weighting scheme, which, in turn impact the
outcome of the summarization task.

5 Conclusion
Explanation methods are a promising approach to leverage hidden knowledge about
the workings of neural networks and black-box systems, promoting transparency and
interpretability of the results in the light of the new data protection EU directive on the
“right of explanation”. This papers attempted to review the state of art of explainability
methods focusing on intertwine between image processing and natural language pro-
cessing fields in a way to promote fruitful development of new explanation framework.
Especially, the paper highlights the implicit link between the two research fields through,
e.g., automatic image annotation, visual question-answer systems, Text-To-Image gen-
eration, multimedia analytics in addition to the overall input-output like system analysis.
On the other hand, this review has also identified several NLP research fields that would
benefit from visual explainability based approach. This includes, wordsense disambigua-
tion, sentiment analysis, argumentation theory, automatic text summarization and topical
modelling.
There are several interesting future research directions to explore further. An inter-
esting direction is semi-supervised learning of the model using a large set of partially
annotated data. For instance, we can exploit a small number of fully annotated images
and a large number of partially annotated images (e.g. images with only text descrip-
tions), which allows the developed model to exploit large-scale datasets, such as the
Google Conceptual Caption dataset. The paper also opens up new research directions in
multimedia analytics, text summarization and abstract argumentation logic.
 AI Explainability 269

Acknowledgment. This work is partly supported by the H2020 YoungRes (# 823701) project,
which is gratefully acknowledged.

References
1. Oxborough, C., Cameron, E.: Explainable AI, PWC report (2020). https://www.pwc.co.uk/
services/risk-assurance/insights/explainable-ai.html Accessed July 2020
2. Grice, H.P.: Logic and Conversation. In: Syntax and Semantics 3: Speech arts, pp. 41–58
(1975)
3. Conati, C., Porayska-Pomsta, K., Mavrikis, M.: AI in education needs interpretable machine
learning: lessons from open learner modelling. arXiv preprint arXiv:1807.00154 (2018)
4. Goodman, B., Flaxman, S.: European union regulations on algorithmic decision making and
a “right to explanation.” AI Mag. 38(3), 50 (2017)
5. Abdul, A., Vermeulen, J., Wang, D., Lim, B.Y., Kankanhalli, M.: Trends and trajectories for
explainable, accountable and intelligible systems: an HCI research agenda. In Proceedings of
the 2018 CHI. Association for Computing Machinery, Montreal, Canada (2018). https://doi.
org/10.1145/3173574.3174156
6. Doshi-velez, F., Kim, B.: A Roadmap for a Rigorous Science of Interpretability. CoRR,
abs/1702.08608 (2017). https://arxiv.org/abs/1702.08608
7. Samek, W., Wiegand, T., Müller, K.R.: Explainable artificial intelligence: understanding,
visualizing and interpreting deep learning models. ITU J. ICT Discoveries Spec. (1), 39–48
(2017)
8. Lewis, D.: Causal Explanation. In: Philosophical Papers. Vol II. Oxford University Press,
New York, Chapter Twenty two, pp. 214–240 (1986)
9. Halpern, J.Y., Pearl, J.: Causes and explanations : a structural-model approach. Part II :
Explanations. 56(4), 889–911 (2005). https://doi.org/10.1093/bjps/axi148
10. Miller, T.: Explanation in artificial intelligence: Insights from the social sciences. Artif. Intell.
267, 1–38 (2019). https://doi.org/10.1016/j.artint.2018.07.007
11. Arrieta, Del Ser et al. Explainable artificial intelligence (XAI): concepts, taxonomies,
opportunities and challenges toward responsible AI. Inf. Fusion 58, 82–115 (2019)
12. Gunning, D.: Explainable artificial intelligence (XAI ). Technical report, pp. 1–18 (2017)
13. Lipton, Z.C.: The mythos of model interpretability. Queue 16(3), 30:31–30:57 (2018)
14. Kim, B., Doshi-Velez, F.: Introduction to interpretable machine learning. In Proceedings of
the CVPR 2018 Tutorial on Interpretable Machine Learning for Computer Vision, Salt Lake
City, UT, USA (2018)
15. Jolliffe, I.: Principal component analysis. In International Encyclopedia of Statistical Science,
pp. 1094–1096. Springer, Berlin (2011). https://doi.org/10.1007/978-3-642-04898-2_455
16. Maaten, L.Y.D., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605
(2008)
17. Kim, B., Khanna, R., Koyejo, O.O.: Examples are not enough, learn to criticize! Criticism
for interpretability. In: Advances in Neural Information Processing Systems. MIT Press:
Cambridge, pp. 2280–2288 (2016)
18. Hartigan, J.A., Wong, M.A.: A k-means clustering algorithm. J. R. Stat. Soc. Ser. C (Appl.
Stat.) 28, 100–108 (1979)
19. Adadi, A., Berrada, M.: Peeking inside the black-box: a survey on explainable artificial
intelligence (XAI). IEEE Access 6, 52138–52160 (2018)
20. Molnar, C.: Interpretable machine learning (2019). https://christophm.github.io/interpretable-
ml-book/. Accessed July 2020
270 M. Oussalah

21. Yang, C., Rangarajan, A., Ranka, S.: Global model interpretation via recursive partitioning
(2018). arXiv:1802.04253
22. Nguyen, A., Dosovitskiy, A., Yosinski, J., Brox, T., Clune, J.: Synthesizing the preferred
inputs for neurons in neural networks via deep generator networks. Proc. Adv. Neural Inf.
Process. Syst. (NIPS), 3387–3395 (2016)
23. Ribeiro, M.T., Singh, S., Guestrin, C.: Why should I trust you?: Explaining the predic-
tions of any classifier, In: Proceedings of 22nd ACM SIGKDD International Conference
on Knowledge Discovery Data Mining (2016)
24. Lundberg, S.M., Lee, S.L.: A unified approach to interpreting model predictions. Proc. Adv.
Neural Inf. Process. Syst. 4768–4777 (2017)
25. Cortez, P., Embrechts, M.J.: Opening black box data mining models using sensitivity analysis.
In: Proceedings of IEEE Symposium on Computational Intelligence Data Mining (CIDM),
pp. 341–348 (2011)
26. Smilkov, D., Thorat, N., Kim, B., ViØgas F., Wattenberg, M.: SmoothGrad: removing noise
by adding noise (2017). https://arxiv.org/abs/1706.03825
27. Green D.P., Kern, H.L.: Modeling heterogeneous treatment effects in large-scale experiments
using Bayesian additive regression trees. In: Proceedings of Annual Summer Meeting Society
for Political Methodology, pp. 1–40 (2010)
28. Bastani, O., Kim, C., Bastani, H.: Interpretability via model extraction (2017). https://arxiv.
org/abs/1706.09773
29. Thiagarajan, J.J., Kailkhura, B., Sattigeri,P., Ramamurthy, K.N.: TreeView: peeking into deep
neural networks via feature-space partitioning. (2016) https://arxiv.org/abs/1611.07429
30. Goldstein, A., Kapelner, A., Bleich, J., Pitkin, E.: Peeking inside the black box: visualizing
statistical learning with plots of individual conditional expectation, J. Comput. Graph. Statist.
24(1), 44–65 (2015)
31. Frank, E., Witten, I.H.: Generating accurate rule sets without global optimization. In: ICML
1998, pp. 144–151 (1998)
32. Robnik-Šikonja, M., Kononenko, L.: Explaining classifications for individual instances. IEEE
Trans. Knowl. Data Eng. 20(5), 589–600 (2008)
33. Etchells, T.A., Lisboa, P.J.G.: Orthogonal search-based rule extraction (OSRE) for trained
neural networks: a practical and efficient approach. IEEE Trans. Neural Netw. 17(2), 374–384
(2006)
34. Tan, S., Caruana, R., Hooker, G., Lou, Y.: Detecting bias in black-box models using transparent
model distillation. https://arxiv.org/abs/1710.06169 (2018)
35. Xu, K., Park, D.H., Yi, D.H., Sutton, C.: Interpreting deep classifier by visual distillation of
dark knowledge (2018). https://arxiv.org/abs/1803.04042
36. Cortez, P., Embrechts, M.J.: Using sensitivity analysis and visualization techniques to open
black box data mining models. Inf. Sci. 225, 1–7 (2013)
37. Fisher, A., Rudin, C., Dominici, F.: Model class reliance: variable importance measures for
any machine learning model class, from the ‘Rashomon’ perspective (2018). https://arxiv.org/
abs/1801.01489
38. Casalicchio, G., Molnar, C., Bischl, B.: Visualizing the feature importance for black box
models (2018). https://arxiv.org/abs/1804.06620
39. Schetinin, V., et al.: Confident interpretation of Bayesian decision tree ensembles for clinical
applications. IEEE Trans. Inf. Technol. Biomed. 11(3), 312 (2007)
40. Hara, S., Hayashi, K.: Making tree ensembles interpretable (2016). https://arxiv.org/abs/1606.
05390
41. Ribeiro, M.T., Singh, S., Guestrin, C.: Anchors: high-precision model-agnostic explanations.
In: Proceedings of AAAI Conference on Artificial Intelligence, pp. 1–9 (2018)
 AI Explainability 271

42. García, S., Fernández, A., Herrera, F.: Enhancing the effectiveness and interpretability of deci-
sion tree and rule induction classifiers with evolutionary training set selection over imbalanced
problems. Appl. Soft Comput. 9(4), 1304–1314 (2009)
43. Wang, F., Rudin, C.: Falling rule lists. In: Proceedings of 18th International Confrence
on Artificial Intelligence on Statistics (AISTATS), San Diego, CA, USA: JMLR W&CP,
pp. 1013–1022 (2015)
44. Ras, G., Van Gerven, M., Haselager, P.: Explanation methods in deep learning: Users, values,
concerns and challenges (2018). https://arxiv.org/abs/1803.07517
45. Johansson, U., König, R., Niklasson, I.: The truth is in there—rule extraction from opaque
models using genetic programming. In: Proceedings of FLAIRS Conference, pp. 658–663
(2004)
46. Wachter, S., Mittelstadt, B., Russell, C.: Counterfactual explanations without opening the
black box: automated decisions and the GDPR (2017). https://arxiv.org/abs/1711.00399
47. Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., Samek, W.: On pixel-wise
explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS
ONE 10(7), e0130140 (2015)
48. Thiagarajan, J.J., Kailkhura, B., Sattigeri,P., Ramamurthy, K.N.: TreeView: peeking into deep
neural networks via feature-space partitioning (2016). arXiv preprint arXiv:1611.07429
49. Wang, T., Rudin,C., Velez-Doshi, F., Liu, Y., Klamp, E., MacNeille, P.: Bayesian rule sets for
interpretable classification. In: IEEE 16th International Conference on Data Mining (ICDM),
pp. 1269–1274 (2016)
50. Boz, O.: Extracting decision trees from trained neural networks. In: Proceedings of the Eighth
ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 456–
461. ACM (2002)
51. Shrikumar, A., Greenside, P., Shcherbina, A., Kundaje, A.: Not just a black box: learning
important features through propagating activation differences (2016). arXiv:1605.01713
52. Zhou, A.M., Gan, J.Q.: Low-level interpretability and high-level interpretability: a unified
view of data-driven interpretable fuzzy system modelling. Fuzzy Sets Syst. 159(23), 3091–
3131 (2008)
53. https://cloud.google.com/vision
54. Szegedy, C., et al.: Going deeper with convolutions. In: CPRV’15 (2004). arXiv preprint
arxiv:1409.4842
55. Xu, H., Saenko, K.: Ask, attend and answer: exploring question-guided spatial attention for
visual question answering. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016.
LNCS, vol. 9911, pp. 451–466. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-
46478-7_28
56. Lu, J., Yang, J., Batra, D.: Parikh. Hierarchical question image co-attention for visual question
answering. In: Advances in Neural Information Processing Systems (NIPS2016), pp 289–297
(2016)
57. Reed S. et al.: Generative adversarial text to image synthesis. In: ICML 2016, pp. 1060–1069
(2016)
58. Reed, S.E., Akata, Z., Mohan, S., Tenka, S., Schiele, B., Lee, H.: Learning what and where
to draw. In: NIPS, pp. 217–225 (2016)
59. Welinder, P., et al.: Caltech-UCSD Birds 200. Technical report. CNS-TR-2010- 001,
California Institute of Technology (2010)
60. Nilsback, M.E., Zisserman, A.: Automated flower classification over a large number of
classes. In: Proceedings of the Indian Conference on Computer Vision, Graphics and Image
Processing, pp. 722–729 (2008)
272 M. Oussalah

61. Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick,
C.L.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B.,
Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014).
https://doi.org/10.1007/978-3-319-10602-1_48
62. Lesk, M.: Automatic sense disambiguation using machine readable dictionaries: how to tell
a pine cone from an ice cream cone. In: Proceedings of SIGDOC, pp. 24–26 (1986)
63. Mihalcea, R.: Knowledge-based methods for WSD. In: Word Sense Disambiguation: Algo-
rithms and Applications, Text, Speech and Language Technology, pp. 107–132. Springer,
Dordrecht (2006). https://doi.org/10.1007/978-1-4020-4809-8_5
64. Navigli, R., Lapata, M.: Graph connectivity measures for unsupervised word sense disam-
biguation. In: IJCAI International Joint Conference on Artificial Intelligence, Hyderabad,
India, pp. 1683–1688 (2007)
65. Dung, P.M.: On the acceptability of arguments and its fundamental role in nonmonotonic
reasoning, logic programming and n-person games. Artif. Intell. 77(2), 321–357 (1995)
66. Cocarascu, O., Stylianou, A., Cyras K., Toni, F.: Data-empowered argumentation for dialecti-
cally explainable predictions. In: 24th European Conference on Artificial Intelligence – ECAI
(2020)
67. Tsytsarau, M., Palpanas, T.: Survey on mining subjective data on the web. Data Min. Knowl.
Discov. 24, 478–514 (2012)
68. Arras, L., Horn, F., Montavon, G., Muller, K.R., Samek W.: Explaining predictions of non-
linear classifiers in NLP. arXiv preprint arXiv:1606.07298 (2016)
69. Zhang Y., Lai, G., Zhang, M., Zhang, Y., Liu, Y., Ma, S.: Explicit factor models for explainable
recommendation based on phrase-level sentiment analysis. In: Proceedings of the 37th ACM
SIGIR, pp. 83–92 (2014)
70. Sherstov, A.A., Stone, P.: Improving action selection in MDP’s via knowledge transfer. AAAI
5, 1024–1029 (2005)
71. Blei, D.M., Lafferty, J.D.: TopicMmodels. Chapman & Hall/CRC (2009)
72. Lau, H.J., Newman, D., Baldwin, T.: Machine reading tea leaves: automatically evaluating
topic coherence and topic model quality. In: EAC (2014)
73. Dieng, A.B., Ruiz, F.R., Blei, D.M.: Topic Modelling in Embedding Spaces (2019). arXiv:
1907.04907v1 cs.IR
74. Hu, Y., Boyd-Graber, J., Satinoff, B., Smith, A.: Interactive topic modeling. Mach. Learn. 95,
423–469 (2013)
75. Nenkova A., McKeown K.: A survey of text summarization techniques. In: Aggarwal, C.,
Zhai, C. (eds.) Mining Text Data. Springer, Boston, MA (2012). https://doi.org/10.1007/978-
1-4614-3223-4_3
76. Mihalcea, R., Tarau, P.: Textrank: bringing order into text. In: Proceedings of the conference
on empirical methods in natural language processing (2004)
77. Gong, Y., Liu, X.: Generic text summarization using relevance measure and latent semantic
analysis. In: Proceedings of the 24th ACM SIGIR, pp. 19– 25 (2001)
78. Steinberger, J., Jezek, K.: Using latent semantic analysis in text summarization and summary
evaluation. Proc. ISIM 4, 93–100 (2004)
79. Hong, S., Yang, D., Choi, J., Lee, H.: Interpretable text-to-image synthesis with hierarchical
semantic layout generation. In: Samek, et al. (eds.): Explainable AI: Interpreting, Explaining
and Visualizing Deep Learning. Springer, Germany (2019). https://doi.org/10.1007/978-3-
030-28954-6
80. Mohamed, M., Oussalah, M.: A hybrid approach for paraphrase identification based on
knowledge-enriched semantic heuristics. Lang. Resour. Eval. 54(2), 457–485 (2019). https://
doi.org/10.1007/s10579-019-09466-4
 AI Explainability 273

81. Mohamed, M., Oussalah, M.: SRL-ESA-TextSum: a text summarization approach based on
semantic role labeling and explicit semantic analysis. Inf. Process. Manage. (2020). https://
doi.org/10.1016/j.ipm.2019.04.003
82. Buchanan, B.G., Shortliffe, E.H.: Rule Based Expert Systems: The MYCIN Experiment of
the Stanford Heuristic Programming Project. Addison-Wesley, Reading, MA (1984)

View publication stats