NPTEL
Fundamentals of Affective Computing
Jainendra Shukla
[email protected] A-410, R & D Block
Computer Science and Engineering
Human-Centered Design
A Definition
The field of affective computing encompasses
NPTEL
both the creation of and interaction with
machine systems that sense, recognize,
respond to, and influence emotions (Picard,
1997; Picard and Klein, 2002).
HAL: Do you read me?
2001: A Space Odyssey: A Conversation with
NPTEL
HAL (1968)
Course Topics
Course Topics
NPTEL
An exciting vision of AI
● Speech,
NPTEL
● Facial recognition,
● Lip reading,
● Systems monitoring,
● Regulation and control,
● Game play, and
● Social skills
Affect Sensing
● Affect sensing refers to a system that can recognize
emotion by receiving data through signals and
NPTEL
patterns (Picard, 1997).
● To accomplish this task, a computer would need to be
equipped with hardware and software.
● Affect-sensing systems can be classified by modalities,
each of which has a unique signature.
Affect Sensing
● Affect-sensing systems can be classified by modalities:
○ Facial activity,
NPTEL
○ Posture, Gesture,
○ Hand tension,
○ Vocal,
○ Textual,
○ Physiological Signals
■ Electrodermal activity (EDA),
■ Electroencephalography (EEG) etc.
Applications (Picard , 1997)
1. Detection: Systems that detect the emotions of the
user
NPTEL
2. Expression: Systems that express what a human
would perceive as an emotion (e.g., an avatar, robot,
and animated conversational agent)
3. Perception: Systems that actually ‘feel’ an emotion.”
Health Care Applications
● Individuals with
Asperger syndrome (AS)
NPTEL
or high functioning
autism (HFA)
○ Mobile applications
like SymTrend and
Autism Track
(HandHold
Adaptive)
HealthCare Applications: Visharanti
NPTEL
PPG & GSR data No Stress
Classifier
Stress
k
ac
db
Wearable Device
e
Fe Recommend an activity
Recommendation
system
Feedback for
recommended activity
Health Care Applications
● Post-traumatic stress
disorder (PTSD)
NPTEL
○ StartleMart,
○ Virtual Vietnam,
Virtual Iraq, Virtual
Afghanistan
Education Applications
● EngageMe (Darnell, 2014).
○ This system uses skin conductance data collected
NPTEL
from students in a classroom, along with video
feeds, to help the teacher reflect on his or her
classes.
● Subtle Stone (Balaam, 2009)
○ Wireless, hand-held, squeezable ball that allows
students to communicate their affective and
motivational experiences to their teachers in real
time.
● China Wants to Raise Smarter Students
Other: Preparing you for interview
● MACH (My Automated Conversation coacH)
NPTEL
KISMET
NPTEL
Case Study: It’s all about Experience
NPTEL
It’s all about User Experience
● UX is a broad concept that encompasses many aspects
of a user’s interaction with your product and
NPTEL
company; namely “how they feel about you.”
○ Marketing Strategy by Steve Jobs
● Usability is more the practical side of things, the main
question being answered is “ease of use.”
● The combination of the two leads to a better
understanding of emotional experience and
interaction with your brand and hopefully adds to a
memorable, positive experience.
○ But, how do you test for both?
Participatory Design/Acceptance Test
● Participatory design is a process that includes the
stakeholders in the early stages of design.
NPTEL
○ Participatory design lends itself to user-centered
design innovation.
■ PICTIVE approach (Muller, 1991)
● Acceptance Testing
○ During Beta testing the end-user (intended real
user) validates the product for functionality,
usability, reliability, and compatibility.
■ Self-reporting in form of focus groups/ surveys
Problems!
● Users have to articulate exactly how they feel about
the products in the form of focus groups and surveys,
NPTEL
which can be plagued by issues like participant bias,
recall bias etc.
● Advanced UX/Neuromarketing
○ Application of Affective Computing/Neuroscience
to marketing.
■ How customers use their mind while
responding to products?
■ How this impacts their decision making?
● To gain new perspectives and have a more accurate
Advanced UX/Neuromarketing
● For the purpose of this class, we will focus on
web-based interactions
NPTEL
○ Some main considerations when embarking on
advanced UX & usability testing with Affective
Computing.
● The primary elements of our case study will focus on
how to employ Affective Computing techniques for
web-based testing.
Advanced UX/Neuromarketing
NPTEL
Ethical Considerations
● Emotional Manipulation: Is it ethical for
computers to detect, recognize, and then attempt to
NPTEL
modify certain behaviors?
● Privacy: Emotions, perhaps more so than thoughts,
are ultimately personal and private (Picard, 2003).
● Emotional Dependency: Could having users
routinely use these moral agents create
codependence?
References
1. Researchers criticize AI software that predicts emotions
2. Picard, R.W., 1997. Affective Computing, first ed. MIT Press,
NPTEL
Cambridge, MA.
3. SymTrend. (n.d.). Symptom tracking, charting, & reminding.
https://www.symtrend.com/tw/public/
4. HandHold Adaptive. (n.d.). Autism track.
http://www.handholdadaptive.com/AutismTrack.
html
1. Holmgard, C., Yannakakis, G.N., Karstoft, K.-I., Andersen, H.S., 2013.
Stress detection for PTSD via the Startlemart game. Proceedings 2013
Human Association Conference on Affective Computing and Intelligent
Interaction (ACII). IEEE, Geneva, Switzerland, pp. 523–528.
References
1. Supporting affective communication in the classroom with the Subtle
Stone
NPTEL
2. Darnell, S.S., 2014. EngageME: a tool to simplify the conveyance of
complicated data. Proceedings CHI’14 extended abstracts on human
factors in computing systems. ACM Press, Toronto, Canada, pp.
359–362.
3. Picard, R.W., 2003. Affective computing: challenges. Int. J. Hum.
Comput. Stud. 59 (1–2), 55–64.
4. MACH: my automated conversation coach
5. Michael J. Muller. 1991. PICTIVE—an exploration in participatory
design. In Proceedings of the SIGCHI Conference on Human Factors in
Computing Systems (CHI '91). Association for Computing Machinery,
New York, NY, USA, 225–231.