Experimental Methodology and Tools in Affective
Computing
Introduction
In affective computing, designing robust experiments is essential to collect valid and ethical data for
emotional modeling. This document covers the entire pipeline from preparing for human-subject
research to deploying experimental tools.
Institutional Review Board (IRB) and Ethical Approval
Before conducting human-subject research, it is mandatory to obtain ethical approval from an
Institutional Review Board (IRB) or equivalent.
Key IRB Submission Documents
Draft/Abstract of Proposal:
Clear research methodology
Types of data and stimuli
Participant recruitment methods
Experiment lab conditions
Informed Consent Form:
Legal document confirming participant’s voluntary involvement
Templates are available from APA and international agencies
Privacy and Anonymity Clause:
Data protection using national/international IT guidelines
Risk and Benefit Analysis:
Clearly outline psychological/physical risks and expected benefits
Recruitment Strategy:
Include sample flyers, invitations, or email templates
Data Collection Instruments:
Page 1 of 5
Surveys, sensors, experimental protocols, interview questions, etc.
Important: Never begin data collection without IRB approval. Doing so is unethical and may
invalidate your research.
IRB Evaluation Criteria
Risk minimization or justification via benefits
Equitable subject selection
Informed consent procedures
Data monitoring strategies
Privacy/confidentiality measures
Additional safeguards for vulnerable populations (e.g., minors, people with disabilities)
Experimental Design
1. Study Concept Development
Define emotion types to be studied
Choose stimuli (images, music, video, etc.)
Select data collection modalities (e.g., EEG, skin conductance)
2. Study Design Types
Within-Subject Design:
Same participants across multiple conditions (e.g., VR vs PC)
Fewer participants needed, but longer session duration
Between-Subject Design:
Different participants for each condition
Requires more participants but shorter sessions
Mixed-Model Design:
Combines within- and between-subject elements
3. Group Allocation
Typically includes a control group and a treatment group
Depends on number of independent variables to be analyzed
4. Sample Size
Page 2 of 5
Ideal: 30 participants per group
More is better for statistical/machine learning robustness
5. Evaluation Methods (for Ground Truth)
Self-Assessment:
Directly asking participants to report emotional state (e.g., after viewing an image)
Interviews:
Post-task discussion to infer emotional responses
Psychophysiology:
Use of physiological signals (EEG, ECG, GSR)
Observation:
Expert analysis of video recordings
Task Performance:
Behavioral response under emotional stimuli (e.g., solving arithmetic under
positive/negative music)
6. Study Location
Controlled Lab Setup:
Full environmental control (light, sound, screen distance)
Naturalistic Setting:
E.g., outdoor/real-world setups to enhance ecological validity
7. Hardware and Equipment
Stimulus presentation systems (PC, mobile, headphones)
Sensor integration (e.g., EEG, heart rate monitors)
8. Failure Contingencies
Pre-plan alternatives for sensor/hardware failures
Example: If one sensor fails, continue with remaining two
9. Recruitment Criteria
Page 3 of 5
Inclusion: Age, background, etc.
Exclusion: Recent illnesses (e.g., post-COVID recovery)
Tools in Affective Computing
Tools are categorized into five major areas:
1. Data Collection
Tools: PsychoPy, OpenSesame, SuperLab, E-Prime, Presentation, DMDX, Paradigm
Example: PsychoPy
Free, cross-platform
GUI Builder Interface and Python scripting support
Supports integration with eye trackers and EEG devices
2. Data Annotation
Self-Assessment: SAM (Self-Assessment Manikin)
Measures Valence, Arousal, Dominance (PAD model)
Audio: ELAN, Wavesurfer, Praat, Speechalyzer
Video: ELAN, Cowlog, ANVIL, Gtrace, ChronoViz
Text: Whissell Dictionary of Affect in Language
3. Signal Processing and Analysis
Audio: Praat, OpenEAR, OpenSmile, Wavesurfer
Physiological: EEGLab, PyEEG, ELAN, AuBT
Image/Video: OpenCV, OpenPose, Kinect SDK, MediaPipe
Example: MediaPipe (Google)
Cross-platform ML framework
Built-in models for face detection, pose estimation, gesture tracking
4. Affect Classification (Machine Learning)
Popular Tools: WEKA, AutoML, SPSS, PRTools, LibSVM, SVMlight, MATLAB, R, RapidMiner, SciPy,
HTK, SAS
Example: WEKA
Open-source with GUI
Supports classification, clustering, feature selection, visualization
Page 4 of 5
5. Affect Expression
Tools: MARY, GRETA, Festival, VHML, SmartBody, FaceFX, Xface, Horde3D, Haptek, ROS, ICT
Virtual Human Toolkit, SOAR, ACT-R
Example: ICT Virtual Human Toolkit
Modular libraries to build conversational virtual agents
Customizable avatars with verbal and non-verbal behavior modules
Conclusion
Experimental methodology is foundational for affective computing. From IRB preparation to study
design and tool usage, each step ensures data quality and ethical compliance. Utilizing the right set of
tools can significantly streamline experiment execution, data collection, and emotion modeling.
Page 5 of 5
