VRET

1. Overall Goal & Context

• Thesis Title: Feasibility Study on Using ‘Behind the Ear’-EEG to Detect Arousal in Virtual Reality Exposure Therapy
• Primary Objective: To investigate whether ‘Behind the Ear’-EEG (BTE-EEG) is a feasible method for detecting physiological arousal in participants undergoing exposure therapy for specific phobias (arachnophobia, acrophobia) within virtual reality (VR) / augmented reality (AR) environments.
• Validation: Pulse rate (derived from PPG) and Galvanic Skin Response (GSR) were measured concurrently to act as validation signals and provide a more comprehensive assessment of arousal.
• Motivation: Traditional scalp EEG setups can be cumbersome and prone to noise/artifacts when combined with VR headsets. BTE-EEG offers a potentially simpler, faster alternative with less direct interference from the headset. The ultimate goal is to inform the potential use of such signals for dynamically adjusting VRET intensity.

2. Methodology Summary

• Two Experimental Studies:
  • Study 1: Focused on feasibility (RQ2). Participants underwent gradual, user-controlled exposure to progressively more intense stimuli in both spider (AR) and height (VR) scenarios. Arousal was expected to increase stepwise.
  • Study 2: Focused on sensitivity to environmental changes (RQ3). Participants experienced automatic, alternating high-arousal and low-arousal conditions (30s each) in both scenarios while remaining relatively stationary. Arousal was expected to fluctuate with the conditions.
• VR Environments:
  • Arachnophobia: Augmented Reality (AR) using Meta Quest 3 passthrough, projecting procedural spiders onto the real floor.
  • Acrophobia: Immersive Virtual Reality (VR) city scene with a tall building and a controllable plank/avatar.
• Participants: Recruited from Aarhus University networks. Screened for age (>18), absence of diagnosed anxiety disorders, and ability to follow instructions. Severity of phobia assessed via SMSP-A (Study 1, pre-visit) or self-report (Study 2, during recruitment). Individuals with severe/extreme phobias were excluded.

3. Data Acquisition

• Signals Measured: BTE-EEG, PPG (for Pulse Rate & SpO2), GSR.
• Sensors:
  • EEG: 5 Ag/AgCl electrodes (2 behind each ear, 1 Fpz reference, 1 neck ground).
  • PPG: g.SpO2 sensor (finger).
  • GSR: g.Sensor (two finger electrodes).
• Amplifier: TMSi Mobita system.
• Setup: Portable setup with amplifier in a backpack (Fig 3.1).
• Sampling Rate: 250 Hz.

4. Data Processing (Chapter 3.4) - Detailed Breakdown

This chapter details the crucial steps to transform raw data into analyzable features.

4.1 Data Alignment Strategy (Addressing Your Question)

• Yes, data alignment was performed and essential.
• How: Alignment was achieved through a multi-pronged approach:
  1. Common Amplifier: All biosignals (EEG, PPG, GSR) were recorded simultaneously by the single TMSi Mobita amplifier, providing an initial synchronized timeline within the raw .Poly5 file.
  2. Manual Triggers: Investigators manually inserted time markers into the Mobita recording for major experimental phases (e.g., start/end of baseline, alpha test segments, calibration, overall exposure blocks, equipment removal - see Tables 3.2 & 3.4).
  3. Automatic VR Triggers: The Unity VR/AR applications automatically logged timestamps whenever an exposure condition changed (user-triggered in Study 1, time-triggered every 30s in Study 2).
  4. Integration Step: During pre-processing (Section 3.4.1, Fig 3.2), the timestamps from the separate VR log file were read and corresponding event markers were inserted into the main biosignal (.set) data file.
• Result: This process created a single, unified dataset where the precise timing of VR events is synchronized with the physiological recordings, enabling accurate segmentation and analysis per condition.

4.2 Data Pre-processing (Section 3.4.1)

• Goal: Clean raw data, ensure consistency, prepare for feature extraction.
• Initial Steps (Fig 3.2):
  • Format Conversion: .Poly5 (Mobita) -> .set (EEGLAB/MNE compatible) via MATLAB.
  • Trigger Integration: Automatic VR triggers added to .set file.
  • Bad Channel Removal: Unusable channels discarded.
  • Modality Separation: Data split into EEG, PPG, GSR streams.
• EEG Pre-processing Pipeline (Fig 3.3):
  • Re-referencing: To Fpz electrode.
  • Filtering: 50 Hz Notch filter (powerline noise) + 2-100 Hz Bandpass filter (drift, high-freq noise/EMG).
  • Artifact Handling:
    • EOG (Blinks): Removed using MNE functions (visualized in Fig 3.4).
    • EMG (Muscle): Segments exceeding 3x standard deviation excluded (no interpolation).
  • Standardization: Data scaled relative to participant’s baseline standard deviation.
• PPG Pre-processing Pipeline (Fig 3.5):
  • Re-referencing: To sensor’s ‘Ref’ channel.
  • Calibration: Applied using recorded 0-100% signals.
  • Filtering: 50 Hz Notch + 30 Hz Lowpass.
  • Error Correction: Interpolation for pulse rates < 35 bpm.
  • Smoothing: Gaussian filter (1s window).
• GSR Pre-processing Pipeline (Fig 3.6):
  • Re-referencing: To sensor’s ‘Ref’ channel.
  • Calibration Check: Value of 1 µS step signal verified.
  • Filtering: 50 Hz Notch + 30 Hz Lowpass.
  • Smoothing: Gaussian filter (8 sample window).
  • Unit Conversion: Voltage (V) -> Microsiemens (µS).

4.3 Data Post-processing / Feature Extraction (Section 3.4.2)

• Goal: Calculate quantitative metrics related to arousal from the cleaned signals for each condition.
• EEG Features (Fig 3.7):
  • Method: Welch’s method applied to 10 subsegments within each condition per channel.
  • Features: Median Alpha Power, Median Beta Power, Power & RMS of Delta-Beta Correlation. (Fig 3.7 illustrates the subsegment/median power concept).
• PPG Feature (Fig 3.8):
  • Feature: Mean Pulse Rate per condition (standardized to baseline).
• GSR Features (Fig 3.9 - Addressing Your Question):
  • Method: Signal decomposed into tonic/phasic using Neurokit2. Peaks (SCRs) > 0.05µS detected in the phasic component.
  • Features (Derived from Phasic):
    • SCR Frequency: Number of detected SCR peaks per second per condition (standardized). (Illustrated by top trace/label in Fig 3.9).
    • SCR Mean Amplitude: Average amplitude (µS) of detected SCR peaks per condition (standardized). (Illustrated by bottom trace/label in Fig 3.9).
  • Clarification: Fig 3.9 shows the final features calculated from the phasic peaks, not the raw tonic/phasic traces themselves.

4.4 Statistical Analysis (Section 3.4.3)

• Method: Permutation testing (10,000 permutations).
• Comparisons:
  • Study 1: Baseline vs. Arousal Conditions.
  • Study 2: Low-Arousal vs. High-Arousal Conditions.
• Note: No correction for multiple comparisons applied due to exploratory nature.

4.5 Clarification on Figure 3.3 Branching (Addressing Your Question)

• The two post-processing branches in Fig 3.3 represent analysis applied to two different datasets:
  1. Left Branch: Data from the specific alpha activity validation test (eyes open/closed).
  2. Right Branch: Data from the main experiment (baseline + VR exposure conditions).
• It does not represent two different ways of processing the same data.

5. Result

5. Results (Page 45-59)

This section presents the findings derived from the processed data (as detailed in Section 3.4) collected during the experiments (described in Section 4). The results are organized by study and aim to directly address the research questions: RQ2 (Feasibility of BTE-EEG for arousal detection in VRET) for Study 1, and RQ3 (Detecting arousal changes from VR modulation using BTE-EEG) for Study 2.

1. Study 1 Results (5.1): Focuses on whether BTE-EEG can detect arousal when comparing baseline measurements to progressively challenging VR exposure conditions.
   • Participants (5.1.1): Details the participant pool (11 recruited), exclusions due to data quality (resulting in 10 analyzable for arachnophobia, 8 for acrophobia), participant demographics and phobia scores (Table 5.1), and presents a study flowchart (Fig 5.1). Two representative participants (sub-004, sub-005) are chosen to illustrate typical data patterns.
   • EEG Findings (5.1.2 - 5.1.4): Presents median Alpha power (Fig 5.2), median Beta power (Fig 5.3), and Delta-Beta Correlation (Fig 5.4) across conditions for the representative participants. These plots show variable individual patterns, sometimes showing increasing trends with exposure (e.g., beta power for sub-004), other times less clear patterns.
   • PPG/GSR Findings (5.1.5): Shows standardized pulse rate, SCR frequency, and SCR amplitude for the representative participants (Fig 5.5), allowing for visual inspection of trends relative to baseline.
   • Statistical Findings (5.1.6): Summarizes the key statistical outcomes from permutation tests comparing baseline to exposure conditions (Table 5.2). A notable finding is that 55.5% of participant-scenario combinations showed a significant increase in beta power during exposure compared to baseline, with this effect being particularly strong (83.3%) for participants reporting fear related to the specific scenario stimulus. Alpha power results were mixed, with more participants showing a significant increase (38.8%) than a decrease (5.5%) during exposure compared to baseline.
2. Study 2 Results (5.2): Focuses on whether BTE-EEG and other biosignals can detect differences between automatically alternating high-arousal and low-arousal conditions within the VR scenarios.
   • Participants: Details the 4 participants (2 spider fear, 2 height fear) with no data exclusions (Table 5.3, Fig 5.6).
   • EEG Findings (5.2.1 - 5.2.3): Presents median Alpha power (Fig 5.7), median Beta power (Fig 5.8), and Delta-Beta Correlation (Fig 5.9) across the alternating conditions. The plots reveal different patterns between subjects and conditions, sometimes showing expected modulations (e.g., beta power changes between high/low states), increased variance in high-arousal states, or effects related to transitions between conditions (Table 5.4).
   • PPG/GSR Findings (5.2.4): Displays the trends for pulse rate, SCR frequency, and SCR amplitude across all participants and conditions (Fig 5.10). Visual inspection suggests no clear trend for pulse rate, but clearer tendencies for higher SCR frequency (and to some extent, amplitude) during high-arousal conditions compared to low-arousal ones.
   • Statistical Findings (5.2.5): Summarizes the permutation test results comparing high- vs. low-arousal conditions (Table 5.5). Beta power was significantly higher in high-arousal conditions in 50% (4/8) of the scenarios. Alpha power was significantly higher in high-arousal conditions in 62.5% (5/8) of scenarios (note potential discrepancy with table footnote interpretation). Pulse rate and GSR measures did not show consistent statistically significant differences between the high- and low-arousal states across scenarios.

6. Key Results Summary

• Mixed Findings: BTE-EEG shows potential but results are not consistently clear across all participants or metrics.
• Beta Power: Appeared most sensitive. Significantly increased during exposure vs. baseline in 55.5% of Study 1 cases (83.3% for those categorized as ‘scared’). Significantly higher in high-arousal vs. low-arousal conditions in 50% of Study 2 scenarios.
• Alpha Power: Varied results. Increased in some, decreased in others during exposure (Study 1). Significantly higher in high-arousal conditions in 62.5% of Study 2 scenarios (contrary to simple relaxation hypothesis).
• PPG/GSR: Showed some expected trends (e.g., higher SCR frequency in high arousal in Study 2) but often lacked statistical significance in Study 2 comparisons between high/low arousal conditions.
• Variability: Increased variance noted in EEG/GSR during high-arousal conditions (Study 2).

7. Discussion Points & Limitations

• BTE-EEG Potential: Beta power seems promising, but needs more investigation. Alpha power interpretation is complex.
• Limitations:
  • Baseline differences (Study 1: seated vs. standing exposure).
  • Transition phase confounders (putting on HMD).
  • Potential for habituation not accounted for.
  • Observer bias potential (Study 2 arachnophobia).
  • Lack of multiple comparison correction.
  • Small sample size (especially Study 2).
  • Noise/artifacts inherent in mobile setups.

8. Conclusion

• The thesis demonstrates that BTE-EEG (particularly beta power) shows sensitivity to arousal changes in VRET, but the findings are mixed and not consistently supported by significant changes in PPG/GSR between conditions (especially in Study 2).
• It is not possible to definitively conclude from this work alone that BTE-EEG can reliably replace other measures or robustly drive VRET adjustments yet.
• The study highlights the potential and identifies key challenges, suggesting BTE-EEG warrants further investigation with refined protocols and larger sample sizes.

Appendix: abbreviation

Experiment in detail

Phase 1: Pre-Experiment Preparation

1. Participant Recruitment:
   • Participants were recruited primarily from the investigators’ personal and professional networks within Aarhus University.
2. VR Environment Development:
   • Two distinct VR scenarios were developed beforehand using the Unity game engine and C#:
     • Arachnophobia Scenario: Utilized Augmented Reality (AR) via the Meta Quest 3’s passthrough capabilities. Spiders (based on Philipp Schofield’s ‘Procedural Spider’ asset, with modified animations and movement constraints) were projected onto the participant’s real-world floor.
     • Acrophobia Scenario: A fully immersive Virtual Reality (VR) environment depicting a high-altitude city scene (using the ‘Real New York City, Vol. 1’ asset pack). It included a narrow plank extending from a tall building and a user-controlled avatar (from Mixamo) to enhance presence.
3. Initial Screening (Study 1 Only):
   • Participants selected for Study 1 received a questionnaire based on the “Severity Measure for Specific Phobia - Adults” (SMSP-A) two to three weeks before their scheduled lab visit.
   • This questionnaire assessed their fear levels for both spiders and heights.
   • Purpose: To categorize participants (“scared” vs. “not scared” for each scenario) and to exclude individuals with severe/extreme phobias (criteria iii) without making them overly conscious of their fear immediately before the experiment.
4. Initial Screening (Study 2 Only):
   • Due to time constraints, the pre-visit SMSP-A questionnaire was not used for Study 2 participants.
   • Exclusion based on severe/extreme phobia (criteria iii) was determined by the investigators based on the participant’s self-reported fear during the initial contact/scheduling phase.

Phase 2: Participant Arrival and Setup (Common to Both Studies)

1. Arrival and Consent:
   • Upon arrival at the lab, the participant received detailed information about the study’s purpose, procedure, duration, potential risks/discomforts, and data handling.
   • The participant signed an informed consent form if they agreed to proceed.
2. Exclusion Check:
   • Investigators confirmed general exclusion criteria: participant is 18 years or older, does not have a diagnosed psychiatric anxiety disorder, and is capable of understanding instructions and completing the experiment.
3. Biosensor Application and Setup (See Figure 3.1):
   • The investigators prepared and applied the biosensors:
     • BTE-EEG: Five Ag/AgCl electrodes were attached. Two active electrodes behind the left ear, two active electrodes behind the right ear, and one reference electrode (Fpz) on the forehead. A ground electrode was placed on the neck (likely mastoid or clavicle). Electrode sites were likely prepped (cleaned, possibly abraded slightly) and conductive gel/paste applied for good contact.
     • PPG: The g.SpO2 sensor was placed on the participant’s ring finger.
     • GSR: Two g.Sensor electrodes were attached to the index and middle fingers of one hand (likely the non-dominant hand).
   • Hardware Connection: All sensor leads were connected to the portable TMSi Mobita amplifier.
   • Backpack: The Mobita amplifier and associated wiring were placed inside a backpack worn by the participant. This ensured portability and reduced interference during movement (especially relevant for Study 1).
4. Sensor Calibration and Signal Check:
   • The measurement system was turned on.
   • PPG/SpO2: The sensor was turned on, and investigators waited for a stable signal indication (Manual Trigger 1 & 2 in Tables 3.2/3.4). The system recorded the calibration sequence (alternating 0-100% signals) for later use in processing (Figure 3.5).
   • GSR: The sensor was calibrated, involving checking the response to the built-in 1 µS step signal (Manual Trigger 3; Figure 3.6).
   • EEG signal quality was likely visually inspected for excessive noise or bad channels at this stage.

Phase 3: Baseline Measurements

1. Alpha Activity Validation Test (Common to Both Studies):
   • The participant sat comfortably.
   • They performed a standardized task: alternating 30 seconds with eyes closed and 30 seconds with eyes open, repeated three times (total 3 minutes).
   • Manual triggers marked each eyes-open/closed transition and the end of the test (Triggers 4-10).
   • Purpose: To record baseline alpha activity changes, validating the EEG setup’s ability to detect known physiological responses.
2. Main Baseline Recording:
   • Study 1: The participant remained seated and was instructed to relax and stay still for 3 minutes. No VR headset was worn. Manual triggers marked the start and end (Triggers 11 & 12 in Table 3.2).
   • Study 2: The participant was instructed to stand up. They put on the Meta Quest 3 VR headset, which likely displayed a neutral view (e.g., passthrough for arachnophobia, simple scene for acrophobia). They stood relatively still for 3 minutes. Manual triggers marked the start and end (Triggers 11 & 12 in Table 3.4). This difference accounts for the standing posture during the Study 2 exposures.

Phase 4: VR Exposure Sessions

1. Scenario Order:
   • The participant was first exposed to the VR scenario (Arachnophobia or Acrophobia) they had reported fearing least.
2. VR Headset Preparation:
   • The participant put on the Meta Quest 3 headset (if not already on for Study 2 baseline). Hand controllers were given.
   • Video recording of the VR view (or participant) might be initiated (mentioned in Table 3.1).
3. Exposure - Scenario 1 (Least Feared):
   • Manual trigger indicated the start of exposure conditions (Trigger 13).
   • Study 1 (Gradual, User-Controlled):
     • Arachnophobia (AR): Participant progressed through the conditions shown in Fig 4.4 (box -> small free -> multiple sizes -> closer -> large) by pressing a controller button when ready. Automatic triggers logged condition changes.
     • Acrophobia (VR): Participant progressed through conditions in Fig 4.5 (edge -> look down -> plank -> gap -> air) by pressing a button. Avatar mirrored movements. Automatic triggers logged condition changes.
     • Reset Option: Participants could reset if anxiety became too high.
   • Study 2 (Alternating, Automatic):
     • Participant remained relatively stationary.
     • Arachnophobia (AR): Conditions shown in Fig 4.6 (small -> grows -> shrinks -> 2 spiders -> 1 small -> large furry) transitioned automatically every 30 seconds. Automatic triggers logged changes.
     • Acrophobia (VR): Conditions shown in Fig 4.7 (low/wide -> high/wide -> low/wide -> high/narrow -> low/narrow -> high/air -> low/ground) transitioned automatically every 30 seconds. Automatic triggers logged changes.
   • Manual trigger indicated the end of all exposure conditions for the scenario (Trigger 14).
4. Recovery Baseline (Study 2 Only):
   • Immediately following the end of exposure conditions, the participant remained standing and wearing the headset for a 1-minute recovery period.
   • Manual triggers marked the start and end of recovery (Triggers 15 & 16 in Table 3.4).
5. Break and Preparation for Scenario 2:
   • A short break was provided.
   • Study 1: Sensors might be checked/recalibrated, and the baseline procedure (Step 10 - seated) was repeated to account for any baseline shift caused by the first exposure.
   • Study 2: Sensors checked, participant prepared for the second scenario (likely remained standing). The baseline (Step 10 - standing) was repeated.
6. Exposure - Scenario 2 (Most Feared):
   • The procedure from Step 13 was repeated for the scenario the participant feared more, following the specific protocol for either Study 1 or Study 2.
7. Final Recovery Baseline (Study 2 Only):
   • The 1-minute recovery baseline (Step 14) was repeated after the second scenario.

Phase 5: Conclusion

1. Equipment Removal:
   • The VR headset and all biosensors were carefully removed from the participant. Manual trigger marked this (e.g., Trigger ‘Unmount equipment’).
2. Debriefing:
   • The participant was thanked for their time and contribution.
   • Investigators likely checked on the participant’s well-being and answered any questions they might have had.
3. End of Session:
   • Final manual trigger marked the absolute end of the recording/session (e.g., Trigger ‘Finish’).
   • Raw data files (.Poly5), manual trigger logs, automatic trigger logs (from VR), and any observational notes were saved for later processing.

Q&A

Why focus on the EEG-GSR relationship, not PPG?

Why assume beta waves, not alpha waves?