Using Autonomic Responses to Infer Emotional State

Project Collaborators (sorted by Institute): 
Section on Neurobiology of Learning and Memory, Laboratory of Neuropsychology, NIMH
Project Brief: 

Emotions play an important role in decision making. Emotions also influence our autonomic nervous system presumably through the amygdala and other regions of the brain. It has been hypothesized that feedback from autonomic nervous activity enhances learning and decision making. Well-learnt associations are used to predict or anticipate outcomes of related events, thereby increasing the chances of making a correct decision and ultimately increasing the deciding organism’s chances of survival. Animal research in decision making must account for emotional responses without the benefit of using language to probe the subject's experiences. Autonomic measures provide an indirect readout of emotional state, providing researchers insight into the animal's experiences (anticipation and receipt of rewards) during an experiment. 

According to neuroeconomic theory, the goal of the nervous system, in evolutionary terms, is to maximize the inclusive fitness of the organism. Recent studies in humans investigated the interplay among valuation, decision making and brain activity. However, animal studies that could provide more detailed studies of the brain are rare, hampered by the difficulty in making “clean” recordings and the difficulty in training cognitive valuation tasks.

Our NIMH collaborators had three primary aims: 1) devise a set of computational tools that automatically mitigate the effects of noise in autonomic recordings of ECG and pupil size, 2) develop tools to demonstrate that a rhesus monkey could learn arbitrarily-assigned values of visual symbols in a decision-making game, and 3) apply these tools to analyze the monkey autonomic nervous activity before and during acquisition of various rewards from the game. While NIMH had the resources to conduct the experiments (including the ECG and pupil size recordings), the nature of the experiment resulted in low quality signals with high noise levels.

SPIS staff developed signal processing algorithms to improve the signal quality of the pupil size and ECG data before analysis by our NIMH collaborators. In addition to reducing the noise to acceptable levels, we performed post-processing tasks such as pupil size data validation (e.g. eliminating trials in which a blink is detected) and heart rate variability analysis.

Publications based on this work include:

Visualization of learning task

The experiment requires the monkey to associate different symbols with varying amounts of rewards (e.g., juice). The monkey is seated before a computer screen and three buttons. The monkey presses the center button to start a trial and fixates on the center of the screen. After a variable amount of time, an image appears either to the right or left of the center. Then, after a second variable time delay another image appears on the opposing side. The monkey presses the left or right button corresponding to the image which is worth more. If the correct image is selected, the monkey receives the larger of the two possible rewards.

Example of ECG signal processing

A) Original ECG data corrupted by noise such as physical movement artifacts. b) Application of bandpass filtering removes the respiratory cycle from the ECG data. C) Application of the Hilbert Transform preferentially amplifies the RS wave for heart beat identification. D) Final heart rate based on the inverse of the inter-beat time interval.

Example of pupil data processing

A) Original pupil data contaminated with blinks (pupil size approaching zero). Blinks which occur during a trial (fixation period followed by image selection) invalidate that trial. B) Pupil data after removal of blink data while maintaining saccadic motion. Blinks are detected with a thresholding technique and distinguished from saccadic motion by analyzing the slopes of those deviations. Interpolation across each isolated blink results in an accurate representation of eye position, without blink contamination.