Penn State University scientists have invented a sticker that successfully identifies human emotions with nearly 90% accuracy. Designed to interpret several biological signals, including temperature and heart rate, the new sticker could help medical professionals treating people with difficulty communicating or psychiatric professionals trying to interpret and treat complex emotional conditions.
“This is a new and improved way to understand our emotions by looking at multiple body signals at once,” said Huanyu “Larry” Cheng, an Associate Professor of Engineering Science and Mechanics at Penn State and lead author on the paper outlining the team’s work in a statement.
The Growing Intersection Between Technology and Human Emotions
When novelty items like mood rings or emotion bracelets first arrived in the mid-twentieth century, they were relatively simple devices that changed color depending on changes in body temperature. While a fun distraction, these devices were far from accurate in interpreting actual human emotions.
More recently, scientists have been exploring ways to analyze and decipher individual emotional states beyond simply asking patients how they feel. Some have looked at the role that the food we eat plays in emotions, while others are exploring emotions at their source. More practical attempts to integrate technology and human emotions have included designing an app to reduce negative thoughts, using brain scans to read people’s minds, or even attempts at using psychology to defend against cyberattacks.
In a new study published in Nano Letters, Penn State scientists have taken these approaches to a new level by measuring physical responses to interpret emotional states. Unlike previous efforts to read human emotions that relied on the electrical properties of skin or facial expressions, this approach uses several disparate inputs to create the most complete and accurate interpretation of emotional states.
“Relying only on facial expressions to understand emotions can be misleading,” Cheng explained. “People often don’t visibly show how they truly feel, so that’s why we’re combining facial expression analysis with other important physiological signals, which will ultimately lead to better mental health monitoring and support.”
Wide Range of Inputs Aids Accurate Emotion Analysis
According to a statement from the research team behind the novel invention, their approach involves a set of sticker-like patches that attach directly to the skin. Made with layers of flexible metals like platinum and gold cut into wave-like shapes, the device can maintain performance and conductivity when stretched or pulled.
The device was also built with several layers of materials that can change the flow of electricity when temperatures change. Along with body temperature, the patch is equipped with a suite of sensors designed to track humidity, heart rate, and blood oxygen levels, which the researchers note “are associated with emotional states.” These include hollow nanotubes made of carbon atoms that can detect humidity directly.
To complement these biometric sensors, the system also incorporates a separate patch attached directly to the face. By collecting facial expression data and analyzing it in combination with physiological data, the team says their system can “better distinguish between genuine emotions and acted ones.” Notably, the team designed their device so that the sensors all act separately and are not affected by the other incorporated sensors. This design increases the chances of a more accurate emotional diagnosis.
“We’ve engineered this device to measure these different signals independently, without them interfering with each other, providing a much clearer and more accurate picture of what’s happening beneath the surface,” said Libo Gao, co-corresponding author on the paper and an associate professor at Xiamen University.
Finally, along with its suite of sensors, the team equipped their emotion reading patch with a wireless interface to send data directly to the cloud for a clinician to examine for rapid evaluation. The team says the device does not collect any personal data, helping users maintain privacy.
Device Successfully Deciphers Emotions with 88.83% Accuracy
To test their device’s accuracy, the researchers enlisted eight volunteers. Each volunteer was asked to “perform” six common facial expressions depicting distinct, individual emotional states: happiness, surprise, fear, sadness, anger, and disgust. After performing each emotional state 100 times, the team fed the data into a specially programmed AI model that was specially designed to spot and interpret outward signs of human emotion.
When the team asked three additional volunteers to help confirm their work by performing the six emotions, the device could classify them with 96.28% accuracy. Still, the team wanted to see how their system would interpret actual emotions instead of performed emotions. This meant showing the same volunteers a series of video clips designed to elicit specific emotional responses.
According to the team’s statement, the device accurately identified 88.83% of the actual emotional states experienced during the video clips. The device’s integrated sensors confirmed this data, showing that “the psychological responses were consistent with known links between emotions and psychological reactions, such as increases in skin temperature and heart rate during surprise and anger.”
Device Could Fill ‘Gaps in Care’, allow for ‘Proactive Support’
Although the device is not commercially available, the team says it could be valuable in several clinical settings. For example, it could help caregivers communicate with patients who cannot talk. The device could also assist patients who are struggling with mental health issues or honesty.
“This technology has the potential to help people who are struggling with their mental health, but maybe aren’t being fully honest with others or even themselves about how much they are struggling,” said Yangbo Yuan, co-author on the paper and doctoral student at Penn State studying engineering science and mechanics.
“This sensor can serve a vital function in bridging gaps in access to care,” Cheng added. “Given the rising stress levels in modern society, the ability to monitor emotions can provide early indicators of debilitating conditions and allow for proactive support.”
The team also believes their device could help bridge “cultural or social gaps” that make it difficult for caregivers to interpret their emotional state. The team also sees potential for preventative care. According to Cheng, by keeping track of biological signals associated with human emotions, “it could be possible to detect problems like anxiety or depression earlier in their progression.”
“While still in the research and development phase, this device is a significant step forward in our ability to monitor and understand human emotions, potentially paving the way for more proactive and personalized approaches to mental health care,” Cheng said.
Christopher Plain is a Science Fiction and Fantasy novelist and Head Science Writer at The Debrief. Follow and connect with him on X, learn about his books at plainfiction.com, or email him directly at christopher@thedebrief.org.