PhD Student Explores Emotional Biosensing Technologies
UC Berkeley School of Information faculty and students are doing incredible research at the intersection of technology and the social sciences. The unique environment of the I School encourages students to use their diverse backgrounds to take the lead in interdisciplinary areas and challenge the status quo.
Ph.D. student Noura Howell tells us about her research with emotional biosensory data.
Noura Howell is a Ph.D. student at the School of Information and a member of the BioSENSE Lab. Her focus is on exploring relationships with emotional biosensory data. Howell is passionate about biosensing technologies and imagines uses for them beyond attempts to calculate human emotion.
Howell’s research was partly inspired by her experience at MIT, where she studied software engineering. Howell spent time working as a software engineer, including a stint at Intel where her focus was in user interface design, and also worked in the MIT Media Lab, where she was exposed to research in the field and became interested in conducting her own research in graduate school.
Howell believes in engaging new technology while also working to understand its potential harms. Howell expressed that computer science often focuses on the technical aspects of developing new technologies, but falls short of evaluating their social implications. In contrast, the I School promotes an environment that supports her to consider both in her research on emotional biosensing tools and their social effects on individuals.
Recently, Howell had two papers accepted at CHI ’18: one on her research on clothing that prompts social interpretation of emotion, and one on representing laughter with physical media.
Ripple
Howell, along with her faculty advisor Prof. Kimiko Ryokai, I School alumna Prof. Laura Devendorf (Ph.D. 2016, now an Assistant Professor at the University of Colorado, Boulder), and co-authors Tomás Alfonso Vega Gálvez, and Rundong Tian, are the team behind Ripple, clothing that ambiguously and indirectly displays emotion (citation). With Ripple, the scholars are attempting to create something that differs from existing emotional biosensing tools on the market, which tend to represent people’s emotions in terms of discrete categories like “happy,” “sad,” or “stressed.”
Howell explains that other biosensing tools neglect the importance of context and culture when they represent people’s emotions in terms of discrete categories. For example, cheerfully saying “hi” to a coworker is a very different kind of happiness from seeing a child’s first steps, but both might be represented as “happy.”
Get Berkeley I School’s stories in your inbox
Join Medium for free to get updates from this writer.
“The paper explores how data, algorithms, and technology claim to ‘know’ how we feel, and how that can influence our social relationships and sense of self, for better or worse,” Howell said.
The paper explores how data, algorithms, and technology claim to ‘know’ how we feel, and how that can influence our social relationships and sense of self, for better or worse.
Ripple uses color-changing technology to display emotion in a subtle, indefinite way. This biosensing clothing uses skin conductance sensors that are loosely related to excitement, both positive and negative, to allow the wearer to interpret their feelings rather than trying to automatically categorize their feelings. The color changing display then becomes a resource for individuals to reflect upon their emotions.
Tangible Laughter
Another part of Howell’s research in biosensing is exploring the physical manifestation of laughter. Professor Ryokai, Elena Durán López, Noura Howell, Jon Gillick, and Prof. Bamman developed a machine learning algorithm that detects and measures an individual’s laughter through audio capture (citation). The algorithm quantifies the length and relative loudness of laughter, which is then translated into a tangible model of chocolate. When the number of instances of laughter increases, the chocolate is thicker, creating a three-dimensional chocolate “bar graph.”
The I School fosters interdisciplinary collaboration between faculty and students. Professor Ryokai and Howell focus on design, HCI and UX research, while Professor Bamman and his student, Jon Gilick, focus on data science, machine learning, NLP, and digital humanities.
The team envisions this as a celebratory representation of collected biosensory data; a person could potentially gift bars of chocolate that serve as a reminder of good memories. The project also included the ability to record a person’s laughter and then bottle it to save for later. Opening the bottle simply causes the recording of the laughter to play. In this way, emotional biosensing tools also have the potential to help us cherish moments.
CHI ’18
This research was published and presented at the ACM CHI Conference on Human Factors in Computing Systems. This conference brings together the best minds in research to discuss the latest in interactive technology. Howell presented on Ripple and Prof. Ryokai presented on the physical representation of laughter. “Presenting at this conference,” Howell said, “was a wonderful opportunity to reach a broad audience of researchers at the top of the field of Human-Computer Interaction, including designers, technologists, ethnographers, and social scientists.”
More Information:
Reported on by Nayaab Ahsan, Rhetoric ’19, UC Berkeley. Produced by Caitlin Appert-Nguyen.
