Health and Behavior: Detecting Emotion in Multi-Party Conversations

Led by Jason O. Hallstrom, Ph.D.

Jason O. Hallstrom, Ph.D., serves as Director of Florida Atlantic University’s Institute for Sensing and Embedded Network Systems Engineering (I-SENSE), which coordinates FAU’s activities in Sensing and Smart Systems. He holds a B.S. in Systems Analysis, an M.A. in Economics, and M.S. and Ph.D. degrees in Computer and Information Science. His expertise is in the design, deployment, and management of wireless sensing infrastructure, with an emphasis on sensor platform design. His work is currently supported through the National Science Foundation, the Department of Energy, and NOAA (via subcontract from Coastal Carolina University) and was previously supported through the NSF, EPA, USDA, NASA, and other entities.  

This REU project is focused on characterizing human emotion through a combination of speech analysis, posture monitoring, and machine learning.

This REU project is focused on characterizing human emotion through a combination of speech analysis, posture monitoring, and machine learning. The research training experience will center around a toolkit for monitoring human emotion during multi-party conversations. The associated uses are broad, ranging from clinical use, to mediation support, to emotional intelligence training. REU participants will work as part of a small team to design, implement, and evaluate a prototype of this toolkit. Participants will gain a variety of skills, ranging from speech processing, to motion tracking, to neural networks and machine learning.