SPIDERS+: A Light-Weight, Wireless, and Low-Cost Glasses-Based Wearable Platform for Emotion Sensing and Bio-Signal Acquisition
Video
Team Information
Team Members
Jingping Nie, PhD Candidate, Department of Electrical Engineering, School of Engineering & Applied Sciences (SEAS), Columbia Engineering
Yanchen Liu, PhD Candidate, Department of Electrical Engineering, School of Engineering & Applied Sciences (SEAS), Columbia Engineering
Faculty Advisor: Xiaofan (Fred) Jiang, Associate Professor, Department of Electrical Engineering, School of Engineering & Applied Sciences (SEAS), Columbia Engineering; and Chair, Data Science Institute Smart Cities Center
Abstract
We present a System for Processing In-situ Bio-signal Data for Emotion Recognition and Sensing (SPIDERS+) -- a low-cost, wireless, glasses-based platform for continuous in-situ monitoring of user's facial expressions (apparent emotions) and real emotions. We present algorithms to provide four core functions to detect eye shape and eyebrow movements, pupillometry, zygomaticus muscle movements, and head movements, using the bio-signals acquired from three non-contact sensors (IR camera, proximity sensor, and IMU). SPIDERS+ distinguishes between different classes of apparent and real emotion states based on the aforementioned four bio-signals. We prototype advanced functionalities including facial expression detection and real emotion classification with a facial expression detector based on landmarks and optical flow that leverages changes in a user's eyebrows and eye shapes to achieve the average true positive rate up to 83.87%, as well as a pupillometry-based real emotion classifier with higher accuracy than existing low-cost wearable platforms that use sensors requiring skin contact. In addition, SPIDERS+ also includes optional contact-based sensors such as photoplethysmography (PPG) and multi-channel electroencephalogram (EEG) to monitor other bio-signals. SPIDERS+ costs less than $20 to assemble and can continuously run for up to 9 hours before recharging. We design, implement and evaluate accompanying desktop and smartphone applications to demonstrate that SPIDERS+ is a truly wireless and portable platform that can impact a wide range of applications, where the knowledge of the user's emotional state is critical.
Contact this Team
Team Contact: Jingping Nie (use form to send email)
