Earphone tracks facial expressions, even with a face mask
An ear-mounted device, called C-Face, could express emotions to online users without holding cameras in front of their faces – this comes as much of the world engages in remote work or learning.
The device continuously track full facial expressions by observing the contour of the cheeks - and can then translate expressions into emojis or silent speech commands.
It could also be used to direct a computer system, such as a music player, using only facial cues.
Because it works by detecting muscle movement, C-Face can capture facial expressions even when users are wearing masks.
The device consists of two miniature RGB cameras -- digital cameras that capture red, green and bands of light -- positioned below each ear with headphones or earphones. The cameras record changes in facial contours caused when facial muscles move.
Images are captured and reconstructed using computer vision and a deep learning model.
These reconstructed facial expressions represented by 42 feature points can also be translated to eight emojis, including "natural," "angry" and "kissy-face," as well as eight silent speech commands designed to control a music device, such as "play," "next song" and "volume up."
The device continuously track full facial expressions by observing the contour of the cheeks - and can then translate expressions into emojis or silent speech commands.
It could also be used to direct a computer system, such as a music player, using only facial cues.
Because it works by detecting muscle movement, C-Face can capture facial expressions even when users are wearing masks.
The device consists of two miniature RGB cameras -- digital cameras that capture red, green and bands of light -- positioned below each ear with headphones or earphones. The cameras record changes in facial contours caused when facial muscles move.
Images are captured and reconstructed using computer vision and a deep learning model.
These reconstructed facial expressions represented by 42 feature points can also be translated to eight emojis, including "natural," "angry" and "kissy-face," as well as eight silent speech commands designed to control a music device, such as "play," "next song" and "volume up."