BACKGROUND
In the digital age, people increasingly tend to hide their true selves on social media, virtual communications, and in the metaverse, using filters, avatars, and carefully crafted digital personas to create idealized versions of themselves. This concealment provides individuals with a sense of security and control, but also creates a disconnect between inner emotions and outward expressions. In this highly fluid information era, are we expressing our true selves or hiding them?This project aims to explore this social phenomenon by utilizing TouchDesigner and MediaPipe for real-time emotion visualization. The system detects users' facial expressions through MediaPipe and generates dynamic digital masks in TouchDesigner, artistically representing their emotional states. Unlike the filters on social media that enhance or beautify reality, this project seeks to reveal the emotions intentionally concealed, externalizing hidden feelings through abstract and fluid visual effects.
SKETCHES
WORK IN PROGRESS
Demo of Visualizations
Mock Try Up
What's next?
Increase the interactivity and fun: Enable hand gesture interaction in TouchDesigner to manipulate the properties of the mask (such as color and lines), creating visual distortions.
Enhance visualization: Use 3D mapping to project onto a real mask, allowing the audience to experience a more immersive effect.
Debugging and user testing: Identify and fix issues through testing, followed by revisions based on the problems discovered.
Set Up
3D Model
A table holds a laptop and monitor, with a physical mask positioned at the front. The user stands in
front of the table, facing the laptop for facial detection. The monitor is placed to the side or mounted on
the ceiling. A projector casts visuals onto the mask.
Work in progress 2
Hand Interaction Ideation
White Mask:
Left Hand: The distance between the thumb and index finger controls particle size. Right Hand: The distance between the thumb and index finger affects how the mask appears to rotate or shift when viewed from different perspectives.
Red Mask:
Left Hand: Controls the outer axis, affecting the overall visual perspective of the scene. Right Hand: Controls the rotation angle of the particle system along a circular path. The more you open your hand, the larger the angle of rotation, creating a stronger visual sense of movement.
Green Mask:
Left Hand: Controls the transparency level of the visual element — the more the hand is opened, the stronger the glow. Right Hand: Controls the period (rhythmic cycle) of the particle’s animation based on audio input — Greater distance = longer period = slower rhythm. Smaller distance = shorter period = faster rhythm
Hand Interaction
Final Exhibition
Full Interaction