Let’s get weird with TouchDesigner! This workshop is about designing and speculating playful interactions that use face, body, and gestures as control interface. We'll work with TouchDesigner’s MediaPipe plugin implemented by Torin Blakensmith as a technical foundation, while exploring interactive, graphic, and interface design.
An introduction to perhaps unorthodox ways to think about novel game controls and screen-based interactivity, we'll prototype gestural interfaces that could exist in alternative presents or plausible futures with TouchDesigner, with a focus on embodied experiences and a 2D collage approach to making visuals.
For more workshops like this, visit Tandon at The Yard's 2025 Summer Workshops page.
Who Is This For
- Creative coders, VJs, generative-art fans exploring non-traditional inputs
- Game & interaction designers seeking novel, embodied control schemes
- Performers & media artists wanting camera-based triggers for shows or installations
- Basic TouchDesigner navigation is helpful; no prior computer-vision experience required.
Materials
- Laptop with the latest TouchDesigner (non-commercial license OK)
- Download Torin Blakensmith’s MediaPipe for TD plug-in (link provided after sign-up)
- Built-in or USB webcam (laptop camera is fine)
10:00 – 10:30
Welcome & Ice Breaker
Quick introductions, ice-breaker that gets us moving around in the space
10:30 – 11:00
Speculative Interface
Alternative game control, OS design, and desktop performance - exploring different modalities of design and interactivity
11:00 – 12:00
MediaPipe Setup & 2D Basics in TD
Plugin installation, hand/face tracking fundamentals, layering techniques in TD that feel more like graphic design
12:00 – 12:30
Break
12:30 – 1:30
Build Session 1 – Gesture-Controlled Pixel Photo Booth
Follow-along example, building a pixel art style photobooth controlled by hand movements, working with real-time composition
1:30 – 2:15
Speculative Everything
Designing for alternative presents, relationship between tools and art, a critique of digital aesthetics, and a little bit AI for creative projects :)
2:15 – 2:30
Break
2:30 – 3:00
Design Workshop
Brainstorming and rapid feedback for your project ideas
3:00 – 4:30
Build Session 2 – Project Jam
Open experimentation time, build your project, debugging
4:30 – 5:00
Demo, share, play!
Outcomes
Participants will:
- Install & run MediaPipe inside TouchDesigner for hand/face/body tracking.
- Prototype at least one gesture-controlled visual experience.
- Apply 2-D collage techniques for quick, graphic-style compositions.
- Develop speculative concepts that rethink how we perform with screens.
- Take away a project file, design sketches, and resources for further exploration.
Presenter
Viola He (@v10101a) is a Shanghai/NYC based educator, performer, interdisciplinary artist, and culture organizer, exploring pathways towards alternative structures, systems and interfaces. They create immersive experiences, installations, XR theatre, performances and performance systems using algorithmic approaches to enhance, alternate, and obfuscate audio-visual assets, investigating new sonic and visual aesthetics across digital and physical mediums.
Viola is an Assistant Professor of Interactive Media Art at NYU Shanghai, as well as an organizing artist in leaderless collective Livecode.NYC, in which they continue to research, organize, teach, and discuss the relationship between community and technology. They have performed in SXSW, Under the Radar Festival, Festival Tsonami, Hong Kong International Poetry Festival, as well as various clubs and venues in Shanghai, Tokyo, Los Angeles, SF, NYC, and more
