DrawTalking: Towards Building Interactive Worlds by Sketching and Speaking

Karl Toby Rosenberg, Rubaiat Habib Kazi, Li-Yi Wei, Haijun Xia, Ken Perlin

Abstract: We introduce DrawTalking, a prototype system enabling an approach that empowers users to build interactive worlds by sketching and speaking. The approach emphasizes user control and flexibility, and gives programming-like capability without requiring code. An early open-ended study shows the mechanics resonate and are applicable to many creative-exploratory use cases, with the potential to inspire and inform research in future natural interfaces for creative exploration and authoring.

Video Presentation: https://www.youtube.com/watch?v=r3F9vLH3tjM

Demos Video Figure: https://youtu.be/9NGgctM7eVA?si=Yndz7i-9H_Yy0FEu

Full-length Paper Pre-print: https://arxiv.org/abs/2401.05631

Short paper ACM DL: https://dl.acm.org/doi/10.1145/3613905.3651089

Leave a Comment

Your email address will not be published. Required fields are marked *