Tacit knowledge is a type of knowledge often existing in one’s subconscious and accessed through mechanisms like muscle memory. Such knowledge is pervasive in creative and technical practices yet remains difficult to observe or codify. In this talk, I’ll describe how sensor-based approaches can be used to support: (1) how we identify and understand tacit skills, and (2) how we can use these insights to design smart interactive tools.
In the first part of the talk, I will describe a case study documenting the “emotional rollercoaster” that typifies the tacit practice of debugging. Using physical sensors, biosignals, and activity logs of 17 programmers using Jupyter notebooks, I will show how codebook segmentation can be used to distill hundreds of hours of dense activity data into a visually parsable representation we term process chromatograms. In the context of programming, I will describe ways chromatograms can serve as an ethnographic descriptor, knowledge mining technique, an evaluation metric, or a design-informing visualization.
In the second part of the talk, I will describe the challenges of building sensor-driven interactions in a living glass workshop. I will explain the development of a sensing infrastructure capable of training ad-hoc convolutional neural networks (CNNs) on bespoke acoustic datasets and outline how this system navigates traditional challenges of ubiquitous computing. Through a set of smart tools, I will describe a vision for enabling information repositories of tacit knowledge to drive skill acquisition, distributed learning, and sustain creative development.
Dr. Cesar Torres is the director of The Hybrid Atelier and an Assistant Professor of Computer Science & Engineering at The University of Texas at Arlington. As a design researcher, Cesar synthesizes new media and craft theory into the software and hardware design of creative, tangible user interfaces. He has received multiple best paper awards at top-tier venues within HCI and has been recognized with the NSF and Adobe/GEM Consortium Graduate Fellowships and a Microsoft Research Dissertation Grant during his Ph.D. He serves on the program committees for ACM Designing Interactive Systems (DIS) and Creativity and Cognition (C&C), while previously serving on committees for ACM CHI and ACM TEI. He received his Ph.D. in Computer Science from UC Berkeley in 2019, and a B.A. in Art Practice and B.S. in Computer Science from Stanford University in 2013.