gpTea
intergenerational storytelling through LLM tea making
2023
For: MIT 4.043 Designing Intelligence
Role: Designer, Hardware prototyping, microcontroller programming
Partner: Kelly Fang (MIT)
Featured on:
Overview
We introduce “gpTea,” a generative story-making tea set that uses large language models to transform tea drinking into an interactive storytelling experience. Unlike traditional digital AI interfaces, gpTea explores Large Language Objects (LLOs) interaction paradigms through physical AI objects. By embedding AI in a tea set, gpTea brings the user’s experience with AI to the speed, ritual, and affordances of tea drinking.
It prompts users to asynchronously share untold stories and memories, fostering reflective communication. The teapot engages users in dialogue, asks insightful questions, and connects their stories, revealing hidden insights.
Additionally, gpTea employs a diffusion model to generate and display real-time imagery. We detail our design methodologies, prototype fabrication, and initial user feedback, showcasing gpTea as a case study to embedded-AI objects. Our work highlights the potential for more physical,
Design Space
In characterizing gpTea we use two key dimensions in our design space: one to represent the range of AI products from highly anthropomorphized robots to agentless tools, and the other to distinguish between embedded and mediated touchpoints.
Because gpTea closely resembles the original object and preserves its original interactions (design rule #1), its touchpoint is co-located and facilitates direct manipulation. For example, lifting the cup will start recording and placing it on the tray will end recording. However, because the LLM does not speak as if it is a teapot (design rule #5), it is not anthropomorphic yet it is not completely agentless as the teapot has a voice that can draw connections and tell stories. At the same time, gpTea may be regarded as a communication tool and memory storing device.
Diagram of Design Space with Anthropomorphic to Tool on the x axis and Decoupled Tochpoints to cCo-located Touchpoints in y axis
Finding the right interaction
Our design process began with the creation of a user journey flow chart depicting a traditional Chinese tea conversation between two individuals. We identified several actions that facilitated communication between the participants and derived generalized insights on how these interactions could be adapted for gpTea.
Interestingly, we observed that the arrangement and sequence of the objects played a crucial role in facilitating verbal and nonverbal exchanges, revealing nuanced behaviors such as tapping fingers on the table, pouring tea for others, bringing the cup close to the body, and lifting it up and down. These findings became our inspiration for our key interactions.
Embedded Touchpoints
The integration of a digital interface with the physical form factor of a tea set is epitomized by the teacup, which is equipped with an embedded screen and serves as the focal point of user interaction. This setup facilitates the rituals of tea drinking, story sharing, and digital storytelling. Our design focuses on creating a seamless and hands-on experience when interfacing with a LLM, enabling physcial manipulation of the digital elements rather than relying solely on verbal commands. This approach enhances user engagement by merging traditional and familiar physical interactions with digital functionalities.
The cup, chosen as the center of all interactions, adheres to our design rules, perfectly functioning as a teacup with a design language closely following that of a traditional cup. The system employs a single, multi-modal interface that dynamically switches functionality based on the user’s interactions and the narrative flow. It serves as both an input and output device, being the sole touchpoint with the user.
AI + Interaction Pipeline
In designing gpTea, we opted to “chain” several LLM prompts together. We broke the complex task of facilitating conversation and storytelling into subtasks, each mapped to a dedicated natural language prompt. Figure 18 shows how each of the six steps in the user journey corresponds to an input or output from a LLM prompt.Diagram showing integration of inputs and outputs of prompt chaining with user journey
Hardware Design
gpTea’s hardware comprises three components: the cup, the tray, and the teapot. The cup features a 1.28-inch round display, an ESP32 microcontroller, a microphone, and a LiPo battery (refer Figure 8, 9). The body of the cup incorporates a magnifying lens into its geometry, blending the display with the liquid content once the cup is filled (refer Figure 10, 11). The tray houses a speaker, a stepper motor that manipulates the teapot’s rotation via a timing belt, and a rotary encoder equipped with two haptic motors for user feedback and one Hall effect sensor for detecting the presence of the cup. All the electronics housed in the tray are connected to an Arduino Nano microcontroller.
Explode View
Prototypes
Initial concept mockup
Form finding
Functioning cup display
LED tray
Full works-like prototype
Motor and timing belt testing
Teapot integration testing
New tray design
All prototypes
Rotary encoder testing
New teapot with machined bearing pin
Teapot mounted on stem
Pour Test