Industrial Designer Interaction Designer

TEAM

Kelly Fang (MIT)

TIMELINE

6 weeks (Mar – May 2024)

TOOLS

ESP-32 Vercel MongoDB GPT-3.5 Turbo Dall-E

Overview

Featured on ,

“gpTea” is a generative story-making tea set that uses large language models to transform tea drinking into an interactive storytelling experience.

Unlike traditional digital AI interfaces, gpTea explores Large Language Objects (LLOs) interaction paradigms through physical AI objects. By embedding AI in a tea set, gpTea brings the user’s experience with AI to the speed, ritual, and affordances of tea drinking.

Diagram shows how a collective memory is share between two users mediated by the device

It prompts users to asynchronously share untold stories and memories, fostering reflective communication. The teapot engages users in dialogue, asks insightful questions, and connects their stories, revealing hidden insights.

Additionally, gpTea employs a diffusion model to generate and display real-time imagery. We detail our design methodologies, prototype fabrication, and initial user feedback, showcasing gpTea as a case study to embedded-AI objects.

Motorized pouring

Design Space
In characterizing gpTea we use two key dimensions in our design space: one to represent the range of AI products from highly anthropomorphized robots to agentless tools, and the other to distinguish between embedded and mediated touchpoints.

Because gpTea closely resembles the original object and preserves its original interactions, its touchpoint is co-located and facilitates direct manipulation. For example, lifting the cup will start recording and placing it on the tray will end recording. However, because the LLM does not speak as if it is a teapot, it is not anthropomorphic yet it is not completely agentless as the teapot has a voice that can draw connections and tell stories. At the same time, gpTea may be regarded as a communication tool and memory storing device.

Diagram of Design Space with Anthropomorphic to Tool on the x axis and Decoupled Tochpoints to cCo-located Touchpoints in y axis

Diagram highlights design insights for interaction during traditional tea drinking.

Finding the right interaction
Our design process began with the creation of a user journey flow chart depicting a traditional Chinese tea conversation between two individuals. We identified several actions that facilitated communication between the participants and derived generalized insights on how these interactions could be adapted for gpTea.

Interestingly, we observed that the arrangement and sequence of the objects played a crucial role in facilitating verbal and nonverbal exchanges, revealing nuanced behaviors such as tapping fingers on the table, pouring tea for others, bringing the cup close to the body, and lifting it up and down. These findings became our inspiration for our key interactions.

User Journey Storyboarding with gpTea

Embedded Touchpoints
The integration of a digital interface with the physical form factor of a tea set is epitomized by the teacup, which is equipped with an embedded screen and serves as the focal point of user interaction. This setup facilitates the rituals of tea drinking, story sharing, and digital storytelling. Our design focuses on creating a seamless and hands-on experience when interfacing with a LLM, enabling physcial manipulation of the digital elements rather than relying solely on verbal commands. This approach enhances user engagement by merging traditional and familiar physical interactions with digital functionalities.

The cup, chosen as the center of all interactions, adheres to our design rules, perfectly functioning as a teacup with a design language closely following that of a traditional cup. The system employs a single, multi-modal interface that dynamically switches functionality based on the user’s interactions and the narrative flow. It serves as both an input and output device, being the sole touchpoint with the user.

A clock animation for loading and a wobbly circle for AI voice.

Flashing red dot for voice recording

Cup becomes a rotary dial when mounted on tray

Sequence shots of tea pouring into cup and coloring the displayed image

AI + Interaction Pipeline
In designing gpTea, we opted to “chain” several LLM prompts together. We broke the complex task of facilitating conversation and storytelling into subtasks, each mapped to a dedicated natural language prompt. Figure 18 shows how each of the six steps in the user journey corresponds to an input or output from a LLM prompt.

Diagram showing integration of inputs and outputs of prompt chaining with user journey

Hardware Design
gpTea’s hardware comprises three components: the cup, the tray, and the teapot. The cup features a 1.28-inch round display, an ESP32 microcontroller, a microphone, and a LiPo battery. The body of the cup incorporates a magnifying lens into its geometry, blending the display with the liquid content once the cup is filled. The tray houses a speaker, a stepper motor that manipulates the teapot’s rotation via a timing belt, and a rotary encoder equipped with two haptic motors for user feedback and one Hall effect sensor for detecting the presence of the cup. All the electronics housed in the tray are connected to an Arduino Nano microcontroller.

Explode View

Cup with a built in magnifying lens geometry to help blend the display

Lens not distoring (left); Lens distorting (right)

Prototypes

Initial concept mockup

Form finding

Functioning cup display

LED tray

Full works-like prototype

Motor and timing belt testing

Teapot integration testing

New tray design

All prototypes

Rotary encoder testing

New teapot with machined bearing pin

Teapot mounted on stem

Pour Test

Kevin Tang

Selected Works

About

CV