Making Music as If Drawing
SOUND AND DRAW is a sound interface developed to generate sound from the act of drawing lines, reconnecting bodily movement and auditory experience. Apple Pencil pressure is linked to volume, while tilt and rotation can shape timbre and effects such as reverb. These mappings connect drawing gestures to changes in sound. Rather than functioning simply as a musical instrument app or drawing tool, it explores an experience in which drawing itself is transformed into sound as a trace of thought and bodily presence.
The project investigates UI design, sound logic, drawing expression, and mechanisms for recording and loop playback as one integrated system, searching for a new sense of performance in digital environments. In particular, it aims for interactions in which the fading trajectory of lines and the repeated playback of sound in sync with strokes make the presence of the body perceptible through both vision and hearing.
Drawn lines are recorded as elements on a single timeline, synchronized with sound.
On the timeline, each line can be visually checked in relation to the timing of its sound, allowing its position to be adjusted, duplicated, or deleted. When a line on the screen is tapped, the corresponding item on the timeline is highlighted, enabling the user to move back and forth between the drawn trace and its temporal structure while editing.
Throughout the development process, I repeatedly examined how different ways of drawing affect the impression of sound. I explored a structure in which visual differences—such as the length, speed, overlap, and rhythm of lines—directly emerge as differences in the performance experience.
In the production process, I refined the relationship between drawing and sound based on feedback from a wide range of people, including artists from Japan and abroad, musicians, and users with little experience in music production.
Rather than simply adding more functions, I adjusted the density of the UI, the way lines fade, and how they appear during loop playback, designing an experience in which drawn traces remain as memories of sound.
Type
Personal Project / Research
Year
2026
