Sound design for communication and sonification, 6 credits
The course is planned to be given as a distance course using (primarily) Zoom for all sessions. The course will consist of lectures, seminars, or sessions via Zoom for the first half, and then a project period where groups of two (or more) course participants work with sonification (and visualization) of data. Also the project part of the course will also be performed in distance mode.
There is one example of sonification and visualization, with source code, further down if you are curious about sonification.
PhD course
Sonification, i.e., transformation of data into sound or mapping of data characteristics to sound parameters, is the auditory equivalent to visualization and can be used to communicate data, and support visual perception and visual analysis of data. Visualization can utilize visual parameters such as color, shape, size, and proximity for presenting data, and in sonification sound parameters like pitch, tempo, sound level, and harmonic intervals, can be used to present data in relation to visual parameters. Even though previous research has explored how data can be mapped to auditory parameters, there is a need to understand how sounds can be designed, what tools are available, and how these design choices and tools might affect the outcome.
This PhD level course provides the basis in sound design for communication and sonification. The course will be divided into two parts, where the first part focuses on sound design and communicative/sonification approaches, while the second part will focus on utilizing these concepts and approaches by interdisciplinary team projects.
I look forward to meet you at the course!
Unfortunately, I've been forced to postpone the course start until the academic year 2023 - 2024.
Prerequisites
To be eligible for the course, you need to have a basic knowledge in either:
- visualization
- sonification
- interaction design
and have a genuine interest in data communication, or work in a domain where multimodal data analysis is needed and wanted. Admission is by application only (see below); get in touch with the course organizer if you have any questions or consider applying.
Expected learning outcomes
- Familiarity with basic techniques for sound design
- Familiarity with communication-oriented sound design
- Experience of sonification of data
- Experience of sound design for sonification
Course contents
Sound design for context
- Communication-oriented design approaches
Jonas Löwgren (web) - Sound and interaction design/cognitive sciences
Mattias Arvola (web) - Sound for learning
Konrad Schönborn (web) - Social and cultural aspects of sound
Ingemar Grandin (web) - Sound design for communication
Niklas Rönnberg (mail, web) - Sonic interaction design
Sandra Pauletto (web) - Visualization (and sonification)
Wolfgang Aigner (web) - Sonification - with focus on audification and parameter mapping
Thomas Hermann (web) - Evaluation techniques
Camilla Forsell (web)
Basic sound design
- Sound recording
- Sound editing, and effects
- Sound synthesis methods
- Interactive sound synthesis environments
- Data mapping approaches
- Auditory parameters
- Sound transformations
- Spatial mapping and positional audio
Niklas Rönnberg (mail, web)
Course organization
The team then executes a sound design project under supervision, involving the following phases:
- identifying communicative intention (e.g., identification of challenges in data visualization or identification of aims with communication),
- ideation and synthetization of sonification concept,
- detailing of a partial prototype,
- evaluation of concept and prototype relative to communicative intention,
- revising of concept and prototype,
- prevention of concept and prototype.
The course consists of two parts.
The first part is a series of sessions (lectures, seminars, workshops, etc) introducing the basic concepts of sound design and communication-oriented sound design. The sessions are focused on overview, enabling the student to identify and learn more about specific concepts and techniques as called for in the specific project (i.e., the second part of the course).
The second part is interdisciplinary team project utilizing the content from the first part with the aim to develop a sonification of data. Each team defines a brief for a communication-oriented sound design project, ideally drawing on ongoing research represented within the team.
Examination

Active participation in the first part of the course.
Adequate contribution to team project demonstrating the fulfillment of the expected learning outcomes.
Admission
PhD students, or post-docs, who are interested in taking the course need to get in touch with the course organizer. Please provide a résumé and a brief motivation for your interest in the course. Please register your interest to participate no later than Month, Year; notification of admission will be given no later than Date.
Schedule
The course is planned for autumn/spring 2023-2024. The course start has been postponed due to my current workload/work situation.
Resources
Sound synthesis
- Sound design - the art and practice of creating sound
- Synthesis - a basic understanding
- The Basics - of Sound Synthesis
Graphic & UX tools
- GIMP - image editor
- Inkscape - vector graphics editor
- Processing - software sketchbook and coding
- p5.js - JavaScript library for creative coding
- d3.js - manipulating documents based on data
- Bootstrap - JavaScript for responsive web
- React - JavaScript library for building user interfaces
- Chart.js - JavaScript charting for designers & developers
- Unity - platform for creating interactive, real-time content
- Unreal engine - real-time 3D creation tool
Sound design tools
- SuperCollider - audio synthesis and algorithmic composition
- Csound - sound and music computing system
- Pure Data - visual programming language for multimedia
- Web Audio API - controlling audio on the Web
- Audacity - multi-track audio editor and recorder
Communication
Positional audio
- Surround sound - sound on a plane around a listener
- 3D sound localization - sound in a three-dimensional space around a listener
- Head-related transfer function - sound localization cues created via the geometry of a person’s head, face, and pinna
Food for thoughts
- The Sound of Data
A gentle introduction to sonification for historians by Shawn Graham. - Data Sonification
A new cosmic triad of sound, Molly Porter, NASA. - The Data Visualisation Catalogue
a project developed by Severino Ribecca. - The TimeViz Browser
A Visual Survey of Visualization Techniques for Time-Oriented Data by Christian Tominski and Wolfgang Aigner. - Text Visualization Browser
A Visual Survey of Text Visualization Techniques by Kostiantyn Kucher and Andreas Kerren.
Some references and inspiration
- The Sonification Handbook
Edited by Thomas Hermann, Andy Hunt, and John G. Neuhoff. - Data Sonification Archive
Curated by Sara Lenzi, Paolo Ciuccarelli, Yuan Hua, and Houjiang Liu. - International communicty for auditory display
The community involved in the International conference on auditory display. - ICAD Proceedings
Georgia Tech SMARTech repository. - Interactive Sonification
an emerging subfield of sonification and auditory display research.
One example
This example is in no way tested or evaluated, neither the visualization nor the sonification or the combination of these. One way of assessing the sonification could be by using the BUZZ questionnaire.
Project files
In this example, the average monthly temperature recordings in Stockholm, Sweden, the years 1950 to 2020, are both visualized and sonified.
The visualization is done in Processing (see link to the code to the right) uses a line graph to represent the monthly average, using a color map to further show the differences in temperature as the year progresses. It also represents the yearly average as a circle, similar to a scatter plot, using the same color map but slightly compressed to support perception of temperature changes. A linear trend line, based on regression analysis, are used with the same color map as the yearly averages to support interpretation of the temperature change. The trend line is also mapped to the playback speed.
The sonification is done in SuperCollider (see link to the code to the right), and uses five synth definitions: 1) the chord synth making a harmonious background chord (Csus4), 2) the bell synth plocking tones according to the yearly average temperature, 3) the sub synth making a low bass tone, 4) the noise synth used to emphasize higher temperatures, and 5) the output effect with additional reverberation. The montly averages, the yearly averages, as well as the temperature trend are mapped to multiple auditory parameters in this example, such as pitch, loudness, and timbre (for a fully detailed description see the links to the right).