Sound design for communication and sonification, 6 credits

PhD course

Sonification, i.e., transformation of data into sound or mapping of data characteristics to sound parameters, is the auditory equivalent to visualization and can be used to communicate data, and support visual perception and visual analysis of data. Visualization can utilize visual parameters such as color, shape, size, and proximity for presenting data, and in sonification sound parameters like pitch, tempo, sound level, and harmonic intervals, can be used to present data in relation to visual parameters. Even though previous research has explored how data can be mapped to auditory parameters, there is a need to understand how sounds can be designed, what tools are available, and how these design choices and tools might affect the outcome.

This PhD level course provides the basis in sound design for communication and sonification. The course will be divided into two parts, where the first part focuses on sound design and communicative/sonification approaches, while the second part will focus on utilizing these concepts and approaches by interdisciplinary team projects.

I look forward to meet you at the course!

Unfortunately, I've been forced to postpone the course start until the academic year 2023 - 2024.

/Niklas

Prerequisites

To be eligible for the course, you need to have a basic knowledge in either:

  • visualization
  • sonification
  • interaction design

and have a genuine interest in data communication, or work in a domain where multimodal data analysis is needed and wanted. Admission is by application only (see below); get in touch with the course organizer if you have any questions or consider applying.

Expected learning outcomes

  • Familiarity with basic techniques for sound design
  • Familiarity with communication-oriented sound design
  • Experience of sonification of data
  • Experience of sound design for sonification

Course contents

Basic sound design

  • Sound recording
  • Sound editing, and effects
  • Sound synthesis methods
  • Interactive sound synthesis environments
  • Data mapping approaches
  • Auditory parameters
  • Sound transformations
  • Spatial mapping and positional audio

    • Niklas Rönnberg (mail, web)

Course organization

The course consists of two parts.

The first part is a series of sessions (lectures, seminars, workshops, etc) introducing the basic concepts of sound design and communication-oriented sound design. The sessions are focused on overview, enabling the student to identify and learn more about specific concepts and techniques as called for in the specific project (i.e., the second part of the course).

The second part is interdisciplinary team project utilizing the content from the first part with the aim to develop a sonification of data. Each team defines a brief for a communication-oriented sound design project, ideally drawing on ongoing research represented within the team.

Examination

Active participation in the first part of the course.

Adequate contribution to team project demonstrating the fulfillment of the expected learning outcomes.

Admission

PhD students, or post-docs, who are interested in taking the course need to get in touch with the course organizer. Please provide a résumé and a brief motivation for your interest in the course. Please register your interest to participate no later than Month, Year; notification of admission will be given no later than Date.

Schedule

The course is planned for autumn/spring 2023-2024. The course start has been postponed due to my current workload/work situation.

Resources

Sound design tools

  • SuperCollider - audio synthesis and algorithmic composition
  • Csound - sound and music computing system
  • Pure Data - visual programming language for multimedia
  • Web Audio API - controlling audio on the Web
  • Audacity - multi-track audio editor and recorder

Communication

  • OSC - Open Sound Control
  • MIDI - Musical Instrument Digital Interface

Positional audio

Food for thoughts

Some references and inspiration

One example

In this example, the average monthly temperature recordings in Stockholm, Sweden, the years 1950 to 2020, are both visualized and sonified.

The visualization is done in Processing (see link to the code to the right) uses a line graph to represent the monthly average, using a color map to further show the differences in temperature as the year progresses. It also represents the yearly average as a circle, similar to a scatter plot, using the same color map but slightly compressed to support perception of temperature changes. A linear trend line, based on regression analysis, are used with the same color map as the yearly averages to support interpretation of the temperature change. The trend line is also mapped to the playback speed.

The sonification is done in SuperCollider (see link to the code to the right), and uses five synth definitions: 1) the chord synth making a harmonious background chord (Csus4), 2) the bell synth plocking tones according to the yearly average temperature, 3) the sub synth making a low bass tone, 4) the noise synth used to emphasize higher temperatures, and 5) the output effect with additional reverberation. The montly averages, the yearly averages, as well as the temperature trend are mapped to multiple auditory parameters in this example, such as pitch, loudness, and timbre (for a fully detailed description see the links to the right).