Martin Crowley
UX designer

The Story of
GraspAbleton

Exploratory sketching - Laptop music performances can be dull!

Exploratory sketching - Laptop music performances can be dull!

The Problem

Laptop-based music performance has several pitfalls for both performer and the audience.

  • Traditional controls such as knobs, faders and buttons do not provide expressive means of control for the performer.

  • Audiences do not find the spectacle of "a guy on stage with a laptop" visually stimulating.

Research focused on exploring solutions for Ableton Live, the most popular software for electronic music performance.

My roles

  • User Research

  • Hardware UX Design

  • Product Design

  • Development

Research
Gig setup of one interviewee. Notice that laptop is hidden under table.

Gig setup of one interviewee. Notice that laptop is hidden under table.

User Interviews

To understand if laptops created notable performance issues, I conducted interviews with three "laptop musicians".

This produced some rich insights into how musicians themselves perceived the laptop:

  • Laptops give the impression of a DJ set rather than a live performance.

  • A laptop does not have the same visual appeal as a dedicated musical instrument.

Clearly, laptops are not a visually thrilling spectacle. An improved design would need to appeal more to a watching audience.

Survey was shared to a Facebook Ableton user group

Survey was shared to a Facebook Ableton user group

User Survey

To delve deeper into both the performer and the audiences' perception of laptop performance, a survey was shared via Facebook.

This provided some valuable insights:

  • The laptop screen can distract musicians from their performance.

  • Laptops and controllers obscure the performer's gestures from the audience forcing them to exaggerate movements in order to be seen.

Future designs would focus on a screen-less means of control that makes movements openly visible.

Clockwise from left: ReacTable, Skal and Ideo's C60 Music Player

Clockwise from left: ReacTable, Skal and Ideo's C60 Music Player

Competitive Benchmark

To understand the possibilities afforded by TUIs, I began reviewing past successful TUIs and identified several benefits.

Benefits for the performer:

  • The open tabletop design can provide the performer with more freedom to move.

  • Novel means of control where virtually any object can become part of the performance.

Benefits for the Audience:

  • The tabletop design opens up the performance more to a watching audience.

  • The "graspable" objects can form a highly visually stimulating element within the performance.

Graphical User Interface vs Tangible User Interface

Graphical User Interface vs Tangible User Interface

Literature Review

I did not have time to interview an audience, so I instead reviewed past academic studies on audience perception of laptop music.

Key findings:

  • Audiences can find laptop performance inauthentic: "Is the performer checking his/her emails up there?"

  • Laptops are more associated with mundane administrative tasks than musical ones.

Further reading suggested that 'Tangible User Interfaces' (TUIs) - graspable interfaces that physically represent digital media - offer fertile ground to explore solutions.

Analysis
Some issues highlighted by research and how TUIs can address these

Some issues highlighted by research and how TUIs can address these

A Way Forward

Why?

I began synthesising the research conducted to date by summarising issues and examining if a TUI based approach could alleviate each one.

I established that TUIs could help the user to achieve their goals as follows:

  • Make the performer's movement overt.

  • Create a spectacle for the audience.

  • Omit the laptop from view.

I could now begin to earmarked certain technologies that should be explored, namely:

  • Camera tracking.

  • Wireless technologies such as RFID/NFC.

Sketching in Hardware: Several quick and dirty early prototypes allowed me to explore the research area.

Sketching in Hardware: Several quick and dirty early prototypes allowed me to explore the research area.

Methodology

I next needed to establish the methods by which the problem areas would be explored.

With platforms like Arduino allowing for rapid prototyping, I decided to adhere to a 'Sketching in Hardware' approach whereby:

  • Sketches and concepts would be interactive in nature.

  • These sketches would be continually user-tested whenever and however possible.

Design would focus on 3 key main functions within electronic music performance:

  • Changing musical patterns.

  • Live mixing.

  • Soloing/muting.

Iterative Design & Testing

RFID tags taped to different faces of dice trigger different drum patterns in Ableton

Early Prototyping

To explore solutions to the three desired areas of functionality, basic modular prototypes were built.

In this example, the prototype explores how the different faces of dice can be used to playfully change drum patterns and create interesting rhythms.

Video posts on social media provoked discussion and critique

Video posts on social media provoked discussion and critique

Feedback by any means possible!

This was a time-sensitive project, so quick and cheap feedback on design concepts was sought at every opportunity.

Sharing videos on social media was a rich source of early feedback. These posts helped in choosing the best designs to proceed with.

Group testing session

Group testing session

Usability Testing

Basic working prototypes were put into the hands of people, regardless of musical ability. This gave a good indication of how engaging and usable they were. Also, observation and group discussion provided insights into new ways that the prototypes could be interacted with.

Prototyping explored usable objects and forming merging past prototypes into a system

Prototyping explored usable objects and forming merging past prototypes into a system

Mid Fidelity Prototyping

With basic several basic prototypes earmarked for inclusion, focus now shifted to merging them into a fully-fledged music control system.

In order to establish the types of objects that could be used in music performance, this phase also made use of playful objects.

The test candidates were given free reign to explore the performance space and objects

The test candidates were given free reign to explore the performance space and objects

User Testing - Mid Fidelity

In order to establish if they system was musically performable, I put the prototype into the hands of actual musicians.

Two testing sessions took place with Ableton performers as test subjects. Candidates were not told how they should use the system but instead allowed to freely interacting with a wide range of objects in whatever way they pleased. These sessions provided new insights into how the system could be 'played'.

Final Build
Digital drawings of of final design

Digital drawings of of final design

Digital & 3D Design

Sketches were transformed into more refined 3D designs, which helped to plan for subsequent digital fabrication.

Precision required: Laser cutting and 3D printing

Precision required: Laser cutting and 3D printing

Digital Fabrication

With designs finalised, digital fabrication techniques such as laser cutting and 3D printing aided in building quickly and precisely.

Top-down view of internal electronics

Top-down view of internal electronics

Electronics & Code

Electronic components such as RFID readers, buttons and cables that had been previously tested on breadboard were now permanently soldered.

The code was modified to accommodate eight RFID readers and microcontrollers all at once.

GraspAbleton's visual design matches that of Ableton

GraspAbleton's visual design matches that of Ableton

Visual Design Considerations

As a product intended to compliment Ableton Live, it made sense that its design should fall in line with Ableton's brand aesthetic. Thus, generous use of grey was incorporated.

The Ableton logo and font was also modified and adapted as the logo of GraspAbleton itself.

Product demonstration video

Final Product

The system was finalised in time for the graduate D.A.W.N. exhibition in the Department of Computer Science at the University of Limerick.

A brief video demonstration of the system was made, focusing on its functionality and its advantages for musical performance.

Reflection
Final evaluation - group collaboration and audience impressions

Final evaluation - group collaboration and audience impressions

Final Evaluation

The exhibition served as an excellent opportunity to learn more about the audience perception of a performance. This evaluation also produced some unpredicted insights:

Audience Perception:

  • The spectacle of the performance was generally lauded

  • While the functionality of the musical instruments was obvious, it was not clear what the dice did.

Unforeseen Insights:

  • System has massive potential as a collaborative musical tool.

  • System was particularly enjoyed by children.

Key Takeaways

  • There is much potential in marrying TUIs with music performance platforms such as Ableton Live.

  • There is value in a sketching in hardware approach to electronic musical instrument design, as long as prototypes are not complex or overly time consuming to build.

  • In a large scale project with a short time, appropriately targetted social media can be leveraged to get quick and dirty feedback on design concepts.