Exploratory sketching - Laptop music performances can be dull!
The Story of
GraspAbleton

The Problem
Laptop-based music performance has several pitfalls for both performer and the audience.
Issues
Traditional controls such as knobs, faders and buttons do not provide expressive means of control for the performer.
Audiences do not find the spectacle of "a guy on stage with a laptop" visually stimulating.
Research focused on exploring solutions for Ableton Live, the most popular software for electronic music performance.
My roles
User Research
Hardware UX Design
Product Design
Development

Gig setup of one interviewee. Notice that laptop is hidden under table.
Interviewing Users
I conducted interviews with laptop musicians to understand possible performance issues. This produced some rich insights.
The laptop is not synonymous with music:
“A laptop is too every day and miscellaneous to have a machine you use to do emails and online banking alongside a bunch of purpose-built music machines."
An audience can negatively perceive laptops on stage:
"We often hide the laptop during performances to prevent the audience from thinking the music was coming from a laptop, and therefore rid any potential suspicion that it could be just a DJ/mp3 show instead of a 'live' show.”
The laptop as a distraction:
“To distance ourselves from a focal point which may unnecessarily occupy our attention and distract ourselves from focusing on the gear”

Survey was shared to a Facebook Ableton user group
User Survey
A short survey was shared with a Facebook Ableton performer group. It consisted of both open-ended and likert scale questions. It received 36 responses.
Insights
The laptop screen distracts users from their performance.
Ableton controllers obscure the performer's gestures and actions from the audience.
Exaggeration of movement required to communicate actions to the audience.
The laptop on stage can be detrimental to the energy between the audience and performer.

Graphical User Interface vs Tangible User Interface
Literature Review
Scientific papers provided some interesting insights into the perception of laptops in music performance and possible workarounds.
Insights
Audiences may doubt the authenticity of laptop-based performances: "Is the performer checking his/her emails up there?"
Laptops are associated with mundane administrative tasks. Performances can be perceived as dull and uninspiring.
Tangible User Interfaces (TUIs) - graspable interfaces that physically represent digital media - offer fertile ground to explore solutions.

Clockwise from left: ReacTable, Skal and Ideo's C60 Music Player
Competitive Analysis
Examining past Tangible User Interfaces helped to get a picture of what was possible.
Benefits of TUIs
An open tabletop design can provide the performer with more freedom to move.
A tabletop design also opens up the performance more to a watching audience.
Virtually any object that the performer chooses can become part of the performance.
These objects can form a highly visually stimulating element within the performance.

Some issues highlighted by research and how TUIs can address these
A Way Forward
Goals
Make the performer's movement overt.
Create a spectacle for the audience.
Omit the laptop from view.
Technologies to explore
Camera tracking and wireless technologies such as RFID/NFC open huge possibilities!

Sketching in Hardware: Several quick and dirty early prototypes allowed me to explore the research area.
Methodology
Design would focus on the possibilities afforded by Tangible User Interfaces.
With microcontrollers such as Arduino allowing one to quickly build quick and dirty interactive prototypes, design followed a 'Sketching in Hardware' approach.
Sketching in Hardware
Sketches and concepts would be interactive in nature.
These would be continually user-tested whenever possible.
Design would focus on 3 key main functions within electronic music performance
Changing musical patterns
Live mixing
Soloing/muting
RFID tags taped to different faces of dice trigger different drum patterns in Ableton
Prototyping - Round 1
Basic modular prototypes were built to explore solutions to the three desired areas of functionality.
In this example, the prototype explores how the different faces of dice can be used to playfully change drum patterns and create interesting rhythms.

Video posts on social media provoked discussion and critique
Feedback by any means possible!
This was a time-sensitive project, so quick and cheap feedback on design concepts was sought by any means.
Sharing videos on social media was a rich source of early feedback.
These posts helped in choosing the best designs to proceed with.

Group testing session
User Testing - Round 1
Basic working prototypes were put into the hands of people, regardless of musical ability.
This gave a good indication of how engaging and usable they were. Also, observation and group discussion provided insights into new ways that the prototypes could be interacted with.

Prototyping explored usable objects and forming merging past prototypes into a system
Prototyping - Round 2
With basic several basic prototypes earmarked for inclusion, focus now shifted to merging them into a fully-fledged music control system.
DJ-ing with Objects
A DJ-ing prototype was built to playfully explore how different objects that could potentially be used in a music performance system.

The test candidates were given free reign to explore the performance space and objects
User Testing - Round 2
It was now time to put a performable music system in the hands of actual musicians.
Two testing sessions took place with Ableton performers as test subjects.
Each was told how the system worked, however, they were not told how they should use the system and allowed to freely interacting with a wide range of objects in whatever way they pleased.
These sessions provided new insights into how the system could be 'played'.

Digital drawings of of final design
Digital & 3D Design
Sketches were transformed into more refined 3D designs, which helped to plan for subsequent digital fabrication.

Precision required: Laser cutting and 3D printing
Digital Fabrication
With designs finalised, digital fabrication techniques such as laser cutting and 3D printing aided in building quickly and precisely.

Top-down view of internal electronics
Electronics & Code
Electronic components such as RFID readers, buttons and cables that had been previously tested on breadboard were now permanently soldered.
The code was modified to accommodate eight RFID readers and microcontrollers all at once.

GraspAbleton's visual design matches that of Ableton
Visual Design Considerations
As a product intended to compliment Ableton Live, it made sense that its design should fall in line with Ableton's brand aesthetic. Thus, generous use of grey was incorporated.
The Ableton logo and font was also modified and adapted as the logo of GraspAbleton itself.

Finished system on display
Final Product
The system was finalised in time for the graduate D.A.W.N. exhibition in the Department of Computer Science at the University of Limerick.
Product demonstration video
Product Demonstration
A brief video demonstration of the system was made, focusing on its functionality and its advantages for musical performance.

Final evaluation - group collaboration and audience impressions
Final Evaluation
The exhibition served as an excellent opportunity to learn more about the audience perception of a performance. This evaluation also produced some unpredicted insights.
Audience Perception
The spectacle of the performance was generally lauded
While the functionality of the musical instruments was obvious, it was not clear what the dice did.
Unforeseen Insights
System has massive potential as a collaborative musical tool.
System was particularly enjoyed by children.
Key Learnings
There is much potential in marrying TUIs with music performance platforms such as Ableton Live.
There is value in a sketching in hardware approach to electronic musical instrument design, as long as prototypes are not complex or overly time consuming to build.
In a large scale project with a short time, appropriately targetted social media can be leveraged to get quick and dirty feedback on design concepts.