ControllAR: Appropriation of Visual Feedback on Control Surfaces

Florent Berthaut (1) , Alex Jones (2)

1 MINT2 - Méthodes et outils pour l'Interaction à gestes
2 UWE Bristol - University of the West of England [Bristol]

Despite the development of touchscreens, many expert systems for working with digital multimedia content, such as in music composition and performance, video editing or visual performance, still rely on control surfaces. This can be due to the accuracy and appropriateness of their sensors, the haptic feedback that they offer, and most importantly the way they can be adapted to the specific subset of gestures and tasks that users need to perform. On the other hand, visual feedback on controllers remains limited and/or fixed, preventing similar personalizing. In this paper, we propose ControllAR, a novel system that facilitates the appropriation of rich visual feedback on control surfaces through remixing of graphical user interfaces and augmented reality display. We then use our system to study current and potential appropriation of visual feedback in the case of digital musical instruments and derive guidelines for designers and developers.

Keywords: ControllAR; Augmented Reality; Visual Feedback; Appropriation; Hardware controllers; Control surfaces

Domains: Sound [cs.SD]; Human-Computer Interaction [cs.HC]; Musicology and performing arts

Florent Berthaut, Alex Jones. ControllAR: Appropriation of Visual Feedback on Control Surfaces. Interactive Spaces and Surfaces, ACM, Nov 2016, Niagara Falls, Canada. ⟨10.1145/2992154.2992170⟩. ⟨hal-01356239⟩