Gesturally Realized Audio, Speech and Song Performance

The purpose of this project is to develop an instrument capable of being used in live performance for the presentation of new music/new media works involving realtime synthesis of speech and song, as well as supplying controlled audio playback and video manipulation. All of this is achieved through gestural control by way of cybergloves and position trackers.
GRASSP is an expansion of GloveTalk II (Fels and Hinton), a gestural speech synthesis system that uses a Cyberglove, custom-designed left hand glove, a Polhemus tracker, and a foot pedal.

GRASSP expands on the capabilities of GloveTalk in a number of ways:

  • it has been refactored in Max/MSP to take advantage of the realtime performance possibilites of that language;
  • besides speech synthesis it can control sound diffusion and video processing through the use of the UBCToolbox (Max/MSP/Jitter);
  • it has tested the ability of a skilled user to synthesize speech without the use of machine learning;
  • it has both speech and song modes, providing for a greater variety of timbre and emotion in performance.
Currently we are developing a portable version of GRASSP using a custom-designed backpack/harness system and conductive fabrics to replace the foot pedal. A portable audio system is part of the design. Commercially available hardware is used whenever possible to allow for possible replication by others.

Future developments of this project include a move towards articulatory synthesis methods combined with projected facial images.

GRASSP is supported by the Social Sciences and Humanities Research Council of Canada

Current Team Members

Bob Pritchard, School of Music, UBC
Sid Fels, Dept. of Electrical and Computer Engineering, UBC
Marguerite Witvoet, Research Assistant: GRASSP performer, pianist, vocal coach
Francois Houle, Research Assistant: clarinet performer/improvisor
Keith Hamel, School of Music, UBC
Helene Day-Fraser, (MA cand.) Textile and Design Specialist, Emily Carr Institute of Art and Design
Meryn Cadell, librettist, Dept. of Creative Writing, UBC
David Litke, (DMA cand.) School of Music, UBC
Jennifer Butler, (DMA cand.) School of Music, UBC
Allison Lenters, programmer, Dept. of Computer Science
Ying Yin, programmer, Dept. of Electrical and Computer Engineering

Previous Members

Laetitia Queyranne, seamstress
Stuart Hadfield, programmer
Jason Horbach, programmer
Rob Felix, programmer
Michael Ma, programmer
Sarah Min, programmer
Raphael Snitzman, programmer
Leo Wong, programmer