15.10 / GesTCom: A sensor-based environment for the analysis, processing and real-time control of complex piano notation through multimodal recordings @ IRCAM, séminaire recherche et technologie


Séminaire recherche & technologie
IRCAM 15.10, 12h00-13h00, live streaming at: video.ircam.fr
Pavlos Antoniadis


GesTCom: A sensor-based environment for the analysis, processing and real-time control of complex piano notation through multimodal recordings

In my PhD thesis I proposed a novel paradigm of pianists’ interaction with complex music notation by the name embodied navigation. Its novelty lies in rethinking the classic notion of interpretation as interaction, and performance itself as a dynamic system. The primacy of performers’ embodied experience and the inherent plasticity of music notation are the paradigm’s central features: Embodiment is shown to shape constantly the comprehension of the notation and to transform notation in real time.

The GesTCom (Gesture Cutting through Textual Complexity) has been developed at IRCAM since 2014 in collaboration with the Interaction-Son-Musique-Mouvement team. It materializes the embodied navigation paradigm into a dedicated interactive system. It is a modular sensor-based environment for the analysis, processing and real-time control of complex piano notation through multimodal recordings. In terms of hardware, it comprises systems for the capture of movement, audio, video, MIDI and capacitive data from sensors on the piano keys. In terms of software, it is equipped with modules for the capture, analysis and control of the multimodal data; and modules for the augmentation and interactive control of music notation. Each of these systems functions both as stand-alone and integrated in the general methodology of embodied navigation.

After an overview of the theoretical framework in embodied cognition and human machine interaction, the talk will focus on GesTCom. I will present its technical features, its initial goals as to representation and interaction, its current applications, including: performance analysis, embodied interactive learning, contemporary composition, free improvisation, piano pedagogy and score-following; and its future directions, including: dissemination in selected communities of performers, web-based collaborative learning, further refinement with the integration of machine learning and MIR techniques, application in studies of sensorimotor learning and prediction, and creation of interactive systems which learn along with the performer in a human, embodied way.


ircam 15.10


short introduction


full presentation



Dr Pavlos Antoniadis (PhD University of Strasbourg-IRCAM, MA University of California, San Diego) is a pianist, musicologist and technologist, based in Berlin and Strasbourg. He performs complex contemporary and experimental music, studies embodied cognition and develops tools for technology-enhanced learning. He is a member of the Interaction-Son-Musique-Mouvement team at IRCAM, as well as of LabEx GREAM, Université de Strasbourg, where he also taught seminars on computer music and contemporary performance practice.
As a researcher, Pavlos was a Musical Research Residency fellow at Ircam in 2014, where under the supervision of Frédéric Bevilacqua and Dominique Fober (Grame, Lyon) he developed GesTCom (Gesture Cutting through Textual Complexity), a sensor-based environment for the gestural control and analysis of complex piano notation, for example of works by Xenakis or Ferneyhough. He has published in English, German and French on embodied cognition, gesture capture and contemporary piano performance and has been invited for lecture-performances in several institutions (HfM Dresden, INMM Darmstadt, IRCAM Paris, Orcim Gent, Goldsmiths London, Trinity Dublin, Aristoteleio Thessaloniki, Cité de la Musique et de la Danse Strasbourg, Hong Kong University, Yamanashi Gakuin University Kofu, Wocmat Taiwan, LEAD Dijon, SARC Belfast, UdK Berlin).
As a pianist, he has performed in Europe, America and Asia with the new music ensembles Work in Progress-Berlin, Kammerensemble Neue Musik Berlin, Phorminx, ERGON, Accroche Note, as well as a soloist with the Thessaloniki State Symphony Orchestra. He has participated in recordings for Mode (2015 German Recording Critics Award / Deutscheschallplattenkritikspreis) and Wergo records. As a soloist, he has worked with composers Mark Andre, Helmut Lachenmann, Brian Ferneyhough, Wolfgang Rihm, Tristan Murail, Richard Barrett, Walter Zimmermann, and has premiered solo piano works by James Erber, Nicolas Tzortzis, Andrew R. Noble, Luis Antunes Pena, Dominik Karski and others. His programming features complex contemporary works, often with electronics and theatrical elements, and often in eclectic dialogue with older repertoire from Cabezón to Bartók. He is also active in free improvisation with the composer and improviser Panos Ghikas (London) and has collaborated with the physical theatre group Zero Point (Athens).
Pavlos holds degrees in piano performance (Piano Diploma, Nakas Athens; Koninklijk Conservatorium Den Haag; MA, UC San Diego) and musicology (BA, Athens National University, PhD Université de Strasbourg in co-direction with the IRCAM, under Pierre Michel and Frédéric Bevilacqua). He has studied on LabEx GREAM, IRCAM, Fulbright, UC San Diego, Nakas conservatory, Internationale Ensemble Modern Academie Frankfurt and Impuls Academie Gratz scholarships. His main piano teachers were Chryssi Partheniade, Ian Pace, Geoffrey Douglas Madge and Aleck Karis.



12.10 / Des algorithmes au geste extatique: analyse assistée par ordinateur de l’acte performatif dans l’œuvre pour piano solo MISTS d’Iannis Xenakis  @ Université de Strasbourg, journée “Penser le musique en acte”



Le 12 octobre 2018
De 14h30 à 15h00
Misha (université de Strasbourg), 5 allée du Général Rouvillois, 67000 Strasbourg, Tram C/E/F arrêt Observatoire, salle Europe, Entrée libre


Des algorithmes au geste extatique: analyse assistée par ordinateur de l’acte performatif dans l’œuvre pour piano solo MISTS d’Iannis Xenakis 

Mists de Iannis Xenakis, une œuvre composée en 1980, expose les théories centrales du compositeur et offre un cas d’étude pertinent pour mettre en évidence des corrélations entre les structures algorithmiques profondes et leur incarnation dans la performance. Je vais structurer ma présentation comme suit :

  1. Présentation de la corrélation entre les structures basiques de la pièce “hors-temps” par rapport à la texture et ses manifestations performatives
  2. Analyse entre divers types de navigation et structures profondes “en-temps”
  3. Analyse par rapport à la navigation pendant l’apprentissage, selon quatre scénarios d’interaction différents