mathematical theory mathematical theory of gestures in music guerino mazzola u minnesota &...
TRANSCRIPT
Mathematical Theory Mathematical Theory of Gestures of Gestures
in Musicin Music
Guerino MazzolaGuerino MazzolaU Minnesota & ZürichU Minnesota & Zü[email protected] [email protected] [email protected] [email protected] www.encyclospace.org www.encyclospace.org
DANS CES MURS VOUÉS AUX MERVEILLES DANS CES MURS VOUÉS AUX MERVEILLES J’ACCUEILLE ET GARDE LES OUVRAGES J’ACCUEILLE ET GARDE LES OUVRAGES DE LA MAIN PRODIGIEUSE DE L’ARTISTE DE LA MAIN PRODIGIEUSE DE L’ARTISTE
ÉGALE ET RIVALE DE SA PENSÉE ÉGALE ET RIVALE DE SA PENSÉE L’UNE N’EST RIEN SANS L’AUTREL’UNE N’EST RIEN SANS L’AUTRE
(Paul Valéry, Palais Chaillot)(Paul Valéry, Palais Chaillot)
DANS CES MURS VOUÉS AUX MERVEILLES DANS CES MURS VOUÉS AUX MERVEILLES J’ACCUEILLE ET GARDE LES OUVRAGES J’ACCUEILLE ET GARDE LES OUVRAGES DE LA MAIN PRODIGIEUSE DE L’ARTISTE DE LA MAIN PRODIGIEUSE DE L’ARTISTE
ÉGALE ET RIVALE DE SA PENSÉE ÉGALE ET RIVALE DE SA PENSÉE L’UNE N’EST RIEN SANS L’AUTREL’UNE N’EST RIEN SANS L’AUTRE
(Paul Valéry, Palais Chaillot)(Paul Valéry, Palais Chaillot)
LA VÉRITÉDU BEAU
DANSLA MUSIQUE
Guerino Mazzola
• MotivationMotivation- performance - and music theory- performance - and music theory- gestural music and painting- gestural music and painting- French philosophy- French philosophy- Embodied AI- Embodied AI
• SpeculumSpeculum - the musical oniontology- the musical oniontology- classification of global compositions and networks- classification of global compositions and networks
• GesturesGestures- categories of gestures- categories of gestures- hypergestures- hypergestures- the Escher theorem and free jazz- the Escher theorem and free jazz
• SymbolsSymbols- homotopy- homotopy- gestoids- gestoids- finitely generated abelian groups and networks- finitely generated abelian groups and networks
• MotivationMotivation- performance - and music theory- performance - and music theory- gestural music and painting- gestural music and painting- French philosophy- French philosophy- Embodied AI- Embodied AI
• SpeculumSpeculum - the musical oniontology- the musical oniontology- classification of global compositions and networks- classification of global compositions and networks
• GesturesGestures- categories of gestures- categories of gestures- hypergestures- hypergestures- the Escher theorem and free jazz- the Escher theorem and free jazz
• SymbolsSymbols- homotopy- homotopy- gestoids- gestoids- finitely generated abelian groups and networks- finitely generated abelian groups and networks
Theodor W. AdornoTheodor W. Adorno(„Zu einer Theorie der musikalischen („Zu einer Theorie der musikalischen Reproduktion“ 1946):Reproduktion“ 1946):
Danach wäre die Aufgabe des Interpreten, Danach wäre die Aufgabe des Interpreten, NotenNoten so zu betrachen, bis sie dem so zu betrachen, bis sie dem insistenten Blick in Originalmanuskripte insistenten Blick in Originalmanuskripte sich verwandeln; nicht aber als sich verwandeln; nicht aber als Bilder der Seelenregung des Autors — Bilder der Seelenregung des Autors — sie sind auch dies, aber nur akzidentiell — sie sind auch dies, aber nur akzidentiell — sondern als die seismographischen Kurven, sondern als die seismographischen Kurven, die der Körper der Musik selber in seinen die der Körper der Musik selber in seinen gestischen Erschütterungengestischen Erschütterungen hinterlassen hat. hinterlassen hat.
Gestures in Performance TheoryGestures in Performance Theory
Gestures in Performance TheoryGestures in Performance Theory
Jürgen Uhde & Renate Wieland Jürgen Uhde & Renate Wieland („Forschendes Üben“ 2002):(„Forschendes Üben“ 2002):
Affekte waren ursprünglich ja Affekte waren ursprünglich ja Handlungen, bezogen auf ein Objekt Handlungen, bezogen auf ein Objekt draussen, im Prozess der Verinnerlichung draussen, im Prozess der Verinnerlichung haben sie sich von ihrem Gegenstand haben sie sich von ihrem Gegenstand gelöst, aber immer noch sind sie bestimmt gelöst, aber immer noch sind sie bestimmt von den Koordinaten des Raumes. (...) Es von den Koordinaten des Raumes. (...) Es gibt mithin etwas wie gibt mithin etwas wie gestische (Raum-)Koordinatengestische (Raum-)Koordinaten..
David LewinDavid Lewin(„Generalized Musical Intervals(„Generalized Musical Intervalsand Transformations“ 1987):and Transformations“ 1987):
If I am at If I am at ss and wish to get to and wish to get to tt, , what characteristic what characteristic gesturegesture should I should I performperform in order to in order to arrive there?arrive there?
Musical Transformational Theory Musical Transformational Theory
Robert S. HattenRobert S. Hatten(„Interpreting Musical Gestures, („Interpreting Musical Gestures, Topics, and Tropes“ 2004)Topics, and Tropes“ 2004)
Given the importance of gesture Given the importance of gesture to interpretation, why do we not to interpretation, why do we not have a comprehensive have a comprehensive theory theory of gesture in musicof gesture in music??
Music Theory Music Theory
Cecil TaylorCecil Taylor
The body is in no way supposed The body is in no way supposed to get involved in Western musicto get involved in Western music. I try to I try to imitate on the piano the imitate on the piano the leapsleaps in space a dancer makes. in space a dancer makes.
Jazz Jazz
ll
hh
ee
soundsoundeventsevents
scorescore
analysisanalysis
instrumental-instrumental-interfaceinterface
√√thawthaw
instrumentalizeinstrumentalizeinstrumentalizeinstrumentalize
positionposition
pitchpitch
timetime
gesturesgestures
Musical theory/notation
Musical theory/notation
Instruments/playing actionInstruments/
playing action Musical soundMusical sound
Tellef Kvifte: Instruments andTellef Kvifte: Instruments andthe electronic Age.the electronic Age.
Solum forlag, Oslo, 1988Solum forlag, Oslo, 1988
The marks are made, The marks are made, and you survey the and you survey the thing like you would a thing like you would a sort of sort of graphgraph. And you . And you see within this graph see within this graph the possibilities of all the possibilities of all types of types of fact being fact being plantedplanted....
David Sylvester: Interview with Francis David Sylvester: Interview with Francis Bacon: The Brutality of FactBacon: The Brutality of Fact. . Thames and Hudson, New York 1975Thames and Hudson, New York 1975
Francis BaconFrancis Bacon
Painting Painting
Charles Alunni (1951 -):Charles Alunni (1951 -):Ce n‘est pas la règle qui gouverne l‘action Ce n‘est pas la règle qui gouverne l‘action diagrammatique, mais l‘action qui fait émerger la règle.diagrammatique, mais l‘action qui fait émerger la règle.
Jean Cavaillès (1903 - 1944):Jean Cavaillès (1903 - 1944):Comprendre est attraper le geste et pouvoir continuer.Comprendre est attraper le geste et pouvoir continuer.
Gilles Deleuze (1925 - 1995):Gilles Deleuze (1925 - 1995): Francis Bacon. La logique de la sensation. Francis Bacon. La logique de la sensation. EEditions de la Différence, Paris 1981ditions de la Différence, Paris 1981
„graph“ „graph“ „diagramme“ „diagramme“ „geste“ „geste“
Zur Anzeige wird der QuickTime™ Dekompressor „YUV420 codec“
benötigt.
Stumpy: AI Lab, U ZurichStumpy: AI Lab, U Zurich
Embodied AI Embodied AI
„„Cheap design“
Cheap design“
• MotivationMotivation- performance - and music theory- performance - and music theory- gestural music and painting- gestural music and painting- French philosophy- French philosophy- Embodied AI- Embodied AI
• SpeculumSpeculum - the musical oniontology- the musical oniontology- classification of global compositions and networks- classification of global compositions and networks
• GesturesGestures- categories of gestures- categories of gestures- hypergestures- hypergestures- the Escher theorem and free jazz- the Escher theorem and free jazz
• SymbolsSymbols- homotopy- homotopy- gestoids- gestoids- finitely generated abelian groups and networks- finitely generated abelian groups and networks
The Oniontology of MusicThe Oniontology of Music
FactsFactssignssigns
realit
ies
realit
ies
communicationcommunication
ProcessesProcesses
GesturesGestures
Classify!
Classify!
The category The category GloComGloComAA of of global objectiveglobal objective A-addressed A-addressed
compositionscompositions has has
objectsobjects K KII, i.e., coverings of sets K by atlases I of local objective, i.e., coverings of sets K by atlases I of local objectiveA-addressed compositions with manifold gluing conditionsA-addressed compositions with manifold gluing conditions
and manifold and manifold morphisms morphisms ff: K: KII LLJJ, , includingincluding and and
compatible with atlas morphisms compatible with atlas morphisms : I : I J J
I
IV
II
VIV
IIIVII
factsfacts
Theorem (global addressed geometric classification)Theorem (global addressed geometric classification)
Let A = locally free of finite rank over commutative ring RLet A = locally free of finite rank over commutative ring R
Consider the Consider the objectiveobjective global compositions K global compositions KII at A with (*): at A with (*):
• the chart modules R.Kthe chart modules R.Ki i areare locally free of finite ranklocally free of finite rank • the function modules the function modules (K(Kii) are) are projective projective
There is a subscheme There is a subscheme JJn* n* of a projective R-scheme of finite type of a projective R-scheme of finite type
whose points whose points : Spec(S): Spec(S)JJn* n* parametrize the isomorphism parametrize the isomorphism
classesclasses of objective global compositions at address S of objective global compositions at address SRRA with (*).A with (*).
Theorem (global addressed geometric classification)Theorem (global addressed geometric classification)
Let A = locally free of finite rank over commutative ring RLet A = locally free of finite rank over commutative ring R
Consider the Consider the objectiveobjective global compositions K global compositions KII at A with (*): at A with (*):
• the chart modules R.Kthe chart modules R.Ki i areare locally free of finite ranklocally free of finite rank • the function modules the function modules (K(Kii) are) are projective projective
There is a subscheme There is a subscheme JJn* n* of a projective R-scheme of finite type of a projective R-scheme of finite type
whose points whose points : Spec(S): Spec(S)JJn* n* parametrize the isomorphism parametrize the isomorphism
classesclasses of objective global compositions at address S of objective global compositions at address SRRA with (*).A with (*).
KKI I can be can be reconstructedreconstructed from the from the coefficient system of retracted functions on free global compositions coefficient system of retracted functions on free global compositions
res*nres*n(K(KII) ) nn((AAn*n*) )
Fact: This construction is a special case of a Fact: This construction is a special case of a local networklocal network. .
Global compositions are classified by Global compositions are classified by limitslimits of powerset of powerset denotatorsdenotators..
KKI I can be can be reconstructedreconstructed from the from the coefficient system of retracted functions on free global compositions coefficient system of retracted functions on free global compositions
res*nres*n(K(KII) ) nn((AAn*n*) )
Fact: This construction is a special case of a Fact: This construction is a special case of a local networklocal network. .
Global compositions are classified by Global compositions are classified by limitslimits of powerset of powerset denotatorsdenotators..
ŸŸ1212
ŸŸ1212
ŸŸ1212
ŸŸ1212
TT44
TT22
TT55.-1.-1 TT1111.-1.-1DD
33 77
22 44
processesprocesses
A A global networkglobal network
Local and Global Limit Denotators and the Local and Global Limit Denotators and the Classification of Global Compositions.Classification of Global Compositions.COLLOQUIUM ON MATHEMATHICAL MUSIC THEORY COLLOQUIUM ON MATHEMATHICAL MUSIC THEORY H. Fripertinger, L. Reich (Eds.) H. Fripertinger, L. Reich (Eds.) Grazer Math. Ber., ISSN 1016–7692 Bericht Nr. 347 (2005), Grazer Math. Ber., ISSN 1016–7692 Bericht Nr. 347 (2005),
Global Networks in Computer Science?Global Networks in Computer Science?http://www.encyclospace.org/talks/networks.ppthttp://www.encyclospace.org/talks/networks.ppt
TheoremTheorem Given address A in Given address A in ModMod, we have a , we have a verification functorverification functor
|?|: |?|: AALfLfModModredred
AAGlobGlob
from the category from the category AALfLfModModredred of reduced, A-addressed locally flat of reduced, A-addressed locally flat
global networks to the category global networks to the category AAGlobGlob of A-addressed global of A-addressed global compositions.compositions.
TheoremTheorem Given address A in Given address A in ModMod, we have a , we have a verification functorverification functor
|?|: |?|: AALfLfModModredred
AAGlobGlob
from the category from the category AALfLfModModredred of reduced, A-addressed locally flat of reduced, A-addressed locally flat
global networks to the category global networks to the category AAGlobGlob of A-addressed global of A-addressed global compositions.compositions.
CorollaryCorollary There are non-interpretable global networks in There are non-interpretable global networks in AALfLfModMod
redred
CorollaryCorollary There are non-interpretable global networks in There are non-interpretable global networks in AALfLfModMod
redred
Gesture Theory in Computer Music Research:Gesture Theory in Computer Music Research:
• Frédéric Bevilacqua • Claude Cadoz Claude Cadoz • Antonio Camurri Antonio Camurri • Rolf Inge GodøyRolf Inge Godøy• Stefan Müller • Norbert Schnell• Koji Shibuya• McAgnus Todd• Marcelo WanderleyMarcelo Wanderley• etc.etc.
Gesture Theory in Computer Music Research:Gesture Theory in Computer Music Research:
• Frédéric Bevilacqua • Claude Cadoz Claude Cadoz • Antonio Camurri Antonio Camurri • Rolf Inge GodøyRolf Inge Godøy• Stefan Müller • Norbert Schnell• Koji Shibuya• McAgnus Todd• Marcelo WanderleyMarcelo Wanderley• etc.etc.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Koji Shibuya‘s RyukokuRyukokuviolin robotviolin robot
• factsfacts: complete classification of : complete classification of addressed global compositionsaddressed global compositions
• processesprocesses: relative classification : relative classification of global networks via a of global networks via a functor to global compositionsfunctor to global compositions
• gesturesgestures: no mathematical theory: no mathematical theory
Mathematical Music TheoryMathematical Music Theory
• MotivationMotivation- performance - and music theory- performance - and music theory- gestural music and painting- gestural music and painting- French philosophy- French philosophy- Embodied AI- Embodied AI
• SpeculumSpeculum - the musical oniontology- the musical oniontology- classification of global compositions and networks- classification of global compositions and networks
• GesturesGestures- categories of gestures- categories of gestures- hypergestures- hypergestures- the Escher theorem and free jazz- the Escher theorem and free jazz
• SymbolsSymbols- homotopy- homotopy- gestoids- gestoids- finitely generated abelian groups and networks- finitely generated abelian groups and networks
Zur Anzeige wird der QuickTime™ Dekompressor „Animation“
benötigt.
aa1111x+ax+a1212y+ay+a1313z = az = a
aa2121x+ax+a2222y+ay+a2323z = bz = b
aa3131x+ax+a3232y+ay+a3333z = cz = c
aa1111 a a1212 a a1313
aa2121 a a2222 a a2323
aa3131 a a3232 a a3333
xxyyzz
aabbcc
==
rotationrotation matrix equationmatrix equation
algebra compactifies gestures to symbolic formulasalgebra compactifies gestures to symbolic formulas
„„attempt at resuscitation“attempt at resuscitation“
Peter Gabriel: Symbolic formulas via digraphs = „quiver algebras“Peter Gabriel: Symbolic formulas via digraphs = „quiver algebras“
SS
PP
TT
KK
TTXX
RRKK
RRK K = R[X]= R[X]polynomial algebrapolynomial algebra
mathematics of transformational theorymathematics of transformational theory
Graphs are only the „skeleton“ of gestures, the „flesh“ is missing.Graphs are only the „skeleton“ of gestures, the „flesh“ is missing.
??
(Local) Gesture(Local) Gesture = = morphism g: morphism g: D D of digraphs with values in a of digraphs with values in a spatial digraphspatial digraph of a topological space X of a topological space X(= digraph of continuous curves in X)(= digraph of continuous curves in X)
XX
XX
DD
positionposition
pitchpitch
timetime
XXgg
bodybody
skeletonskeleton
A gesture A gesture morphism morphism u:u: gg h is a digraph morphism u, h is a digraph morphism u, such that there is a continuous map f: X such that there is a continuous map f: X Y which Y whichdefines a commutative diagram: defines a commutative diagram:
ff
DD
EE
XX
YY
gg
hh
uu
GG((gg, , hh))category category GG of (local) gestures of (local) gestures
Advantage: Digraphs have an inherent (intuitionistic) logic, Advantage: Digraphs have an inherent (intuitionistic) logic, because the category of digraphs is a topos.because the category of digraphs is a topos.
This is not only cheap, but free design!This is not only cheap, but free design!
Advantage: Digraphs have an inherent (intuitionistic) logic, Advantage: Digraphs have an inherent (intuitionistic) logic, because the category of digraphs is a topos.because the category of digraphs is a topos.
This is not only cheap, but free design!This is not only cheap, but free design!
A A global gestureglobal gesture(only bodies shown)(only bodies shown)
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
Z Y
Xxx
zz yy
11(t(t))
66(t(t))
22(t(t))
33(t(t))
44(t(t))
55(t(t))
One hand One hand product product = = 1122334455
66
of 6 gestural curves of 6 gestural curves in space-time in space-time (x,y,z;e) of piano(x,y,z;e) of piano
j = 1, 2, ... 5: j = 1, 2, ... 5: tips of fingerstips of fingers
j = 6: j = 6: the carpus the carpus
e = timee = time
11 = = ŸŸ
Zur Anzeige wird der QuickTime™ Dekompressor „H.264“
benötigt.
Stefan MüllerStefan Müller
DD
pp
real forms?real forms?tip spacetip space
positionposition
pitchpitch
onsetonset
Renate Wieland & Renate Wieland & Jürgen Uhde:Jürgen Uhde:
Forschendes ÜbenForschendes Üben
Die Klangberührung Die Klangberührung ist das Ziel der ist das Ziel der
zusammenfassenden zusammenfassenden Geste, der Anschlag ist Geste, der Anschlag ist
sozusagen sozusagen die Geste in der Gestedie Geste in der Geste..
circlecircle
knotknot
„„loop of loops “loop of loops “
Hypergestures! Hypergestures!
DigraphDigraph((FF, , ) = ) = topological space of (local) gestures oftopological space of (local) gestures ofof digraph of digraph FF with values in a with values in a spatial digraph . spatial digraph . Notation:Notation: F F @@XXXX
XX
Zur Anzeige wird der QuickTime™ Dekompressor „Animation“
benötigt.
spacespace
spacespace
timetime
ET-dance gesture ET-dance gesture
EE
gg
hh
hypergesture impossible!hypergesture impossible!
gg
hh
morphism exists!morphism exists!
Gestural mapsGestural maps are particular continuous maps are particular continuous maps
(u,v):(u,v): F F @@X X G G @@YY
canonically induced by a pair of maps canonically induced by a pair of maps u: u: GG FF (digraphs)(digraphs)v: X v: X Y Y (continuous)(continuous)
The The categorycategory HHGG == HHGG11 of of hypergestureshypergestures has has
1) hypergestures as objects and1) hypergestures as objects and2) gestural maps as continuous maps.2) gestural maps as continuous maps.
The The categorycategory HHGGn n of of n-foldn-fold hypergestureshypergestures has has
1) objects: n-fold hypergestures1) objects: n-fold hypergestures g: g: FFn n FFn-1n-1@ ... @ ... FF11@ @ X X
2) (n-1)-fold gestural maps as continuous maps. 2) (n-1)-fold gestural maps as continuous maps.
Have chain of successively refined gestural categoriesHave chain of successively refined gestural categories
G G HHGG == HHGG11 HHGG2 2 ...... HHGGn n HHGGn+1 n+1 ......
which represent the granularity of gestural relations, which represent the granularity of gestural relations, much as in differential geometry, where the categories ofmuch as in differential geometry, where the categories ofn-times differentiable manifolds do.n-times differentiable manifolds do.
E.g. gluing local gestures to global gestures E.g. gluing local gestures to global gestures in quasi-anatomic joints in quasi-anatomic joints
Proposition Proposition (Escher Theorem)(Escher Theorem)Given a topological space X, a sequence of digraphs Given a topological space X, a sequence of digraphs
FF1 1 , , FF22, ..., ... FFnn
and a permutation and a permutation of 1, 2,... n. of 1, 2,... n.
Then there is a homeomorphismThen there is a homeomorphism
FF11@ @ ...... FFnn@@X X FF(1)(1)@ @ ...... F F (n)(n)@@X X
gg
hh
kk
Comprendre est Comprendre est
attraper le geste et attraper le geste et
pouvoir continuer.pouvoir continuer.
• MotivationMotivation- performance - and music theory- performance - and music theory- gestural music and painting- gestural music and painting- French philosophy- French philosophy- Embodied AI- Embodied AI
• SpeculumSpeculum - the musical oniontology- the musical oniontology- classification of global compositions and networks- classification of global compositions and networks
• GesturesGestures- categories of gestures- categories of gestures- hypergestures- hypergestures- the Escher theorem and free jazz- the Escher theorem and free jazz
• SymbolsSymbols- homotopy- homotopy- gestoids- gestoids- finitely generated abelian groups and networks- finitely generated abelian groups and networks
homotopichomotopiccurvescurves
XX
Gestoids:Gestoids: From Gestures to Symbols From Gestures to Symbols
0 10 1
0 1
compositioncompositionof homotopicof homotopiccurvescurvesis associativeis associative
XX
AlgebraicAlgebraicTopology
Topology
The homotopy classes of curves of a gesture gThe homotopy classes of curves of a gesture gdefine the define the GestoidGestoid G Ggg of a gesture g. of a gesture g.
This consists of the This consists of the linear combinationslinear combinations
nn a annccnn
of homotopy classes cof homotopy classes cnn of curves between given points x, y of curves between given points x, y
of gesture g.of gesture g.
yy
xx
eei2i2tt
——
ii——
11
ii
X = SX = S11
GGg g ¬ ¬ 11(S(S11))
fundamental groupfundamental group
11(S(S11) ) ŸŸ
eei2i2ntnt ~ n ~ n
~ ~ Fourier formulaFourier formula f(t) = f(t) = nn a an n eei2i2nt nt nn a ann eei2i2ntnt
g:g:
11(X) (X) ŸŸn n ? ?
finitely generated abelian groups?finitely generated abelian groups?
11(X) (X) ŸŸn n ? ?
finitely generated abelian groups?finitely generated abelian groups?
Zur Anzeige wird der QuickTime™ Dekompressor „Animation“
benötigt.
FF
2F2F
3F3F
4F4F
Fourier balletFourier ballet
QEDQED
ZZnn
LLn,1n,1
SS33
11
action of action of ŸŸnn
11((LLn,1n,1) ) ŸŸnn
11((SS33) ) 00
gestures gestures = ?= ?
string theory of musicstring theory of music
gestures gestures = ?= ?
string theory of musicstring theory of music