Musical Gestures Musical Gestures and their and their

Diagrammatic LogicDiagrammatic Logic

Guerino MazzolaGuerino MazzolaU & ETH Zürich   U & ETH Zürich   guerino@mazzola.ch     guerino@mazzola.ch     www.encyclospace.org        www.encyclospace.org        

DANS CES MURS VOUÉS AUX MERVEILLES DANS CES MURS VOUÉS AUX MERVEILLES J’ACCUEILLE ET GARDE LES OUVRAGES J’ACCUEILLE ET GARDE LES OUVRAGES DE LA MAIN PRODIGIEUSE DE L’ARTISTE DE LA MAIN PRODIGIEUSE DE L’ARTISTE

ÉGALE ET RIVALE DE SA PENSÉE ÉGALE ET RIVALE DE SA PENSÉE L’UNE N’EST RIEN SANS L’AUTREL’UNE N’EST RIEN SANS L’AUTRE

(Paul Valéry, Palais Chaillot)(Paul Valéry, Palais Chaillot)

DANS CES MURS VOUÉS AUX MERVEILLES DANS CES MURS VOUÉS AUX MERVEILLES J’ACCUEILLE ET GARDE LES OUVRAGES J’ACCUEILLE ET GARDE LES OUVRAGES DE LA MAIN PRODIGIEUSE DE L’ARTISTE DE LA MAIN PRODIGIEUSE DE L’ARTISTE

ÉGALE ET RIVALE DE SA PENSÉE ÉGALE ET RIVALE DE SA PENSÉE L’UNE N’EST RIEN SANS L’AUTREL’UNE N’EST RIEN SANS L’AUTRE

(Paul Valéry, Palais Chaillot)(Paul Valéry, Palais Chaillot)

musiquemusique

mathématique

mathématique

LA VERITÉDU BEAU

DANSLA MUSIQUE

Guerino Mazzola

summer 2006summer 2006

gestegestegestegesteformuleformuleformuleformule

harmonieharmoniede gestesde gestes

~

compositioncompositionde formulesde formules

~musiquemusique

mathématique

mathématique

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Ryukoku Ryukoku violin robotviolin robot

WasedaWasedawabot IIwabot II

• Musical GesturesMusical Gestures

• Gesture CategoriesGesture Categories

• Diagram LogicDiagram Logic

• Musical GesturesMusical Gestures

• Gesture CategoriesGesture Categories

• Diagram LogicDiagram Logic

ll

hh

ee

sonicsoniceventsevents

scorescore

analysisanalysis

instrumentalinstrumentalinterfaceinterface

√√thawthaw freeze (MIDI)freeze (MIDI)

instrumentalizeinstrumentalizeinstrumentalizeinstrumentalize

gestualizegestualizegestualizegestualize

positionposition

pitchpitch

timetime

gesturesgestures

Ceslaw Marek:Ceslaw Marek:Lehre des Lehre des

KlavierspielsKlavierspielsAtlantis-VerlagAtlantis-VerlagZürich 1972/77Zürich 1972/77

Folie 2

Every No play is a cross sectionEvery No play is a cross sectionof the life of one person, the of the life of one person, the shiteshite..

The shite is an appearance (demon, etc.)The shite is an appearance (demon, etc.)and a subject = one of the five elements and a subject = one of the five elements

(fire, water, wood, earth, metal) (fire, water, wood, earth, metal)

The The wakiwaki is is A kind of co-sub-A kind of co-sub-

ject and ject and mirror person mirror person

of the shite.of the shite.

The No gestures are reduced to the The No gestures are reduced to the katakata units units and made symbolic.and made symbolic.

This enables a richer communication than with This enables a richer communication than with common gestures.common gestures.

Important:Important:

• Shite weaves a texture of fantasy usingShite weaves a texture of fantasy using curves.curves.

• Waki describes reality usingWaki describes reality usingstraight linesstraight lines..

——positionposition

pitchpitch

timetime 00

11

11

22

2 2 + + 11

t.t.

22

11

1 1 22

√√gesturesgestures

√√scorescore

hh

eell

HH

EE

LL

positionposition

pitchpitch

EE

Symbolic scoreSymbolic score(a) Without(a) Withoutfingering fingering annotation annotation

(b) with (b) with fingeringfingeringannotation annotation

PhD thesis of Stefan Müller PhD thesis of Stefan Müller (Mazzola G & Müller S: ICMC 2003)(Mazzola G & Müller S: ICMC 2003)

C3C3

DIN8996DIN8996

Independent Independent symbolicsymbolicgesture curvesgesture curvesfor fingers for fingers 22et et 33

Curve parameter tCurve parameter ton horizontal axison horizontal axis

QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.

Z Y

Xxx

zz yy

bb55

bb22

dd55

dd22

11(t(t))

66(t(t))

22(t(t))

33(t(t))

44(t(t))

55(t(t))

One hand One hand product product = = 112233445566

of 6 gestural curves in space-time (x,y,z;e) of pianoof 6 gestural curves in space-time (x,y,z;e) of pianoj = 1, 2, ... 5: tips of fingers,j = 1, 2, ... 5: tips of fingers,j = 6: the carpus, j = 6: the carpus, 6 6 = = rootroot

parameter t parameter t sequence of points: sequence of points:

(t) = ((t) = (11(t),...,(t),...,66(t))(t))

two base vectorstwo base vectorsof fingersof fingersdd22, d, d55

from carpus.from carpus.

e = timee = time

Geometric constraints: six boxesGeometric constraints: six boxes

The Newton condition for fingers or carpus j isThe Newton condition for fingers or carpus j is

mmjj dd2 2 spacespacej j /de/de22(t)(t) < K < Kj j

for all 0 ≤ t ≤ 1. for all 0 ≤ t ≤ 1.

Have masses mHave masses mjj and and

maximal forces Kmaximal forces Kjj

for fingers/carpus j.for fingers/carpus j.

Have masses mHave masses mjj and and

maximal forces Kmaximal forces Kjj

for fingers/carpus j.for fingers/carpus j.

dd2 2 spacespace3 3 /de/de22

Use cubic polynomials for gestural coordinates, i.e., 76 Use cubic polynomials for gestural coordinates, i.e., 76 variables of coefficients:variables of coefficients:

xxjj(t)(t) = x= xj,3 j,3 tt33 + x + xj,2 j,2 tt22 + x + xj,1 j,1 t + xt + xj,0j,0

yyjj(t)(t) = y= yj,3 j,3 tt33 + y + yj,2 j,2 tt22 + y + yj,1 j,1 t + yt + yj,0j,0

zzjj(t) = z(t) = zj,3 j,3 tt33 + z + zj,2 j,2 tt22 + z + zj,1 j,1 t + zt + zj,0j,0

ee(t) (t) = e= e3 3 tt33 + e + e2 2 tt22 + e + e1 1 t + et + e00

Geometric and physical constraints Geometric and physical constraints polynomial inequalities: P(t) > 0 for all 0 ≤ t ≤ 1.polynomial inequalities: P(t) > 0 for all 0 ≤ t ≤ 1.

These inequalities are guaranteed by These inequalities are guaranteed by Sturm chainsSturm chains..

Symbolic gestural curveSymbolic gestural curve

Physical gestural curvePhysical gestural curve

fingers 2, 3: geometric constraintsfingers 2, 3: geometric constraints fingers 2, 3: physical constraintsfingers 2, 3: physical constraints

Gestural interpretation of Carl Czerny‘s op. 500Gestural interpretation of Carl Czerny‘s op. 500

Zur Anzeige wird der QuickTime™ Dekompressor „H.264“

benötigt.

• Musical GesturesMusical Gestures

• Gesture CategoriesGesture Categories

• Diagram LogicDiagram Logic

vv

xx

ww

yy

cc

aa

bb

dd

QuiverQuiver = category of quivers = category of quivers(= digraphs, diagram schemes, etc.) (= digraphs, diagram schemes, etc.)

DD = A V = A Vhh

tt

x = t(a) x = t(a)

y = h(a)y = h(a)

aa

EE = B W = B Wh‘h‘

t‘t‘

u q

DD

QuiverQuiver((DD,, E E))

A gesture A gesture morphism morphism u:u: gg h is a quiver morphism u, h is a quiver morphism u, such that there is a continuous map f: X such that there is a continuous map f: X Y which Y whichdefines a commutative diagram: defines a commutative diagram:

ff

DD

EE

XX

YY

gg

hh

uu

GestureGesture((gg, , hh))category of (local) gesturescategory of (local) gestures

(Local) Gesture(Local) Gesture = = morphism g: morphism g: D D of quivers with values in a of quivers with values in a spatial quiverspatial quiver of a metric space X of a metric space X(= quiver of continuous curves in X)(= quiver of continuous curves in X)

XX

XX

DD

positionposition

pitchpitch

timetime

XXgg

A A global gestureglobal gesturebeing coveredbeing covered

by threeby threelocal gestureslocal gestures

Hypergestures! Hypergestures!

QuiverQuiver((FF, , ) = ) = metric space of (local) gestures ofmetric space of (local) gestures ofof quiver of quiver FF with values in a with values in a spatial quiver .spatial quiver .

XX

XX

Zur Anzeige wird der QuickTime™ Dekompressor „Animation“

benötigt.

FF

rr

EE

ss

ttRenate Wieland & Jürgen Uhde:Renate Wieland & Jürgen Uhde:Forschendes ÜbenForschendes Üben

Die Klangberührung ist das Ziel Die Klangberührung ist das Ziel der zusammenfassenden Geste, der zusammenfassenden Geste, der Anschlag ist sozusagen der Anschlag ist sozusagen die Geste in der Gestedie Geste in der Geste..

EE

gg

hh

Hypergesture impossible!

gg

hh

Morphism exists!

• Musical GesturesMusical Gestures

• Gesture CategoriesGesture Categories

• Diagram LogicDiagram Logic

D D E E

D D ++ E E

1 1 = =

0 0 = Ø = Ø

DDEE QuiverQuiver(( ,, D DEE)) QuiverQuiver(( ,, D DEE))

≈ ≈ QuiverQuiver((E E ,, D D))

The category The category QuiverQuiver is a is a topostopos

Alexander GrothendieckAlexander Grothendieck

≈ ≈ QuiverQuiver((E E ,, D D))

=

TT

vv

xx

ww

yy

In particular:The set Sub(DD) of subquiversof a quiver DD is a Heyting algebra: have „QuiverQuiver logic“.

Ergo:

Local/global gestures,ANNs,Klumpenhouwer-nets,and global networksenable logicaloperators (, , ,)

In particular:The set Sub(DD) of subquiversof a quiver DD is a Heyting algebra: have „QuiverQuiver logic“.

Ergo:

Local/global gestures,ANNs,Klumpenhouwer-nets,and global networksenable logicaloperators (, , ,)

Subobject classifierSubobject classifier

Heyting logic on set Sub(g) of subgestures of g Heyting logic on set Sub(g) of subgestures of g

h, k h, k Sub(g) Sub(g)h h k = h k = h k kh h k = h k = h k k h h k (complicated) k (complicated) h = h h = h Ø Ø tertium datur: h ≤ tertium datur: h ≤ h h

u: gu: g11 g g22

Sub(u): Sub(gSub(u): Sub(g22) ) Sub(g Sub(g11) )

homomorphism of Heyting algebras homomorphism of Heyting algebras = contravariant functor = contravariant functor

Sub: Sub: GestureGesture HeytingHeyting

Heyting logic on set Sub(g) of subgestures of g Heyting logic on set Sub(g) of subgestures of g

h, k h, k Sub(g) Sub(g)h h k = h k = h k kh h k = h k = h k k h h k (complicated) k (complicated) h = h h = h Ø Ø tertium datur: h ≤ tertium datur: h ≤ h h

u: gu: g11 g g22

Sub(u): Sub(gSub(u): Sub(g22) ) Sub(g Sub(g11) )

homomorphism of Heyting algebras homomorphism of Heyting algebras = contravariant functor = contravariant functor

Sub: Sub: GestureGesture HeytingHeyting

VIIVII

II

IIIIII

VV

IIIIVIVI

IVIV

cc

dd

ee

ffgg

aa

bb

C-major hypergestureC-major hypergesture

FingersFingers = = QuiverQuiver((FF, ), ) XX

FingersFingers

F F = =

VV

II

II

VIVI

IVIV

==

•Investigate the possible role and semantics of gestural logic in concrete contexts

such as local/global musical/robot gesturesand more specific environments...(and more generally: Quiver logic for ANNs,Klumpenhouwer-nets,global networks).

•Investigate a (formal) propositional/predicate language of gestureswith values in Heyting algebras of quivers.

Problems: