treequencer: collaborative rhythm sequencing … collaborative rhythm sequencing - a comparative...

TreeQuencer: Collaborative Rhythm Sequencing -A Comparative Study

Niklas KlügelTechnische Universität

MünchenDepartment of Informatics

Boltzmannstrasse 385748 Garching bei München

[email protected]

Gerhard HagererTechnische Universität



[email protected]

Georg GrohTechnische Universität



[email protected]

ABSTRACTIn this contribution we will show three prototypical appli-cations that allow users to collaboratively create rhythmicstructures with successively more degrees of freedom to gen-erate rhythmic complexity. By means of a user study weanalyze the impact of this on the users’ satisfaction andfurther compare it to data logged during the experimentsthat allow us to measure the rhythmic complexity created.

KeywordsCollaborative Music Making, Creativity Support, User Study

1. INTRODUCTIONIn the last century the importance of rhythm has under-gone a remarkable transformation from an equal to the mostvisceral element in the musical trinity of melody, rhythmand harmony - in both the high art and popular music[8]. Rhythm is social, it binds communities through dancesthey may share, such as in tribal or rave cultures, but alsothrough participation in drum circles [10]. This participa-tory facet of rhythm, the collaborative creation of music, es-pecially if intrinsically motivated, can be a highly engagingactivity, the joy of the groove [12]. The related phenomenonof Group Flow [11, p. 158] is beneficial for social creativity,it is a motivator and means for the group to innovate in acreative task. In a musical context, it has been shown em-pirically to lead to more valuable results [6]. The personalexperience of Flow itself, defined as “a holistic sensationthat people feel when they act with total involvement” [1],is an enabler for the empathic involvement with music andstimulator for the implicit learning process [9].

We see these social factors as highly beneficial for musicalcreativity and therefore aim at supporting the collaborativecreation of rhythmic structures while fostering Group Flowwith help of a computer system. In such a setting, thecomputer can be seen as a mediator for the collaboration,helping to communicate musical intent or to facilitate therealization of musical ideas. Furthermore, in the context ofgroup creativity, the heterogeneity of individuals, and theiroverall knowledge diversity, experience, and expertise arekey elements [2]. The integration of these elements is alsopart of the mediating role of the computer. Focusing on

Permission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a fee.NIME’14, June 30 – July 03, 2014, Goldsmiths, University of London, UK.Copyright remains with the author(s).

the creative task itself, we aim at investigating the extentof rhythmic complexity and expressivity that users wouldwant to pursue without being either underwhelmed or over-whelmed, as this is detrimental to the experience of GroupFlow. Especially regarding the support for heterogeneousgroups, this question is essential.

However, to our best knowledge, sufficient empirical re-sults do not exist. Therefore we aim to perform a com-parative study with several prototypical applications thatsuccessively give more degrees of freedom to the users withregard to shaping rhythmic structures and then evaluatetheir responses. We will furthermore measure the rhythmiccomplexity of the created rhythmic structures and comparethem to these responses. In this regard, we try to shed somelight into this issue such that later applications may be ableto perform their role of the mediator more intelligently. Wemake use of a multi-touch table as shared interface, sincethis offers means that simplify social communication proto-cols [15, 4].

There have been several contributions that addressed thecollaborative creation of rhythmic forms. One can distin-guish between real time input driven systems such as Beat-Bugs [14] which allow users to directly perform percussivepatterns that are then recorded into the system and se-quencer based approaches such as the reacTable [5] or ThePlanets [7]. In case of the latter, users can modify the musi-cal events asynchronously to the produced audio. We con-sider this temporal decoupling as beneficial since it concep-tually allows for a larger variety of access points for collab-oration and the mediating role of the IT-system. Thereforewe design our prototypical applications as collaborative se-quencing environments.

Additionally, we want to support melodic elements (e.g.bass sounds, simple harmonics) rather than primarily fo-cusing on percussive ones, since we believe embedding theshared composition in a more musically comprehensive con-text reflects a more realistic use case. Furthermore, we seekto allow the modification of timbres in real time as tim-bre itself is an important part of the musical structure [3],especially for modern popular music (e.g. dance music).

2. CONCEPTOne of the most prevalent sequencing concepts in electronicmusic production is the step sequencer which dates backto the early 20th century (e.g. Raymond Scott’s Circle Ma-chine) and has been employed in various drum machines(e.g. Roland TR-808) but also in modern Digital AudioWorkstations (e.g. Fruity Loops). In general, a musical du-ration is divided into a discrete set of units, the steps. Atevery step a musical event can be set to be triggered. Thisstep sequence pattern is then played back in a loop. Inmany cases a 4/4 bar is divided into 8 or 16 steps, mean-

Proceedings of the International Conference on New Interfaces for Musical Expression

50

Figure 1: Screenshots of prototype 1 and 2

ing that events can happen at most at granularity of 1/8thor 1/16th. In this regard, a step sequencer is very limitedin terms of rhythmic expressivity since only one time lineexists that is fixed in length and because the musical subdi-visions of a bar can only be even multiples of the durationof a step (e.g. triplets can not be used). We will make useof this core concept and successively extend it to allow formore degrees of freedom in regard to rhythmic expressivityfor our prototypes.

The first extension allows to create polymeters such thatlooping rhythmic structures with even or odd overlappingelements can be constructed. This differs from the originalconcept as now several time lines (step sequencers) with thesame pulse but different length are necessary.

The second extension concerns the duration of subdivi-sions in order to construct polyrhythms. This can be en-abled by allowing each of the time lines in the polymeter tohave a different pulse frequency. This makes it possible tocreate complex rhythmic patterns.

If we disregard the pulse frequency for a moment, thenthe step sequencer metaphor can be expressed as a graphconstruct in which nodes represent audible musical events(such as a hit of a bass drum) and edges express the suc-cession of these events. In this way, a step sequencer with 8steps can be seen a simple linear list of 8 nodes that are con-nected by edges. Similarily, a polymeter can be expressedas a tree that is to be traversed starting at the root. Wewill make use of this perspective on rhythmic constructs asinteraction metaphor for our prototypes because in this waywe can coherently express time-event relationships for all ofour prototypes.

3. PROTOTYPESIn our prototypes, nodes represent sound generators di-rectly, we chose to use different abstract three dimensionalforms and colors to differentiate between them in the visu-alization (cp. fig. 1). These forms can be interacted with inthe following way to change the graph structure and timbre:

• Dragging a form across the shared workspace (re-)connects it to the nearest form, thus this is used to

construct the graph. Dragging forms outside of theworkspace deletes them.

• A pinch gesture on the forms scales it and modifiesits loudness. Forms can be muted when scaled to aminimum value, thus constructing rests.

• Rotation (2D or 3D using a multi-touch gesture) mod-ifies synthesis parameters. Each form has 3 modifiableparameters associated. The global amount of a rota-tion of a form is indicated as one of three arc segmentsaround it.

Edges of the graph are indicated as simple lines, the pro-gression of the sequence(s) is indicated by small circles thattravel along the edges from node to node at the pace of thepulse duration of the sequence. In this way it is visually in-dicated which forms are about to be triggered next. If thisindicator reaches a form, the sound generator is triggeredto produce an aural result. This is also shown visually bychanging the brightness of the form according to the ampli-tude level of the generated sound.

We used six different sound generators that all allow tosynthesize a large variety of different sounds. They canbe controlled in real time therefore influencing the sonicoutput immediately. In this way users are able to alter thesynthesized timbres and alter the musical result even whenthe rhythmic structure is static.

The first prototype p1, which is an adoption of a sim-ple step sequencer, consists of 8 subdivisions represented asstatic, non-audible forms to which forms can be attachedto. Only for this prototype no other graph constructionsare allowed. Thus, only rhythmic constructions equivalentto that of a step sequencer can be manifested.

The second prototype p2 allows the construction of treestructures. The root of the tree is represented as non-audible form. Sub-trees leading from the root are loopedaccording to the longest path from root to leaf node (eachwith the same pulse). This construction allows for parallelevents to happen in parallel paths.

The third prototype p3 is a modification of the secondone, where the pulse interval can be changed per sub-tree toa multiple of 1/8th. This is set by changing the distance ofthe first node of the sub-tree to the center of the workspace.

The global tempo in each prototype can be altered withone of four sliders placed around the workspace.

4. EVALUATIONFor the evaluation we will make use of both a user studyas well as tracking the use of the application by logging theinteraction. With the latter it is possible to reconstruct therhythmic patterns that have been created and therefore toapply measures for rhythmic complexity but also to analyzewhether users alternatively preferred to shape the timbre ofsounds via measuring the duration of gestures. 31 personstook part of our user study in groups of 3-4 people. Withrespect to their demography, their age was between 19 and43 years, while 24 were male and 7 female. A large portionof our participants had a musical background (21 had per-forming experience). This is also reflected in the averagetime per week a test person makes music (9.4 hours). Weperformed the study by first presenting the second proto-type and then the first followed by the third. Each of thesesessions took 10 minutes after a short introduction aboutthe basic usage. We tried to minimize any necessary in-structions as we wanted to force participants to explore theprototypes. After the evaluation, participants were asked tofill out a questionnaire that contained questions regardingpersonal information and self-assessment, individual experi-ence, group and interaction workflow as well as general feed-back. Most questions used a 5-level Likert scale. Using the


51

Figure 2: Plots of the logged data: a) Weighted Note To Beat Distance, b) the duration of gestures duringinteraction and the questionnaire: c) subjective assessment, d) perceived musical complexity, e) manner ofchanging objects (high means experimental, low means targetted) and f) the changed attributes (high meanstimbre, low means rhythm) for the respective prototypes p1 (green), p2 (red), p3 (blue)

logging data from the interaction with the applications, thegraph structure at every time instant of the session can bereconstructed. This can be further transformed into a pat-tern notation in which every sound generator class is givenits own staff and where all the trigger onsets of their re-spective multiple of a 1/8th are recorded. The length of thepattern is determined by the number of steps it takes untilthe whole rhythmic structure repeats (the smallest commonmultiple of the constituent patterns). After this transfor-mation it is possible to apply rhythmic complexity measuresfor the whole pattern. We used the Weighted Note to BeatDistance measure since it was shown in [13] to agree withhuman perception and because it reflected more of the dy-namic variation of the rhythmic structures in our data setcompared to other measures.

5. RESULTS & CONCLUSIONIn general, the feedback to all of our prototypes was verypositive (cp. fig. 2c) with some of the groups playingup to one hour longer with the prototypes after the ex-periments. Several observations were made during the ex-periments themselves or when watching the correspondingrecorded video material:

• Flow: Participants frequently moved to the beat bynodding head, clapping hands, tapping feet on theground or the fingers on the edges of the tabletop.Even though people sometimes showed how much theyenjoyed the rhythms by facial expression, many ofthese actions were done apparently unconsciously. Theparticipants were especially engaged with the applica-tion when modifying timbres and discovering sounds(as they were shown only a small subset of the soundpalette in the introduction). This was accompanied byshowing emotions of amusement via verbal statementsfacial expressions (e.g. grinning). This shows that theexploration of sounds further caused constructive in-volvement leading to more engagement and curiosity,which are characteristics of a flow experience.

• Group Flow: Participants frequently showed not onlypositive emotions with regard to their own actions butalso in regard to actions of their collaborators. Weobserved that the articulation of Flow of several par-ticipants at the same time led to a more extrovertedexpression of engagement such as vividly dancing tothe produced music, collectively clapping to the beator energetic laughing. In this sense it can be claimedthat at times collaborators felt to be ’in the groove’with each other.

• Styles of interaction: In all groups a division of la-bor was observable. In many groups a single partic-ipant took the role of a coordinator, expressing hisideas while others helped executing them or assign-ing roles (as in a band setting) to his collaborators.This was done in an equally directed and undirected

manner. The degree of verbal communication varied,some groups debated about the next steps while oth-ers did not talk at all but merely used gestures andeye-contact.

• Frequent Processes: We frequently observed the samesequence of steps that groups performed repeatedly tocreate their compositions. Namely exploration, whereseveral persons experiment in parallel in a loosely cou-pled way. Followed by restriction, where after com-plaining about the non-musical quality of the auraloutput the group decides to remove all nodes and torestart the construction. Planning after a consultingconversation about improvement suggestions with thedivision of roles and working methods to avoid ear-lier mistakes. Constructing while the pace of the con-versation fades and the interaction and collaborationbecomes goal oriented. And finally reception of the ac-complished if the musical result is felt worth listeningwhile the group shows positive emotions of joy.

We now look at the data generated by logging the users’interaction with the prototypes. Regarding the questionwhether the additional degrees of freedom for rhythmicalcomplexity have been used in a consequent way, the answeris negative on average (fig. 2a). The median values of thereconstructed rhythmical complexity values for each proto-type show that the complexity generated by prototype twois less than for prototype one, although the opposite resultwould have been expected: for prototype one the maximumamount of rhythmic complexity that is producible is limiteddue to the fixed amount of ”steps” and prototype two allowsmore the degrees of freedom. However, in accordance withexpectation, the rhythmic complexity created with proto-type three surpasses the results of the other two. Theseresults agree with the results of the questionnaire elementsasking for the perceived complexity of the created musicper prototype (fig. 2c). One can conclude that the user’sperception and the measurement of rhythmic complexitymatch, which not only indicates the validity of WNBD asan appropriate measure but also that the subjective user’sassessment is also valid objectively. However, one questionis whether the rhythmical complexities are voluntary or in-voluntary results. For this we will take a closer look at theresults of the questionnaire.

The duration of gestures can be used as measure forwhether users preferred to alter the timbre of sounds inreal time rather than changing the rhythmic structure ofthe sound forms. Figure (fig. 2b) shows the mean of theduration of gestures for each prototype. We chose the meanhere since it is susceptible for outliers and thus may betterrepresent extended gesture input. Here it is apparent that inall applications timbral transformations of the sound formswere performed. However, regarding prototype one it is alsovisible that users may also have performed a large amountof rhythmical modifications.


52

For the evaluation of the questionnaire, we calculatedSpearman’s R as correlation coefficients for pais of answers.Regarding the collaboration itself, a high correlation is ap-parent between the ability to implement own ideas and fos-tering one’s own creativity (0.71) as well as between theability to implement own ideas and the preference to usethe applications collaboratively (0.52). This means thatusers saw their collaborators and the applications as aids topursue their own ideas. A highly positive correlation alsoexists between enjoying collaboration, on the one hand, andeither evaluating the three apps as fostering creativity (0.55)or discovering new musical means (0.56) on the other hand.Users that expressed curiosity when working with the appli-cations also expressed being inspired by the work of otherusers (0.61). Furthermore users who evaluated the musi-cal outcome as harmonious or danceable liked not only themusical outcome in general (0.62, 0.52) but also liked the ex-perimentation with sounds in a collaborative setting (0.36,0.50). This may also lead to the conclusion that the collabo-ration may foster experimentation rather than the targetedconstruction of sound forms. This is apparent in the cor-relation between musical skills (a score accumulated fromseveral biographical questions assigned to every user) andmore critical evaluations of our prototypes: the higher themusical skill the more disappointment (0.37), confinement(0.45), and overstrain (0.46) for all applications were ex-pressed, especially regarding the general assessment of pro-totype two (0.38). However, it seems unlikely that musicallyskilled users evaluated the applications negatively becauseof the actions of their collaborators, since the higher themusical skill the more positively collaborators and the inter-action with them was seen (0.25). Many of our higher musi-cally skilled participants attended a conservatory and thusmay have substantial experience regarding musical collabo-rations. Given the correlation (0.32) that musically skilledusers preferred to use the application three for a more tar-geted construction of sound forms and that users havingstated that the applications exceeded their expectations es-pecially assessed application one positively (0.65), it can bestated that for skilled users, application two is seen as tooconstrictive (corr. btw. skills and general assessment: -0.34)and application three as too difficult to comprehend (corr.btw. skills and comprehensibility: -0.23). The assessment ofapplication one correlates positively with enjoying the ex-perimentation with sounds (0.52). Additionally, there arepositive correlations between the comprehensibility of themusical results generated with application one, on the onehand, and being able to implement own ideas (0.31) or ex-periencing pleasure (0.30) on the other hand. This com-prehensibility of prototype one also correlates to perceivingthe music as rhythmic (0.40) and to being inspired by othercollaborators (0.39). This leads to the conclusion that ap-plication one in general is seen as more comprehensible andtherefore traceable for collaboration. Additionally it seemssuitable for sound-forms whose articulation is more rootedin timbre than in rhythmic (complexity) as can be found intoday’s dance music.

We assume that musically skilled users are adept to copewith and to create more rhythmically complex music. Inthis regard, a good prototype would allow such users tocreate rhythmically complex music compared to less skilledusers. To gain more insight into such a relationship be-tween musical skills and created rhythmical complexity, weperformed a linear regression for these variables for eachprototype. For prototype two, the slope of the best fitstraight line is negative (coeff. -0.159), which may indi-cate that skilled users are not be able to express themselvesaccordingly while less skilled users may have created the

complexity involuntarily. For prototype three the slope hasno clear trend (coeff. 0.05), which may indicate that skilledusers were unable to use their expertise for more targetedexploration. For prototype one, the slope has a stronger up-wards trend (coeff. 0.24). Thus, from the viewpoint of theassumption previously made, this shows that this prototypeis the one that scales best with users’ musical skills. Thismore targeted interaction with prototype one, especially re-garding the construction of the rhythmic structure, is alsoapparent in the users’ evaluation of how they performedtheir changes (fig. 2e) and what was changed (fig. 2f). Fur-thermore this is also supported by the observation that theduration of gesture interaction includes a large portion ofshort events.

Concluding, it is to say that it is important to gathermore empirical data in order to be able to make further re-liable statements investigating the appropriate level of ex-pressivity for collaborative composition support. This is ourintention for future work.

6. REFERENCES[1] M. Csikszentmihalyi. Flow: The Psychology of

Optimal Experience. Harper and Row, New York, NY,USA, 1990.

[2] G. Fischer. Distances and diversity: sources for socialcreativity. In Proc. 5th Conf. Creat. Cogn., pages128–136. ACM, 2005.

[3] P. Holmes. An exploration of musical communicationthrough expressive use of timbre: The performer’sperspective. Psychol. Music, Mar. 2011.

[4] E. Hornecker, P. Marshall, and Y. Rogers. From entryto access: how shareability comes about. InProc. conf. on Designing pleasurable products andinterfaces, pages 328–342. ACM, 2007.

[5] S. Jorda, M. Kaltenbrunner, G. Geiger, andM. Alonso. The reacTable: a tangible tabletopmusical instrument and collaborative workbench. InSIGGRAPH 2006 Sketches, page 91, New York, NY,USA, 2006. ACM.

[6] R. MacDonald. Creativity and flow in musicalcomposition: an empirical investigation. Psychol.Music, 34(3):292–306, July 2006.

[7] M. Meier and M. Schranner. The Planets. (NIME2010):501–504, 2010.

[8] B. Neill. Pleasure beats: rhythm and the aesthetics ofcurrent electronic music. Leonardo Music J., 12:3–6,2002.

[9] L. Nijs. The musical instrument of natural extensionof the musician. Witwatersrand University Press,2009.

[10] J. Pareles. The Rhythm Century: The UnstoppableBeat. New York Times, pages 2–3, 1998.

[11] R. K. Sawyer. Group creativity: musical performanceand collaboration. Psychol. Music, 34(2):148–165,Apr. 2006.

[12] B. Swift. A Chasing After the Wind : Experience inComputer-supported Group Musicmaking. OHZCHI,pages 1–6, 2009.

[13] E. Thul. Measuring the Complexity of MusicalRhythm. PhD thesis, McGill University, 2008.

[14] G. Weinberg, R. Aimi, and K. Jennings. The BeatbugNetwork: A Rhythmisc System for InterdependentGroup Collaboration. Proc. NIME 2002, pages107–111, 2002.

[15] D. Wigdor and D. Wixon. Brave NUI World:Designing Natural User Interfaces for Touch andGesture. Morgan Kaufmann, 2011.


53

treequencer: collaborative rhythm sequencing … collaborative rhythm sequencing - a comparative...

Documents