HarT: Smart Tool for Harmonic ContentTransformations

Diogo Cocharro and Gilberto Bernardes

Sound and Music Computing Group, INESC TEC

{dmfc, gba} @inesctec.pt

Abstract. We present HarT, a software tool for assisting users recreat-

ing existing music content by harmonic transformations while retaining

rhythmic and harmonic function structure. Our main contribution is the

exploration of the perceptually-motivated Tonal Interval Space to ana-

lyze the harmonic content of user-input MIDI files according to the func-

tion each vertical music aggregate plays within the structure, as well as

providing a ranking order of harmonic related chords to a given function.

These strategies are then applied to guide harmonic transformations of

the music content in an intuitive user-navigable map.

Keywords: tonal harmony; music content analysis, generative music,

digital music interfaces

1 Introduction

Based on the assumption that music analysis sits at the core of the music practice— in the sense it provides an interpretation for musical structures which can thensupport music composition — we present HarT (HARmonic Transformations),a system which embeds the first steps towards a smart tool capable of capturingthe harmonic function vertical pitch aggregates play within the musical structureas a means to recreate the musical surface.

In leveraging harmonic function analysis by parsimonious and perceptual-motivated interpretations in the Tonal Interval Space [1], we enable users moreacquainted with technology than music theory to recreate the ever-growing (sym-bolic) music repositories — manifested in formats such as MIDI and MusicXML.From these harmonic function interpretations, our system allows the manipula-tion of the consonance of each vertical aggregate from an input MIDI file whileretaining the rhythmic and tonal function structure of the original music.

Furthermore, our system instantiates a new approach within generative mu-sic, which merges knowledge-based [7] and learning-from-example approaches [4]and addresses the long identified lack of long-term structure in generative mu-sic system [5]. Fig. 1 provides the system architecture which we detail in theremainder of this paper. In Sec. 2, we present the segmentation, key detectionand harmonic function analysis of a MIDI input file. In Sec. 3, we describe ourmethod for transforming the content of annotated MIDI input files. In Sec. 4,we state conclusions and future work.

Proc. of the 13th International Symposium on CMMR, Matosinhos, Portugal, Sept. 25-28, 2017

560

2 Cocharro and Bernardes

Fig. 1. Illustration of the HarT system architecture, which is divided in two major

blocks: Harmonic Function Analysis and Harmonic Content Transformations.

2 Harmonic Function Analysis

A threefold operation chain is used to annotate the harmonic functions of aMIDI input file. We start by 1) segmenting the input into vertical aggregates,which then provides the input elements for 2) estimating the global key of theinput file, and then 3) extracting the harmonic function of each aggregate.

Following [8], we segment a MIDI input file as sequences of vertical pitchaggregates, dubbed ”salami slices”, at each note onset and o↵set. Next, we usethe salami slices as input to a system which estimates the key from a symbolicmusic input in the Tonal Interval Space [1], detailed at length in [2].

Based on the estimated key of a musical phrase we then perform an automaticanalysis of each slice to annotate it with the harmonic function it plays within themusical structure according to the tonic (T), subdominant (SD), and dominant(D), harmonic categories, which can be traced back to Riemman’s functionaltheory of harmony [6]. To this end, we use the perceptually-motivated TonalInterval Space [1], by finding the Euclidean distance to the Tonal Interval Vectorscorresponding to the tonic (I), sub-dominant (IV) and dominant (V) triads ofthe estimated key.

Each of these categories comprises several chords with di↵erent degrees ofconsonance equated in the Tonal Interval Space to Euclidean distances. For ex-ample, the tonic harmonic category includes the tonic, mediant, and submediantdegrees. Fig. 2 shows an example of such analysis compared to expert annota-tions [3]. 1

3 Harmonic Content Transformations

We guide users in the intuitive transformation of the harmonic content of eachslice according to a ranking cost of all possible n-element combinations of all 12pitch classes {0�11}. To preserve the same density as our input slice, n is equal tothe number of notes in each slice. The ranking cost of all n-element combinationsfor each slice is computed as the (user-defined) weighted sum of three TonalInterval Space’s metrics: 1) the Euclidean distance from the harmonic categoryof the slice, 2) the Euclidean distance from the center of the space (a metricof consonance [1]), and 3) the Euclidean distance from the previous slice. The

1

Several more musical analysis are available at: http://bit.ly/2pOo7Uy.

Proc. of the 13th International Symposium on CMMR, Matosinhos, Portugal, Sept. 25-28, 2017

561

HarT: Smart Tool for Harmonic Content Transformations 3

Fig. 2. Comparative analysis of the first phrase of Bach’s choral BWV 274 by an expert

musicologist [3] and the harmonic categories computed from the Tonal Interval Space.

Vertical lines separate the salami slices.

first two are responsible for the perceptual (stability) ranking of the n-elementpitch-class combinations within each harmonic category function and the thirdpromotes parsimonious voice-leading [1]. For each slice, the c = 20 combinationswith minimum cost will then be mapped to a slider which exposes to the userthe ranked perceptual order of vertical aggregates to the harmonic content of theslice (see Fig. 3). The user can then navigate in a space of harmonic possibilitiesand transform the harmonic content of each slice from the MIDI input file witha refined degree of control over the perceptual distance to the harmonic content.

Fig. 3. HarT system User interface. The left side represents the distribution of the

generated pitch configurations (circles) in relation to the harmonic function categories

and the original harmonic segment. The right side shows the user slider with the

position of the original harmony (dashed ellipse) and the current harmonic selection

(closed ellipse).

Proc. of the 13th International Symposium on CMMR, Matosinhos, Portugal, Sept. 25-28, 2017

562

4 Cocharro and Bernardes

Using the method detailed in [2], the surface instantiation of the trans-formed slice is then distributed in the pitch range to convey voice-leading prin-ciples embodied in Western tonal harmony. The HarT system is implemented inMax/MSP2. Fig. 3 illustrates the interface and work-flow of the application.

4 Conclusions

We presented HarT, a software tool for assisting non-expert users in recreatingthe harmonic content of MIDI files by transforming the stability of vertical pitchaggregates. The harmonic transformations are guided by metrics on the TonalInterval Space, which allow users to automatically estimate the harmonic func-tion of the MIDI input vertical aggregates, as well as guarantee their preservationin the transformations. A graphical user interface enables users to navigate in alarge space of harmonic possibilities for a given pitch aggregate. Supporting ma-terial related to the harmonic category analysis and the chord stability rankingwithin each harmony category is available at: http://bit.ly/2pTJpQI.

Acknowledgements Project TEC4Growth - Pervasive Intelligence, Enhancersand Proofs of Concept with Industrial Impact/NORTE-01-0145-FEDER-000020is financed by the North Portugal Regional Operational Programme (NORTE2020), under the PORTUGAL 2020 Partnership Agreement, and through the Eu-ropean Regional Development Fund (ERDF). This research is also supported bythe Portuguese Foundation for Science and Technology under the post-doctoralgrant SFRH/BPD/109457/2015.

References

1. Bernardes, G., Cocharro, D., Caetano, M., Guedes, C., Davies, M.: A multi-level

tonal interval space for modelling pitch relatedness and musical consonance. Journal

of New Music Research 45(4), 281–294 (2016)

2. Bernardes, G., Cocharro, D., Guedes, C., Davies, M.: Harmony generation driven by

a perceptually motivated tonal interval space. ACM Computers in Entertainment

14(2) (2017)

3. Czarnecki, C.: JS Bach, 413 Chorales Analyzed: A Study of the Harmony of JS

Bach. Seezar Publications (2013)

4. Granroth-Wilding, M., Steedman, M.: A robust parser-interpreter for jazz chord

sequences. Journal of New Music Research 43(4), 355–374 (2014)

5. Hutchings, P., McCormack, J.: Using Autonomous Agents to Improvise Music Com-

positions in Real-Time, pp. 114–127. Springer International Publishing (2017)

6. Riemann, H.: Vereinfachte Harmonielehre, oder die Lehre von den tonalen Funktio-

nen der Akkorde. 1896. Tr. H. Bewerunge, Augener, London (1893)

7. Rybnik, M., Homenda, W.: Extension of knowledge-driven harmonization model for

tonal music. In: Int. Joint Conference on Neural Networks. pp. 1–8 (June 2012)

8. White, C., Quinn, I.: The Yale-classical archives corpus. Empirical Musicology Re-

view 11(1), 50–58 (2016)

2

http://www.cycling74.com

Proc. of the 13th International Symposium on CMMR, Matosinhos, Portugal, Sept. 25-28, 2017

563