evoiasp 2008 presentation

Outline

Hybrid Genetic Algorithm based on Gene FragmentCompetition for Polyphonic Music Transcription

Gustavo Reis1 Nuno Fonseca1 Francisco Fernandez2

Anibal Ferreira3

1School of Technology and ManagementPolytechnic Institute of Leiria, Portugal

2University of Extremadura,Spain

3University of Porto, Portugal

EvoIASP, Evo* 27th March 2008, Naples, Italy

Reis, Fonseca, Fernandez, Ferreira Hybrid GA based on Gene Fragment Competition

Outline

1 Abstract

2 Introduction

3 Proposed ApproachGene Fragment CompetitionSimple Example

4 Polyphonic Music Transcription ProblemAutomatic Music TranscriptionGenetic Algorithm ApproachApplying gene fragment competition to music transcription

5 Experiments and Results

6 Conclusions and Future Work

Outline

1 Abstract

2 Introduction

Outline

1 Abstract

2 Introduction

Outline

1 Abstract

2 Introduction

Outline

1 Abstract

2 Introduction

Outline

1 Abstract

2 Introduction

AbstractIntroduction

Proposed ApproachPolyphonic Music Transcription Problem

Experiments and ResultsConclusions and Future Work

Outline

1 Abstract

2 Introduction

Abstract

Gene Fragment Competition

Present the Gene Fragment Competition concept that can beused with Hybrid Genetic Algorithms specially in signal andimage processing

The need for a new semi-local non-exhaustive method

Memetic Algorithms have shown great success in real-lifeproblems by adding local search operators to improve thequality of the already achieved “good” solutions during theevolutionary process.

Traditional local search operators don’t perform well in highlydemanding evaluation processes.

Abstract

Our proposed Approach

A tradeoff between classical Genetic Algorithms andtraditional Memetic Algorithms, performing aquasi-global/quasi-local search by means of gene fragmentevaluation and selection.

The applicability of this hybrid Genetic Algorithm to thesignal processing problem of Polyphonic Music Transcription isshown.

The results obtained show the feasibility of the approach.

Abstract

Outline

1 Abstract

2 Introduction

Introduction

Exploration vs Exploitation

Although Genetic Algorithms (GAs) are very good at rapidlyidentifying good areas of the search space (exploration), theyare often less good at refining near-optimal solutions(exploitation).

Therefore hybrid GAs using local search can search moreefficiently by incorporating a more systematic search in thevicinity of “good” solutions [Hart04].

Introduction

Exploration vs Exploitation

Although Genetic Algorithms (GAs) are very good at rapidlyidentifying good areas of the search space (exploration), theyare often less good at refining near-optimal solutions(exploitation).

Therefore hybrid GAs using local search can search moreefficiently by incorporating a more systematic search in thevicinity of “good” solutions [Hart04].

Introduction

One Max Problem

When a Genetic Algorithm (GA) is applied to the “OneMax”problem, near-optimal solutions are quickly found butconvergence to the optimal solution is slow.

A bit-flipping hill-climber could be quickly applied within eachgeneration for the OneMax to ensure fast convergence.

Introduction

One Max Problem

When a Genetic Algorithm (GA) is applied to the “OneMax”problem, near-optimal solutions are quickly found butconvergence to the optimal solution is slow.

A bit-flipping hill-climber could be quickly applied within eachgeneration for the OneMax to ensure fast convergence.

Introduction

Memetic Algorithms

Memetic Algorithms (MAs) are a class of stochastic globalsearch heuristics in which Evolutionary Algorithms-basedapproaches are combined with local search techniques toimprove the quality of the solutions created byevolution[Hart04].

Memetic Algorithms go a further step by combining therobustness of GAs on identifying good areas of the searchspace with local search for refining near-optimal solutions.

Particularly, they not only converge to high quality solutions,but also search more efficiently than their conventionalcounterparts.

Introduction

Memetic Algorithms

Introduction

Memetic Algorithms

Introduction

A new problem with traditional local search operators arises due tothe cost of evaluation

If the calculation of the fitness function is heavy, having localsearch operators changing each individual several times meanslots of individual evaluations thus lots of computational cost.

In problems that demand high computational cost in fitnessevaluation this might be a prohibitive solution.

Introduction

A new problem with traditional local search operators arises due tothe cost of evaluation

If the calculation of the fitness function is heavy, having localsearch operators changing each individual several times meanslots of individual evaluations thus lots of computational cost.

In problems that demand high computational cost in fitnessevaluation this might be a prohibitive solution.

Introduction

Gene Fragment Evaluation

Fitness evaluation allows us to measure the whole “quality” ofeach individual, and in many cases, it is obtained by simplycombining the values of the evaluations of each gene or genefragment.

But when the evaluation of gene fragments is possible, thesefragment values are not taken on consideration duringrecombination.

Introduction

Gene Fragment Evaluation

Fitness evaluation allows us to measure the whole “quality” ofeach individual, and in many cases, it is obtained by simplycombining the values of the evaluations of each gene or genefragment.

But when the evaluation of gene fragments is possible, thesefragment values are not taken on consideration duringrecombination.

Introduction

A different approach to recombination, that takes advantageof gene/fragment evaluation and gene/fragment selection as away to speed up the process, especially when evaluation ofindividuals is a demanding computational task.

Introduction

The presented method arises from the work of the authors onthe field of Automatic Music Transcription using GeneticAlgorithms.

In this kind of approach the individuals evaluation demandslots of digital signal processing based on a high number ofFFTs, which results on a heavy computational cost makingimpracticable the use of traditional Memetic Algorithms, butstill needing a special operator to increase significantly theperformance.

Introduction

The presented method arises from the work of the authors onthe field of Automatic Music Transcription using GeneticAlgorithms.

In this kind of approach the individuals evaluation demandslots of digital signal processing based on a high number ofFFTs, which results on a heavy computational cost makingimpracticable the use of traditional Memetic Algorithms, butstill needing a special operator to increase significantly theperformance.

Gene Fragment CompetitionSimple Example

Outline

1 Abstract

2 Introduction

Classic GA (a) vs Traditional MA (b) vs Gene FragmentCompetition (c)

Find the Sequence Problem

Individuals Encoding

Fitness Function

Fitness =

genes∑i=1

|O(i)− X (i)| (1)

Fitness Values of Individual1 and Individual2

Breeding of a new born individual

Automatic Music TranscriptionGenetic Algorithm ApproachApplying gene fragment competition to music transcription

Outline

1 Abstract

2 Introduction

Automatic Music Transcription

Monophonic Music Transcription

Polyphonic Music Transcription

Music transcription is a very difficult problem, not only fromthe computational point of view but also in a musical viewsince it can only by addressed by the most skilled musicians.

Traditional approaches to Automatic Music Transcription tryto extract the information directly from audio source signal(using frequency analysis, autocorrelation functions[Klapuri00, Klapuri04], and other digital signal processingtechniques [Marolt04, Dixon00, Bello03, Walmsley99, Goto04,Gomez03]).

Music transcription is a very difficult problem, not only fromthe computational point of view but also in a musical viewsince it can only by addressed by the most skilled musicians.

Traditional approaches to Automatic Music Transcription tryto extract the information directly from audio source signal(using frequency analysis, autocorrelation functions[Klapuri00, Klapuri04], and other digital signal processingtechniques [Marolt04, Dixon00, Bello03, Walmsley99, Goto04,Gomez03]).

Music Transcription as a Search Space Problem

Automatic Music Transcription can be considered as a searchspace problem where the goal is to find the sequence of notesthat best models our audio signal[Lu06].

Instead of trying to deconstruct the audio signal, a searchspace approach will try to find a solution which allows thecreation of a similar audio signal.

Usually search space approaches are not addressed in musictranscription problems due to the huge size of the searchspace.

Nevertheless genetic algorithms[Holland75] have proven to bean excellent tool to find solutions in extremely large searchspaces, since they only need to use a very small subset of theentire search space.

Genetic Algorithm Approach

To apply genetic algorithm to the music transcription problemthere are some important considerations regarding: theencoding of the individuals, the creation of the initialpopulation, recombination and mutation and also the fitnessfunction (as reviewed by Reis et al.[Reis07]).

Each individual (chromosome) will correspond to a candidatesolution, and is made of a sequence of note events (thenumber of notes will most likely be different from individual toindividual, i.e.: the number of genes is not fixed). Each note,acting as a gene, will have the information needed torepresent that note event (e.g. note, start time, duration, anddynamics).

To apply genetic algorithm to the music transcription problemthere are some important considerations regarding: theencoding of the individuals, the creation of the initialpopulation, recombination and mutation and also the fitnessfunction (as reviewed by Reis et al.[Reis07]).

Each individual (chromosome) will correspond to a candidatesolution, and is made of a sequence of note events (thenumber of notes will most likely be different from individual toindividual, i.e.: the number of genes is not fixed). Each note,acting as a gene, will have the information needed torepresent that note event (e.g. note, start time, duration, anddynamics).

Encoding of the individuals

Starting Population Process

Mutation Operators

note change (±1octave, ± half tone)

start position (up to± 0.5 second change)

duration (from 50%to 150%)

velocity (up to ± 16 in a scale of128)

event split (split in two with asilent between)

event remove

new event (random or eventduplication with different note)

Two Additional Mutation Operators

There are 2 mutation operators (with lower probability) that areapplied equally to all events: velocity change (up to ± 4 in a scaleof 128) and duration (from 50% to 150%)

Fitness Evaluation

To evaluate an individual, a synthesizer is used to convert thenote sequence into an audio stream, which will be comparedwith the original audio stream.

Each note has to pass through an internal synthesizer whichin our case is made of a simple sampler, using piano samples.

Fitness Evaluation

The original stream and the synthesized one are compared in thefrequency domain. The streams are segmented in time frames with4096 samples (fs = 44.100kHz) and an overlapping of 75%. AFFT is calculated for each frame and the differences are computed.

Fitness Function

Fitness =tmax∑t=0

fs2∑

f =27.5Hz

||O (t, f ) | − |X (t, f ) ||f

Fitness Evaluation

The original stream and the synthesized one are compared in thefrequency domain. The streams are segmented in time frames with4096 samples (fs = 44.100kHz) and an overlapping of 75%. AFFT is calculated for each frame and the differences are computed.

Fitness Function

Fitness =tmax∑t=0

fs2∑

f =27.5Hz

||O (t, f ) | − |X (t, f ) ||f

Algorithm Parameters

Parameters ValuesPopulation 100

Survivor Selection Best 100 individuals (population size)

Crossover probability 75%

Mutation probability 1%

Note minimal duration 20 ms

Applying Gene Fragment Competition to MusicTranscription

Remarks

It must be possible to evaluate gene or gene fragments.

Each gene represents a musical note and it is not possible toevaluate each note, especially in polyphonic parts.

The overall fitness of each individual is obtained by adding theFFT differences over the different time frames.

It is possible to evaluate time frames (for instance, it ispossible to evaluate the behavior of an individual on a timefragment between time=2.0s and time=4.0s).

Remarks

It is possible to map that time interval to the genes (notes)acting on that time fragment.

If some notes on that fragment began before or end after thetime frontiers, the note is split, and only the inside part areconsidered.

Later, during the recombination merge phase, if a note endson the exact same time that a similar note begins, notes aremerged as one, since the algorithm considers that a previoussplit happened.

Remarks

Selection Operators

For global selection, the “deterministic tournament” methodwas used, but in fragment selection a “non-deterministictournament” method was used as a means to preserve somebiodiversity.

Regarding fragment size, that in this case is measure inseconds since we consider time fragments, the value of 5seconds was used.

Selection Operators

For global selection, the “deterministic tournament” methodwas used, but in fragment selection a “non-deterministictournament” method was used as a means to preserve somebiodiversity.

Regarding fragment size, that in this case is measure inseconds since we consider time fragments, the value of 5seconds was used.

Selection Operators

Increasing the fragment size should decrease the impact of theoperator, and decreasing fragment size has the side effect ofsplitting too much the notes.

For each individual, a cache feature was implemented, thatstored the fitness values for each time frame, which meansthat evaluating a time fragment is simply done by addingfitness values of its internal time frames.

Selection Operators

Increasing the fragment size should decrease the impact of theoperator, and decreasing fragment size has the side effect ofsplitting too much the notes.

For each individual, a cache feature was implemented, thatstored the fitness values for each time frame, which meansthat evaluating a time fragment is simply done by addingfitness values of its internal time frames.

Outline

1 Abstract

2 Introduction

Experiments and Results

New Parameters

Classic GASelection Deterministic Tournament (5 individuals)

Recombination 1-point time crossover(with eventual note split)

Gene FragmentIndividual Selection Deterministic Tournament (5 individuals)

Fragment Selection Tournament (5 individuals)

Fragment size 5 secondsa

aResulting in 6 fragments on 30 seconds audio files.

To analyze the impact of the presented method, several testswere made.

The proposed method was applied on our music transcriptionapproach on an audio file with the first 30 seconds of thepiano performance of the Schubert’s Impromptu No.2 in EMinor.

Each bank of tests was created with 1000 generations, andwith at least 4 different runs (the values presented correspondto the average values).

First bank of tests

In the first bank of tests, there were 3 different scenarios:classic GA approach, static fragment size and dynamicfragment size of the proposed method.

Average fitness values of over 1000 generations.

First bank of tests

In the first bank of tests, there were 3 different scenarios:classic GA approach, static fragment size and dynamicfragment size of the proposed method.

Average fitness values of over 1000 generations.

First bank of tests

Tests were made also with another audio file to confirm theearlier results.

A 30s seconds audio file of Mozart’s Piano Sonata n. 17 in Bflat K570 was used.

Once again the proposed method presents an increase ofperformance.

In this test the performance difference between static anddynamic fragment size also increases comparatively with theinitial test.

First bank of tests

Average Fitness Values

Very significative gain in performance

It is important to say that in both tests by applying our operatorwith dynamic size, we have achieved in only 500 generations thesame results the classical GA achieves in 1000 generations, whichis a very significative gain in performance.

Average Fitness Values

Very significative gain in performance

It is important to say that in both tests by applying our operatorwith dynamic size, we have achieved in only 500 generations thesame results the classical GA achieves in 1000 generations, whichis a very significative gain in performance.

Outline

1 Abstract

2 Introduction

Conclusions and Future Work

This paper has presented a Gene Fragment Competition as anew technique for improving quality of results when applyingGAs to several signal and image processing problems.

Although the proposed method presents some requirementsthat are not fulfilled on several GA applications (capability ofevaluating fragment of genes), it is shown that at least insome scenarios can achieve an important performanceincrease.

In the future, additional tests must be made to have a betterunderstanding of the impact of the operator applied ondifferent problem applications.

Thanks for your attention

Questions?

evoiasp 2008 presentation

Documents