the art, joy, and power of creating musical programs (jfugue at sxsw interactive 2015)

59
The Art, Joy, & Power of Creating Musical Programs David Koelle @dmkoelle

Upload: david-koelle

Post on 17-Jul-2015

419 views

Category:

Software


0 download

TRANSCRIPT

The Art, Joy, & Power ofCreating Musical Programs

David Koelle @dmkoelle #JFugue

play(“C”);

Musically-Inclined Programmers!

Programming-Inclined Musicians!

Anyone who wants to infuse fun into programming!

Who is JFugue for?

import org.jfugue.player.Player;

public class HelloWorld {public static void main(String[] args) {

Player player = new Player();player.play(“C”);

}}

“Hello, World!” in JFugue

import org.jfugue.player.Player;

public class HelloWorld {public static void main(String[] args) {

Player player = new Player();player.play(“C”);

}}

“Hello, World!” in JFugue

Sequencer sequencer = MidiSystem.getSequencer();Sequence sequence = sequencer.getSequence();Track track = sequence.createTrack();ShortMessage onMessage = new ShortMessage();onMessage.setMessage(ShortMessage.NOTE_ON, 0, 60, 128);MidiEvent noteOnEvent = new MidiEvent(onMessage, 0);track.add(noteOnEvent);ShortMessage offMessage = new ShortMessage();offMessage.setMessage(ShortMessage.NOTE_OFF, 0, 60, 128);MidiEvent noteOffEvent = new MidiEvent(offMessage, 200);track.add(noteOffEvent);sequencer.start();try { Thread.sleep(track.ticks());} catch (InterruptedException e) { Thread.currentThread().interrupt();}

“Hello, World!” without JFugue

public class TwelveBarBlues { public static void main(String[] args) { Pattern pattern = new ChordProgression(“I IV V”) .distribute(“7%6”) .allChordsAs(“$0 $0 $0 $0 $1 $1 $0 $0 $2 $1 $0 $0”) .eachChordAs(“$0i $1i $2i $3i $4i $3i $2i $1i”) .getPattern() .setInstrument(“Acoustic Bass”) .setTempo(120); new Player().play(pattern); }}

Advanced JFugue… still fits on one slide

Real-World Examples

Log Files Classic Novels

Log Files Classic Novels

Brian Tarbox

“I want to know the status of

my video servers without

staring at a screen.”

time

video server log files

time

video server log files

video server log files

video stream is created

video stream has failed

video stream is tricked

video stream is destroyed

video content is added

SNMP trap emitted

Interesting Events in Video On Demand

(a video stream is content that is being played)

video stream is created

video stream has failed

video stream is tricked

video stream is destroyed

video content is added

SNMP trap emitted

Interesting Events in Video On Demand

video stream is created

video stream is destroyed

video stream has failed

[BASS_DRUM]s

[ACOUSTIC_SNARE]s

[CRASH_CYMBAL_1]s

Mapping Events to Music

Listening to Video on Demand Servers

Learn more at Log4JFugue.org

Winner, 2010 Duke’s

Choice Award

Integrates with Spring Framework

Scala version now available

Log Files Classic Novels

Hannah Davis

“I want to create a new

interactive experience to

perceive classic novels.”

TransProse creates music from literature

Songs are based on the emotional content of a novel

Introducing TransProse

unique word prevalence

average sentence length

book length

punctuation

presence of main characters

descriptive words

chapters/sections

changes between tones/moods

narrator perspective

pacing, plot action/movement

locations/environment

overall mood/tone/emotion

real world or mental landscape

smaller individual scenes

character ages, voices

time period

average syllable count

amount of dialog

Textual Things to Analyze

unique word prevalence

average sentence length

book length

punctuation

presence of main characters

descriptive words

chapters/sections

changes between tones/moods

narrator perspective

pacing, plot action/movement

locations/environment

overall mood/tone/emotion

real world or mental landscape

smaller individual scenes

character ages, voices

time period

average syllable count

amount of dialog

Textual Things to Analyze

Emolex – 14K tagged words

The Art of Making the Music

Stable Tones

Emotion Counts

Musical Considerations

Melodic Shape

minor/major key

tempo

note duration

harmony

pitch

changing between keys

different instrument voices

rhythm

time signature

melody

lyrics/vocals

# of notes in key used(complexity)

Musical Elements to Use

minor/major key

tempo

note duration

harmony

pitch

changing between keys

different instrument voices

rhythm

time signature

melody

lyrics/vocals

# of notes in key used(complexity)

Musical Elements to Use

amount of emotion

low emotion counts

high emotion counts

spikes in emotion

active:passive emotions

positive:negative emotion

amt of emotion per section

total emotions in chapter

prevalent emotions in novel

stability of note from root note

more consonant notes

more dissonant notes

interesting melodic movement

tempo

major/minor key and octave

note duration

main melody

additional melodies

Mappings From Analysis to Music

Learn more at MusicFromText.com

TIMEDiscoveryWiredio9NBCEngadget

Fast CompanyPopular ScienceCNETDer SpiegelLe Monde…

Lots of pressMore complex music

Motifs for main characters

Additional instruments and

genres

Future Work

More Real-World Examples

Interactive Experiences & GamesAlgorithmic/Generative Music

Music Based on DataEducation

Art & DesignSoftware Support for Cultural Music

General Categories

SoundOfShopping.com

Turning barcodes into music

“I'm analyzing microblogs like YikYak, Tumblr and Twitter to get trending topics. I'm trying to translate text to quantitative

data (semantics, topic, gender, time of day, etc.) to then interpolate and put

through a genetic algorithm to generate music.”

-Tyler

Music from Live-Streaming Data

Making Music from a Chess Match

Soundtrack for Astrophotography Show

Based on 6000 naked-eye stars:declination (latitude) pitch

right ascension (longitude) timemagnitude volume and duration

“Since JFugue can turn MIDI into text, algorithmically created music can be made

using the same technology your phone uses for predicting the next word”

[email protected]

JFugue and Markov chaining

A Tour of the Magic of JFugue

Easy to UseBased on Music TheoryFun Things are Baked In

Consistent “System”Event-Based Architecture

Five Bits of JFugue Magic

Play music with two lines of code:

Player player = new Player();player.play(“C”);

Ease of Use

Music is easy to specify and intentionally human-readable

C Middle-CC#6h C# note, 6th octave, half durationEbMAJ^w E-flat major chord, first inversion,

whole duration

Ease of Use

Music is easy to specify and intentionally human-readable

I[Flute] Change instrument to FluteT[Allegro] Change tempo to Allegro (120bpm)m560.8q Play a 560.8 Hz tone at ¼-durationRq Rest (takes duration just like a note)

Ease of Use

Pattern melodyVerseOne = new Pattern("T115 V0 I[Synth_Choir]") .add("Bi B B Ai Gi Bihi Bi Bi Bi Aqi F#ih Rhi F#i Ai Bi").add("Aqit Ehtq Rhs Gi Aqi+C#6qi Bi+D6i Aih+C#6ih Rhqi B5i").add("Bi B B Ai Gi Bihi Bi Bi Bi Aqi F#ih Rhi F#i Ai Bi").add("Aqit Ehtq Rhs Gi Aqi+C#6qi Bi+D6i Aih+C#6ih Rh B D6");

Ease of Use

public class TwelveBarBlues { public static void main(String[] args) { Pattern pattern = new ChordProgression(“I IV V”) .distribute(“7%6”) .allChordsAs(“$0 $0 $0 $0 $1 $1 $0 $0 $2 $1 $0 $0”) .eachChordAs(“$0i $1i $2i $3i $4i $3i $2i $1i”) .getPattern() .setInstrument(“Acoustic Bass”) .setTempo(120); new Player().play(pattern); }}

Ease of “Expression”

Define a Scale in terms of IntervalsDefine a Chord in terms of IntervalsCreate a “I IV V” Chord ProgressionSet the Root of a Chord ProgressionGet the Chords from a Chord ProgressionGet the Notes from a Chord…

Based on Music Theory

Rhythm rhythm = new Rhythm().addLayer("O..oO...O..oOO..").addLayer("..S...S...S...S.").addLayer("````````````````").addLayer("...............+");

new Player() .play(rhythm.getPattern().repeat(2));

Fun Things are Baked In

Pattern prog = ChordProgression .fromChords("C#4min E4maj B3maj A3maj") .eachChordAs("$_i $_i Ri $_i") .getPattern() .setInstrument("GUITAR") .setVoice(0);

Rhythm rhythm = new Rhythm() .addLayer("..X...X...X...XO");

new Player() .play(new Pattern(prog, rhythm).repeat(2));

Consistent “System”

MidiParser parser = new MidiParser();

LilyPondListener listener = new LilyPondListener();

parser.addListener(listener);

parser.parse(midi sequence);

listener.displayEngravedSheetMusic();

Event-Based Architecture

StaccatoParser parser = new StaccatoParser();

LilyPondListener listener = new LilyPondListener();

parser.addListener(listener);

parser.parse(“C D E F G A B”);

listener.displayEngravedSheetMusic();

Event-Based Architecture

StaccatoParser parser = new StaccatoParser();

MusicXmlListener listener = new MusicXmlListener();

parser.addListener(listener);

parser.parse(“C D E F G A B”);

listener.save(new File(“my_song.xml”));

Event-Based Architecture

MidiParser parser = new MidiParser();MyParserListener listener = new MyParserListener();parser.addParserListener(listener);parser.parse(MidiSystem.getSequence(new File(“some midi file")));System.out.println("There are "+listener.counter+" 'C' notes in this music.");

class MyParserListener extends ParserListenerAdapter { public int counter; @Override public void onNoteParsed(Note note) { if (note.getPositionInOctave() == 0) { counter++; } }}

Event-Based Architecture

Sending to / Receiving from MIDI Devices

Creating New Patterns in Real-Time

Sending Musical Events “Before” They Occur

Extending the Staccato Parser

Using Replacement Maps to Play Solfege

Writing Functions for Musical Effects (e.g., Trill)

Scratching the Surface…

Making Music from DNA

Making Music from DNA

Many people have done this many ways…

http://www.mimg.ucla.edu/faculty/miller_jh/gene2music/previouswork.html

Download JFugue

Make beautiful and creative things!

http://www.jfugue.org

Optionally:

Contribute to JFugue!

Live coding, Tools for Melodic Structure, Making Awesome Music…

What Next?

Thank You!

David Koelle @dmkoelle#JFugue