the art, joy, and power of creating musical programs (jfugue at sxsw interactive 2015)
TRANSCRIPT
Musically-Inclined Programmers!
Programming-Inclined Musicians!
Anyone who wants to infuse fun into programming!
Who is JFugue for?
import org.jfugue.player.Player;
public class HelloWorld {public static void main(String[] args) {
Player player = new Player();player.play(“C”);
}}
“Hello, World!” in JFugue
import org.jfugue.player.Player;
public class HelloWorld {public static void main(String[] args) {
Player player = new Player();player.play(“C”);
}}
“Hello, World!” in JFugue
Sequencer sequencer = MidiSystem.getSequencer();Sequence sequence = sequencer.getSequence();Track track = sequence.createTrack();ShortMessage onMessage = new ShortMessage();onMessage.setMessage(ShortMessage.NOTE_ON, 0, 60, 128);MidiEvent noteOnEvent = new MidiEvent(onMessage, 0);track.add(noteOnEvent);ShortMessage offMessage = new ShortMessage();offMessage.setMessage(ShortMessage.NOTE_OFF, 0, 60, 128);MidiEvent noteOffEvent = new MidiEvent(offMessage, 200);track.add(noteOffEvent);sequencer.start();try { Thread.sleep(track.ticks());} catch (InterruptedException e) { Thread.currentThread().interrupt();}
“Hello, World!” without JFugue
public class TwelveBarBlues { public static void main(String[] args) { Pattern pattern = new ChordProgression(“I IV V”) .distribute(“7%6”) .allChordsAs(“$0 $0 $0 $0 $1 $1 $0 $0 $2 $1 $0 $0”) .eachChordAs(“$0i $1i $2i $3i $4i $3i $2i $1i”) .getPattern() .setInstrument(“Acoustic Bass”) .setTempo(120); new Player().play(pattern); }}
Advanced JFugue… still fits on one slide
video stream is created
video stream has failed
video stream is tricked
video stream is destroyed
video content is added
SNMP trap emitted
Interesting Events in Video On Demand
(a video stream is content that is being played)
video stream is created
video stream has failed
video stream is tricked
video stream is destroyed
video content is added
SNMP trap emitted
Interesting Events in Video On Demand
video stream is created
video stream is destroyed
video stream has failed
[BASS_DRUM]s
[ACOUSTIC_SNARE]s
[CRASH_CYMBAL_1]s
Mapping Events to Music
Learn more at Log4JFugue.org
Winner, 2010 Duke’s
Choice Award
Integrates with Spring Framework
Scala version now available
TransProse creates music from literature
Songs are based on the emotional content of a novel
Introducing TransProse
unique word prevalence
average sentence length
book length
punctuation
presence of main characters
descriptive words
chapters/sections
changes between tones/moods
narrator perspective
pacing, plot action/movement
locations/environment
overall mood/tone/emotion
real world or mental landscape
smaller individual scenes
character ages, voices
time period
average syllable count
amount of dialog
Textual Things to Analyze
unique word prevalence
average sentence length
book length
punctuation
presence of main characters
descriptive words
chapters/sections
changes between tones/moods
narrator perspective
pacing, plot action/movement
locations/environment
overall mood/tone/emotion
real world or mental landscape
smaller individual scenes
character ages, voices
time period
average syllable count
amount of dialog
Textual Things to Analyze
minor/major key
tempo
note duration
harmony
pitch
changing between keys
different instrument voices
rhythm
time signature
melody
lyrics/vocals
# of notes in key used(complexity)
Musical Elements to Use
minor/major key
tempo
note duration
harmony
pitch
changing between keys
different instrument voices
rhythm
time signature
melody
lyrics/vocals
# of notes in key used(complexity)
Musical Elements to Use
amount of emotion
low emotion counts
high emotion counts
spikes in emotion
active:passive emotions
positive:negative emotion
amt of emotion per section
total emotions in chapter
prevalent emotions in novel
stability of note from root note
more consonant notes
more dissonant notes
interesting melodic movement
tempo
major/minor key and octave
note duration
main melody
additional melodies
Mappings From Analysis to Music
Learn more at MusicFromText.com
TIMEDiscoveryWiredio9NBCEngadget
Fast CompanyPopular ScienceCNETDer SpiegelLe Monde…
Lots of pressMore complex music
Motifs for main characters
Additional instruments and
genres
Future Work
Interactive Experiences & GamesAlgorithmic/Generative Music
Music Based on DataEducation
Art & DesignSoftware Support for Cultural Music
General Categories
“I'm analyzing microblogs like YikYak, Tumblr and Twitter to get trending topics. I'm trying to translate text to quantitative
data (semantics, topic, gender, time of day, etc.) to then interpolate and put
through a genetic algorithm to generate music.”
-Tyler
Music from Live-Streaming Data
Soundtrack for Astrophotography Show
Based on 6000 naked-eye stars:declination (latitude) pitch
right ascension (longitude) timemagnitude volume and duration
“Since JFugue can turn MIDI into text, algorithmically created music can be made
using the same technology your phone uses for predicting the next word”
JFugue and Markov chaining
Easy to UseBased on Music TheoryFun Things are Baked In
Consistent “System”Event-Based Architecture
Five Bits of JFugue Magic
Music is easy to specify and intentionally human-readable
C Middle-CC#6h C# note, 6th octave, half durationEbMAJ^w E-flat major chord, first inversion,
whole duration
Ease of Use
Music is easy to specify and intentionally human-readable
I[Flute] Change instrument to FluteT[Allegro] Change tempo to Allegro (120bpm)m560.8q Play a 560.8 Hz tone at ¼-durationRq Rest (takes duration just like a note)
Ease of Use
Pattern melodyVerseOne = new Pattern("T115 V0 I[Synth_Choir]") .add("Bi B B Ai Gi Bihi Bi Bi Bi Aqi F#ih Rhi F#i Ai Bi").add("Aqit Ehtq Rhs Gi Aqi+C#6qi Bi+D6i Aih+C#6ih Rhqi B5i").add("Bi B B Ai Gi Bihi Bi Bi Bi Aqi F#ih Rhi F#i Ai Bi").add("Aqit Ehtq Rhs Gi Aqi+C#6qi Bi+D6i Aih+C#6ih Rh B D6");
Ease of Use
public class TwelveBarBlues { public static void main(String[] args) { Pattern pattern = new ChordProgression(“I IV V”) .distribute(“7%6”) .allChordsAs(“$0 $0 $0 $0 $1 $1 $0 $0 $2 $1 $0 $0”) .eachChordAs(“$0i $1i $2i $3i $4i $3i $2i $1i”) .getPattern() .setInstrument(“Acoustic Bass”) .setTempo(120); new Player().play(pattern); }}
Ease of “Expression”
Define a Scale in terms of IntervalsDefine a Chord in terms of IntervalsCreate a “I IV V” Chord ProgressionSet the Root of a Chord ProgressionGet the Chords from a Chord ProgressionGet the Notes from a Chord…
Based on Music Theory
Rhythm rhythm = new Rhythm().addLayer("O..oO...O..oOO..").addLayer("..S...S...S...S.").addLayer("````````````````").addLayer("...............+");
new Player() .play(rhythm.getPattern().repeat(2));
Fun Things are Baked In
Pattern prog = ChordProgression .fromChords("C#4min E4maj B3maj A3maj") .eachChordAs("$_i $_i Ri $_i") .getPattern() .setInstrument("GUITAR") .setVoice(0);
Rhythm rhythm = new Rhythm() .addLayer("..X...X...X...XO");
new Player() .play(new Pattern(prog, rhythm).repeat(2));
Consistent “System”
MidiParser parser = new MidiParser();
LilyPondListener listener = new LilyPondListener();
parser.addListener(listener);
parser.parse(midi sequence);
listener.displayEngravedSheetMusic();
Event-Based Architecture
StaccatoParser parser = new StaccatoParser();
LilyPondListener listener = new LilyPondListener();
parser.addListener(listener);
parser.parse(“C D E F G A B”);
listener.displayEngravedSheetMusic();
Event-Based Architecture
StaccatoParser parser = new StaccatoParser();
MusicXmlListener listener = new MusicXmlListener();
parser.addListener(listener);
parser.parse(“C D E F G A B”);
listener.save(new File(“my_song.xml”));
Event-Based Architecture
MidiParser parser = new MidiParser();MyParserListener listener = new MyParserListener();parser.addParserListener(listener);parser.parse(MidiSystem.getSequence(new File(“some midi file")));System.out.println("There are "+listener.counter+" 'C' notes in this music.");
class MyParserListener extends ParserListenerAdapter { public int counter; @Override public void onNoteParsed(Note note) { if (note.getPositionInOctave() == 0) { counter++; } }}
Event-Based Architecture
Sending to / Receiving from MIDI Devices
Creating New Patterns in Real-Time
Sending Musical Events “Before” They Occur
Extending the Staccato Parser
Using Replacement Maps to Play Solfege
Writing Functions for Musical Effects (e.g., Trill)
Scratching the Surface…
Making Music from DNA
Many people have done this many ways…
http://www.mimg.ucla.edu/faculty/miller_jh/gene2music/previouswork.html
Download JFugue
Make beautiful and creative things!
http://www.jfugue.org
Optionally:
Contribute to JFugue!
Live coding, Tools for Melodic Structure, Making Awesome Music…
What Next?