aws deepcomposer - developer guide · chord feature generates a simple triad chord taking the note...

AWS DeepComposerDeveloper Guide

AWS DeepComposer Developer Guide

AWS DeepComposer: Developer GuideCopyright © 2020 Amazon Web Services, Inc. and/or its affiliates. All rights reserved.

Amazon's trademarks and trade dress may not be used in connection with any product or service that is notAmazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages ordiscredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who mayor may not be affiliated with, connected to, or sponsored by Amazon.


Table of ContentsWhat is AWS DeepComposer? ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Concepts and Terminology .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Keyboard hardware .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Setup .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Keyboard features .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

How it works .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Get started .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Pre-trained models ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Sample melody (pre-trained models) ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Custom melody (pre-trained models) ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Custom Models ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Sample melody (custom trained models) ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Custom melody (custom trained models) ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Evaluating trained models ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16Browser support ... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20Document history .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21AWS glossary .... . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

iii

AWS DeepComposer Developer GuideConcepts and Terminology

What is AWS DeepComposer?AWS DeepComposer provides a creative, hands-on experience for learning generative AI and machinelearning. With generative AI, one of the biggest recent advancements in artificial intelligence, you cancreate a new dataset based on a training dataset. With AWS DeepComposer, you can experiment withdifferent generative AI architectures and models in a musical setting by creating and transformingmusical inputs and accompaniments.

Regardless of your experience with machine learning (ML) or music, you can use AWS DeepComposer todevelop a working knowledge of generative AI. AWS DeepComposer includes learning capsules, samplecode, and training data to help you understand and use generative AI models.

To get started with AWS DeepComposer, start the AWS DeepComposer Music studio, choose one of thesample melodies, and choose a pretrained model. After you generate a composition, you can change theinstruments, download your new composition, and share it with friends on SoundCloud.

To flex your creativity, record a custom melody using either the console-based keyboard or the AWSDeepComposer keyboard. To dive deeper, start training custom models using training data provided byAWS. Want more? Learn how to create your own generative adversarial networks (GANs) by using theexamples in Jupyter notebooks for Amazon SageMaker.

Important - Browser requirementsAWS DeepComposer fully supports the Chrome browser. Other browsers offer limited supportfor the AWS DeepComposer console and hardware. For more information about browsercompatibility, see Browser support (p. 20).

Topics• AWS DeepComposerconcepts and terminology (p. 1)• AWS DeepComposer keyboard (p. 3)

AWS DeepComposerconcepts and terminologyAWS DeepComposer builds on the following concepts and uses the following terminology.

AWS DeepComposer keyboard

Also known as the AWS DeepComposer physical keyboard or hardware keyboard. You can connectit to a computer with access to the AWS DeepComposer console, play and record a short melodies.Then use a recorded melody with a supported AWS DeepComposer generative AI technique.

Virtual keyboard

Also known as the AWS DeepComposer physical keyboard or hardware keyboard. You can connectthe keyboard to a computer that has access to the AWS DeepComposer console, play and record ashort melody, and feed the melody to a supported generative AI algorithm to create new musicalcompositions.

compositions

Sequences of notes, melodies, harmonies, and rhythms that make up a musical work.convolutional neural network (CNN)

A type of neural network commonly used for image and video recognition tasks. A CNN filtersand summarizes the data during training to learn patterns. This mathematical process is calledconvolution.

1

https://console.aws.amazon.com/deepcomposer/home?region=us-east-1#musicStudio

AWS DeepComposer Developer GuideConcepts and Terminology

decoder

A process that takes an input vector, commonly a features matrix, and transforms the vector into anoutput. A decoder can be used in conjunction with an encoder. AWS DeepComposer uses an encoder-decoder network for some generative AI tasks.

drum in pattern

A ratio of the total number of notes in a drum track to a predetermined beat popular in 4/4 time.empty bar rate

The ratio of empty bars to the total number of bars.encoder

A process used to translate data from one format to another. AWS DeepComposer uses multipleencoders. For example, it uses an encoder to translate an input MIDI file into a piano roll image.Often, an encoder is used with a decoder.

epoch

One complete pass through the training data by the neural network. For example, if you have 10,000music tracks in the training data set, one epoch represents one pass through all 10,000 tracks. Thenumber of epochs required for a model to converge varies based on the training set.

generative adversarial network (GAN)

Two neural networks that consists of a generator and a discriminator. In AWS DeepComposer, thegenerator learns to compose music as realistic as possible with feedback from the discriminator. Thediscriminator treats the generator's output as unrealistic as possible while holding the input trainingsample as the ground truth. Training proceeds with the generator searching for its network weightsby minimizing the chances that its generations differ from the training samples, and with thediscriminator searching for the both networks' weights by minimizing the chances it misjudges thetraining samples as realistic compositions and the generator's outputs as unrealistic compositions.The efforts made by both the generator and the discriminator push the generator's network weightsin opposite directions. This is the essence of the adversarial role that the discriminator takes againstthe generator. Learning proceeds until equilibrium is reached, where the generator improves itsoutput to such a degree that the discriminator can no longer distinguish between the generatedcomposition and realistic music.

inference

Predictions generated by a model.hyperparameters

Algorithm-dependent variables controlled by the user. They are often tuned to a best fit for aspecific problem that you are trying to model.

in scale ratio

The ratio of the average number of notes in a bar, the key signature, to the total number of notes.learning rate

A hyperparameter used for training neural networks that controls how much the weights and biasesare updated during training.

loss function

Evaluates how good an algorithm is at modeling the data. For example, if a model consistentlypredicts values very different from the actual values, it would return a large loss. Depending on thealgorithm used for training, more than one loss function might be used.

model

An artifact that is created while training with a machine learning algorithm. The algorithm usedto train the model finds patterns in the training data that map the input data attributes to the

2

AWS DeepComposer Developer GuideKeyboard hardware

target (the answer that you want to predict). The algorithm outputs a machine learning model thatcaptures these patterns.

MuseGAN

A generative adversarial network (GAN) architecture built specifically to generate music. Like otherGANs, MuseGAN is made of both a discriminator and a generator that uses a CNN. The MuseGANarchitecture is available in AWS DeepComposer. To learn more about the MuseGAN architecture, see creating the MuseGAN architecture.

Music studio

A component of the AWS DeepComposer console that provides access to a console-based keyboardfor playing, recording, and composing music. You don't need to train a model to use Music studio.With Music studio, you can try out AWS DeepComposer before purchasing a AWS DeepComposerphysical keyboard.

neural network network

Also known as an artificial neural network. A collection of connected units or nodes that are used tobuild an information model based on biological systems. Each node is called an artificial neuron. Anartificial neuron mimics a biological neuron in that it receives an input (stimulus), becomes activatedif the input signal is strong enough (activation), and produces an output predicated upon the inputand activation. Neural networks are widely used in machine learning because an artificial neuralnetwork can serve as a general-purpose approximation to any function.

pitches used

A metric that captures the average number of notes in each bar.

polyphonic rate

The ratio of the number of time steps where the number of pitches being played is greater than the"threshold" number of allowable time steps.

Rhythm assist

An AWS DeepComposer function that automatically corrects the timing of musical notes in yourinput. Use Rhythm assist when your melody contains the right notes, but isn't perfectly in time withthe beat. Rhythm assist is available in Music studio.

virtual keyboard

Also known as the AWS DeepComposer console-based keyboard. The virtual keyboard is available inthe AWS DeepComposer music studio. The virtual keyboard runs on any computer or mobile deviceconnected to the AWS Cloud. You can use the virtual keyboard to play and record a short melody,then use the recorded melody with a supported AWS DeepComposer generative AI technique tocreate new musical compositions.

AWS DeepComposer keyboardThe AWS DeepComposer keyboard is a music keyboard that you can use to get hands-on experiencelearning generative AI and machine learning (ML). You can connect it to any computer that has access tothe AWS DeepComposer console, play and record a short melody, and feed the melody to a supportedgenerative AI architecture supported by AWS DeepComposer.

Setting up the AWS DeepComposer keyboardTo get started using your AWS DeepComposer keyboard, link your device inside the AWS DeepComposerconsole. To do so, follow the steps outlined below.

3

https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/17286/15668

https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/17286/15668

https://console.aws.amazon.com/deepcomposer/home?region=us-east-1#deepComposerKeyboard


AWS DeepComposer Developer GuideSetup

Connecting the AWS DeepComposer keyboard

1. Open the link your keyboard section of the AWS DeepComposer console.

2. Use the included USB cable to connect the keyboard to your computer.

3. On the back of the keyboard, locate the 8- or 16-digit alphanumeric serial number (S/N or DSN).

4. In step 2 in the AWS DeepComposer console, enter the alphanumeric serial number.


4


AWS DeepComposer Developer GuideKeyboard features

AWS DeepComposer keyboard features

5

AWS DeepComposer Developer GuideKeyboard features

The AWS DeepComposer keyboard is 32-key, 2 octave keyboard. It has several built-in features that aredesigned to increase the usability of the AWS DeepComposer keyboard.

Arpeggiator (arp)

To create an arpeggio, use this button to step through a group of notes in a particular sequence tocreate an arpeggio.

Auto-chord (chord)

Use this button to generate a chord by playing a single note. When you play a note, the auto-chordfeature generates a simple triad chord using the note provided as the root note.

Modulation

Use this wheel to modulate the synthetic audio signal to create a vibrato effect.Octave adjust

The default starting position of the AWS DeepComposer keyboard is the C4 and C3 octaves. Theoctave down button adjusts the keyboard down one or more octaves. The octave up button adjuststhe keyboard up one or more octaves.

Pitch

Use this wheel to slightly bend (change) the pitch of a note up or down.Playback

Use this button to playback a melody that you just recoreded.Record

Use this button to begin recording a melody on your keyboard. To stop recording, tap the buttonagain.

Sustain switch interface

You can plug any universal MIDI-compliant sustain pedal into your keyboard through this port.Tap tempo

Tap this button to set the arpeggio tempo manually.USB cable port

Use this port to connect the physical keyboard to the AWS DeepComposer console (Music studio).Volume

Use this slider enables to adjust the volume on your keyboard.

6


How AWS DeepComposer worksAt a fundamental level, music composition corresponds to sequences of notes of intricate tempo anddynamics. Musical compositions are categorized into genres based on discernible differences in thedistribution different musical elements. To compose music for specific genre you have learn that genrespecific distribution. At its core that is what the different generative AI techniques in AWS DeepComposerallow you to do.

This process involves figuring out the appropriate patterns and how to arrange them together incohesive way. Aspiring composers typically have to spend several years training to learn these patterns.When composers graduate, they have developed an acute appreciation of the distributions of differentmusical elements in the genres they studied. Their understanding might be built on non-quantitativeterms but they have built a quantitative distribution into their brains. They have developed a a naturalneural network model of music composition and mastered ways to apply the model with greatproficiency.

In machine learning, it's analogous to teaching a machine to compose music of a given genre. Themachine learns the compositional features or patterns from a known musical collection to develop apractical understanding of the distribution of musical elements. The knowledge is built into an artificialneural network and represented by a set of optimal network weights of a chosen architecture.

7

AWS DeepComposer Developer GuidePre-trained models

Get started with AWSDeepComposer

The AWS DeepComposer music studio is where generative AI models and your creativity come together.To generate accompaniment tracks, you use a trained model. AWS DeepComposer allows you to use pre-trained models to get started and to learn the basics of GANs. You can then use training data that weprovide to train your own custom models based on either the U-Net or MuseGAN architecture.

To generate musical compositions, you can use the AWS DeepComposer built-in console-based keyboardor a registered AWS DeepComposer keyboard.

To share or download your new compositions, you can open the Compositions window and share yourmusic on SoundCloud, downloading an MP3 file, or a MIDI file.

NoteTo use the AWS DeepComposer console and other AWS services, you need an AWS account.If you don't have an account, see aws.amazon.com and choose Create an AWS Account. Fordetailed instructions, see Create and Activate an AWS Account.As a best practice, you should also create an AWS Identity and Access Management (IAM) userwith administrator permissions and use that for all work that does not require root credentials.Create a password for console access, and access keys to use command line tools. For moreinformation, see Creating Your First IAM Admin User and Group in the IAM User Guide.

After you've created your AWS account, sign in to the AWS DeepComposer console with yourAWS account or user credentials. The exercises in this section assume you've signed in to the AWSDeepComposer console.


Topics

• Using a pre-trained model (p. 8)

• Using a custom model (p. 11)

• Learn more about training and evaluating AWS DeepComposer models (p. 16)

Using a pre-trained modelAWS DeepComposer has five genre-based pre-trained models that use the MuseGAN architecture. TheMuseGAN architecture is specifically designed for generating music. The MuseGAN is composed of ashared network, which learns a high-level representation of the song, and a series of private networks tolearn how to generate individual accompaniment tracks.

Learning capsules provide additional information about the structure of GANs and our implementationof the MuseGAN architecture. You can either use one of the sample input melodies or record a custominput melody with the AWS DeepComposer pre-trained models.

8

https://console.aws.amazon.com/deepcomposer/home?region=us-east-1#learningCapsules



https://console.aws.amazon.com/deepcomposer/home?region=us-east-1#compositionList

https://aws.amazon.com/

http://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/

https://docs.aws.amazon.com/IAM/latest/UserGuide/getting-started_create-admin-group.html

https://console.aws.amazon.com/deepcomposer

AWS DeepComposer Developer GuideSample melody (pre-trained models)

Topics

• Using a pre-trained model with a sample input melody (p. 9)

• Using a pre-trained model with a custom melody (p. 9)

Using a pre-trained model with a sample inputmelodyTo use a pre-trained model with a sample input melody

1. Open the AWS DeepComposer console.

2. In the primary navigation pane, choose Music studio.

3. In Music studio, the sample input melody Twinkle, Twinkle Little Star is preselected. To use adifferent sample input melody, choose one of the available sample melodies.

4. In the Model parameters section, for Models, choose one of the five available pre-trained modelsbychoosing the down arrow next to Select a model.

NoteIf you generate a new composition, the accompaniment tracks are rewritten. Savingindividual accompaniment tracks is not supported. Additionally, you can't change the drumtrack accompaniment.

5. To listen to the accompaniment tracks you have just generated choose the play playback icon.

6. To download or share the composition, open the navigation pane and choose Compositions.Navigate to the composition that you want to download or share. Choose the down arrow next toActions to access the available download and sharing options.

Next steps

• Launch the Introduction to Generative adversarial networks Learning capsule

• Try to train a custom model using AWS DeepComposer

• Start to examine evaluating custom models in AWS DeepComposer.

Using a pre-trained model with a custom melodyTo record a custom melody, you can use either the console-based keyboard or the AWS DeepComposerkeyboard. To make recording a melody easier, the console-based keyboard has the option to enable hotkeys. Turning on hot keys, allows you to use your keyboard when you are recording a custom melody. Toidentify which notes correspond to which keys, the keys are outlined and show the corresponding letterinside. The console-based keyboard also supports a metronome. The metronome is set to 100 beats perminute (BPM). You can use Rhythm assist to help correct the timing of your recorded melody. This canalso improve the accompaniment tracks.

9



https://console.aws.amazon.com/deepcomposer/home?region=us-east-1#learningCapsules/introToGANs

https://docs.aws.amazon.com/deepcomposer/latest/devguide/get-started-learn-from-pre-trained-models.html

AWS DeepComposer Developer GuideCustom melody (pre-trained models)

To use the registered AWS DeepComposer keyboard plug your keyboard into your computer. After youhave successfully connected your AWS DeepComposer keyboard, a green status bar appears. The AWSDeepComposer keyboard has many recording tools built into the device.

To record a custom input melody and generate a composition with a pre-trained model


2. In the primary navigation pane, choose Music studio.

3. Choose the record playback icon.

4. Use either the console-based keyboard or the AWS DeepComposer keyboard to record yourcomposition.

NoteIf you want to use Rhythm assist, enable the toggle prior to choosing Generatecomposition. If you have already generated a new composition, enabling Rhythm assist works but rewrites the accompaniment tracks.

5. In the Model parameters section, for Models, choose one of the five available pre-trained models bychoosing the down arrow under Select input model.

6. Choose Generate Composition to perform inference based on the pre-trained model. After yourcomposition has finished, it is automatically saved.

NoteIf you generate a new musical composition, the accompaniment tracks are rewritten. Savingindividual accompaniment tracks is not supported by AWS DeepComposer. Changes to thedrum track accompaniment are also not supported.


8. To access your composition, open the navigation pane and choose Compositions.

10


https://docs.aws.amazon.com/deepcomposer/latest/devguide/what-it-is-keyboard.html



AWS DeepComposer Developer GuideCustom Models

9. To download or share a composition, open the navigation pane and choose Compositions. Navigateto the composition that you want to download or share. Choose the down arrow next to Actions toaccess the available download and sharing options.

Next steps



Using a custom modelTraining a custom GAN and evaluating your trained model is a great way to learn about working withand writing GANs. AWS DeepComposer provides training data for two GAN architectures, U-Net andMuseGAN. When training a custom model, you can choose from different hyperparameters to potentiallyimprove your training outcome. You can use either one of the sample input melodies or a custom inputmelody with a custom trained model. Training custom models can take up to 8 hours.

Topics

• Using a custom trained model with a sample input melody (p. 11)

• Using a custom trained model with a custom input melody (p. 13)

Using a custom trained model with a sample inputmelodyYou can train your own AWS DeepComposer model using a supported architecture in the AWSDeepComposer console.

NoteBefore proceeding further, make sure you have signed up for an AWS account and signed in tothe AWS DeepComposer console (p. 8).

To train your custom model in the AWS DeepComposer console


2. In the primary navigation pane, choose Getting started.

3. On the Getting started page, choose Train a model.

4. On the Train a model page, choose one of the supported AWS DeepComposer generative algorithmarchitectures.

11




AWS DeepComposer Developer GuideSample melody (custom trained models)

NoteBoth MuseGAN and U-Net are based on convolutional neural networks (CNNs). These CNNsgenerate a high-level representation of music, which is used to generate the output tracks.

5. After you choose a generative algorithm, for Training dataset, choose one of the available trainingdatasets.

6. For Hyperparameters, use or modify the default values for Epochs, Learning rate, or Ratio ofgenerator vs. discriminator. For more information about hyperparameter tuning, see

NoteTraining a model takes approximately 8 hours. The parameters for Epoch need toaccommodate a minimum of 8 hours of training. Decreasing the number of training epochsdecreases the training time required, but might also decrease the accuracy of the model.

7. For Model details, enter a name for your model. The model is listed by this name in the AWSDeepComposer Music studio.

8. Choose Start Training.

9. To check the status of your model's training, open the navigation pane and choose Model. Thestatus of your model's training appears under Models. After the model is trained, the status changesto Training complete.

After training has completed, your model automatically appears in the AWS DeepComposer Music studio.To find your trained model, in Model parameters, choose the down arrow under Model.

12


AWS DeepComposer Developer GuideCustom melody (custom trained models)

To generate a composition with a custom model with a sample input melody in AWSDeepComposer Music studio


2. In the navigation pane, choose Music studio.

3. In the AWS DeepComposer Music studio, the sample input melody Twinkle, Twinkle Little Star ispreselected. To use a different sample input melody, choose one of the available sample melodiesfrom the dropdown.

4. In the Model parameters section, in Models ,choose the down arrow next to Select a model. Yourcustom trained model is listed in the Custom model section.

5. Choose Generate Composition to perform inference based on the custom trained model. After yourcomposition has finished, it is automatically saved.

NoteIf you generate a new composition, the accompaniment tracks are rewritten. Savingindividual accompaniment tracks is not supported by AWS DeepComposer. Additionally, youcan't change the drum track accompaniment.


7. To access your compositions, open the navigation pane (on the left) and choose Compositions.

8. To download or share a composition, open the navigation pane and choose Compositions. Navigateto the composition that you want to download or share. Choose the down arrow next to Actions toaccess the available download and sharing options.

Next steps



Using a custom trained model with a custom inputmelodyYou can train your own AWS DeepComposer model using a supported architecture in the AWSDeepComposer console.

NoteBefore proceeding further, make sure you have signed up for an AWS account and signed in tothe AWS DeepComposer console (p. 8).

To train your custom model in the AWS DeepComposer console


2. In the navigation pane, choose Getting started.

3. On the Getting started page, choose Train a model.

4. On the Train a model page, choose one of the supported AWS DeepComposer generative algorithmarchitectures.

13







NoteBoth MuseGAN and U-Net are based on convolutional neural networks (CNNs). These CNNsgenerate a high-level representation of music, which is used to generate the output tracks.

5. After you choose a generative algorithm, for Training dataset, choose one of theavailable datasets.6. For Hyperparameters, use or modify the default values for Epochs, Learning rate or Ratio of

generator vs. discriminator. For more information about hyperparameter tuning, see.

NoteThe default parameters for Epoch need approximately 8 hrs to train a model. Decreasingthe number of training epochs will decrease the training time required, but may alsodecrease the accuracy of the model.

7. For Model details, enter a name for your model. The model is listed by this name in the AWSDeepComposer Music studio.

8. Choose Start Training.9. To check the status of your model's training, open the navigation pane and then choose Model. The

status of your model's training appears under Models. After the model has been trained, the statuschanges to Training complete.

After training has completed, your model automatically appears in the AWS DeepComposer Music studio.To find your trained model, in Model parameters, choose the down arrow under Model.

To record a custom input melody, you can use either the console-based keyboard or the AWSDeepComposer keyboard. To make recording a melody easier, the console-based keyboard has the optionto enable Hot keys. Turning on hot keys, allows you to use your keyboard when you are recording a

14



custom melody. To identify which notes correspond to which keys, the keys have been outlined with thecorresponding letter inside the console. The console-based keyboard also supports a Metronome. Themetronome is set to 100 BPM. Rhythm assist is used to help correct the timing of your recorded melody.This can help improve the accompaniment tracks being created.

To use your registered AWS DeepComposer keyboard plug your keyboard into your computer. Once youhave successfully connected your AWS DeepComposer keyboard a green status bar will appear. The AWSDeepComposer keyboard has many recording tools built into the device.

To generate a composition with a custom model and a custom input melody

1. In AWS DeepComposer Music studio,choose the record playback icon.

2. Record your composition using either the console-based keyboard or the AWS DeepComposerkeyboard.

NoteIf you want to use Rhythm assist , enable it before choosing Generate composition. If youhave already generated a new composition, enabling Rhythm assist works, but deletes theaccompaniment tracks that have been generated.

3. In the Model parameters section, for Models, choose your custom model by choosing the downarrow next to Select a model. Your custom trained model can be found under the Custom modelsection.

4. Choose Generate Composition to perform inference on the custom model. After your compositionhas finished, it is automatically saved.

5. To access your compositions, open the navigation pane (on the left) and choose Compositions.

NoteIf you generate a new composition, the accompaniment tracks are rewritten. Savingindividual accompaniment tracks is not supported by AWS DeepComposer. Additionally, youcan't change the drum track accompaniment.

15


https://docs.aws.amazon.com/deepcomposer/latest/devguide/what-it-is-keyboard.html


AWS DeepComposer Developer GuideEvaluating trained models

6. To download or share a composition, open the navigation pane and choose Compositions. Navigateto the composition you want to download or share. Choose the down arrow next to Actions toaccess the available download and sharing options.

Next steps



Learn more about training and evaluating AWSDeepComposer models

By examining the training results of a trained model, you can learn what makes a good model and howto assess the training performance of different models.

This section walks you through using the AWS DeepComposer console to examine a trained model'straining results, including the loss graph that shows how the training of a model converges, howthe structure similarity index progresses over the training epochs, and how the computed metrics ofgenerated samples evolve over time in the training epochs.

To learn more about training and evaluating a AWS DeepComposer model


2. In the navigation pane, choose Models.

3. On the Models page, choose a model from the list of trained models.

4. On the model's training results page under Loss, examine the Loss graph and the Epoch Explorer toascertain the training performance of the selected model.

16





The generator loss and discriminator loss are superimposed onto the same graph, but on differentscales. The scale of the generator loss is shown on the left side and the scale of the discriminatorloss is shown on the right. In this particular training job, the generator loss plateaus around the50th epoch, when it stops significantly improving its capability to generate realistic music. Thediscriminator loss shows a similar behavior, but is less noisy after it plateaus. You can verify this bylistening to the training sample output generated at fixed intervals of training epochs. The trainingsample outputs and the evaluation sample input are available in Epoch explorer. For every displayedsample output from the 100th epoch on, the generated accompaniment tracks sound like real music.

5. On the model's training results page under Epoch vs similarity distance, choose Structuresimilarity index tab to examine how the similarity index progresses from epoch to epoch and howgenerated samples become more similar to the input samples.

17


The structure similarity index measures how similar a generated music sample is to a training musicsample. If the two are identical, the index is 1. If there is no similarity, the index is 0. Successfultraining should have the structure similarity index converge to a positive value. In this particulartraining job, the index appears to converge to 0.025 and the fluctuations fall within +/-0.005 afterthe 100th epoch.

6. On the model's training results page under Summary of dataset, choose a metric from Computemetric.

NoteSupported metrics include Drum in pattern, Pitches used, Pitch classes used, Empty barrate, Polyphonic range, and In scale ratio. The definition and a summary description ofeach metric is displayed in the AWS DeepComposer console after you choose the metric.

18


19


Browser support for AWSDeepComposer

Browser support for the AWS DeepComposer console varies, depending on the feature that you areaccessing. The Web Audio API and WebMIDI API are used to connect the keyboard and play songs in theconsole respectively.

The following table shows which features are available in different browsers.

Browser Version Physical KeyboardPlayback

Digital KeyboardPlayback

Playing Music

Chrome Latest 3 Yes Yes Yes

Firefox Latest 3 No Yes Yes

Safari Latest 3 No Yes Yes

Internet Explorer 11 No No No

Edge Latest Yes Yes Yes

Opera Latest Yes Yes Yes

AWS DeepComposer console functions

• Physical Keyboard Playback – To record your own musical tracks the WebMIDI API operation isrequired, which limits some browser playback.

• Digital Keyboard Playback – To use the digital keyboard inside the AWS DeepComposer console theWebAudio API operation is required, which limits some browser playback.

• Playing Music – To play music inside the console the WebAudio API operation is required, which limitssome browser compatibility.

20

https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API

https://www.w3.org/TR/webmidi/


Document history for AWSDeepComposer Developer Guide

The following table describes the important changes to the documentation since the initial release ofAWS DeepComposer.

update-history-change update-history-description update-history-date

Initial release AWSDeepComposer DeveloperGuide (p. 21)

Initial release of thedocumentation.

December 2, 2019

21


AWS glossaryFor the latest AWS terminology, see the AWS glossary in the AWS General Reference.

22

https://docs.aws.amazon.com/general/latest/gr/glos-chap.html

aws deepcomposer - developer guide · chord feature generates a simple triad chord taking the note...

Documents