That's what happens on this channel, I guess.

So this is sponsored by Spell.

Thank you so much to Spell for the sponsorship.

What you're about to watch is an edited version

of a livestream that happened a couple weeks ago.

We have a guest educator and artist,

Brooklyn based educator and artist Nabil Hassein.

I recommend you check out his website

linked in this video's description and learn more

about his background and his current work,

and all sorts of wonderful stuff that he is up to.

So what you're going to see, from beginning to end

in this video, is the process

for taking a corpus of text,

training a machine learning model,

this particular model is called LSTM,

long short term memory neural network.

Nabil will explain that a bit more in the video

and offer you some resources to learn about it.

Train a model to learn about that text.

Train it in the cloud, on Spell,

you go to spell.run slash coding train

if you want to sign up for that service

and follow along with the tutorial.

And then download the train model,

then bring that train model into the browser,

into JavaScript, generate new text in the style of

the original text that the model was trained on.

So you're going to see the full process for this tutorial.

Probably, if you've never watched

any of my videos before you're new to coding,

you might want to watch some of my workflow videos

that show you how to set up your environment

you're going to need, you'll need a Python environment,

you're going to need a code editor

and know how to run a webpage in your browser

that you're developing locally on your computer.

But I have videos that show all that stuff.

I also have a video that introduces the Spell platform

and gives you some background about how that works.

Alright, so I hope you enjoy this video.

If you make something with this, please share it with me.

I would love to see what kind of crazy, and interesting,

and wacky, and original, and fun, and playful projects

you are inspired to make by learning how to do this.

Thank you again to Nabil for being here to make

this tutorial and to Spell for the sponsorship.

Okay, bye bye.

- Alright, hello everyone, I'm Nabil,

thanks Dan for this great intro

and thank Spell for paying me to make

this video or to do this livestream.

So I have here kind of an outline

of what I plan to go through,

so I guess I'll start by going ahead

and introducing myself.

So I already said hi, I'm Nabil,

I live in Brooklyn, I'm a freelance

technologist, educator, do some other things.

Again, thank you Spell for sponsoring this video.

So this livestream is about how to train

an LSTM model using the Spell platform,

so on some remote machine somewhere,

and then how to use that model that we've trained

using a library called ml5.js,

which is a browser based front end library

for using machine learning models.

So what I'm going to do in this video,

I've practiced most of this,

I'm going to try to do a few things

truly live for you here today.

I'm going to kind of extend a project that I did

actually at the School for Poetic Computation,

which Dan mentioned last summer.

The way that that project works is there's a bunch

of random, it'll generate random rhymes.

Right now, this, what I have live on the web,

what I'm actually showing from my website

is based on a Markov model,

so it's not really machine learning,

it's just probabilistic predicting

the next character based on the previous ones.

Then you can click this all day

and it'll keep coming up with more and more rhymes.

The video in general, as you know,

is about training an LSTM model using Spell

and then using it in the browser

via a library called ml5.js.

So let's go ahead and get into it.

So the next thing, so I'm not really going to talk

to you much in this video about the theory

of neural networks or what is an LSTM really,

but I figure I should probably say something.

First of all, LSTM stands for long short term memory.

It's a specific type of recurrent neural network,

and what is useful about recurrent neural networks

or RNNs compared to some other types

of neural networks is the way

that their architecture includes loops,

and that can be useful for kind of

keeping data around in the network,

so to speak, which is very useful

for applications involving

natural language, human language,

because context matters so much in language.

Predicting the next character or the next word,

you might get a much better prediction

if you actually remember what was said

even some while ago, maybe like

much earlier in a long sentence.

I have a few quick references here,

which, by now are a little old,

but these are what I read to learn

a little bit about recurrent neural networks.

So there's this blog post called The Unreasonable

Effective of Recurrent Neural Networks,

and there's this other blog post

called Understanding LSTMs.

So yeah, this gives a little bit of overview

of kind of the same stuff I was just talking about.

Humans don't start their thinking from scratch every second.

You understand each word based on

your understanding of previous words,

and that's what we want our network to do as well,

which is why we're going to use this LSTM model.

I know that before I had the chance,

while preparing for this video,

to watch a video that Dan made kind of

giving an overview of the Spell platform,

so a link that video will also

be added to the video description

and you can kind of get into a little

bit more depth about using Spell.

And I'll also mention some things

about using Spell as we go through this.

Okay, so when you want to do a project like this,

the first thing that you have to do

is get your corpus of data.

So in this case, since I was getting song lyrics,

I used a site called Genius.com,

which you might be familiar with.

It's a popular lyrics website,

it has some other features too

but the main thing I use it for,

and I think most people use it for,

is reading lyrics.

So what I'm going to do, I'm going to try to do

everything kind of from scratch, so to speak,

so that you should be able to follow along in theory.

What I'm going to do, this is a folder that I used to prepare.

What I'm going to do is just make a new folder

called Spell Livestream, and I'm going to

do everything from inside of this folder,

which just lives somewhere on my computer.

So right now this folder is empty.

And so the first thing that I'm going to do is just

clone my generative DOOM repository from GitHub.

There's only actually one file in there

that I care about so maybe not actually clone

the whole repository, let me just get that one file.

Okay, so I'm just going to

where this is, it's in data, oh but did I push it up?

I have so many branches here.

Okay, why don't I use the one that I have on my computer.

So I'm just going to copy a file

that I have on my computer into this folder.

So where's that, in Spell demo slash

generative DOOM slash data.

I have a file called input.txt that I just moved,

that I just brought a copy of into my current directory.

We can just check it out really quickly,

oops, less input.txt.

So you can see this is just the list of lyrics.

Okay, this is my corpus.

It's worth noting that the data set

I'm actually using for this example isn't that big.

We can check the size of it with the command

line utility du for this usage,

past the human readable flag so that we can actually

tell how big this file is.

It's about 308 kilobytes, so it's not huge.

Normally when you're training machine learning models,

kind of, the more data the better,

but I got all the lyrics I could find by this artist,

this is really the most that I could get.

So that's what we're going to use.

Cool, so it's also worth noting

that you have to clean the data before you train it,

so I can actually go ahead and show

the code that I used to get these lyrics.

I'm not going to go into full depth, but again,

it's on my GitHub if you want to check it out.

So let's put it over here.

So I happen to do my scripting using Python.

You can do this in any language,

you can do web scripting using Node.js

or Ruby or whatever your favorite language is.

But I happen to have already used before

a Python library called BeautifulSoup,

which is very useful for web scripting.

It so happens that Genius.com happens to keep

their lyrics and their URLs follow a pattern like this,

genius.com slash the name of the artist,

which I substituted in here,

and the name of the song,

which I substituted in here,

and then I used another Python library

called Requests to just go ahead

and fetch all these different things.

So this is the basic idea,

I'm not going to go into full depth,

but I just kind of hard coded a lot of names

of different songs into here,

and then I have a main loop which basically

just loops through each artist's name

because DOOM has actually recorded

under many different names, so I can't just

use the same artist's name all the time.

And then the same thing for the albums

and then finally the songs in order

to go ahead and just fetch all of this data.

The thing is that when you just go directly to some

lyrics website, like when you fetch

the data on the page, you end up getting

a lot of other data that you don't

really care about in the HTML,

and so an important step is to clean the data

so that when you're training the model

you're only training it on the actual

corpus that you care about

and you're not training it on the angle brackets

of HTML tags or something like that

that you don't actually want.

So again, I think I have most of the code

that I used to clean it on the GitHub,

I think it is there, but if not there are

other resources that you can use

to learn more about data cleaning.

Again this video is really about

training machine learning models using Spell

and then using them in the browser.

So let's get back to that.

I wanted to mention Project Gutenberg

is another resource that has lots of free text

that's in the public domain that you can just use.