Placeholder Image

字幕列表 影片播放

  • [MUSIC]

  • Stanford University.

  • >> Okay everyone.

  • We're ready.

  • Okay well welcome to CS224N in Linguistics 284.

  • This is kind of amazing.

  • Thank you for everyone who's here that's involved and also the people who don't fit

  • in here and the people who are seeing it online on SCPD.

  • Yeah it's totally amazing the number of people who've signed up to do this class

  • and so in some sense it seems like you don't need any advertisements for

  • why the combination of natural language process and

  • deep learning is a good thing to learn about.

  • But nonetheless today, this class is really going

  • to give some of that advertisement, so I'm Christopher Manning.

  • So what we're gonna do is I'm gonna start off by saying

  • a bit of stuff about what natural language processing is and what deep learning is,

  • and then after that we'll spend a few minutes on the course logistics.

  • And a word from my co-instructor, Richard.

  • And then, get through some more material on why is language

  • understanding difficult, and then starting to do an intro to deep learning for NLP.

  • So we've gotten off to a rocky start today,

  • cause I guess we started about ten minutes late because of that fire alarm going off.

  • Fortunately, there's actually not a lot of hard content in this first lecture.

  • This first lecture is really to explain what an NLP class is and say

  • some motivational content about how and why deep learning is changing the world.

  • That's going to change immediately on the Thursday lecture because for

  • the Thursday lecture is then we're gonna start with sort of vectors and

  • derivatives and chain rules and all of that stuff.

  • So you should get mentally prepared for

  • that change of level between the two lectures.

  • Okay, so first of all what is natural language processing?

  • So natural language processing, that's the sort of computer scientist's name for

  • the field.

  • Essentially synonymous with computational linguistics which is

  • sort of the linguist's name of the field.

  • And so it's in this intersection of computer science and linguistics and

  • artificial intelligence.

  • Where what we're trying to do is get computers to

  • do clever things with human languages to be able to understand and

  • express themselves in human languages the way that human beings do.

  • So natural language processing counts as a part of artificial intelligence.

  • And there are obviously other important parts of artificial intelligence,

  • of doing computer vision, and robotics,

  • and knowledge representation, reasoning and so on.

  • But language has had a very special part of artificial intelligence,

  • and that's because that language has been this very distinctive properties of

  • human beings, and we think and go about the world largely in terms of language.

  • So lots of creatures around the planet have pretty good vision systems,

  • but human beings are alone for language.

  • And when we think about how we express our ideas and go about doing things that

  • language is largely our tool for thinking and our tool for communication.

  • So it's been one of the key technologies that people have thought

  • about in artificial intelligence and it's the one that we're going to look at today.

  • So our goal is how can we get computers to process or

  • understand human languages in order to perform tasks that are useful.

  • So that could be things like making appointments, or buying things, or

  • it could be more highfalutin goals of sort of, understanding the state of the world.

  • And so this is a space in which there's starting to be a huge amount of commercial

  • activity in various directions, some of things like making appointments.

  • A lot of it in the direction of question answering.

  • So, luckily for people who do language, the arrival of mobile has just been super,

  • super friendly in terms of the importance of language has gone way way higher.

  • And so now really all of the huge tech firms whether it's Siri,

  • Google Assistant, Facebook and Cortana.

  • But what they're furiously doing is

  • putting out products that use natural language to communicate with users.

  • And that's an extremely compelling thing to do.

  • It's extremely compelling on phones because phones have these dinky

  • little keyboards that are really hard to type things on.

  • And a lot of you guys are very fast at texting, I know that, but

  • really a lot of those problems are much worse for a lot of other people.

  • So it's a lot harder to put in Chinese characters than it is to put in

  • English letters.

  • It's a lot harder if you're elderly.

  • It's a lot harder if you've got low levels of literacy.

  • But then there are also being new vistas opening up.

  • So Amazon has had this amazing success with Alexa, which is really shown

  • the utility of having devices that are just ambient in the environment, and

  • that again you can communicate with by talking to them.

  • As a quick shout-out for Apple, I mean, really,

  • we do have Apple to thank for launching Siri.

  • It was, essentially, Apple taking the bet on saying we can

  • turn human language into consumer technology that

  • really did set off this arms race every other company is now engaging on.

  • Okay, I just sort of loosely said meaning.

  • One of the things that we'll talk about more is meaning is a kind of a complex,

  • hard thing and it's hard to know what it means to understand fully meaning.

  • At any rate that's certainly a very tough goal which people refer to as AI-complete

  • and it involves all forms of our understanding of the world.

  • So a lot of the time when we say understand the meaning,

  • we might be happy if we sort of half understood the meaning.

  • And we'll talk about different ways that we can hope to do that.

  • Okay, so one of the other things that we hope that you'll get in

  • this class is sort of a bit of appreciation for human language and

  • what it's levels are and how it's processed.

  • Now obviously we're not gonna do a huge amount of that if you really wanna

  • learn a lot about that.

  • There are lots of classes that you can take in the linguistics department and

  • learn much more about it.

  • But I really hope you can at least sort of get a bit of a high level of

  • understanding.

  • So this is kind of the picture that people traditionally have given for

  • levels of language.

  • So at the beginning there's input.

  • So input would commonly be speech.

  • And then you're doing phonetic and

  • phonological analysis to understand that speech.

  • Though commonly it is also text.

  • And then there's some processing that's done there which has

  • sort of been a bit marginal from a linguistics point of view, OCR,

  • working out the tokenization of the words.

  • But then what we do is go through a series of processing steps

  • where we work out complex words like incomprehensible,

  • it has the in in front and the ible at the end.

  • And that sort of morphological analysis, the parts of words.

  • And then we try and

  • understand the structure of sentences, that syntactic analysis.

  • So if I have a sentence like 'I sat on the bench',

  • that 'I' is the subject of the verb 'sat', and the 'on the bench' is the location.

  • Then after that we attempt to do semantic understanding.

  • And that's semantic interpretation's working out the meaning of sentences.

  • But simply knowing the meaning of the words of a sentence isn't

  • sufficient to actually really understand human language.

  • A lot is conveyed by the context in which language is used.

  • And so that then leads into areas like pragmatics and discourse processing.

  • So in this class, where we're gonna spend most of our time is in that middle

  • piece of syntactic analysis and semantic interpretation.

  • And that's sort of bulk of our natural language processing class.

  • We will say a little bit right at the top left where this discussion,

  • speech signal analysis.

  • And interestingly, that was actually the first place where deep learning

  • really proved itself as super, super useful for tasks involving human language.

  • Okay, so applications of Natural Language Processing are now

  • really spreading out thick and fast.

  • And every day you're variously using applications of

  • Natural Language Processing.

  • And they vary on a spectrum.

  • So they vary from very simple ones to much more complex ones.

  • So at the low level, there are things like spell checkings, or

  • doing the kind of autocomplete on your phone.

  • So that's a sort of a primitive language understanding task.

  • Variously, when you're doing web searches,

  • your search engine is considering synonyms, and things like that for you.

  • And, well, that's also a language understanding task.

  • But what we are gonna be more interested in is trying to

  • push our language understanding computers up to more complex tasks.

  • So some of the next level up kind of tasks that we're actually gonna want to have

  • computers look at text information, be it websites, newspapers or whatever.

  • And get the information out of it, to actually understand the text well enough

  • that they know what it's talking about to at least some extent.

  • And so that could be things like expecting particular kinds of information, like

  • products and their prices or people and what jobs they have and things like that.

  • Or it could be doing other related tasks to understanding the document,

  • such as working out the reading level or intended audience of the document.

  • Or whether this tweet is saying something positive or

  • negative about this person, company, band or whatever.

  • And then going even a higher level than that, what we'd like our computers

  • to be able to do is complete whole level language understanding tasks.

  • And some of the prominent tasks of that kind that we're going to talk about.

  • Machine translation, going from one human language to another human language.

  • Building spoken dialogue systems, so you can chat to a computer and

  • have a natural conversation, just as you do with human beings.

  • Or having computers that can actually exploit the knowledge of the world