CS 5740 SP21

Time: MoWe 1:35PM - 2:50PM
Listing: CS 5740

Instructor: Yoav Artzi
Teaching assistant: Ge Gao
Graders: Kuan-Ting Liu, Wenyi Chu, and Cheng Wang

Canvas 📋 | iCal 📆 | Forum 🗣

Schedule

Subscribe to the course calendar! 📆

Topic and assignment dates are tentative.

Assignments and Exam

Release Date Due Date (all 11:59pm)
Assignment 1 Mar 01 Mar 18
Assignment 2 Mar 22 Apr 06
Assignment 3 Apr 07 Apr 29
Assignment 4 (due: 8am) Apr 28 May 15
Take-home Exam May 17 May 21

Lectures

Slides for previous semesters are available online. The websites for previous semesters are available here. While we update the material between semesters, the majority of the slides are carried from one semester to the next. All reading are optional. Bold readings are higher priority. J&Mxx, Gxx, and M&Sxx refer to recommended course readings.

Feb 08 Introduction   NLP (circa 2001)
Feb 10 " PDF
Text classification PDF M&S 7.4,16.2-16.3, Collins: Naive Bayes (Sec 1-4), Collins: Log Linear (Sec 2), MaxEnt, Baselines, CNN Classification Naive Bayes prior derivation
Feb 15 " PDF
Feb 17 " PDF
Feb 22 " PDF
Feb 24 " PDF
Mar 01 Neural networks PDF Primer, Back-prop, Deep Averaging Networks, Gradient Checks (briefly), Gradient Checks (in details)
Mar 03 " PDF
Mar 08 Computation graphs PDF NN Tips, Intro to Computation Graphs
Mar 10 No class
Mar 15 Meta NLP PDF
Mar 17 Lexical semantics and embeddings PDF w2v explained, word2vec, word2vec phrases, Hill2016, Turney2010
Mar 22 " PDF
Mar 24 Language modeling PDF J&M 4, M&S 6, Collins: LM, Smoothing, Char RNN
Mar 29 " PDF
Mar 31 " PDF
Sequence modeling PDF J&M 5.1-5.3, 6, M&S 3.1, 9, 10.1-10.3, Collins: HMM, Collins: MEMMs (Sec 3), Collins: CRF (sec 4), Collins: Forward-backward, SOTA Taggers, TnT Tagger, Stanford Tagger
Apr 05 " PDF
Apr 07 " PDF
Apr 12 " PDF
Apr 14 " PDF
Recurrent neural networks PDF G14, BPTT, RNN Tutorial, Effectiveness, Luong2015, Einsum
Apr 19 " PDF
Apr 21 " PDF
Apr 26 No class
Apr 28 Machine translation PDF Bleu score
Self-Attention and Transformers PDF Annotated Transformer, Illustrated Transformer, Vaswani2017
May 03 " PDF
May 05 Contextualized representations PDF BERT, The Illustrated BERT, ELMo, and co., Chen2019, BERTScore
May 10 " PDF
May 12 Guest Lecture by Joel Tetreault (Dataminr)

Upcoming Topics

As we schedule a topic, it will be moved to the schedule table above.