Time: MoWe 1:35PM - 2:50PM
Listing: CS 5740
Instructor: Yoav Artzi
Teaching assistant: Ge Gao
Graders: Kuan-Ting Liu, Wenyi Chu, and Cheng Wang
Canvas 📋 | iCal 📆 | Forum 🗣
Subscribe to the course calendar! 📆
Topic and assignment dates are tentative.
| Release Date | Due Date (all 11:59pm) | |
|---|---|---|
| Assignment 1 | Mar 01 | Mar 18 |
| Assignment 2 | Mar 22 | Apr 06 |
| Assignment 3 | Apr 07 | Apr 26 |
| Assignment 4 | Apr 27 | May 13 |
| Take-home Exam | May 18 | May 21 |
Slides for previous semesters are available online. The websites for previous semesters are available here. While we update the material between semesters, the majority of the slides are carried from one semester to the next. All reading are optional. Bold readings are higher priority. J&Mxx, Gxx, and M&Sxx refer to recommended course readings.
| Date | Topic | Board | Recommended Readings |
|---|---|---|---|
| Feb 08 | Introduction | NLP (circa 2001) | |
| Feb 10 | " | ||
| Text classification | M&S 7.4,16.2-16.3, Collins: Naive Bayes (Sec 1-4), Collins: Log Linear (Sec 2), MaxEnt, Baselines, CNN Classification Naive Bayes prior derivation | ||
| Feb 15 | " | ||
| Feb 17 | " | ||
| Feb 22 | " | ||
| Feb 24 | " | ||
| Mar 01 | Neural networks | Primer, Back-prop, Deep Averaging Networks, Gradient Checks (briefly), Gradient Checks (in details) | |
| Mar 03 | Computation graphs | NN Tips, Intro to Computation Graphs | |
| Mar 08 | |||
| Mar 10 | No class | ||
| Mar 15 | |||
| Mar 17 | |||
| Mar 22 | |||
| Mar 24 | |||
| Mar 29 | |||
| Mar 31 | |||
| Apr 05 | |||
| Apr 07 | |||
| Apr 12 | |||
| Apr 14 | |||
| Apr 19 | |||
| Apr 21 | |||
| Apr 26 | No class | ||
| Apr 28 | |||
| May 03 | |||
| May 05 | |||
| May 10 | |||
| May 12 |
As we schedule a topic, it will be moved to the schedule table above.
| Topic | Recommended Readings |
|---|---|
| Meta NLP | |
| Lexical semantics and embeddings | w2v explained, word2vec, word2vec phrases, Hill2016, Turney2010 |
| Language modeling | J&M 4, M&S 6, Collins: LM, Smoothing, Char RNN |
| Sequence modeling | J&M 5.1-5.3, 6, M&S 3.1, 9, 10.1-10.3, Collins: HMM, Collins: MEMMs (Sec 3), Collins: CRF (sec 4), Collins: Forward-backward, SOTA Taggers, TnT Tagger, Stanford Tagger |
| Recurrent neural networks | G14, BPTT, RNN Tutorial, Effectiveness, Luong2015, Einsum |
| Convolutional neural networks | G13, Kim2014, Jacovi2018 |
| Self-Attention and Transformers | Annotated Transformer, Illustrated Transformer, Vaswani2017 |
| Contextualized representations | BERT, The Illustrated BERT, ELMo, and co., Chen2019, BERTScore |
| Dependency parsing | J&M 12.7, Nivre2003, Chen2014 |
| Constituency parsing | J&M 12.1-12.6, 13.1-13.4, 14.1-14.4, M&S 11, 12.1, Collins: PCFGs, Eisner: Inside-outside, Collins: Inside-outside |