CS 5740 SP21

Time: MoWe 1:35PM - 2:50PM
Listing: CS 5740

Instructor: Yoav Artzi
Teaching assistant: Ge Gao
Graders: Kuan-Ting Liu, Wenyi Chu, and Cheng Wang

Canvas 📋 | iCal 📆 | Forum 🗣


Subscribe to the course calendar! 📆

Topic and assignment dates are tentative.

Assignments and Exam

  Release Date Due Date (all 11:59pm)
Assignment 1 Mar 01 Mar 18
Assignment 2 Mar 22 Apr 06
Assignment 3 Apr 07 Apr 29
Assignment 4 (due: 8am) Apr 28 May 15
Take-home Exam May 17 May 21


Slides for previous semesters are available online. The websites for previous semesters are available here. While we update the material between semesters, the majority of the slides are carried from one semester to the next. All reading are optional. Bold readings are higher priority. J&Mxx, Gxx, and M&Sxx refer to recommended course readings.

Date Topic Board Recommended Readings
Feb 08 Introduction   NLP (circa 2001)
Feb 10 " PDF  
  Text classification PDF M&S 7.4,16.2-16.3, Collins: Naive Bayes (Sec 1-4), Collins: Log Linear (Sec 2), MaxEnt, Baselines, CNN Classification Naive Bayes prior derivation
Feb 15 " PDF  
Feb 17 " PDF  
Feb 22 " PDF  
Feb 24 " PDF  
Mar 01 Neural networks PDF Primer, Back-prop, Deep Averaging Networks, Gradient Checks (briefly), Gradient Checks (in details)
Mar 03 " PDF  
Mar 08 Computation graphs PDF NN Tips, Intro to Computation Graphs
Mar 10 No class    
Mar 15 Meta NLP PDF  
Mar 17 Lexical semantics and embeddings PDF w2v explained, word2vec, word2vec phrases, Hill2016, Turney2010
Mar 22 " PDF  
Mar 24 Language modeling PDF J&M 4, M&S 6, Collins: LM, Smoothing, Char RNN
Mar 29 " PDF  
Mar 31 " PDF  
  Sequence modeling PDF J&M 5.1-5.3, 6, M&S 3.1, 9, 10.1-10.3, Collins: HMM, Collins: MEMMs (Sec 3), Collins: CRF (sec 4), Collins: Forward-backward, SOTA Taggers, TnT Tagger, Stanford Tagger
Apr 05 " PDF  
Apr 07 " PDF  
Apr 12 " PDF  
Apr 14 " PDF  
  Recurrent neural networks PDF G14, BPTT, RNN Tutorial, Effectiveness, Luong2015, Einsum
Apr 19 " PDF  
Apr 21 " PDF  
Apr 26 No class    
Apr 28 Machine translation PDF Bleu score
  Self-Attention and Transformers PDF Annotated Transformer, Illustrated Transformer, Vaswani2017
May 03 " PDF  
May 05 Contextualized representations PDF BERT, The Illustrated BERT, ELMo, and co., Chen2019, BERTScore
May 10 " PDF  
May 12 Guest Lecture by Joel Tetreault (Dataminr)    

Upcoming Topics

As we schedule a topic, it will be moved to the schedule table above.

Topic Recommended Readings
Convolutional neural networks G13, Kim2014, Jacovi2018
Dependency parsing J&M 12.7, Nivre2003, Chen2014
Constituency parsing J&M 12.1-12.6, 13.1-13.4, 14.1-14.4, M&S 11, 12.1, Collins: PCFGs, Eisner: Inside-outside, Collins: Inside-outside