CS 5740 SP21

Time: MoWe 1:35PM - 2:50PM
Listing: CS 5740

Instructor: Yoav Artzi
Teaching assistant: Ge Gao
Graders: Kuan-Ting Liu, Wenyi Chu, and Cheng Wang

Canvas 📋 | iCal 📆 | Forum 🗣


Subscribe to the course calendar! 📆

Topic and assignment dates are tentative.

Assignments and Exam

  Release Date Due Date (all 11:59pm)
Assignment 1 Mar 01 Mar 18
Assignment 2 Mar 22 Apr 06
Assignment 3 Apr 07 Apr 26
Assignment 4 Apr 27 May 13
Take-home Exam May 18 May 21


Slides for previous semesters are available online. The websites for previous semesters are available here. While we update the material between semesters, the majority of the slides are carried from one semester to the next. All reading are optional. Bold readings are higher priority. J&Mxx, Gxx, and M&Sxx refer to recommended course readings.

Date Topic Board Recommended Readings
Feb 08 Introduction   NLP (circa 2001)
Feb 10 " PDF  
  Text classification PDF M&S 7.4,16.2-16.3, Collins: Naive Bayes (Sec 1-4), Collins: Log Linear (Sec 2), MaxEnt, Baselines, CNN Classification Naive Bayes prior derivation
Feb 15 " PDF  
Feb 17 " PDF  
Feb 22 " PDF  
Feb 24 " PDF  
Mar 01 Neural networks   Primer, Back-prop, Deep Averaging Networks, Gradient Checks (briefly), Gradient Checks (in details)
Mar 03 Computation graphs   NN Tips, Intro to Computation Graphs
Mar 08      
Mar 10 No class    
Mar 15      
Mar 17      
Mar 22      
Mar 24      
Mar 29      
Mar 31      
Apr 05      
Apr 07      
Apr 12      
Apr 14      
Apr 19      
Apr 21      
Apr 26 No class    
Apr 28      
May 03      
May 05      
May 10      
May 12      

Upcoming Topics

As we schedule a topic, it will be moved to the schedule table above.

Topic Recommended Readings
Meta NLP  
Lexical semantics and embeddings w2v explained, word2vec, word2vec phrases, Hill2016, Turney2010
Language modeling J&M 4, M&S 6, Collins: LM, Smoothing, Char RNN
Sequence modeling J&M 5.1-5.3, 6, M&S 3.1, 9, 10.1-10.3, Collins: HMM, Collins: MEMMs (Sec 3), Collins: CRF (sec 4), Collins: Forward-backward, SOTA Taggers, TnT Tagger, Stanford Tagger
Recurrent neural networks G14, BPTT, RNN Tutorial, Effectiveness, Luong2015, Einsum
Convolutional neural networks G13, Kim2014, Jacovi2018
Self-Attention and Transformers Annotated Transformer, Illustrated Transformer, Vaswani2017
Contextualized representations BERT, The Illustrated BERT, ELMo, and co., Chen2019, BERTScore
Dependency parsing J&M 12.7, Nivre2003, Chen2014
Constituency parsing J&M 12.1-12.6, 13.1-13.4, 14.1-14.4, M&S 11, 12.1, Collins: PCFGs, Eisner: Inside-outside, Collins: Inside-outside