CS 5740 SP24

Time: MoWe 1:25PM - 2:40PM
Room: Bloomberg 131
Listing: CS 5740

Instructor: Yoav Artzi
Teaching assistant: Anne Wu and Omer Gul
Graders: Yi Chen, Young He, Vikranth Kanumuru, Subin Yun

Canvas 📋 | Forum 🗣

Schedule

Tentative topics are listed at the bottom of this page. Assignment dates are available on Canvas.

Date Topic Board Extended Readings
Jan 22 Introduction   NLP (circa 2001)
Jan 24 "    
  Warming up!    
  Text classification, data basics, and the perceptron    
Jan 29 "    
  Neural networks basics   Back-prop, Deep Averaging Networks, Gradient Checks (briefly), Gradient Checks (in details)
Jan 31 "    
Feb 05 Learning from raw data    
  Word embeddings   w2v explained, Levy2014, word2vec, word2vec phrases, Hill2016, Turney2010
Feb 07 "    
Feb 12 "    
  Language modeling   Collins: LM, Smoothing, More about smoothing
Feb 14 "    
Feb 19 Tokenization   Mielke2021, BPE notebook, HF summary
  Neural LMs and Transformers   Annotated Transformer, Illustrated Transformer, Vaswani2017
Feb 21 "    
Feb 26 No class ⛷️    
Feb 28 Neural LMs and Transformers    
Mar 04 "    
  Decoding LMs    
  Scaling up to LLMs    
Mar 06 "    
Mar 11 Masked LMs    
  Raw data recap    
  Learning from annotated data    
  Prototypical NLP tasks    
Mar 13 "    
Mar 18 "    
Mar 20 "    
Mar 25 "    
Mar 27 "    
Apr 01 No class ⛱️    
Apr 03 No class 🏖️    
Apr 08 "    
  Aligning LLMs    
Apr 10 "    
Apr 15 "    
Apr 17 Guest lecture: Orhan Firat (Google) – LLMs at Google    
Apr 22 Aligning LLMs    
Apr 24 Guest lecture: Joel Tetreault (Dataminr) – History of NLP    
Apr 29 Working with LLMs: Prompting    
May 01 "    
  Encoder-decoder Pre-training    
May 06 Recurrent neural networks    

Upcoming Topics

As we schedule a topic, it will be moved to the schedule table above.

Topic Extended Readings
Other architectures