Computer techniques for processing human languages, with a focus on textual data. Students will develop a deep and intuitive understanding of quantitative and algorithmic principles behind Natural Language Processing techniques and use those techniques creatively to design language technologies applicable to a wide range of fields.
Athena Title
NLP for Data Science
Non-Traditional Format
This course will be taught 95% or more online.
Prerequisite
STAT 6381E
Semester Course Offered
Offered spring
Grading System
A - F (Traditional)
Student learning Outcomes
Students will be able to describe the components of specific computational models for NLP.
Students will be able to describe and implement simple algorithms for model training and inference, such as expectation maximization, the Viterbi algorithm, and backpropagation through time.
Students will be able to evaluate NLP models quantitatively and qualitatively in terms of characteristics like accuracy, generalization ability, and social impact.
Students will be able to explain why different models work well for different types of tasks and data.
Students will be able to describe how different NLP model architectures are applied to different NLP tasks, such as text classification, part-of-speech tagging, and natural language generation.
Students will be able to apply NLP tasks to applied contexts in their respective fields.
Topical Outline
1. Introduction to NLP: linguistic annotation tasks; downstream NLP tasks; NLP and AI ethics
2. Language as sequences: n-grams and smoothing; word-level tokenization; Viterbi algorithm; byte-pair encoding; WordPiece; evaluating language models and segmentation
3. Static word vectors: sparse and dense word vectors; PPMI; latent semantic analysis; word2vec; GLoVE; imbalance in word embeddings; debiasing
4. Shallow classification: sentiment analysis; word sense disambiguation; the noisy channel model; naïve Bayes; logistic regression classifiers; coreference resolution; ranking models; evaluating chain annotations
6. Classic deep learning methods: word2vec and the projection layer; deep neural classifiers; feed-forward language models; simple recurrent networks; backpropagation through time; bidirectional RNNs; dynamic word embeddings; question-answering systems; seq2seq models and attention; long short-term memory networks
7. Transformers and large language models: pre-training and fine-tuning; self-attention; encoder-only transformers; BERT; chatbots; decoder-only transformers; reinforcement learning from human feedback; retrieval-augmented generation
8. Social impacts of NLP: fairness and representation; environmental costs; security and privacy issues; labor issues; toxicity and hallucination
Institutional Competencies Learning Outcomes
Analytical Thinking
The ability to reason, interpret, analyze, and solve problems from a wide array of authentic contexts.
