33 decks · 30 contact hours · 1,614 pages. New here? The course path tells you what each unit is for.
What NLP is and why it is hard; tasks, corpora and preprocessing; Zipf, Heaps and type token ratio; TF-IDF and document similarity; subword tokenisation (BPE) and edit distance.
Vector space models and the distributional hypothesis; count-based vectors and SVD; word2vec; GloVe and FastText; bias in embeddings.
Perceptrons, activations and loss functions; gradient descent, backpropagation and generalization; feedforward networks.
n-gram models, smoothing and perplexity; neural language models; RNNs and BPTT; LSTM/GRU and seq2seq; evaluating classifiers.
ELMo and the origins of attention; self-attention, multi-head attention, positional encoding and causal masking; BERT vs. GPT; the LLM landscape and scaling.
Decoding strategies and prompting; fine-tuning and PEFT (LoRA/QLoRA); alignment and RLHF; retrieval-augmented generation; machine translation and evaluation metrics.
Bias, harms and mitigation; responsible AI practice; course synthesis.