Super Study Guide: Transformers & Large Language Models

It is divided into 5 parts: Foundations: primer on neural networks and important deep learning concepts for training and evaluation Embeddings: tokenization algorithms, word-embeddings (word2vec) and sentence embeddings (RNN, LSTM, GRU) Transformers: motivation behind its self-attention mechanism, detailed overview on the encoder-decoder architecture and related variations such as BERT, GPT and T5, along with tips and tricks on how to speed up computations Large language models: main techniques to tune Transformer-based models, such as prompt engineering, (parameter efficient) finetuning and preference tuning Applications: most common problems including sentiment extraction, machine translation, retrieval-augmented generation and many more.