SLM Engineering — a practical course on creating Small Language Models (SLM) at the enterprise level. Over four weeks, you will go through the entire development cycle: from data preparation and tokenizer training to Pretraining, Fine-Tuning, RLHF, implementing RAG, inference optimization, and deploying the model in the local enterprise infrastructure. The course is aimed at developers and ML engineers who wish to independently create, train, and implement modern AI systems without relying on cloud services.
Who the course is for
- Developers transitioning into the fields of LLM Engineering, NLP, and Enterprise AI.
- Engineers creating chatbots, intelligent search, and document analysis systems.
- Specialists who want to learn how to independently train language models, not just use ready-made APIs.
- Researchers needing practical skills in industrial AI development.
What you will learn
Transformer Architecture
Study in detail the inner workings of modern language models: the attention mechanism (Attention), Feed Forward blocks, RMSNorm, Rotary Positional Encoding, embeddings, and the interplay of all components within the Transformer architecture.
Data preparation and tokenizer training
Learn to collect, clean, and deduplicate datasets, form data corpora, train your own tokenizers based on BPE and SentencePiece, and prepare data for pretraining and subsequent model tuning.
Pretraining
Build a complete training cycle for a language model from scratch, become familiar with scaling laws, learn to analyze loss functions and evaluate the efficiency of the training process.
Fine-Tuning with LoRA and QLoRA
Master modern parameter-efficient model tuning methods, including LoRA, QLoRA, Supervised Fine-Tuning, and 4-bit quantization. Learn when it is better to use model fine-tuning versus applying RAG or an engineering approach to prompt design.
Model alignment and safety
Study methods like Reward Modeling, DPO, and RLHF, and learn to implement protection against Prompt Injection, safe failure mechanisms, response moderation, and other elements for the safe operation of language models.
RAG and model quality evaluation
Create a full-fledged Retrieval-Augmented Generation system with hybrid search, result ranking, source linking, and using approaches like LLM-as-a-Judge and Golden Sets for model quality evaluation.
Inference and performance optimization
Understand the features of Prefill and Decode stages, the use of KV Cache, model quantization, Continuous Batching, Paged Attention, and deploying high-performance inference servers based on vLLM and llama.cpp.
Production deployment
Learn to create APIs for language models, organize streaming request processing, implement monitoring, track model quality degradation, conduct Canary deployments, and create automatic retraining loops.
Full SLM development cycle
During the course, you will implement the full production pipeline for creating a corporate language model:
- data collection and preparation;
- tokenizer training;
- creating embeddings;
- training a Transformer model;
- pretraining;
- Fine-Tuning and RLHF;
- RAG implementation;
- inference optimization;
- deploying the model in corporate infrastructure.
Practical final project
The final project will involve developing a corporate assistant for analyzing legal agreements. You will create your own small language model capable of automatically extracting and classifying contract clauses, detecting personal data and confidential information, conducting document searches with accurate references to relevant sections, and operating entirely locally without transmitting data to cloud services.
Learning outcomes
After completing the course, you will be able to independently design, train, optimize, and implement Small Language Models (SLM) at the enterprise level. You will gain practical experience with Transformer, LoRA, QLoRA, RLHF, RAG, vLLM, llama.cpp, industrial AI pipelines, and modern LLM development tools. You will also add to your portfolio a fully-realized project ready for real-world corporate use cases.
