Skip to main content

Rafting Trip

34h 26m 24s
English
Paid

Course description

Imagine a winding river of network programming. At its mouth, closer to the sea, you see carefree users happily splashing in the waves, experimenting with web scrapers. As you move further upstream, you observe farms of HTTP servers, then message queue systems, RPC, and distributed objects. However, if you go further, past the last bridge, the landscape becomes grimmer: low-level system programming, sockets, threads, asynchronicity, strange and frightening constructs. Echo servers resonate in the narrowing gorge of complexity. Along the banks, there are remnants of abandoned projects and traces of developers who lost hope. It is here that your week-long journey begins: an attempt to implement the Raft distributed consensus algorithm from scratch. Perhaps—with no guarantees of success. The task of distributed consensus is to make a group of machines work as a single unit, resilient to the failures of individual nodes. This is the foundation for building reliable fault-tolerant systems. Raft is an algorithm that provides such behavior. The goal of the course is simple in formulation but difficult in essence: to implement Raft using only system libraries and engineering thinking. This may turn out to be the most challenging piece of system code you have ever written, tested, and debugged. Nonetheless, the experience will be extremely valuable.
Read more about the course

Why This Course

Implementing Raft is objectively a challenging task that tests an engineer's maturity. Formally, the goal is to write Raft, but in fact, the task is broader: to develop a strategy for solving a complex problem. How to break down a large mechanism into manageable parts? How do these parts interact? Where to start? How to test? Working on this project makes a developer technically stronger and a more mature architect.

Prerequisites

Such a project is often undertaken as part of master's courses on distributed systems. You will need a confident command of the chosen programming language (Rust, Python, Java, Go, etc.), and the ability to test, debug, and work in the terminal. It is also desirable to have experience in network and system programming, as well as working with multithreading. All necessary concepts are covered within the course, but a basic preparation significantly helps.

Learning Format

The course is project-based and requires a significant amount of independent intellectual and practical work. There are no presentations. Each day starts with discussions, demonstrations, and analyses of examples related to specific aspects of the project. However, most of the time is dedicated to individual development.

Development Environment

You are free to choose any programming language. During discussions, examples are usually given in basic Python as executable pseudocode. However, it is important to understand that a successful Raft implementation requires attention to detail. This is why many prefer to use more "strict" and instrumented environments.

Key Topics

Throughout the course, fundamental aspects of parallelism and distributed computing are covered, including:

  • Socket network programming.
  • Message exchange and communication patterns (RPC, queues, etc.).
  • State machines.
  • Formal specifications and modeling (including TLA+).
  • Multithreading.
  • Asynchronous programming.
  • Object-oriented design.
  • Software architecture.
  • Error handling and fault tolerance.

One of the main challenges is testing, monitoring, and debugging systems with nondeterminism and failures. Even in a configuration of five Raft nodes, it can run up to 60 threads in multiple processes, using timers, queues, and channels — this creates extreme cognitive load. A significant part of the course is dedicated to strategies for managing this complexity.

Are You Ready?

Probably not. But that's what makes the journey interesting.

Watch Online

This is a demo lesson (10:00 remaining)

You can watch up to 10 minutes for free. Subscribe to unlock all 10 lessons in this course and access 10,000+ hours of premium content across all courses.

View Pricing
0:00
/
#1: 001 Day 1 Morning

All Course Lessons (10)

#Lesson TitleDurationAccess
1
001 Day 1 Morning Demo
03:29:42
2
002 Day 1 Afternoon
03:24:22
3
003 Day 2 Morning
03:31:35
4
004 Day 2 Afternoon
03:28:03
5
005 Day 3 Morning
03:32:16
6
006 Day 3 Afternoon
03:15:17
7
007 Day 4 Morning
03:21:10
8
008 Day 4 Afternoon
03:31:49
9
009 Day 5 Morning
03:30:31
10
010 Day 5 Afternoon
03:21:39

Unlock unlimited learning

Get instant access to all 9 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.

Learn more about subscription

Comments

0 comments

Want to join the conversation?

Sign in to comment

Similar courses

Modular Monolith Architecture

Modular Monolith Architecture

Sources: Milan Jovanović
This in-depth course will transform the way you build modern systems. You will learn the best practices for applying the Modular Monolith architecture in a real-world scenario. ...
12 hours 48 minutes 50 seconds
Systems Design Fundamentals

Systems Design Fundamentals

Sources: algoexpert
The most important ingredient to success in systems design interviews is having extensive knowledge in the areas of distributed computing, reliability engineering, information s...
10 hours 2 minutes 52 seconds
Digital Project Management

Digital Project Management

Sources: superhi.com
Smooth processes and happy human relationships are key to managing effectively. Learn better ways to work with complexity so you can run projects like a pro.
17 hours 53 minutes 30 seconds
Designing Modern Web-Scale Distributed Services

Designing Modern Web-Scale Distributed Services

Sources: Shivang Sarawagi
Practical study of designing common web services like Netflix and Facebook. Suitable for developers, architects, and managers.
Low Level Design

Low Level Design

Sources: takeUforward
The course is dedicated to low-level design (LLD), a key stage in software development where abstract ideas and...
22 hours 34 minutes 32 seconds