NestJS Microservices: Build a Distributed Job Engine
Course description
Master the art of creating scalable and distributed microservices with NestJS by developing a fully functional Job Engine from scratch—a distributed task processing system. The course will teach you to build microservices the right way: from architecture and design to deployment and scaling in production.
You will start with setting up a monorepo with Nx to efficiently manage multiple services within a single codebase. Using gRPC, you will implement high-performance, low-latency communication between services. Special attention is given to Apache Pulsar—a distributed messaging system that allows multiple consumers to connect to a shared queue for achieving horizontal scaling. To interact with the system, you will create a GraphQL API, providing flexible and convenient data access.
Security is one of the key aspects of the course: you will implement JWT-based authentication with secure HTTP-only cookies. The entire code is built on the principles of object-oriented design and DRY, making the architecture clean, modular, and easily maintainable as the project grows.
You will optimize Docker container builds, apply Nx libraries, and separate package.json for flexible dependency management. You will learn techniques for horizontal scaling so that your services can handle increasing loads. As the engine develops, task state tracking will be implemented, allowing you to monitor the progress of task execution.
For database work, Prisma ORM and Drizzle ORM are used with PostgreSQL for type-safe migrations and optimized queries. The final stage will be deployment in Kubernetes using AWS, a custom domain, and SSL encryption.
By the end of the course, you will have a fully operational distributed Job Engine and a systematic understanding of how to design, develop, and run scalable microservices on NestJS in real production environments.
Watch Online
All Course Lessons (104)
| # | Lesson Title | Duration | Access |
|---|---|---|---|
| 1 | 1.1 -Introduction Demo | 03:40 | |
| 2 | 1.2 -System Architecture | 06:52 | |
| 3 | 1.3 -Prerequisites | 01:36 | |
| 4 | 1.4 -Resources | 01:08 | |
| 5 | 1.5 -Nx Monorepo Setup - Part 1 | 09:56 | |
| 6 | 1.6 -Nx Monorepo Setup - Part 2 | 08:19 | |
| 7 | 1.7 -GitHub Actions CI | 10:40 | |
| 8 | 2.1 -Prisma Schema | 12:59 | |
| 9 | 2.2 -Nx Prisma Targets | 12:43 | |
| 10 | 2.3 -GraphQL & Nx Libs - Part 1 | 09:22 | |
| 11 | 2.4 -GraphQL & Nx Libs - Part 2 | 10:45 | |
| 12 | 2.5 -Users Mutation & Query | 16:11 | |
| 13 | 3.1 -Commit Hooks | 06:45 | |
| 14 | 3.2 -Unit Tests | 05:05 | |
| 15 | 3.3 -Config Module | 02:34 | |
| 16 | 3.4 -Nx Options | 01:14 | |
| 17 | 4.1 -JWT Module | 07:34 | |
| 18 | 4.2 -Config Cleanup | 02:34 | |
| 19 | 4.3 -User Verification | 12:50 | |
| 20 | 4.4 -Sign JWT Cookie | 13:38 | |
| 21 | 4.5 -JWT Strategy | 14:32 | |
| 22 | 4.6 -User Parameter Decorator | 02:57 | |
| 23 | 4.7 -Auth Playground Settings | 00:43 | |
| 24 | 5.1 -Generate Application | 05:26 | |
| 25 | 5.2 -Job Decorator | 09:17 | |
| 26 | 5.3 -Discovery Module | 07:25 | |
| 27 | 5.4 -Jobs Resolver | 16:10 | |
| 28 | 5.5 -Jobs Playground Settings | 00:43 | |
| 29 | 6.1 -Auth Proto | 07:47 | |
| 30 | 6.2 -TS Proto | 13:22 | |
| 31 | 6.3 -gRPC Server | 10:08 | |
| 32 | 6.4 -Auth Controller & Client | 07:32 | |
| 33 | 6.5 -gRPC GQL Auth Guard | 11:08 | |
| 34 | 6.6 -gRPC Auth Strategy | 06:45 | |
| 35 | 7.1 -Pulsar Client | 12:20 | |
| 36 | 7.2 -Abstract Job Producer | 16:51 | |
| 37 | 7.3 -Abstract Consumer | 10:45 | |
| 38 | 7.4 -Job Executor | 11:49 | |
| 39 | 7.5 -JSON GraphQL Scalar | 04:49 | |
| 40 | 7.6 -Message Serialization & Error Handling | 13:32 | |
| 41 | 7.7 -Job Validation | 07:16 | |
| 42 | 8.1 -Pulsar Batch Producer | 07:52 | |
| 43 | 8.2 -Backlog Test Script | 16:50 | |
| 44 | 9.1 -NPM Workspaces | 11:25 | |
| 45 | 9.2 -New Libraries | 08:34 | |
| 46 | 9.3 -Library Dependencies | 08:20 | |
| 47 | 9.4 -Library Webpack | 08:18 | |
| 48 | 9.5 -Update Applications | 09:16 | |
| 49 | 9.6 -Module Alias | 09:10 | |
| 50 | 9.7 -Jobs Dockerfile | 13:35 | |
| 51 | 9.8 -Finish Dockerfiles | 07:50 | |
| 52 | 9.9 -Common Webpack | 09:31 | |
| 53 | 10.1 -Pino Logger | 09:16 | |
| 54 | 10.2 -GraphQL Logger | 08:07 | |
| 55 | 10.3 -gRPC Logger | 08:48 | |
| 56 | 11.1 -Elastic Container Registry | 04:17 | |
| 57 | 11.2 -GitHub Action ECR | 14:06 | |
| 58 | 11.3 -Minikube | 03:30 | |
| 59 | 11.4 -Update Prettier | 01:25 | |
| 60 | 11.5 -Create Helm Chart | 04:16 | |
| 61 | 11.6 -Jobs Deployment | 12:18 | |
| 62 | 11.7 -Registry Creds Addon | 06:36 | |
| 63 | 11.8 -Pulsar & Postgres | 12:10 | |
| 64 | 11.9 -Environment Variables | 09:22 | |
| 65 | 11.10 -Prisma Debian | 02:27 | |
| 66 | 11.11 -Executor & Auth Deployment | 12:42 | |
| 67 | 11.12 -Auth gRPC URL | 02:01 | |
| 68 | 11.13 -Services | 09:23 | |
| 69 | 11.14 -Minikube Service | 06:56 | |
| 70 | 11.15 -Prisma Migrations | 09:32 | |
| 71 | 11.16 -Job Execution | 15:16 | |
| 72 | 12.1 -Fibonacci Script | 09:53 | |
| 73 | 12.2 -Horizontal Scaling | 08:59 | |
| 74 | 13.1 -File Upload | 15:44 | |
| 75 | 13.2 -Products Service | 04:41 | |
| 76 | 13.3 -Drizzle ORM | 15:36 | |
| 77 | 13.4 -Drizzle Kit Migrations | 07:28 | |
| 78 | 13.5 -Products Proto | 07:59 | |
| 79 | 13.6 -Products gRPC | 04:37 | |
| 80 | 13.7 -Load Products Job | 13:37 | |
| 81 | 13.8 -Job File Data | 12:36 | |
| 82 | 13.9 -Load Products Consumer | 03:36 | |
| 83 | 13.10 -Products gRPC Client | 08:48 | |
| 84 | 13.11 -Categories | 12:08 | |
| 85 | 13.12 -Products Dockerfile | 07:52 | |
| 86 | 13.13 -Products Helm | 12:00 | |
| 87 | 13.14 -Products Helm Testing | 07:12 | |
| 88 | 13.15 -Jobs Volume | 08:51 | |
| 89 | 14.1 -Jobs Prisma | 08:31 | |
| 90 | 14.2 -Save Job | 08:11 | |
| 91 | 14.3 -Job Acknowledge | 11:13 | |
| 92 | 14.4 -Executor Acknowledge | 13:44 | |
| 93 | 14.5 -Acknowledge Testing | 03:24 | |
| 94 | 14.6 -Jobs Dockerfile Prisma | 03:11 | |
| 95 | 14.7 -Acknowledge Helm | 10:36 | |
| 96 | 14.8 -Job Status Resolver | 07:49 | |
| 97 | 15.1 -Ingress | 10:20 | |
| 98 | 15.2 -AWS CLI | 03:49 | |
| 99 | 15.3 -Elastic Kubernetes Service | 03:44 | |
| 100 | 15.4 -LB Controller & CSI Driver | 08:22 | |
| 101 | 15.5 -AWS Helm Chart - Part 1 | 09:26 | |
| 102 | 15.6 -AWS Helm Chart - Part 2 | 08:28 | |
| 103 | 15.7 -Horizontal Scaling | 06:45 | |
| 104 | 15.8 -Custom Domain & SSL | 10:08 |
Unlock unlimited learning
Get instant access to all 103 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.
Learn more about subscriptionComments
0 commentsWant to join the conversation?
Sign in to commentSimilar courses
React, NextJS and NestJS: A Rapid Guide - Advanced
NestJS. Architecture and Advanced Patterns
Master NestJS 9 - Node.js Framework 2023