CF

Semantic Log Indexing & Search

53m 37s
English
Paid
Updated September 2026

Semantic Log Indexing & Search is a 16-lesson 53 minutes self-paced course by Andreas Kretz. Keyword search on logs only gets you so far; this course, from Andreas Kretz , builds a full semantic search pipeline instead, one that understands meaning rather than just matching text.

Course facts

Lessons
16
Duration
53 minutes
Level
All levels
Language
English
Updated
2026-09-11
Instructor
Andreas Kretz
Price
Premium

Keyword search on logs only gets you so far; this course, from Andreas Kretz, builds a full semantic search pipeline instead, one that understands meaning rather than just matching text.

Building the Pipeline

Following on from The Hidden Foundation of GenAI, you'll take embeddings from theory into practice: aggregating logs, processing them through a FastAPI service, and storing the resulting embeddings in the qdrant vector database.

Search and Comparison

  • Exploring qdrant collections, points, and cosine similarity search
  • Building a Streamlit dashboard for natural-language log search
  • Comparing semantic search results against traditional SQL queries in DuckDB
  • Refining embeddings and query formulation for better accuracy

Across 16 lessons, the entire stack, FastAPI, qdrant, Streamlit, and DuckDB, gets deployed with Docker Compose, leaving you with a working project you can adapt to your own AI-driven search use case.

Additional

https://github.com/team-data-science/GenAI-DataObservability

Who teaches Semantic Log Indexing & Search? Andreas Kretz

Andreas Kretz thumbnail

Andreas Kretz is a German data engineer and one of the most widely followed independent voices on data engineering as a career discipline. He runs the Plumbers of Data Science brand and has been publishing tutorial material continuously since the field consolidated around the modern lake-house stack (Spark, Kafka, Snowflake, Databricks, Airflow).

His CourseFlix listing is the largest single-author catalog under this source — over thirty courses spanning data-pipeline construction, streaming architectures, the cloud-native data stack on AWS / Azure / GCP, the Python and Scala tooling that dominates the field, and the soft-skills / career side of breaking into data engineering. Material is paid and aimed at engineers transitioning into data work or already-working data engineers picking up specific tools.

What lessons are included in Semantic Log Indexing & Search?

This is a demo lesson (10:00 remaining)

You can watch up to 10 minutes for free. Subscribe to unlock all 16 lessons in this course and access 10,000+ hours of premium content across all courses.

View Pricing
0:00
/
#1: Intro
All Course Lessons (16)
#Lesson TitleDurationAccess
1
Intro Demo
00:44
2
Getting Started: Semantic Search for Your Logs
03:08
3
Dissecting the Pipeline Monitor Architecture: FastAPI, Qdrant & DuckDB
03:50
4
Beginner’s Guide to Qdrant Collections and Similarity Search
03:28
5
Your First Glimpse at the Project Code Structure on GitHub
02:55
6
Building and Launching the Pipeline with Docker Compose
04:37
7
Writing JSON Logs to FastAPI: Bulk Upload Explained
01:42
8
How FastAPI Parses LogEntry Models and Prepares Embeddings
04:37
9
Embeddings 101: Turning Your Logs into Searchable Vectors
02:06
10
Querying Qdrant: From Playground to Streamlit Dashboard
03:55
11
Hands-On Embedding Tuning: Boost Your Log Search Accuracy
03:54
12
Deploying Improved Embeddings and Measuring Improvement
05:35
13
What We Built and Why It Matters
02:53
14
How DuckDB Fits into Your Data Observability Stack
01:28
15
Writing to DuckDB with a Write-Ahead Log
05:03
16
Docker & DuckDB: Implementing WAL to Solve File Lock Errors
03:42
Unlock unlimited learning

Get instant access to all 15 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.

Learn more about subscription

What courses are similar to Semantic Log Indexing & Search?

More courses by Andreas Kretz

Frequently asked questions

What prior knowledge do I need before taking this course?
Before enrolling in this course, it is recommended to have foundational knowledge of generative AI, which is covered in the prerequisite course 'The Hidden Foundation of GenAI'. Familiarity with basic web development, API creation, and database management will also be beneficial as the course involves working with FastAPI, qdrant, and DuckDB.
What kind of projects will I work on during the course?
This course involves developing a data observability project. You will build a pipeline that aggregates logs, processes them using FastAPI, and stores the generated embeddings in qdrant. Additionally, you will create a user-friendly dashboard with Streamlit to facilitate semantic log searches, offering a practical alternative to traditional SQL-based searches.
Who is the target audience for this course?
The course is ideal for data engineers, software developers, and IT professionals interested in enhancing their skills in semantic search technologies and data observability. Individuals looking to apply generative AI concepts to real-world data processing projects will find the course particularly relevant.
How does this course compare in depth and scope to similar courses?
Unlike introductory courses that focus solely on the basics of semantic search, this course offers a comprehensive approach by integrating the entire process from generating embeddings to implementing a full semantic search pipeline. It covers advanced topics such as embedding optimization, vector storage with qdrant, and dashboard creation with Streamlit.
What tools and platforms will I learn to use in this course?
Throughout the course, you will gain hands-on experience with FastAPI for API development, qdrant for vector storage and similarity search, and Streamlit for building interactive dashboards. Additionally, you will work with DuckDB to compare semantic search with traditional SQL queries.
What topics are not covered in this course?
The course does not cover the foundational principles of generative AI in detail; it assumes prior knowledge from 'The Hidden Foundation of GenAI'. It also does not delve into the specifics of machine learning model training or traditional keyword-based search techniques.
What is the expected time commitment for completing the course?
The course consists of 16 structured lessons. The exact runtime is not specified, but students should anticipate dedicating several hours per week to complete the lessons, understand the material, and work on the hands-on project components.