CF

Data Engineering on GCP

1h 17m 33s
English
Paid
Updated September 2026

Data Engineering on GCP is a 23-lesson 1 hour 17 minutes self-paced course by Andreas Kretz. Google Cloud Platform offers a deep toolset for building and scaling data pipelines, and Andreas Kretz built this course around actually shipping one, from raw API data to a finished dashboard.

Course facts

Lessons
23
Duration
1 hour 17 minutes
Level
All levels
Language
English
Updated
2026-09-11
Instructor
Andreas Kretz
Price
Premium

Google Cloud Platform offers a deep toolset for building and scaling data pipelines, and Andreas Kretz built this course around actually shipping one, from raw API data to a finished dashboard.

The project

  • Extract data from an external weather API
  • Process it through a pipeline built on GCP services
  • Store the results in a server database
  • Visualize the outcome in Looker Studio

Services you'll use

MySQL via Cloud SQL for storage, a Linux VM on Compute Engine for database management, Cloud Scheduler for triggering API calls on a schedule, cloud functions for processing, and Pub/Sub for passing data between services.

What you'll take away

Across 23 lessons plus a companion GitHub repo, you'll set up a Google Cloud account (Google currently offers $300 in trial credit for new accounts), build out the full pipeline, write the server functions that store data correctly, and build bubble charts and time-series visualizations in Looker Studio — skills that carry over to AWS and other clouds given how similar the underlying services are.

Additional

Link to the GitHub: https://github.com/team-data-science/Data-Engineering-On-GCP

Who teaches Data Engineering on GCP? Andreas Kretz

Andreas Kretz thumbnail

Andreas Kretz is a German data engineer and one of the most widely followed independent voices on data engineering as a career discipline. He runs the Plumbers of Data Science brand and has been publishing tutorial material continuously since the field consolidated around the modern lake-house stack (Spark, Kafka, Snowflake, Databricks, Airflow).

His CourseFlix listing is the largest single-author catalog under this source — over thirty courses spanning data-pipeline construction, streaming architectures, the cloud-native data stack on AWS / Azure / GCP, the Python and Scala tooling that dominates the field, and the soft-skills / career side of breaking into data engineering. Material is paid and aimed at engineers transitioning into data work or already-working data engineers picking up specific tools.

What lessons are included in Data Engineering on GCP?

This is a demo lesson (10:00 remaining)

You can watch up to 10 minutes for free. Subscribe to unlock all 23 lessons in this course and access 10,000+ hours of premium content across all courses.

View Pricing
0:00
/
#1: Introduction
All Course Lessons (23)
#Lesson TitleDurationAccess
1
Introduction Demo
01:14
2
GitHub & the team
01:31
3
Architecture of this project
03:20
4
Introduction Weather API
02:19
5
Setup Google Cloud Account
02:13
6
Creating the project
02:36
7
Enabling the required APIs
01:35
8
Configure scheduling
02:21
9
Setup VM for database interaction
02:54
10
Setup mysql database
02:17
11
Setup vm client and create database
02:47
12
Creating pub/sub message queue
01:42
13
Create cloud function to pull data form API
04:18
14
Explanation code pull from API
04:21
15
Create function to write to db
07:48
16
Explanation code write data to db
05:57
17
Testing the function
05:52
18
Create function write data to db - pull
03:54
19
Explanation code write data to db - pull
04:34
20
Setup Looker Studio and create bubble chart
02:21
21
Setup Looker Studio and create time series chart
01:58
22
Pipeline Monitoring
06:21
23
Conclusion & Challenges
03:20
Unlock unlimited learning

Get instant access to all 22 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.

Learn more about subscription

What courses are similar to Data Engineering on GCP?

More courses by Andreas Kretz

Frequently asked questions

What are the prerequisites for enrolling in the course?
The course does not explicitly list prerequisites, but familiarity with cloud platforms and basic programming concepts will be beneficial. The course starts with setting up a Google Cloud account and progresses through pipeline creation, which may require some technical understanding.
What types of projects will I work on during the course?
During the course, you will work on a project that involves extracting data from an external weather API, processing it through a pipeline using Google Cloud Platform services, storing it in a MySQL database, and creating visualizations with Looker Studio.
Who is the target audience for this course?
The course is designed for aspiring and current data engineers who want to learn about building data pipelines on Google Cloud Platform. It is also suitable for professionals looking to expand their skills to include cloud-based data processing and visualization.
How does the depth of this course compare to similar courses?
This course provides a project-based approach to learning about Google Cloud Platform, allowing students to gain practical experience in data extraction, processing, storage, and visualization. The focus on a real-world project distinguishes it from other courses that might only cover theoretical aspects.
Which specific tools and platforms are covered in the course?
The course covers several Google Cloud Platform services such as Pub/Sub, Cloud Functions, and Looker Studio. Students will also use MySQL for database storage and learn to set up virtual machines for database interaction.
What topics are not covered in this course?
The course does not cover advanced machine learning algorithms or detailed network security practices. It focuses on building and managing data pipelines using GCP rather than in-depth data science or security topics.
How can the skills learned in this course be applied to other careers or platforms?
The skills learned in the course are applicable to other cloud platforms such as AWS, as many of the concepts and services are similar. Understanding how to build data pipelines and create visualizations is valuable for roles in data engineering, analytics, and cloud architecture across various industries.