Dockerized ETL With AWS, TDengine & Grafana

29m 12s
English
Paid

Course description

Data engineers often need to quickly set up a simple ETL script that just gets the job done. In this project, you will learn how to easily implement such an ETL on AWS: connect live data from a weather API and write it to a TDengine time-series database.
Read more about the course

What you will learn

The Basics of Temporal Databases

You will get acquainted with the basics of working with temporal databases, their architecture, and use cases.

Working with a Public Weather API

You will set up and explore an external weather API, and write a Python script to read data from the API.

Docker ETL on AWS

You will package the script into a Docker container and deploy it as a serverless ETL using Amazon Elastic Container Registry (ECR), Lambda, and EventBridge.

TDengine Setup

You will get familiar with TDengine, set up an instance via the TDengine cloud and configure the database.

Data Visualization in Grafana

You will visualize data from the API stored in TDengine using Grafana. You’ll learn how to connect TDengine to Grafana and create a dashboard for data analysis.

Watch Online

Join premium to watch
Go to premium
# Title Duration
1 Quick note from Andreas before you begin 00:44
2 Introduction 01:27
3 Setup Of The Project 02:53
4 Time Series Data Basics 02:21
5 Big Pros Of Timeseries Databases 02:07
6 About TDengine 01:23
7 Setup Weather API 01:05
8 Code query API 02:42
9 TDengine Setup 03:05
10 Connect Python To TDengine 01:51
11 Lambda Docker Container & Push To ECR 01:56
12 AWS Setup 01:37
13 Create Lambda Function Using Docker image 01:05
14 Schedule Function With EventBridge 01:26
15 Cloud Watch Lambda Events 00:28
16 Grafana Setup 03:02

Comments

0 comments

Want to join the conversation?

Sign in to comment

Similar courses

The Data Engineering Bootcamp: Zero to Mastery

The Data Engineering Bootcamp: Zero to Mastery

Sources: zerotomastery.io
Learn to build streaming pipelines with Apache Kafka and Flink, create data lakes on AWS, run ML workflows on Spark, and integrate LLM models into...
13 hours 23 minutes 15 seconds
The Data Bootcamp: Transform your Data using dbt™

The Data Bootcamp: Transform your Data using dbt™

Sources: udemy
Are you looking for a cutting-edge way to extract load and transform your data? Do you want to know more about dbt™ and how to use it? Well, this is the course
4 hours 10 minutes 51 seconds
SQL & Database Design A-Z™: Learn MS SQL Server + PostgreSQL

SQL & Database Design A-Z™: Learn MS SQL Server + PostgreSQL

Sources: udemy
Are you interested in a career in Data Science or Data Analytics? In that case, inevitably you are going to encounter databases in your work. But how do you int
12 hours 32 minutes 7 seconds
Relational Data Modeling

Relational Data Modeling

Sources: Eka Ponkratova
Relational modeling is widely used in building transactional databases. You might say, "But I'm not planning to become a backend engineer."
1 hour 52 minutes
Case Study in A/B Testing

Case Study in A/B Testing

Sources: LunarTech
Examples from practice in A/B testing - this course will introduce you to the methods of designing, conducting, and analyzing experiments using A/B...
1 hour 56 minutes 17 seconds