Dockerized ETL With AWS, TDengine & Grafana

29m 12s
English
Paid

Course description

Data engineers often need to quickly set up a simple ETL script that just gets the job done. In this project, you will learn how to easily implement such an ETL on AWS: connect live data from a weather API and write it to a TDengine time-series database.
Read more about the course

What you will learn

The Basics of Temporal Databases

You will get acquainted with the basics of working with temporal databases, their architecture, and use cases.

Working with a Public Weather API

You will set up and explore an external weather API, and write a Python script to read data from the API.

Docker ETL on AWS

You will package the script into a Docker container and deploy it as a serverless ETL using Amazon Elastic Container Registry (ECR), Lambda, and EventBridge.

TDengine Setup

You will get familiar with TDengine, set up an instance via the TDengine cloud and configure the database.

Data Visualization in Grafana

You will visualize data from the API stored in TDengine using Grafana. You’ll learn how to connect TDengine to Grafana and create a dashboard for data analysis.

Watch Online

This is a demo lesson (10:00 remaining)

You can watch up to 10 minutes for free. Subscribe to unlock all 16 lessons in this course and access 10,000+ hours of premium content across all courses.

View Pricing

Watch Online Dockerized ETL With AWS, TDengine & Grafana

0:00
/
#1: Quick note from Andreas before you begin

All Course Lessons (16)

#Lesson TitleDurationAccess
1
Quick note from Andreas before you begin Demo
00:44
2
Introduction
01:27
3
Setup Of The Project
02:53
4
Time Series Data Basics
02:21
5
Big Pros Of Timeseries Databases
02:07
6
About TDengine
01:23
7
Setup Weather API
01:05
8
Code query API
02:42
9
TDengine Setup
03:05
10
Connect Python To TDengine
01:51
11
Lambda Docker Container & Push To ECR
01:56
12
AWS Setup
01:37
13
Create Lambda Function Using Docker image
01:05
14
Schedule Function With EventBridge
01:26
15
Cloud Watch Lambda Events
00:28
16
Grafana Setup
03:02

Unlock unlimited learning

Get instant access to all 15 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.

Learn more about subscription

Comments

0 comments

Want to join the conversation?

Sign in to comment

Similar courses

The Data Engineering Bootcamp: Zero to Mastery

The Data Engineering Bootcamp: Zero to Mastery

Sources: zerotomastery.io
Learn to build streaming pipelines with Apache Kafka and Flink, create data lakes on AWS, run ML workflows on Spark, and integrate LLM models into...
13 hours 23 minutes 15 seconds
Machine Learning A-Z : Become Kaggle Master

Machine Learning A-Z : Become Kaggle Master

Sources: udemy
Want to become a good Data Scientist? Then this is a right course for you. This course has been designed by IIT professionals who have mastered in Mathematics and Data Science....
36 hours 23 minutes 54 seconds
Introduction to Data Engineering

Introduction to Data Engineering

Sources: zerotomastery.io
Companies of all sizes have access to enormous amounts of data, but the problem is that the data is often unstructured. In order to answer important...
57 minutes 26 seconds
Statistics for Data Science and Business Analysis

Statistics for Data Science and Business Analysis

Sources: udemy
Is statistics a driving force in the industry you want to enter? Do you want to work as a Marketing Analyst, a Business Intelligence Analyst, a Data Analyst, or
4 hours 49 minutes 30 seconds
Build a Large Language Model (From Scratch)

Build a Large Language Model (From Scratch)

Sources: Sebastian Raschka
"Creating a Large Language Model from Scratch" is a practical guide that will teach you step by step how to create, train, and fine-tune large language models..