Dockerized ETL With AWS, TDengine & Grafana

29m 12s
English
Paid
Data engineers often need to quickly set up a simple ETL script that just gets the job done. In this project, you will learn how to easily implement such an ETL on AWS: connect live data from a weather API and write it to a TDengine time-series database.
Read more about the course

What you will learn

The Basics of Temporal Databases

You will get acquainted with the basics of working with temporal databases, their architecture, and use cases.

Working with a Public Weather API

You will set up and explore an external weather API, and write a Python script to read data from the API.

Docker ETL on AWS

You will package the script into a Docker container and deploy it as a serverless ETL using Amazon Elastic Container Registry (ECR), Lambda, and EventBridge.

TDengine Setup

You will get familiar with TDengine, set up an instance via the TDengine cloud and configure the database.

Data Visualization in Grafana

You will visualize data from the API stored in TDengine using Grafana. You’ll learn how to connect TDengine to Grafana and create a dashboard for data analysis.

Watch Online Dockerized ETL With AWS, TDengine & Grafana

Join premium to watch
Go to premium
# Title Duration
1 Quick note from Andreas before you begin 00:44
2 Introduction 01:27
3 Setup Of The Project 02:53
4 Time Series Data Basics 02:21
5 Big Pros Of Timeseries Databases 02:07
6 About TDengine 01:23
7 Setup Weather API 01:05
8 Code query API 02:42
9 TDengine Setup 03:05
10 Connect Python To TDengine 01:51
11 Lambda Docker Container & Push To ECR 01:56
12 AWS Setup 01:37
13 Create Lambda Function Using Docker image 01:05
14 Schedule Function With EventBridge 01:26
15 Cloud Watch Lambda Events 00:28
16 Grafana Setup 03:02

Similar courses to Dockerized ETL With AWS, TDengine & Grafana

Data Engineering on AWS

Data Engineering on AWSAndreas Kretz

Category: AWS, Data processing and analysis
Duration
Data Engineering with Hadoop

Data Engineering with HadoopSuyog Nagaokar

Category: Data processing and analysis
Duration 7 hours 3 minutes
TensorFlow Developer Certificate in 2023: Zero to Mastery

TensorFlow Developer Certificate in 2023: Zero to Masteryzerotomastery.io

Category: Data processing and analysis
Duration 62 hours 43 minutes 54 seconds
Statistics for Data Science and Business Analysis

Statistics for Data Science and Business Analysisudemy

Category: Data processing and analysis
Duration 4 hours 49 minutes 30 seconds
Statistics Bootcamp (with Python): Zero to Mastery

Statistics Bootcamp (with Python): Zero to Masteryzerotomastery.io

Category: Python, ChatGPT, Data processing and analysis
Duration 20 hours 50 minutes 51 seconds
Mathematical Foundations of Machine Learning

Mathematical Foundations of Machine Learningudemy

Category: Python, Data processing and analysis
Duration 16 hours 25 minutes 26 seconds
Build Fast Masterclass

Build Fast MasterclassBuildFast Academy

Category: Python, Data processing and analysis
Duration 7 hours 22 minutes 11 seconds
PyTorch for Deep Learning and Computer Vision

PyTorch for Deep Learning and Computer Visionudemy

Category: Data processing and analysis
Duration 10 hours 20 minutes 51 seconds
Spark and Python for Big Data with PySpark

Spark and Python for Big Data with PySparkudemy

Category: Python, Data processing and analysis
Duration 10 hours 35 minutes 43 seconds
Time Series Analysis, Forecasting, and Machine Learning

Time Series Analysis, Forecasting, and Machine Learningudemy

Category: Python, Data processing and analysis
Duration 22 hours 47 minutes 45 seconds