Enhance your skills in managing time series data with this comprehensive course. The processing, storage, and visualization of time series data are becoming increasingly important. With IoT data, system logs, and production process statistics, the volume of information needing management is constantly expanding. This course will guide you in building a complete pipeline for handling time series data effectively.
Course Overview
Time series storage systems like InfluxDB and visualization tools such as Grafana are essential for data management and analysis. This course aims to equip you with the skills to implement these technologies in practice.
Learning Outcomes
Throughout this course, you will gain practical skills such as:
- Processing time series data from CSV files, including air quality datasets
- Connecting to external APIs to retrieve real-time weather data
- Writing data to InfluxDB 2.0 and querying it using Python and the Flux language
- Creating and configuring dashboards in Grafana: including server installation, panel configuration, and access management
Course Structure
Introduction
Begin with an overview of the project, and get familiar with the dataset and the InfluxDB interface, including graphs, queries, and data structure. You'll also explore the platform architecture to understand the interaction between components.
Data Schema Design
Learn about the features of relational and time series databases and determine the appropriate scenarios for each. You'll develop skills in designing a storage schema tailored to data characteristics and usage methods.
Environment Setup
Install and launch InfluxDB and Grafana using Docker. Configure the Python library for InfluxDB, create an access token, and set up your development environment in Visual Studio Code.
Working with Test Data
Gain experience in loading test CSV files and weather data into InfluxDB using Python. Discover potential issues during data loading and learn how to resolve them efficiently.
Working with Air Quality Data
Load air quality data into InfluxDB, write queries using Python, and connect Grafana to InfluxDB for visualization. Set up a data source and create a dashboard dedicated to air quality analysis.
Working with External Weather API
Get acquainted with a weather API, obtain an access key, and manage time zones effectively. Connect to an external API, load data into InfluxDB, and visualize it on a Grafana dashboard.
Working with Grafana Dashboards
Delve into the capabilities of Grafana, including multi-user mode, access rights management, and settings for users and organizations. Conclude the project by creating two organizations in Grafana and connecting weather and air quality data, applying your knowledge in practical scenarios.
This course provides essential practical skills for modern analytics and monitoring projects, focusing on time series data storage and visualization tools.
