Data Engineering on AWS

0h 0m 0s
English
Paid

This course is the perfect start for those who want to master cloud technologies and begin working with Amazon Web Services (AWS), one of the most popular platforms for data processing. The course is especially useful for beginner data engineers and those seeking their first job in this field.

Throughout the course, you will create a fully-fledged end-to-end project based on data from an online store. Step by step, you will learn to model data, build pipelines, and work with key AWS tools: Lambda, API Gateway, Kinesis, DynamoDB, Redshift, Glue, and S3.

Read more about the course

What to expect in the course:

  • Data Work
    • Learn the structure and types of data you will be working with. Define the project goals - an important step for successful implementation.
  • Platform and Pipeline Design
    • Get acquainted with the platform architecture and design pipelines: for data loading, storage in S3 (Data Lake), processing in DynamoDB (NoSQL), and Redshift (Data Warehouse). Learn to build pipelines for interfaces and data streaming.
  • Basics of AWS
    • Create an account in AWS, understand access and security management (IAM), get introduced to CloudWatch and the Boto3 library for working with AWS through Python.
  • Data Ingestion Pipeline
    • Create an API via API Gateway, send data to Kinesis, configure IAM, and develop an ingestion pipeline in Python.
  • Data Transfer to S3 (Data Lake)
    • Set up a Lambda function to receive data from Kinesis and save it to S3.
  • Data Transfer to DynamoDB
    • Implement a pipeline for transferring data from Kinesis to DynamoDB - a fast NoSQL database.
  • API for Data Access
    • Create an API for working with data in the database. Learn why direct access from visualization to the database is a bad practice.
  • Data Visualization in Redshift
    • Send streaming data to Redshift via Kinesis Firehose, create a Redshift cluster, configure security, create tables, and set up Firehose. Connect Power BI to Redshift for data analysis.
  • Batch Processing: AWS Glue, S3, and Redshift
    • Master batch data processing: set up and run Glue to write data from S3 to Redshift, understand Crawler and data catalog, and learn to debug processes.

This course will help you gain practical experience in creating streaming and batch pipelines in AWS, as well as mastering key tools for working with cloud data.

Similar courses to Data Engineering on AWS

AWS & Typescript Masterclass - CDK V2, Serverless, React

AWS & Typescript Masterclass - CDK V2, Serverless, Reactudemy

Category: TypeScript, AWS
Duration 10 hours 48 minutes 18 seconds
Choosing Data Stores

Choosing Data StoresAndreas Kretz

Category: Data processing and analysis
Duration 1 hour 25 minutes 31 seconds
Spark and Python for Big Data with PySpark

Spark and Python for Big Data with PySparkudemy

Category: Python, Data processing and analysis
Duration 10 hours 35 minutes 43 seconds
AWS Certified Developer - Associate

AWS Certified Developer - AssociateAdrian Cantrill

Category: AWS
Duration 68 hours 8 minutes 48 seconds
Serverless Framework Bootcamp: Node.js, AWS & Microservices

Serverless Framework Bootcamp: Node.js, AWS & Microservicesudemy

Category: AWS, Node.js
Duration 5 hours 24 minutes 21 seconds
DevOps Deployment Automation with Terraform, AWS and Docker

DevOps Deployment Automation with Terraform, AWS and Dockerudemy

Category: AWS, Docker, Python, Terraform
Duration 10 hours 59 minutes 9 seconds
Case Study in Product Data Science

Case Study in Product Data ScienceLunarTech

Category: Data processing and analysis
Duration 1 hour 4 minutes 47 seconds
Mathematical Foundations of Machine Learning

Mathematical Foundations of Machine Learningudemy

Category: Python, Data processing and analysis
Duration 16 hours 25 minutes 26 seconds
Data Platform & Pipeline Design

Data Platform & Pipeline DesignAndreas Kretz

Category: Data processing and analysis
Duration 1 hour 59 minutes 5 seconds
Production-Ready Serverless

Production-Ready ServerlessYan Cui

Category: AWS, Others
Duration 13 hours 37 minutes 6 seconds