Machine Learning & Containers on AWS

Course Structure

Twitter API

Twitter API is a great place for accessing open data. You will learn how to configure access to the API and retrieve tweets from a user's feed for further processing. We will delve into API configuration and the data format (payload) it returns.

RDS Database

Every platform needs a data storage. You will learn how to set up a Postgres database in Amazon RDS and understand why we will be saving JSON tweets in this database. You will also master working with virtual private clouds (VPC) to make the database accessible from the internet. With PGAdmin, you will create tables and execute queries on the database.

NLP Lambda

For text analysis, we will use a ready-made machine learning algorithm from the Natural Language Toolkit (NLTK) library. You will create a Lambda function to retrieve tweets from the API, determine their sentiment, and save the results in the database.

To run the Lambda function, you will learn to connect the necessary dependencies through layers - how to import prepared K-Layers and create your own layer. You will also learn how to set up an automatic Lambda function trigger using Event Bridge.

Dependency Management and Streamlit Application

For visualizing results, you will create an application using Streamlit. You will set up a local development environment with Anaconda3 and create a conda virtual environment. Using the provided Git repository, you will learn to manage project dependencies with Poetry. We will go through the application code step by step and demonstrate how to run it in a new virtual environment for testing.

Deploying Streamlit Application in ECS

Once the visualization is ready, you will learn to work with Docker images and containers in AWS. You will create an Elastic Container Registry (ECR) and set up AWS CLI. You will learn to create user groups and individual users with restricted access rights in IAM.

After building the Docker image, you will upload it to ECR, configure an ECS Fargate cluster, and deploy your Streamlit application as a task on the platform.