Contact Tracing with Elasticsearch

You will create your own dataset using Python and Pandas, utilizing an open dataset of San Francisco stores containing over 140,000 stores with their names and coordinates. From this dataset, you will select 10,000 stores and generate 100,000 fictional users, each of whom will perform an average of 10 check-ins. After uploading the data to Elasticsearch, you will create a user interface with Streamlit for data visualization.

Your application interface includes:

• Search by store name
• Search by ZIP code to filter stores by area
• Search by business ID for visit analysis
• Search and track by Device ID to see where a specific user has been

In the course of working on the project, you will learn to:

• Transform data and upload it in parquet format to Elasticsearch
• Work with Kibana for index management and document search
• Create an interactive interface with Streamlit featuring controls, Folium maps, and tables
• Configure pages and execute queries to Elasticsearch

Course Program

• Preparing the San Francisco dataset with 10,000 stores
• Generating 100,000 fictional users
• Merging user data with stores
• Creating 1,000,000 app check-ins
• Preparing data for upload to Elasticsearch
• Uploading data to Elasticsearch
• Developing a Streamlit application: maps, filters, tables
• Page setup and working with Elasticsearch queries

Requirements

Before starting, it is recommended to take the course “Log Analysis in Elasticsearch” to understand the basics of working with Elasticsearch. Additionally, due to extensive work with data, it's advisable to complete the lessons on Pandas from the course “Python for Data Engineers”.

The project is designed for a computer with 8 GB of RAM.