1 Python for Data Science Automation: Let's Do This! Demo 02:19 2 The Game Plan: Data Analysis Foundations 00:46 3 The Business Case: Building an Automated Forecast System 00:44 4 Course Project Zip [File Download] 00:45 5 Course Workflow: Tying Specific Actions to the Business Process 02:27 6 Ultimate Python Cheat Sheet: Python Ecosystem in 2 Pages 04:03 7 The Transactional Database Model [PDF Download] 03:35 8 02:38 9 IDE (Integrated Development Environment) Options 02:19 10 01:38 11 Connect VSCode to Your Course Project Files 01:13 12 Conda Env Create: Make the Python Course Environment 04:41 13 Python Select Interpreter: Connect VSCode to Your Python Environment 00:41 14 Conda Env Update: Add Python Packages to Your Environment 01:44 15 Conda Env Export: Review & Share Your Environment 01:12 16 Conda Env List & Remove: List Available Environments & Remove Unnecessary Envs 01:22 17 01:15 18 VSCode Theme Customization 02:07 19 00:44 20 VSCode User & Workspace Settings 04:16 21 VSCode Keyboard Shortcuts 01:17 22 03:23 23 VSCode Jupyter Extension - Jupyter Notebook Support 02:05 24 VSCode Jupyter Extension - Interactive Python 03:35 25 [Optional VSCode Setting] Jupyter: Send Selection to Interactive Window 02:31 26 01:01 27 VSCode Markdown & PDF Extensions 02:43 28 01:09 29 00:41 30 [Optional] VSCode Extensions for R Users 01:27 31 Python Environment Checkpoint [File Download] 03:53 32 Getting Started [File Download] 04:08 33 01:26 34 Import: pandas, numpy, matplotlib.pyplot 03:34 35 Importing From: plotnine, miziani 04:42 36 Importing Functions and Submodules: os, rich 02:10 37 Setting Up Python Interactive 02:45 38 [Reminder | Optional VSCode Setting] Jupyter: Send Selection to Interactive Window 02:31 39 Getting Help Documentation 02:47 40 IMPORTANT VSCODE SETTING: File Paths | jupyter.notebookFileRoot 06:35 41 06:46 42 05:10 43 Exploratory 1: Top 5 Most Frequent Descriptions 03:55 44 Exploratory 2: Plotting the Top 5 Bike Descriptions 06:23 45 Preparing Orderlines for Merge: Drop Column 03:05 46 Merging the Bikes DataFrame 03:34 47 Merging the Bikeshops Data Frame 03:27 48 Datetime: Converting Order Date | Copy vs No Copy 04:52 49 Splitting the Description: Category 1, Category 2, and Frame Material 07:27 50 Splitting Location: City, State 03:05 51 Create the Total Price Column 02:55 52 04:44 53 04:07 54 Reviewing the Data Transformations 01:12 55 Save Your Work: Pickle it. 03:50 56 Pandas Datetime Accessors 02:44 57 Resampling: Working with Pandas Offsets 07:26 58 Quick Plot: Plotting Single Time Series w/ Pandas Matplotlib Backend 01:41 59 Plotnine Visualization: Sales By Month, Part 1 - Geometries 05:53 60 Plotnine Visualization: Sales by Month, Part 2 - Scales & Themes 05:51 61 Resampling Groups: Combine groupby() and resample() 09:23 62 Quick Plot: Plotting Multiple Time Series w/ Pandas Matplotlib Backend 07:24 63 Plotnine Visualization, Part 1: Facetted Sales By Date & Category2 (Group) 08:58 64 Plotnine Visualization, Part 2: Adding Themes & Scales 08:53 65 Writing Files: Pickle, CSV, Excel 04:42 66 Congrats. That was a fun whirlwind. Let's recap. 02:35 67 Getting Started [File Download] 01:22 68 03:41 69 04:00 70 03:26 71 01:47 72 Pandas I/O & SQL Alchemy Overviews 03:02 73 01:24 74 Create the SQLite Database 04:20 75 03:04 76 Create the Database Tables 07:12 77 00:54 78 02:08 79 Getting the Database Table Names 02:36 80 Reading from the Tables with f-strings 01:48 81 [Bonus] VSCode SQLite Extension 03:05 82 Making collect_data(), Part 1: Function Setup 06:40 83 Making collect_data(), Part 2: Read Tables from the Database 08:39 84 Making collect_data(), Part 3: Test the Database Import 01:15 85 Making collect_data(), Part 4: Joining the Data 08:26 86 Making collect_data(), Part 5: Cleaning the Data 1 07:15 87 Making collect_data(), Part 6: Cleaning the Data 2 06:49 88 Making collect_data(), Part 7: VSCode Docstring Generator 03:59 89 Making a Package (my_pandas_extensions): Adding the database module 04:42 90 🥳Congrats! You're learning really powerful concepts. 01:07 91 Getting Started [File Download] 02:25 92 [VSCode Setting] Jupyter: Send Selection to Interactive Window 01:13 93 Package & Function Imports 01:29 94 My Pandas Extensions: Fix FutureWarning Message (regex) 01:29 95 How Python Works: Objects 05:30 96 Pandas DataFrame & Series 02:52 97 04:09 98 Python Builtin Data Structures: Dictionary, List, Tuple 05:54 99 Python Builtin Data Types: Int, Float, Str, Bool, 03:42 100 Casting Basics: Numeric & String Conversions 04:10 101 Casting Sequences: To List, Numpy Array, Pandas Series, & DataFrame 02:41 102 Pandas Series Dtype Conversion 01:44 103 Pandas Data Wrangling Setup 02:09 104 Subsetting Columns by Name 02:17 105 Subsetting by Column Index (Position): iloc[] 01:36 106 Subsetting Columns with Regex (Regular Expressions) 03:38 107 Rearranging a Single Column (Column Subsetting) 02:17 108 Rearranging Multiple Columns (Repetitive Way First) 01:44 109 Rearranging Multiple Columns (List Comprehension) 02:51 110 Data Frame Rearrange: Select Dtypes, Concat, & Drop 06:33 111 03:07 112 Simple Filters with Boolean Series 03:55 113 03:48 114 Filtering with isin() and 03:42 115 Index slicing with df.iloc[] 02:42 116 Getting Distinct Values: Drop duplicates 01:44 117 02:15 118 01:53 119 DataFrame Column Assignment: Calculated Columns 02:26 120 Assign Basics: Lambda Functions 03:11 121 Assign Cookbook: Making a Log Transformation 03:32 122 Assign Cookbook: Searching Text (Boolean Flags) 05:27 123 Assign Cookbook: Even-Width Binning with pd.cut() 03:46 124 Visualizing Binning Strategies with a Pandas Heat Table 03:01 125 Assign Cookbook: Quantile Binning with pd.qcut() 02:36 126 Aggregation Basics (Summarizations) 05:49 127 04:11 128 Groupby + Aggregate Basics (Summarizations) 05:27 129 Groupby + Agg Cookbook (Summary DF 1): Sum & Median Total Price By Category 1 & 2 03:14 130 Groupby + Agg Cookbook (Summary DF 2): Sum Total Price & Quantity By Category 1 & 2 03:24 131 Groupby + Agg Details: Examining the Multilevel Column Index 02:01 132 Groupby + Agg Cookbook (Summary DF 3): Grouping Time Series with Groupby & Resample 04:12 133 Groupby + Apply Basics (Transformations) 03:42 134 Groupby + Apply Cookbook: Transform All Columns by Group 02:35 135 Groupby + Apply Cookbook: Filtering Slices by Group 03:25 136 Renaming Basics: Renaming All Columns with Lambda 04:28 137 Renaming Basics: Targeting Specific Columns 01:21 138 Advanced Renaming: Renaming Multi-Index Columns 05:57 139 Set Up Summarized Data: Revenue by Category 1 05:00 140 06:42 141 Export a Stylized Pandas Table to Excel (Wide Data) 06:09 142 03:31 143 Plotnine - Making a Faceted Horizontal Bar Chart (Tidy Long Data) 04:34 144 Intro to Categorical Data: Sorting the Plotnine Plot 06:09 145 Pivot Table (An awesome function for BI Tables) 07:42 146 Unstack: A programmatic version of pivot() 04:10 147 Stack: A programmatic version of melt() 02:25 148 04:12 149 Concat: Binding DataFrames Rowwise & Columnwise 04:27 150 03:08 151 01:07 152 Set Up Summarized Data: Sales by Category 2 Daily 03:02 153 Apply: Lambda Aggregations vs Transformations 02:22 154 Apply: Broadcasting Aggregations 01:53 155 Grouped Apply: Broadcasting 02:24 156 Grouped Transform: Alternative to Grouped Apply (Fixes Index Issue) 02:03 157 Making a "Data Frame" Function: add_columns() 06:08 158 Pipe: Method chaining our custom function using the pipe 03:12 159 Challenge #1: Data Wrangling with Pandas [File Download] 01:17 160 Method 1: Jupyter VSCode Integration 02:25 161 Method 2: Jupyter Notebooks (Legacy Method) 02:07 162 Method 3: JupyterLab (Next Generation of Jupyter) 03:16 163 03:08 164 Getting Started: Syncing Your JupyterLab Current Working Directory (%cd and %pwd) 05:10 165 03:18 166 08:40 167 Congrats! You've finished your first challenge. 01:37 168 Automating Time Series Forecasting 01:40 169 Getting Started [File Download] 01:49 170 VSCode Extension: Browser Preview 01:41 171 01:40 172 The ProfileReport() Class 01:10 173 Section 1: Profile Overview 03:19 174 Section 2A: Numeric & Date Variables 06:03 175 Section 2B: Categorical (Text) Variables 05:02 176 Sections 3-6: Interactions, Correlations, Missing Values, & Sample 02:52 177 Pandas Extension: df.profile_report() 03:09 178 Exporting the Profile Report as HTML 01:36 179 00:50 180 TimeStamp & Period Conversions 02:59 181 Pandas Datetime Accessors 01:56 182 Date Math: Offsetting Time with TimeDelta's 02:39 183 Date Math: Getting Duration between Two TimeStamps 03:28 184 Creating Date Sequences: pd.date_range() 03:09 185 07:58 186 Resampling (In-Depth): bike_sales_m_df 06:24 187 Grouped Resampling (In-Depth): bike_sales_cat2_m_wide_df 06:38 188 Reorganizing: Adding Comments 01:31 189 Differencing with Lags (Single Time Series) 05:39 190 Differencing with Lags (Multiple Time Series) 02:05 191 Difference from First (Single Time Series) 01:43 192 Difference From First (Multiple Time Series) 00:58 193 Cumulative Expanding Windows (Single Time Series) 03:21 194 Cumulative Expanding Windows (Multiple Time Series) 01:39 195 Moving Average (Single Time Series) 08:15 196 Moving Average (Multiple Time Series) 04:37 197 Next Steps (Where we are headed) 01:17 198 Getting Started [File Download] 01:37 199 Setup: Python Imports & Data 00:46 200 Function Anatomy: pd.Series.max() 03:53 201 01:03 202 01:17 203 Function Anatomy: **kwargs 05:13 204 Detect Outliers: Function Setup 02:19 205 IQR Outlier Method, Part 1 03:37 206 04:07 207 New Argument: IQR Multiplier 01:47 208 New Argument: How? (Both, Upper, Lower) 02:36 209 Checking for Pandas Series Input 02:11 210 Checking IQR Multiplier for Int or Float Type 02:54 211 Checking that IQR Multiplier is a Positive Value 01:10 212 Checking that How is a Valid Option: both, lower, upper 02:19 213 Informative Help Documentation: Adding a Docstring 07:12 214 Testing Our Function: Detecting Outliers within Groups 03:05 215 Extending the Pandas Series Class 02:12 216 Summarize By Time: A handy function for time series wrangling 04:00 217 Setting Up the "Summarize By Time" Function 04:56 218 Handling the Date Column Input 01:31 219 02:03 220 Handling the Time Series Resample 04:14 221 Handling the Aggregation Function Input 03:16 222 Handling the Value Column Input 01:40 223 Forcing the Value Column Input to a List (to generate a data frame) 02:44 224 Bug! Thinking through a solution 02:26 225 Solution: Converting to a Function Dictionary with Zip + Dict 03:52 226 02:02 227 Handling the Period Conversion 02:51 228 Add Fill Missing Capability 02:25 229 Review the Core Functionality 01:25 230 Check Incoming Data: Raising a TypeError 01:50 231 07:28 232 Pandas Flavor: Extending Pandas DataFrame Class 06:23 233 Getting Started [File Download] 03:03 234 04:36 235 How to Google Search like a Pro 01:35 236 02:41 237 Summarizing to get Total Revenue by Month 05:00 238 Summarizing to get Total Revenue by Category 2 & Month 02:42 239 04:59 240 AutoARIMA Applied: Forecaster, Fit, Predict 08:25 241 Adding Confidence Intervals (Prediction Intervals) 02:41 242 Tuple Unpacking (Predictions, Confidence Intervals) 02:39 243 05:28 244 00:24 245 Multiple Time Series Forecasting: AutoARIMA() 03:10 246 For Loop: Iterate Across the DataFrame Columns 02:20 247 For Loop: Modeling AutoARIMA() 05:23 248 For-Loop: Getting the Confidence Intervals 01:32 249 For-Loop: Combine with DataFrame | Actual Values, Predictions, & CIs 04:12 250 For-Loop: Storing the Results (as a Dictionary) 03:36 251 Housekeeping: Appending Variable Types to Variable Names 01:53 252 Visual Forecast Assessment 02:43 253 03:41 254 Setting up the ARIMA Automation Function 03:45 255 Making arima_forecast() | Function Definition 03:19 256 Function Body | Setting Up the Iteration 04:41 257 Training the AutoARIMA() Models 03:02 258 Controlling Progress Bars: tqdm(min_interval) 01:12 259 Making Predictions and Confidence Intervals 02:09 260 Combine Results into a DataFrame 02:24 261 Compose a Prediction Dictionary 01:50 262 Return Results as a Single DataFrame | Rowwise Concatenation 02:37 263 Setting the Column Names of the Output 09:16 264 Drop remaining columns beginning with "level_" 02:51 265 Testing the arima_forecast() function 02:05 266 Creating the forecasting.py module 03:44 267 Docstring: arima_forecast() 01:32 268 Adding Checks: arima_forecast() 06:35 269 Finally - Check Your Forecasts with Grouped Pandas Plotting 02:29 270 Recap: You've just made an ARIMA Forecast Automation! 01:10 271 Introduction to ETS Forecasting (Exponential Smoothing) 02:07 272 Challenge 2 [File Download] 06:07 273 05:18 274 Part 3: Visualization & Reporting 01:25 275 Getting Started [File Download] 00:32 276 03:15 277 Plotnine Anatomy: Imports 02:57 278 Data Summarization: For Plotting Annual Bike Sales 02:54 279 The Plot Canvas: Mapping Columns to Plot Components 07:13 280 04:00 281 Adding a Trend Line: geom_smooth() 03:00 282 01:53 283 01:42 284 Scales: Dollar Format for Y-Axis 03:52 285 Scales: Date Format for X-Axis 02:15 286 02:58 287 01:18 288 Exploring the Plotnine Object 02:24 289 02:20 290 Scatter Plot: Data Manipulation 02:52 291 Scatter Plot: Visualization 03:18 292 Line Plot: Data Manipulation 02:08 293 05:29 294 Data Manipulation, Part 1: No Categorical Ordering 02:49 295 Visualization, Part 1: Without Categorical Ordering 01:35 296 Aside: Introduction to Plotting using Categorical Data Type 09:55 297 Finalizing the Horizontal Bar Chart 01:20 298 Histogram: Data Manipulation 02:48 299 02:15 300 Histogram: Using Fill Aesthetic to Explore Differences by a Category 02:51 301 Histogram: Using Facet Grids to Compare Distributions by Category 02:48 302 Density Plots: Kernel Density Estimation (KDE) using geom_density() 02:55 303 Box Plot: Data Manipulation 02:20 304 07:24 305 Violin Plot with Jitter: geom_violin() and geom_jitter() 03:32 306 Data Manipulation: Add a Total Price Text Column with USD Dollar Format 06:09 307 Creating the Bar Plot: geom_col() and geom_smooth() 03:11 308 Adding Text to a Bar Plot: geom_text() 05:35 309 Highlighting an Outlier with a Label: geom_label() 05:58 310 Finalizing the Plot with Scales and Themes 03:39 311 Sales by Month and Category 2: Data Manipulation 04:41 312 Facets: Adding subplots "facets" with facet_wrap() 06:55 313 Scales: Applying scales to alter x, y, and color mappings 04:34 314 Themes: Theme Customization with Pre-Built Themes | theme_matplotlib() 03:54 315 Theme Elements: Customization with theme() 05:33 316 Plot Title and X/Y-Axis Labels: labs() 04:44 317 01:21 318 02:10 319 Our Forecasting Workflow Recap 05:00 320 Data Preparation: Melting the Value and Prediction Columns 04:51 321 Data Preparation: Fixing the FutureWarning 03:04 322 Visualization: Setting up the canvas with ggplot() 03:35 323 Visualization: Adding geoms and facets 05:43 324 Visualization: Scales and Theme Minimal 05:12 325 Visualization: Customizing the Theme Elements 04:22 326 Making the plot_forecast() Function Definition 03:23 327 Data Wrangling: Implementing the Melt 04:38 328 Handling the Time-Based Column: Converting to TimeStamp 08:52 329 Visualization: Parameterizing the Plot 08:56 330 Testing the Forecast Plot Function Parameters 07:10 331 Testing the Automation Workflow 01:30 332 Reordering the Subplots using Cat Tools 06:15 333 Adding the plot_forecast() function to our forecasting module 03:37 334 Docstring | Testing Our Imported plot_forecast() Function 03:30 335 Getting Started [File Download] 01:56 336 02:33 337 01:37 338 Generating the Forecasting Workflow 05:26 339 Generating the Forecast Visualization 01:39 340 Overview of the Database I/O Process 01:27 341 Preparing the Forecast for Update 05:44 342 Validating the Column Names 06:11 343 Testing the Prep Forecast for Database Function 01:15 344 Setting Up the Write Forecast to Database Function 05:28 345 Modularizing the Data Preparation Step 01:19 346 Specifying SQL Data Types 06:47 347 06:50 348 00:55 349 04:29 350 Creating our Read Forecast Function 06:00 351 Adding Functions to Database Module 04:19 352 03:07 353 Automation Workflow with Database I/O 02:44 354 Forecasting 1: Total Revenue 04:43 355 Fix #1: Reorder Columns in Prep Data Function 04:30 356 Plotting Total Revenue Forecast 01:25 357 Forecasting 2: Revenue by Category 1 05:33 358 Forecasting 3: Revenue by Category 2 04:37 359 Forecasting 4: Forecast Quarterly Revenue by Customer 05:34 360 Fix #2: Prep Data | Add timestamp conversion 01:33 361 Rerun Our Workflow: Success! 03:20 362 03:15 363 Pro-Tip: Saving Intermediate Data 01:46 364 Utility Function: Convert to Datetime 07:00 365 Rerun the Forecast Workflow 03:46 366 Read Forecast from Database 02:02 367 Recap: Debugging is a Skill 03:42 368 Jupyter Automated Reporting 01:16 369 Getting Started [File Download] 02:54 370 The Updated Database Script: Automatically Run Forecasts 05:49 371 python update_database.py 03:28 372 01:23 373 Setting Up the Working Directory 06:14 374 Importing Data and Parameterizing a Header with Markdown 06:12 375 Parameterizing a Paragraph with Markdown 03:56 376 Performance Summary: Pivot Table, Part 1 05:35 377 Performance Summary: Pivot Table, Part 2 02:14 378 Plotting the Forecast: plot_forecast() 02:04 379 01:16 380 02:12 381 03:19 382 Developing Parameters: Game Plan 02:11 383 03:21 384 04:32 385 Part 1: Intro to Pathlib and OS 03:02 386 Part 2: Detecting Directories Exist & Making New Directories 04:42 387 02:57 388 Parameterizing the Jupyter Template 03:49 389 Finishing the Juyter Template Parameterization 03:47 390 The pm.exectute_notebook() function 04:06 391 Setting Up Key Parameters 06:02 392 Iterating without a For-Loop 06:09 393 Iterating with a For-Loop 06:01 394 01:08 395 Setting Up the Report Parameters 03:33 396 01:29 397 String Transformation: Make File Names from Report Titles 06:32 398 02:46 399 Make the Report Directory 04:49 400 Setting Up the For-Loop Parameters 06:23 401 Setting Up Jupyter Notebook Execution (Inside of For-Loop) 04:02 402 Package Resources: Setting Up the Template Path 06:04 403 Integrating the Run Reports Function into Our Package 04:35 404 Getting Started [File Download] 03:32 405 NB Convert Documentation & Installation Requirements 01:50 406 Step 1: Pandoc Installation 01:06 407 Step 2: Tex Installation (MikTex Windows Shown | Mac Use MacTex) 01:06 408 01:34 409 01:04 410 04:22 411 04:23 412 03:21 413 Exporting an HTML Report Programmatically 06:39 414 HTML Automation: Using a For-Loop to Convert All 4 Reports 05:51 415 PDF Automation: Using a For-Loop to Convert All 4 Reports 06:00 416 02:42 417 Integrating glob: Pulling the Jupyter Notebook File Paths 02:47 418 Integrate "Convert to HTML" Report Automation 04:12 419 Test "Convert to HTML" Report Automation 02:32 420 Integrate "Convert to PDF" Report Automation 01:47 421 Test "Convert to PDF" Report Automation 03:40 422 My Pandas Extensions: Upgrade reporting.py with HTML & PDF Reports, Part 1 06:32 423 My Pandas Extensions: Upgrade reporting.py with HTML & PDF Reports, Part 2 04:01 424 Run Forecast Reports Py: Part 1 - The main() function 05:08 425 Run Forecast Reports Py: Part 2 - Adding Timestamps to Folders 06:02 426 Run Forecast Reports Py: Part 3 - Running Reports 02:57 427 Run Forecast Reports Py: Part 4 - Adjusting Folder Automation 03:14 428 Scheduling Python Scripts Bonus!!! 00:28 429 Making the Batch File (.bat) to run our Python Script 02:26 430 Setting up Automated Tasks with Windows Task Scheduler 02:14 431 Debugging Windows Task Scheduler Tasks with Pause 00:40 432 Fixing the SQL Alchemy Connection 01:53 433 Removing the Automation: Disable & Delete 00:24 434 Python Script Setup | SQL Database Absolute Path 02:31 435 03:23 436 Scheduling the Automator App with Calendar 02:01 437 01:11 438 Forecasting 100 Time Series in Python with Sktime 01:33:11 