Big Data Analysis with Python

This Course Includes:

Get hands-on experience of big data analysis with Python with the comprehensive course and lab. The lab provides hands-on learning in analysing data with the use of python, beginning up with the basics to mastering different types of data. The course and lab deal with python data science stack, statistical visualizations, working with big data frameworks, handling missing values and correlation analysis, exploratory data analysis and reproducibility in big data analysis.

Lessons 1: Preface

Lessons 2: The Python Data Science Stack

  • Python Libraries and Packages
  • Using Pandas
  • Data Type Conversion
  • Aggregation and Grouping
  • Exporting Data from Pandas
  • Visualisation with Pandas

Lessons 3: Statistical Visualisations

  • Types of Graphs and When to Use Them
  • Components of a Graph
  • Seaborn
  • Which Tool Should Be Used?
  • Types of Graphs
  • Pandas DataFrames and Grouped Data
  • Changing Plot Design: Modifying Graph Components
  • Exporting Graphs

Lessons 4: Working with Big Data Frameworks

  • Hadoop
  • Spark
  • Writing Parquet Files
  • Handling Unstructured Data

Lessons 5: Diving Deeper with Spark

  • Getting Started with Spark DataFrames
  • Writing Output from Spark DataFrames
  • Exploring Spark DataFrames
  • Data Manipulation with Spark DataFrames
  • Graphs in Spark

Lessons 6: Handling Missing Values and Correlation Analysis

  • Setting up the Jupyter Notebook
  • Missing Values
  • Handling Missing Values in Spark DataFrames
  • Correlation

Lessons 7: Exploratory Data Analysis

  • Defining a Business Problem
  • Translating a Business Problem into Measurable Metrics and Exploratory Data Analysis (EDA)
  • Structured Approach to the Data Science Project Life Cycle

Lessons 8: Reproducibility in Big Data Analysis

  • Reproducibility with Jupyter Notebooks
  • Gathering Data in a Reproducible Way
  • Code Practices and Standards
  • Avoiding Repetition

Lessons 9: Creating a Full Analysis Report

  • Reading Data in Spark from Different Data Sources
  • SQL Operations on a Spark DataFrame
  • Generating Statistical Measurements

Hands-on LAB Activities

The Python Data Science Stack

  • Interacting with the Python Shell
  • Calculating the Square
  • Grouping a DataFrame
  • Applying a Function to a Column
  • Subsetting a DataFrame
  • Slicing and Subsetting
  • Reading Data from a CSV File
  • Viewing the Standard Deviation
  • Calculating the Median Value
  • Calculating the Mean Value

Statistical Visualisations

  • Plotting an Analytical Graph
  • Creating a Graph
  • Creating a Graph for a Mathematical Function
  • Creating a Line Graph Using Seaborn
  • Creating a Line Graph Using pandas
  • Creating a Line Graph Using matplotlib
  • Detecting Outliers
  • Displaying Histograms
  • Using a Box Plot
  • Constructing a Scatterplot
  • Plotting a Line Graph with Styles and Colour
  • Configuring a Title and Labels for Axis Objects
  • Designing a Complete Plot
  • Exporting a Graph to a File on a Disk

Working with Big Data Frameworks

  • Performing DataFrame Operations in Spark
  • Accessing Data with Spark
  • Parsing Text in Spark

Diving Deeper with Spark

  • Creating a DataFrame Using a CSV File
  • Creating a DataFrame from an Existing RDD
  • Specifying the Schema of a DataFrame
  • Removing a Column from a DataFrame
  • Renaming a Column in a DataFrame
  • Adding a Column to a DataFrame
  • Creating a KDE Plot
  • Creating a Linear Model Plot
  • Creating a Bar Chart

Handling Missing Values and Correlation Analysis

  • Filtering Data
  • Counting Missing Values
  • Handling NaN Values
  • Using the Backward and Forward Filling Methods
  • Calculating Correlation Coefficient

Exploratory Data Analysis

  • Generating the Feature Importance of the Target Variable
  • Identifying the Target Variable
  • Plotting a Heatmap
  • Generating a Normal Distribution Plot

Reproducibility in Big Data Analysis

  • Performing Data Reproducibility
  • Preprocessing Missing Values with High Reproducibility
  • Normalisating the Data

Exam FAQs

Installed applications - Python and Jupyter Notebook



Data Analysis - Python


9+ Lessons

Delivery Method:




Scroll to Top