Data Validation and Statistical Analysis with Programming

Programme Outline

Learning Objectives and Structure

1. Perform the statistical validation component for the role of a junior data scientist, Statistician and/or Researcher.
2. Validate dataset statistically to support data analysis and modelling
3. Explore on data distribution and identify statistical anomalies
4. Compute Statistical Indicators for the dataset
5. Check normality assumption for a data series
6. Perform statistical test and validation on dataset
7. Conduct further statistical analysis

Programme Structure: Participants will go through 4 days of training. Class will reconvene on the 5th day for a presentation as part of the course assessment.

Day 1

• Overview of Data Science Pipeline
• What is Data Validation and Statistical Analysis?
• Importance of Statistics in Data Validation for Machine Learning
• Requirements prior to Data Validation phase
• Understand how data scientist leverage on Data Validation and Statistics
• Basics of Statistics and Hypothesis testing

Day 2

• Statistical Test on Dataset Characteristics
• Probability and Expectations
• Central Tendencies and Dispersion
• Central Limit Theorem

Day 3

• Understanding Dataset characteristics or differences
• Parametric Test
• Introduction to Interval and Ratio Data
• Central Limit Theorem
• z – test
• t- test
• Parametric ANOVA (Interval Data or F-Test)
• Understanding Dataset variables relationship
• Spearman r
• Pearson r

Day 4

• Understanding Dataset characteristics or differences
• Non-Parametric Tests
• Introduction to Ordinal Data
• Non-Parametric ANOVA (Ranked Data – Friedman Test)
• Introduction to Categorical Data Analysis
• Goodness of Fit – Chi Square (Categorical Data)

Day 5

• Project Presentation

Assessment

Participants will be assessed via group based project presentation on the 5th session of the course. There will also be formative assessment and case studies to assess a participant’s understanding and competency.

Subject Credits

Upon completion and satisfying the requirements of passing this course, learners will be awarded 12 subject credits.