Statistical Methods Guidance

MoJ Statistical Methodology Training Group

2026-08-11

1 Topics

1.1 Introduction

This document signposts useful statistical methods guidance on key topics for MoJ analysts. The contents have been selected on the basis of providing one or more of the following:

• An accessible introduction to the topic
• ‘Under the bonnet’ theory in accessible as well as technical language
• Key steps to take (including common issues and key assumptions)
• A practical demonstration

Please bear in mind:

• It’s intended to be a ‘live’ document.
• Readers are welcome to make suggestions about content including the addition of other resources and topics. Please do this either by creating a GitHub issue or by completing this MS form.
• It’s deliberately kept brief with further statistical methods resources being signposted in the Online analytical training Trello board.
• Guidance about Analytical Platform and related tools (Git/GitHub, Python, R, SQL etc.) can be found in the Analytical Platform and related tools training guidance.

1.2 Overview sources

Helpful overview sources include:

• Tables providing general guidelines for choosing a statistical test (UCLA) and data analysis examples (UCLA) along with links to R code
• The Data Science Textbook (TIBC) which provides brief overviews of many techniques
• Much liked textbooks covering a wide range of techniques:
  • Discovering statistics using R
  • An Introduction to Statistical Learning with R or Python code

1.3 Exploratory Data Analysis

“Getting familiar with the data”

• Exploratory versus explanatory analysis (story telling with data), a reminder about why data need to be visualised (datasauRus) and recommended chart formats depending on the type of data (From data to viz)
• Helpful textbooks:
  • R for Data Science (EDA section)
  • Discovering statistics using R; see Chapter 4 on exploring data with graphs, Chapter 5 on exploring assumptions, and Chapter 6 on correlation

1.4 Outliers, missing values and data imputation

“Dealing with extreme and missing values”

• Outlier detection and treatment (C4D2 Training)
• Missing data imputation (a chapter from Data Analysis Using Regression and Multilevel/Hierarchical Models (2006) by Andrew Gelman and Jennifer Hill)
• Datacamp courses:
  • Introduction to Anomaly Detection in R
  • Dealing With Missing Data in R
  • Handling Missing Data with Imputations in R
  • Anomaly Detection in Python
  • Dealing with Missing Data in Python

1.5 Statistical inference

“Making inferences about a population based on certain sample characteristics”

• Introductions to statistical inference:
  • Statistical inference (Wikipedia)
  • Discovering statistics using R; see Chapter 1
• Datacamp courses:
  • Foundations of inference in R
  • Foundations of inference in Python

1.6 Hypothesis testing

“Do the sample data sufficiently support a particular population hypothesis?”

• Internal resources:
  • “Old” Government Statistician Group List A Topics; see Hypothesis Testing section on pages 26-30
  • MoJ hypothesis testing workbook with Excel based examples and accompanying data
  • MoJ junior statistician group presentations on parametric hypothesis testing and non-parametric hypothesis testing
• External resources:
  • General guidelines for choosing a statistical test (UCLA)
  • Discovering statistics using R; see Chapter 2 on introduction to testing, Chapter 5 on exploring assumptions, Chapter 6 on correlation, Chapter 9 on comparing two means, and Chapter 15 on non-parametric tests
• Analytical Function courses:
  • Statistics in R and Hypothesis testing in R
  • Hypothesis testing in Python
• Datacamp courses:
  • Hypothesis testing in R
  • Hypothesis testing in Python

1.7 Linear regression

“Modelling the relationship between a continuous dependent variable and explanatory variables by fitting a linear equation to observed data”

• Internal resources:
  • “Old” GSS introduction to multiple regression; see Pages 7-9
  • MoJ quick guide to linear regression
  • MoJ junior statistician group presentation on multiple regression
• External resources:
  • Discovering statistics using R; see Chapter 7
  • An Introduction to Statistical Learning; see Chapter 3 with R or Python code
  • Centre for Multilevel Modelling Online Course Module 3 - Multiple regression; with R code
• Analytical Function courses including linear regression:
  • Statistics in R and Machine learning in R
  • Introduction to machine learning in Python
• Datacamp courses:
  • Introduction to Regression in R, Intermediate Regression in R and Inference for Linear Regression in R
  • Introduction to Regression with statsmodels in Python and Intermediate Regression with statsmodels in Python

1.8 Risk, Odds and Generalised Linear Models

“Risk, odds and the extension of linear modeling ideas to a wider class of response types, such as count data or binary responses”

• Internal resources:
  • “Old” GSS introduction to generalised linear models; see Pages 22-25
  • MoJ quick guide to logistic regression
  • MoJ junior statistician group presentation on generalised linear models
• External resources:
  • Discovering statistics using R; see Chapter 8
  • An Introduction to Statistical Learning; see Chapter 4 with R or Python code
  • Centre for Multilevel Modelling Online Course Module 6 - Regression models for binary responses; with R code
• Analytical Function courses including generalised linear modelling:
  • Statistics in R and Machine learning in R
  • Introduction to machine learning in Python
• Datacamp courses:
  • Introduction to Regression in R, Intermediate Regression in R and Supervised Learning in R: Regression
  • Introduction to Regression with statsmodels in Python, Intermediate Regression with statsmodels in Python, Machine Learning with PySpark, and Linear Classifiers in Python
• For examples of logistic regression modelling in crime/offending contexts along with performance versus machine learning type approaches see the Online analytical training Trello card.

1.9 Survival analysis

“Analysing the expected time to an event of interest”

• For a series of questions that should be considered when analysing time-to-event data see Time-To-Event (TTE) Data Analysis (Columbia University Mailman School of Public Health)
• Introduction to Regression Methods for Public Health Using R; see Chapter 7
• An Introduction to Statistical Learning; see Chapter 11 with R or Python code
• Datacamp courses:
  • Survival Analysis in R
  • Survival Analysis in Python

1.10 Multilevel and cluster robust models

“Models when building in data hierarchies”

• Brief introductions:
  • Introducing multilevel models (NRCM YouTube)
  • Random intercept models (NRCM YouTube)
  • Random coefficient models (NRCM YouTube)
• Centre for multilevel modelling course
• Discovering statistics using R; see Chapter 19
• Clustered standard errors with R (R-bloggers)
• Nonlinear multilevel models: Generalised Additive Mixed Models (GAMMs) using the R package mgcv, these being more interpretable than nonlinear models like Random Forest.
• Datacamp course Hierarchical and Mixed Effects Models in R

1.11 Time series analysis & forecasting

“Analysing a sequence of data points collected over an interval of time”

• Forecasting: Principles and Practice

1.12 Bayesian regression

“The Bayesian approach to linear regression”

• Bayesian Regression Using NumPyro, a practical guide to using Python to infer the distributions of regression coefficients.
• For R programmers see the brms package which provides getting started links.

1.13 Sample size determination

“Choosing an appropriate sample size”

• Bitesize session on sample size calculations in R; drawing on resources on the R sample size calculations Trello card
• MoJ Guide to clustered sample sizes using R at the MoJ Evaluation & Prototyping One-Stop Shop
• Sampling: Design and Analysis (Lohr).
• Further resources signposted in the Sample calculations trello card include:
  • how to reduce the minimum sample size needed in trials
  • sample sizes for difference in difference and regression discontinuity designs

1.14 Survey analysis

“Analysing the results of a survey”

• Analyzing Survey Data in R; essentially someone’s freely accessible notes from the Datacamp course
• Datacamp course: Analyzing Survey Data in R

1.15 Inter-rater reliability analysis

“Measuring the agreement between subjective ratings”

• Introduction to inter-rater reliability analysis (Wikipedia)
• Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial (Hallgren)
• Inter-Rater Reliability for binary, categorical, and ordinal ratings (Statistics by Jim)
• Intraclass Correlation Coefficients for ordinal and continuous ratings (Koo & Li)
• Inter-rater reliability measures in r (Datanovia)

1.16 Evaluation and Prototyping analysis

“In particular, to understand the impact of an intervention”

• MoJ Evaluation & Prototyping One-Stop Shop

1.17 Further sources

• There are a number of Centers of Expertise within Data and Analysis, which offer advice and support.
• Information on specific methods used in the process to produce statistics across the Government Statistical Service (GSS)
• In addition to the internal Statistical Methodology Team, there is also the mostly free GSS Methodology Advice Service
• Lastly, don’t forget that you may be able to get useful help from Microsoft 365 Copilot and ChatGPT on methods. When prompting, be clear and specific, concise, use correct grammar and spelling, and provide an example if necessary.