Prof. Dr. Larry AdamsAcademic, Author & Researcher
Data Science (1) — Level 7 · 14 slides

Presentation · 14 slides

Data Science (1) — Level 7

Research Design, Analytics, Machine Learning and Responsible Deployment

Prof. Dr. Larry Adams

A Level 7 teaching deck on Data Science (1): Research Design, Analytics, Machine Learning and Responsible Deployment.

Free to view online and to download for teaching, with acknowledgement of the author. To reproduce or adapt any part of a deck, please request permission through the contact page beforehand.

  1. Data Science (1) — Level 7

    Data Science

    Research Design, Analytics, Machine Learning and Responsible Deployment

    01
  2. Level 7 perspective

    • Data science combines domain knowledge, statistics, computing, communication and responsible decisions.
    • A technically accurate model can fail through poor data, leakage, misaligned objectives or weak deployment.
    • The goal is reliable, useful and accountable decisions—not complexity for its own sake.
    02
  3. Learning outcomes

    • Design a reproducible workflow from problem framing to deployment.
    • Evaluate data quality, statistical inference, machine-learning models and uncertainty.
    • Address privacy, fairness, security, governance and explainability.
    • Communicate findings and propose monitoring linked to real decisions.
    03
  4. Problem framing

    • Translate stakeholder needs into a decision question, target population and measurable outcome.
    • Define unit of analysis, prediction horizon, intervention and success criteria.
    • Separate descriptive, diagnostic, predictive and causal questions.
    • Set a baseline and estimate the cost of false positives and false negatives.
    04
  5. Data acquisition and quality

    • Document provenance, collection, sampling, permissions and limitations.
    • Check missingness, duplicates, outliers, inconsistent units, drift and label quality.
    • Consider mechanisms behind missing data rather than filling values blindly.
    • Create data dictionaries, validation rules and lineage records.
    05
  6. Exploratory analysis

    • Inspect distributions, relationships, subgroup differences and temporal patterns.
    • Make scales, uncertainty, denominators and exclusions clear in visualisations.
    • Treat exploratory findings as hypotheses, not definitive conclusions.
    • Avoid cherry-picked intervals and aggregation that hides subgroup effects.
    06
  7. Statistics and experimental design

    • Use sampling theory, confidence intervals, hypothesis tests and effect sizes appropriately.
    • Distinguish observational association from causal inference.
    • Design experiments with randomisation, power, pre-specified outcomes and ethical safeguards.
    • Account for confounding, multiple comparisons, selection bias and interference.
    07
  8. Machine-learning workflow

    • Split data by time, group or entity to reflect deployment and prevent leakage.
    • Build a simple baseline before tuning complex models.
    • Tune only within training data and use suitable cross-validation.
    • Document features, transformations, assumptions, model version and evaluation data.
    08
  9. Model evaluation

    • Choose metrics aligned to task and costs: precision, recall, calibration, MAE, RMSE or ranking measures.
    • Evaluate subgroup performance and robustness under distribution shift.
    • Use interpretable baselines and understand the limits of explanation methods.
    • Communicate uncertainty, failure modes and prohibited use cases.
    09
  10. Data engineering and deployment

    • Design repeatable pipelines with testing, version control, orchestration and access controls.
    • Compare batch and real-time inference by latency, reliability, cost and fallback needs.
    • Monitor data quality, drift, model performance, service health and user feedback.
    • Create rollback, incident response and retraining criteria before launch.
    10
  11. Privacy, fairness and governance

    • Minimise collection and define purpose, retention, access and deletion processes.
    • Assess representation gaps, proxy variables, measurement bias and disparate impact.
    • Maintain human accountability and meaningful appeal routes for high-impact decisions.
    • Use documented risk assessment aligned with applicable laws and standards.
    11
  12. Generative AI

    • Evaluate retrieval-augmented generation, embeddings and language models against a defined task.
    • Test factuality, privacy leakage, prompt injection, bias, cost and latency.
    • Keep provenance and human review for high-stakes outputs.
    • Do not assume synthetic data or generated labels are unbiased.
    12
  13. Capstone workshop

    • Scenario: forecast demand and identify factors associated with missed service targets.
    • Create data inventory, target definition, baseline, validation and metric plan.
    • Describe leakage risks, subgroup checks, privacy controls and monitoring.
    • Present a decision brief with limitations and recommended next steps.
    13
  14. End-text references

    • James, G. et al. (2023). An Introduction to Statistical Learning (2nd ed.). Springer.
    • Hastie, T., Tibshirani, R., & Friedman, J. (2009). The Elements of Statistical Learning (2nd ed.). Springer.
    • Wickham, H., & Grolemund, G. (2017). R for Data Science. O’Reilly.
    • Kuhn, M., & Johnson, K. (2013). Applied Predictive Modeling. Springer.
    • NIST (2023). Artificial Intelligence Risk Management Framework (AI RMF 1.0).
    • Gebru, T. et al. (2021). Datasheets for datasets. Communications of the ACM, 64(12), 86–92.
    14
01 / 14