Data Science and Analytics: From Raw Data to Actionable Insights
Free to read online. For a copy of this book, or to quote, highlight or reproduce any part of it, please request permission through the contact page beforehand.
Editorial and publication note
This is a proposed educational series. IEEE-style technical writing and citations do not imply IEEE endorsement, publication, accreditation, certification, or affiliation. Any standard, designation, author credential, publication detail, or institutional relationship must be verified before publication. These Word files are expanded educational drafts and chapter-development manuscripts for editorial review; they are not represented as final peer-reviewed or publication-ready books.
The proposed series title is descriptive only. Do not use IEEE logos or imply IEEE affiliation without authorisation. Any references to standards should be checked in the official IEEE Standards Association catalogue and used only within their stated scope.
About this book
Covers the analytics lifecycle, collection, cleaning, statistics, visualisation, SQL, Python, dashboards, predictive analytics, and decision support.
Who should use this book
This volume is designed for undergraduate, professional, and lifelong learning. Instructors should adapt pacing, depth, tools, and examples to learner needs and local curriculum requirements.
Learning outcomes
- Explain the main concepts and vocabulary introduced throughout the volume.
- Apply a structured method to a relevant practical problem.
- Interpret evidence, identify limitations, and communicate results clearly.
- Consider privacy, fairness, safety, security, and human oversight where relevant.
How to use this book
Read the chapter explanation, complete the activity, answer the review questions, and keep a learning journal recording assumptions, decisions, mistakes, and improvements. For technical work, save code, data dictionaries, test results, and software-version details.
Table of contents
- Chapter 1: The Data Science Lifecycle
- Chapter 2: Defining Problems and Analytical Objectives
- Chapter 3: Data Sources and Collection Methods
- Chapter 4: Data Types, Structures, and Storage
- Chapter 5: Data Cleaning and Quality Assessment
- Chapter 6: Exploratory Data Analysis
- Chapter 7: Descriptive Statistics and Probability
- Chapter 8: Inferential Statistics and Statistical Uncertainty
- Chapter 9: Data Visualisation and Storytelling
- Chapter 10: Spreadsheet Analytics and SQL
- Chapter 11: Python for Data Science
- Chapter 12: Business Intelligence and KPI Dashboards
- Chapter 13: Predictive and Prescriptive Analytics
- Chapter 14: Big Data and Real-Time Analytics
- Chapter 15: Turning Findings into Decisions
- Chapter 16: Integrated Analytics Capstone
Chapter 1: The Data Science Lifecycle
Learning outcomes
By the end of this chapter, the learner should be able to:
- Explain the central concepts introduced in the data science lifecycle.
- Apply the ideas to a clearly defined example or practical task.
- Identify at least one limitation, assumption, or responsible-use consideration.
Core explanation
A typical lifecycle moves from problem framing through collection, preparation, exploration, modelling, evaluation, communication, and monitoring. Projects may loop back when evidence exposes a weak question or data gap.
Worked example and application
Worked example: use a small, fictional dataset with a documented meaning for each column. Check completeness and units, calculate a simple summary, visualise the result, and write one conclusion that is directly supported by the observations. State what the dataset cannot establish.
Practical activity
Map a practical business or public-service question to lifecycle stages and deliverables.
Chapter review
- What is the main idea in 'The Data Science Lifecycle'?
- Which assumptions or data requirements matter most in this topic?
- What could cause the approach to fail or produce a misleading result?
- How would you explain the result or limitation to a non-specialist?
Key takeaways
- Start with a clear question, objective, or system requirement.
- Document inputs, assumptions, methods, and limitations.
- Test the approach with relevant examples and review the consequences of errors.
Chapter 2: Defining Problems and Analytical Objectives
Learning outcomes
By the end of this chapter, the learner should be able to:
- Explain the central concepts introduced in defining problems and analytical objectives.
- Apply the ideas to a clearly defined example or practical task.
- Identify at least one limitation, assumption, or responsible-use consideration.
Core explanation
Translate stakeholder needs into answerable questions and measurable outcomes. Define the unit of analysis, decision owner, constraints, and success criteria before choosing tools.
Worked example and application
Worked example: define a real-world task, list the inputs and constraints, sketch the steps or system components, and test the proposed solution with at least one ordinary case and one edge case. Record the outcome and what you would improve.
Practical activity
Draft a problem statement, three analytical questions, and measurable success criteria.
Chapter review
- What is the main idea in 'Defining Problems and Analytical Objectives'?
- Which assumptions or data requirements matter most in this topic?
- What could cause the approach to fail or produce a misleading result?
- How would you explain the result or limitation to a non-specialist?
Key takeaways
- Start with a clear question, objective, or system requirement.
- Document inputs, assumptions, methods, and limitations.
- Test the approach with relevant examples and review the consequences of errors.
Chapter 3: Data Sources and Collection Methods
Learning outcomes
By the end of this chapter, the learner should be able to:
- Explain the central concepts introduced in data sources and collection methods.
- Apply the ideas to a clearly defined example or practical task.
- Identify at least one limitation, assumption, or responsible-use consideration.
Core explanation
Sources include surveys, sensors, transaction systems, experiments, administrative records, and public datasets. Assess provenance, consent, representativeness, measurement error, and permitted use.
Worked example and application
Worked example: use a small, fictional dataset with a documented meaning for each column. Check completeness and units, calculate a simple summary, visualise the result, and write one conclusion that is directly supported by the observations. State what the dataset cannot establish.
Practical activity
Compare two candidate data sources for relevance, quality, cost, and privacy.
Chapter review
- What is the main idea in 'Data Sources and Collection Methods'?
- Which assumptions or data requirements matter most in this topic?
- What could cause the approach to fail or produce a misleading result?
- How would you explain the result or limitation to a non-specialist?
Key takeaways
- Start with a clear question, objective, or system requirement.
- Document inputs, assumptions, methods, and limitations.
- Test the approach with relevant examples and review the consequences of errors.
Chapter 4: Data Types, Structures, and Storage
Learning outcomes
By the end of this chapter, the learner should be able to:
- Explain the central concepts introduced in data types, structures, and storage.
- Apply the ideas to a clearly defined example or practical task.
- Identify at least one limitation, assumption, or responsible-use consideration.
Core explanation
Data may be structured, semi-structured, or unstructured; numerical and categorical variables need different treatment. Data dictionaries and consistent units improve interoperability.
Worked example and application
Worked example: use a small, fictional dataset with a documented meaning for each column. Check completeness and units, calculate a simple summary, visualise the result, and write one conclusion that is directly supported by the observations. State what the dataset cannot establish.
Practical activity
Create a data dictionary and identify type, unit, permitted values, and missing-value rules.
Chapter review
- What is the main idea in 'Data Types, Structures, and Storage'?
- Which assumptions or data requirements matter most in this topic?
- What could cause the approach to fail or produce a misleading result?
- How would you explain the result or limitation to a non-specialist?
Key takeaways
- Start with a clear question, objective, or system requirement.
- Document inputs, assumptions, methods, and limitations.
- Test the approach with relevant examples and review the consequences of errors.
Chapter 5: Data Cleaning and Quality Assessment
Learning outcomes
By the end of this chapter, the learner should be able to:
- Explain the central concepts introduced in data cleaning and quality assessment.
- Apply the ideas to a clearly defined example or practical task.
- Identify at least one limitation, assumption, or responsible-use consideration.
Core explanation
Cleaning addresses duplicates, inconsistent formats, invalid values, missingness, and outliers. Changes should be documented; do not remove inconvenient observations without a defensible reason.
Worked example and application
Worked example: use a small, fictional dataset with a documented meaning for each column. Check completeness and units, calculate a simple summary, visualise the result, and write one conclusion that is directly supported by the observations. State what the dataset cannot establish.
Practical activity
Create a quality report with counts of missing, duplicated, and invalid records.
Chapter review
- What is the main idea in 'Data Cleaning and Quality Assessment'?
- Which assumptions or data requirements matter most in this topic?
- What could cause the approach to fail or produce a misleading result?
- How would you explain the result or limitation to a non-specialist?
Key takeaways
- Start with a clear question, objective, or system requirement.
- Document inputs, assumptions, methods, and limitations.
- Test the approach with relevant examples and review the consequences of errors.
Chapter 6: Exploratory Data Analysis
Learning outcomes
By the end of this chapter, the learner should be able to:
- Explain the central concepts introduced in exploratory data analysis.
- Apply the ideas to a clearly defined example or practical task.
- Identify at least one limitation, assumption, or responsible-use consideration.
Core explanation
EDA uses summaries and visualisations to understand distributions, relationships, anomalies, and potential data problems. Exploratory findings generate hypotheses but do not automatically confirm them.
Worked example and application
Worked example: use a small, fictional dataset with a documented meaning for each column. Check completeness and units, calculate a simple summary, visualise the result, and write one conclusion that is directly supported by the observations. State what the dataset cannot establish.
Practical activity
Produce an EDA notebook or worksheet with a question-led sequence of charts.
Chapter review
- What is the main idea in 'Exploratory Data Analysis'?
- Which assumptions or data requirements matter most in this topic?
- What could cause the approach to fail or produce a misleading result?
- How would you explain the result or limitation to a non-specialist?
Key takeaways
- Start with a clear question, objective, or system requirement.
- Document inputs, assumptions, methods, and limitations.
- Test the approach with relevant examples and review the consequences of errors.
Chapter 7: Descriptive Statistics and Probability
Learning outcomes
By the end of this chapter, the learner should be able to:
- Explain the central concepts introduced in descriptive statistics and probability.
- Apply the ideas to a clearly defined example or practical task.
- Identify at least one limitation, assumption, or responsible-use consideration.
Core explanation
Means, medians, quantiles, variance, and standard deviation describe distributions. Probability supports reasoning under uncertainty. Choose summaries that fit skewness, outliers, and measurement scale.
Worked example and application
Worked example: use a small, fictional dataset with a documented meaning for each column. Check completeness and units, calculate a simple summary, visualise the result, and write one conclusion that is directly supported by the observations. State what the dataset cannot establish.
Practical activity
Summarise a dataset and explain why each chosen statistic is appropriate.
Chapter review
- What is the main idea in 'Descriptive Statistics and Probability'?
- Which assumptions or data requirements matter most in this topic?
- What could cause the approach to fail or produce a misleading result?
- How would you explain the result or limitation to a non-specialist?
Key takeaways
- Start with a clear question, objective, or system requirement.
- Document inputs, assumptions, methods, and limitations.
- Test the approach with relevant examples and review the consequences of errors.
Chapter 8: Inferential Statistics and Statistical Uncertainty
Learning outcomes
By the end of this chapter, the learner should be able to:
- Explain the central concepts introduced in inferential statistics and statistical uncertainty.
- Apply the ideas to a clearly defined example or practical task.
- Identify at least one limitation, assumption, or responsible-use consideration.
Core explanation
Samples rarely contain all information about a population. Confidence intervals, hypothesis tests, and effect sizes help quantify uncertainty when assumptions are suitable. Statistical significance is not the same as practical importance.
Worked example and application
Worked example: use a small, fictional dataset with a documented meaning for each column. Check completeness and units, calculate a simple summary, visualise the result, and write one conclusion that is directly supported by the observations. State what the dataset cannot establish.
Practical activity
Interpret a fictional confidence interval and distinguish effect size from p-value.
Chapter review
- What is the main idea in 'Inferential Statistics and Statistical Uncertainty'?
- Which assumptions or data requirements matter most in this topic?
- What could cause the approach to fail or produce a misleading result?
- How would you explain the result or limitation to a non-specialist?
Key takeaways
- Start with a clear question, objective, or system requirement.
- Document inputs, assumptions, methods, and limitations.
- Test the approach with relevant examples and review the consequences of errors.
Chapter 9: Data Visualisation and Storytelling
Learning outcomes
By the end of this chapter, the learner should be able to:
- Explain the central concepts introduced in data visualisation and storytelling.
- Apply the ideas to a clearly defined example or practical task.
- Identify at least one limitation, assumption, or responsible-use consideration.
Core explanation
Effective charts make comparisons, distributions, relationships, and changes visible. Select visual encodings that match the question and avoid distorted scales or unsupported causal language.
Worked example and application
Worked example: use a small, fictional dataset with a documented meaning for each column. Check completeness and units, calculate a simple summary, visualise the result, and write one conclusion that is directly supported by the observations. State what the dataset cannot establish.
Practical activity
Create a one-page visual report with a headline, chart, caveat, and recommendation.
Chapter review
- What is the main idea in 'Data Visualisation and Storytelling'?
- Which assumptions or data requirements matter most in this topic?
- What could cause the approach to fail or produce a misleading result?
- How would you explain the result or limitation to a non-specialist?
Key takeaways
- Start with a clear question, objective, or system requirement.
- Document inputs, assumptions, methods, and limitations.
- Test the approach with relevant examples and review the consequences of errors.
Chapter 10: Spreadsheet Analytics and SQL
Learning outcomes
By the end of this chapter, the learner should be able to:
- Explain the central concepts introduced in spreadsheet analytics and sql.
- Apply the ideas to a clearly defined example or practical task.
- Identify at least one limitation, assumption, or responsible-use consideration.
Core explanation
Spreadsheets support rapid calculations and review; SQL retrieves and aggregates relational data. Reproducible queries need clear joins, filters, grouping, and checks for duplicated records.
Worked example and application
Worked example: define a real-world task, list the inputs and constraints, sketch the steps or system components, and test the proposed solution with at least one ordinary case and one edge case. Record the outcome and what you would improve.
Practical activity
Write or sketch SQL to group sales by month and validate totals against a source table.
Chapter review
- What is the main idea in 'Spreadsheet Analytics and SQL'?
- Which assumptions or data requirements matter most in this topic?
- What could cause the approach to fail or produce a misleading result?
- How would you explain the result or limitation to a non-specialist?
Key takeaways
- Start with a clear question, objective, or system requirement.
- Document inputs, assumptions, methods, and limitations.
- Test the approach with relevant examples and review the consequences of errors.
Chapter 11: Python for Data Science
Learning outcomes
By the end of this chapter, the learner should be able to:
- Explain the central concepts introduced in python for data science.
- Apply the ideas to a clearly defined example or practical task.
- Identify at least one limitation, assumption, or responsible-use consideration.
Core explanation
Python libraries support tabular manipulation, numerical analysis, visualisation, and reproducible workflows. Scripts and notebooks should record assumptions, versions, and transformations.
Worked example and application
Worked example: use a small, fictional dataset with a documented meaning for each column. Check completeness and units, calculate a simple summary, visualise the result, and write one conclusion that is directly supported by the observations. State what the dataset cannot establish.
Practical activity
Develop a documented workflow that loads, cleans, summarises, and charts a dataset.
Chapter review
- What is the main idea in 'Python for Data Science'?
- Which assumptions or data requirements matter most in this topic?
- What could cause the approach to fail or produce a misleading result?
- How would you explain the result or limitation to a non-specialist?
Key takeaways
- Start with a clear question, objective, or system requirement.
- Document inputs, assumptions, methods, and limitations.
- Test the approach with relevant examples and review the consequences of errors.
Chapter 12: Business Intelligence and KPI Dashboards
Learning outcomes
By the end of this chapter, the learner should be able to:
- Explain the central concepts introduced in business intelligence and kpi dashboards.
- Apply the ideas to a clearly defined example or practical task.
- Identify at least one limitation, assumption, or responsible-use consideration.
Core explanation
Dashboards track indicators aligned with decisions. Define each KPI precisely, including numerator, denominator, time window, owner, and data source. Avoid dashboards overloaded with low-value metrics.
Worked example and application
Worked example: define a real-world task, list the inputs and constraints, sketch the steps or system components, and test the proposed solution with at least one ordinary case and one edge case. Record the outcome and what you would improve.
Practical activity
Design a dashboard specification for a fictional service and define five KPIs.
Chapter review
- What is the main idea in 'Business Intelligence and KPI Dashboards'?
- Which assumptions or data requirements matter most in this topic?
- What could cause the approach to fail or produce a misleading result?
- How would you explain the result or limitation to a non-specialist?
Key takeaways
- Start with a clear question, objective, or system requirement.
- Document inputs, assumptions, methods, and limitations.
- Test the approach with relevant examples and review the consequences of errors.
Chapter 13: Predictive and Prescriptive Analytics
Learning outcomes
By the end of this chapter, the learner should be able to:
- Explain the central concepts introduced in predictive and prescriptive analytics.
- Apply the ideas to a clearly defined example or practical task.
- Identify at least one limitation, assumption, or responsible-use consideration.
Core explanation
Predictive analytics estimates likely outcomes; prescriptive analytics compares actions under objectives and constraints. Forecasts require evaluation, while recommendations require explicit costs, values, and operational feasibility.
Worked example and application
Worked example: define a real-world task, list the inputs and constraints, sketch the steps or system components, and test the proposed solution with at least one ordinary case and one edge case. Record the outcome and what you would improve.
Practical activity
Compare a forecast with a decision recommendation and list assumptions needed for each.
Chapter review
- What is the main idea in 'Predictive and Prescriptive Analytics'?
- Which assumptions or data requirements matter most in this topic?
- What could cause the approach to fail or produce a misleading result?
- How would you explain the result or limitation to a non-specialist?
Key takeaways
- Start with a clear question, objective, or system requirement.
- Document inputs, assumptions, methods, and limitations.
- Test the approach with relevant examples and review the consequences of errors.
Chapter 14: Big Data and Real-Time Analytics
Learning outcomes
By the end of this chapter, the learner should be able to:
- Explain the central concepts introduced in big data and real-time analytics.
- Apply the ideas to a clearly defined example or practical task.
- Identify at least one limitation, assumption, or responsible-use consideration.
Core explanation
Large-scale analytics introduces distributed storage, streaming, latency, reliability, and cost trade-offs. Scale should solve a real need rather than being pursued for its own sake.
Worked example and application
Worked example: use a small, fictional dataset with a documented meaning for each column. Check completeness and units, calculate a simple summary, visualise the result, and write one conclusion that is directly supported by the observations. State what the dataset cannot establish.
Practical activity
Design a batch-versus-streaming choice for a fictional monitoring problem.
Chapter review
- What is the main idea in 'Big Data and Real-Time Analytics'?
- Which assumptions or data requirements matter most in this topic?
- What could cause the approach to fail or produce a misleading result?
- How would you explain the result or limitation to a non-specialist?
Key takeaways
- Start with a clear question, objective, or system requirement.
- Document inputs, assumptions, methods, and limitations.
- Test the approach with relevant examples and review the consequences of errors.
Chapter 15: Turning Findings into Decisions
Learning outcomes
By the end of this chapter, the learner should be able to:
- Explain the central concepts introduced in turning findings into decisions.
- Apply the ideas to a clearly defined example or practical task.
- Identify at least one limitation, assumption, or responsible-use consideration.
Core explanation
An insight becomes actionable when it has an owner, an option, a rationale, and a way to measure outcomes. Explain uncertainty and separate evidence from recommendation.
Worked example and application
Worked example: define a real-world task, list the inputs and constraints, sketch the steps or system components, and test the proposed solution with at least one ordinary case and one edge case. Record the outcome and what you would improve.
Practical activity
Write an executive brief with findings, limitations, action, owner, and follow-up metric.
Chapter review
- What is the main idea in 'Turning Findings into Decisions'?
- Which assumptions or data requirements matter most in this topic?
- What could cause the approach to fail or produce a misleading result?
- How would you explain the result or limitation to a non-specialist?
Key takeaways
- Start with a clear question, objective, or system requirement.
- Document inputs, assumptions, methods, and limitations.
- Test the approach with relevant examples and review the consequences of errors.
Chapter 16: Integrated Analytics Capstone
Learning outcomes
By the end of this chapter, the learner should be able to:
- Explain the central concepts introduced in integrated analytics capstone.
- Apply the ideas to a clearly defined example or practical task.
- Identify at least one limitation, assumption, or responsible-use consideration.
Core explanation
Combine a question, documented data, quality checks, analysis, visual communication, and recommendations. Include enough detail for another analyst to understand the process.
Worked example and application
Worked example: define a real-world task, list the inputs and constraints, sketch the steps or system components, and test the proposed solution with at least one ordinary case and one edge case. Record the outcome and what you would improve.
Practical activity
Transform a raw dataset into a documented analysis, visual dashboard, and actionable recommendations.
Chapter review
- What is the main idea in 'Integrated Analytics Capstone'?
- Which assumptions or data requirements matter most in this topic?
- What could cause the approach to fail or produce a misleading result?
- How would you explain the result or limitation to a non-specialist?
Key takeaways
- Start with a clear question, objective, or system requirement.
- Document inputs, assumptions, methods, and limitations.
- Test the approach with relevant examples and review the consequences of errors.
Capstone project
Transform a raw dataset into a documented analysis, visual dashboard, and actionable recommendations. Include data provenance, a data dictionary, cleaning log, reproducible analysis, KPI definitions, uncertainty, and an executive brief.
Suggested project deliverables
- A short problem statement and intended audience.
- A description of data, inputs, assumptions, tools, and method.
- Evidence of testing, analysis, or evaluation appropriate to the project.
- A clear explanation of results, limitations, and foreseeable risks.
- A final report or presentation and a reflection on possible improvements.
Assessment guidance
Evaluate problem framing, data quality, statistical reasoning, chart integrity, reproducibility, practical recommendations, and transparent limitations.
Glossary
EDA: Exploratory data analysis: initial examination of data using summaries and visualisations.
KPI: A key performance indicator linked to a defined objective.
Data provenance: Information about where data came from and how it was produced or changed.
Confidence interval: A statistical interval procedure used to express uncertainty about an estimated quantity.
Prescriptive analytics: Analysis that compares or recommends actions under objectives and constraints.
Responsible practice checklist
- Use data and materials only when their source and permitted use are understood.
- Avoid exposing personal, confidential, or credential information.
- Document assumptions, missing information, and important limitations.
- Test ordinary cases, boundary cases, and foreseeable failure conditions.
- Use human review where errors could materially affect people, safety, rights, or resources.
- Do not claim that a system is fair, safe, secure, compliant, or reliable without appropriate evidence.
References and further reading
The source material supplied for this project does not provide a complete, volume-specific bibliography. Before publication, add verified primary sources, textbooks, official documentation, and standards applicable to this volume. Confirm exact titles, editions, publication dates, URLs, licences, and access dates. Do not cite a standard as evidence of conformity unless the required assessment has actually been performed.
For IEEE-style technical manuscripts, use numbered citations in order of first appearance when required by the target venue, and verify the venue's official author instructions.