Prof. Dr. Larry AdamsAcademic, Author & Researcher
How to Conduct Quantitative Research · Chapter 12 of 33

Part II — DESIGN, SAMPLING, ETHICS AND DATA

Chapter 8. Population, Sampling and Sample-Size Planning

8.1 Define the population precisely

Specify the target population, accessible population, unit of analysis, inclusion and exclusion criteria, setting, period and sampling frame. The unit may be an employee, customer, manager, organisation, branch, transaction or country. Confusing individual-level and organisation-level variables can create ecological or atomistic errors.

8.2 Probability and non-probability sampling

MethodDescriptionAdvantagesRisks/requirements
Simple randomEach frame unit has a known, equal selection chanceClear probability basisNeeds a suitable complete frame
SystematicSelect every kth unit after a random startOperationally simplePeriodicity in the list can bias selection
StratifiedSample within meaningful strataCan improve representation/precisionRequires stratum data and correct weighting
ClusterSample groups then units within themCost-efficient for dispersed populationsDesign effect and cluster-level dependence
MultistageSelection occurs in multiple stagesPractical for complex populationsAnalysis must reflect stages and probabilities
ConvenienceRecruit readily available casesFast and inexpensiveSelection bias; limited population inference
Purposive/criterion-basedRecruit cases meeting defined criteriaUseful for specialised populationsInclusion probabilities often unknown
QuotaRecruit to fill category targetsMay improve visible compositionDoes not eliminate selection bias
Snowball/referralParticipants refer othersCan reach hidden networksNetwork dependence and strong selection effects

8.3 Sample size is not one magic number

Sample size depends on the estimand, design, expected effect or precision, variability, significance level, desired power, number of predictors, clustering, repeated measures, missingness, response rate and intended subgroup analyses. Rules such as “ten cases per variable” are not universal justifications. A sample-size calculation should be tied to the primary outcome and primary analysis.

8.4 Common planning approaches

  • Precision-based estimation: choose a confidence level and acceptable margin of error for a proportion or mean.
  • Power analysis: define a meaningful effect size, alpha level, desired power (often 80% or 90%), test and design.
  • Regression planning: consider expected model fit, number of parameters, predictor distributions, collinearity and validation needs.
  • Cluster designs: inflate for design effect, often approximated as 1 + (average cluster size − 1) × intraclass correlation, when assumptions fit.
  • Attrition/nonresponse: recruit above the analyzable target using a defensible response/retention assumption.
  • Complex models: use simulation or specialist guidance when analytic formulas are inadequate.

A response rate alone does not determine bias: a low response rate can sometimes produce limited bias and a high response rate can still be biased. Compare respondents and nonrespondents on available frame variables, document follow-up procedures, and describe the limits of inference.

8.5 Illustrative finite-population formula

For a simple random sample estimating a proportion under assumptions, a common starting formula is n₀ = Z² p(1−p) / e², where Z is the normal critical value for the chosen confidence level, p is the anticipated proportion (0.50 is conservative when unknown), and e is the desired margin of error. For a known finite population N, one adjustment is n = n₀ / [1 + (n₀−1)/N]. This is not appropriate for every design and does not replace power analysis for hypothesis tests.

8.6 Report sampling transparently

State how the frame was obtained, how cases were selected, who was eligible, how many were approached, how many responded, how exclusions were applied, how weights were used, and what population the analysis can reasonably represent. For convenience samples, describe the sample as the achieved sample and be cautious about population-level generalisation.