Prof. Dr. Larry AdamsAcademic, Author & Researcher
How to Conduct Quantitative Research · Chapter 17 of 33

Part III — DATA ANALYSIS AND INTERPRETATION

Chapter 13. Reliability, Validity and Measurement Models

13.1 Reliability

Internal consistency concerns how a set of items behaves as a scale; it is not proof of unidimensionality or validity. Cronbach's alpha is widely reported, but its interpretation depends on assumptions, including the scale's dimensional structure. A value around .70 is sometimes used as a rough guideline in early research, not a universal pass mark. Higher values are not always better, as very high alpha can indicate redundant items. Consider omega when appropriate and available.

13.2 Factor analysis

Exploratory factor analysis (EFA) investigates latent structure when it is not sufficiently established. Confirmatory factor analysis (CFA) evaluates a specified measurement model. Factor analysis requires appropriate sample size, adequate item correlations, considered factor-retention decisions and a theoretically defensible interpretation. KMO and Bartlett's test can be useful diagnostics but do not by themselves establish that factor analysis is valid. Report extraction, rotation, factor-retention criteria, loadings, cross-loadings and decisions transparently.

13.3 Validity evidence

  • Content validity: items adequately cover the intended construct.
  • Response-process evidence: respondents interpret and answer items as intended.
  • Internal structure: dimensionality and item relationships support the proposed score.
  • Convergent evidence: measures expected to relate do relate appropriately.
  • Discriminant evidence: distinct constructs are not empirically indistinguishable.
  • Criterion-related evidence: scores relate to relevant external criteria as expected.
  • Consequential and contextual evidence: use of the scores is appropriate for the intended population and decision.

13.4 Measurement invariance

When comparing groups or time points, assess whether the construct is measured comparably. In latent-variable analysis, configural, metric and scalar invariance address increasingly restrictive aspects of measurement equivalence. If invariance fails, group mean comparisons may be difficult to interpret. Simpler applied projects should at least consider whether language, role, culture or mode of administration changes the meaning of items.