Quantitative Data Collection: Questionnaires and Structured Instruments
❦
Designing the Questionnaire
A questionnaire is the commonest instrument in student research, and its quality determines the quality of the study (Dillman et al., 2014; DeVellis & Thorpe, 2021).
Steps
1. List the constructs from the conceptual framework.
2. Find established scales in the literature and record the source, the number of items, and the reliability reported.
3. Adapt items to the context (without changing their meaning), and obtain permission when required.
4. Choose the response format.
5. Organize into sections: introduction and consent, screening questions, main constructs, demographics (placed last in most cases), and a thank-you.
6. Translate if necessary (see below).
7. Review with experts, pilot, and revise.
Response formats
| Format | Example |
|---|---|
| Likert scale (the most common) | 1 = strongly disagree to 5 = strongly agree (Likert, 1932); 5- and 7-point scales are common |
| Semantic differential | Bipolar adjectives (good-bad) |
| Nominal categories | Gender, sector |
| Ordinal ranges | Age groups, income bands |
| Open-ended | Comments |
Writing Good Items
- One idea per item; avoid double-barrelled questions ("The pay and the promotion are fair").
- Use simple, neutral language; avoid leading and loaded words; avoid double negatives.
- Include reverse-coded items sparingly and check them.
- Avoid sensitive questions without need; use ranges for income.
- Keep the questionnaire short enough to be completed in about 10 to 15 minutes.
- Ensure anonymity where possible.
Language and Translation
In Sri Lanka, respondents may prefer Sinhala or Tamil. Use translation and back-translation (Brislin, 1970): one bilingual person translates into the target language, another translates it back, and the two versions are compared and reconciled; then pilot the translated version. Check that business terms have the same meaning.
Data Collection Modes
| Mode | Advantages | Cautions |
|---|---|---|
| Paper, self-administered | Controlled setting; good for employees at work | Time; data entry |
| Online (Google Forms, Microsoft Forms, Qualtrics) | Fast; automatic data entry; lower cost | Self-selection; internet access; lower response rates |
| Interviewer-administered (face to face or telephone) | Higher response; help with literacy | Interviewer bias; cost |
| Drop-and-collect | Convenient for firms | Follow up for return |
Seek permission from organizations, brief the respondents, protect confidentiality, track the response rate (completed ÷ distributed), and keep a log of distributed, returned, and unusable questionnaires.
Common Method Bias and Other Biases
When all variables are measured by the same respondents at the same time, common method variance can inflate relationships (Podsakoff et al., 2003). Remedies: protect anonymity, separate the measurement of predictors and outcomes where possible, vary scale formats, and test with statistical checks such as Harman's single-factor test or a full collinearity assessment. Also consider non-response bias (compare early and late respondents) and social desirability bias.
Preparing Data for Analysis
1. Codebook: a document listing each variable, its label, its codes, and reverse coding.
2. Data entry: enter into SPSS or a spreadsheet; use one row per respondent and one column per variable; use numeric codes.
3. Cleaning: check for entry errors with frequencies; handle missing data (delete cases if very few and random, or use appropriate imputation) and outliers (check with box plots, z-scores above about 3.29, or Mahalanobis distance).
4. Create scale scores: compute the mean of items for each construct after reverse-coding.
5. Test assumptions: normality (skewness and kurtosis within about ±2, or Shapiro-Wilk tests; with large samples, rely on graphs and the central limit theorem), linearity, homoscedasticity, and multicollinearity (variance inflation factor below 5, or below 10 in a lenient rule) (Tabachnick & Fidell, 2019; Hair et al., 2019).
CHAPTER 10