Appendix B: Notes on methodology
Fieldwork. The fieldwork was conducted by Ipsos MORI. Interviews were conducted face-to-face in respondents home, using Computer Assisted Personal Interviewing (CAPI). With this system, for questions with long or complicated sets of possible responses, respondents were able to look at the list of response options and make their choices by naming a letter.
Selection of respondents. Respondents were chosen by quota sampling, conducted from 157 sample points in England, Scotland, and Wales. Northern Ireland was excluded, as in most polling of British public opinion. (Adding Northern Ireland adds significantly and disproportionately to the expense of polling.) The procedure is designed to produce a representative sample of the adult population of Great Britain, inclusive of all residents aged 15 years and up.
Weighting. While the sample was designed to be representative of the adult population of Britain, further steps are taken prior to analysis to ensure representativeness. Namely, responses are weighted to correct for any divergence between the sample population and the national population. Responses are weighted according to a combination of 2001 Census data, 2010 ONS mid-year population estimates, and 2010 National Readership Survey (NRS) data. Note that the weighted sample size is 985, as opposed to the actual sample of 1002 people.
Noteworthy facts about this sample in relation to others. The Migration Observatory survey was embedded in Ipsos MORI’s weekly “Capibus” survey, administered to a representative sample of the adult population of Great Britain. The sample includes non-citizens as well as citizens. Seven per cent of the sample are non-citizens, while 5% are citizens who were born abroad. This is normal practice for polling and survey organizations; figures commonly cited on opposition to immigration will include foreign-born and foreign-national respondents as well, so the sample is comparable to others on this dimension.
Furthermore, although non-citizens and foreign-born citizens were less likely to want reduced immigration on most of our questions, the results do not change substantively if we focus only on UK-born British citizens. Since UK-born British citizens make up most of the overall sample, their attitudes dominate the overall topline results. The UK-born British responses diverge from that of the overall sample by no more than seven percentage points on any single question. With just two exceptions, support for reducing immigration was three to seven percentage points higher among UK-born British citizens than among the population as a whole. The two exceptions: illegal immigrants, and scientists and researchers. Differences here were not statistically significant. On questions about perceptions of immigrants, there were only three statistically significant differences: UK-born British citizens were more likely than other respondents to say they were thinking about permanent immigrants, asylum seekers, and spouses/civil partners.
Third, the inclusion of Scotland makes a detectable difference in our overall results. Unlike the Citizenship Survey, which includes only England and Wales, the Ipsos MORI sample includes respondents in Scotland. Support for reduced immigration is lower in Scotland, which lowers overall opposition to immigration in our results. We preferred the greater coverage from including Scotland to the greater comparability from excluding it from the sample. Results for only England and Wales can still be generated from our data as needed, since the region for each respondent was recorded.
Fourth, the use of face-to-face interviews prevents any problems with the systematic exclusion of households that use only mobile phones and lack a landline.
Finally, the sample reaches slightly younger people than other surveys, but probably not enough to make any substantive difference to the results. For example, the Citizenship Survey includes residents aged 16 and up, while the Transatlantic Trends Immigration survey includes aged 18 and up. Ipsos MORI’s Capibus polls people as young as 15; this was not specially requested or adopted for our project. Young people are the least likely age group to oppose immigration, so a slightly younger sample will be slightly less supportive of reduced immigration.
Margins of Error and Significance Testing. Each result we report, as in any poll or survey, is actually an estimate with a margin of error around it. The margin of error, or “confidence interval,” gives a sense of the range in which we can confident that the true value lies, given what we find in our sample. More precisely, imagine the same survey repeated many times under the same conditions with an unbiased sample of the population, producing many estimates with accompanying confidence intervals. The “true” value for the whole population would be included within 95% of these confidence intervals. With a sample of 1002 people (985 in the weighted sample), our confidence intervals around our estimates are no more than three percentage points. For sub-groups of respondents, confidence intervals become larger. We treat the sample as an assumed close-enough approximation of a random sample for the purpose of constructing confidence intervals and significance tests. For a 1000-person sample such as the one in this survey, approximate confidence intervals are +/- 3 percentage points for estimates between 30% and 70%, and +/- 2 percentage points for estimates in the ranges of 10%-20% or 80%-90%.
In the report, when we report differences between sub-groups of respondents, or across different questions, these differences are always statistically significant.
