Windrush Lessons Learned Review: Evidence from the Migration Observatory at the University of Oxford

Below is the submission provided by The Migration Observatory to the Home Office “Windrush: lessons learned review call for evidence”

1. The Migration Observatory

The Migration Observatory at the University of Oxford produces impartial, evidence-based analysis of migration and migration policy in the UK. This submission focuses on the role of evidence in preventing undesired policy outcomes such as the problems experienced by some members of the Windrush generation. It does not take a position on the merits of UK migration and enforcement policies themselves.

2. Background and summary

The causes of the Windrush situation were complex and stretch far beyond the use of evidence in policymaking. However, an important part of any strategy to reduce the risks of undesired impacts of policies in the future involves ensuring that the government and those who scrutinise it have the tools to understand what the impacts of policies actually are.

Evaluation of migration and enforcement policies in the UK is currently not conducted systematically, and independent scrutiny is often held back by a lack of data. A more systematic approach to understanding the true impacts of migration-related policies would involve:

• The production of policy-relevant data and research to assess the likely impacts of policies as they are being developed; this includes generating feedback from actual users of the immigration system to make sure that processes they go through work and that documentary requirements are reasonable.
• Independent evaluation for all major policy changes. Evaluation relies on appropriate policy-relevant data, which will generally require a plan to be in place in advance for collecting the necessary information. External scrutiny also requires a richer set of publicly available data on immigration processes, such as data on refusals and reasons for refusal.

The problem of insufficient data and evaluation is not specific to the Home Office and not specific to the UK. Indeed, there are examples within the UK immigration context, such as the settled status process for EU citizens after Brexit or the Migration Advisory Committee for labour migration, in which immigration policy has made extensive use of data and evidence. There are also several areas in which publicly available data have significantly improved. These experiences could usefully be applied across the whole immigration system.

This submission considers these elements in turn, with particular attention to policies that make up the ‘hostile’ or ‘compliant environment’—that is, the set of interior enforcement policies designed to discourage illegal migration and residence by making it difficult to access services and amenities in the UK.

3. Policy-relevant data and research

Programmes of work to generate detailed evidence about the impacts of policy have been used with some success for certain aspects of immigration policymaking, while in others the available evidence is relatively weak.

For example, in-depth independent analysis is frequently conducted in advance of policy development on labour migration by the Migration Advisory Committee (MAC), which has conducted numerous investigations into proposed policy measures from salary thresholds for work visas to the impacts of closing the agricultural workers scheme. This process creates the conditions in which government can draw on newly assembled evidence (e.g. reviews of existing studies, commissioned research, data analysis and stakeholder consultations) from an independent panel to understand the range of impacts a proposed policy might have.

Another example of detailed evidence-gathering has been the internal Home Office programme of work to inform the roll-out of the settled status programme for EU citizens already living in the UK, which has included—crucially—regularly soliciting user feedback. If a similar process for understanding the experience of users were in place across the rest of the immigration system, it is likely that some of the problems facing the Windrush generation would have been identified sooner.

In other areas of policy such as interior enforcement and to some extent family migration the evidence has been relatively limited—whether it comes to rigorous research or detailed user feedback. Government departments must produce impact assessments but they are produced internally, making it unrealistic to expect them to provide a genuinely balanced appraisal of the advantages and disadvantages of different policy choices.

3.1 Lack of evidence on the impacts of interior enforcement policies

The suite of ‘hostile environment’ policies that led to the Windrush situation was developed in the absence of a strong evidence base on the scale or characteristics of irregular migration to the UK, the impacts of interior enforcement policies in reducing or preventing it, and the potential unintended consequences for legal residents.

Scale of irregular migration (what was the problem to be addressed?)

Impact assessments conducted for the 2014 Immigration Act made numerous references to the goal of increasing the number of voluntary departures from the UK and reducing the number of illegal migrants resident here.[1] For example, the opening paragraph of the overarching impact assessment to the 2014 Immigration Bill states that the Government is “determined to reduce illegal immigration” and that proposed measures will “make it more difficult for illegal migrants to live in the UK, encouraging them to depart” (IA HO0097).

However, there is little evidence about the scale or characteristics of the population to be targeted or what kind of reductions could reasonably be expected and impact assessments evaluating the proposed ‘hostile environment’ measures in most cases provide no such estimates.[2]

Drivers of irregular migration (would the proposed policies work?)

Policies to deny services to people without proof of legal residence rely on the assumption that unauthorized migrants will leave the country or not come here in the first place if their ability to undertake regular life transactions or interactions is restricted. Whether this is correct is an empirical question, so it is perhaps surprising that there has not been more work to determine whether the ‘hostile environment’ policies would work in practice.

Statements in impact assessments about the likely effect of policies are vague, e.g. “…the policy is expected to increase the number of voluntary departures from the UK meaning the benefits are likely to be underestimated. The volume of voluntary departures cannot be quantified; however, it is thought that the benefits will exceed the costs” (IA HO0094); and “housing is an important enabler of illegal migration” (IA HO0097).

These evidence gaps were highlighted by the Independent Chief Inspector of Borders and Immigration, whose October 2016 evaluation of measures to restrict access to bank accounts and driving licences argued that: “justification for extending the ‘hostile environment’ measures is based on the conviction that they are ‘right’ in principle, and enjoy broad public support, rather than on any evidence that the measures already introduced are working or needed to be strengthened, since no targets were set for the original measures and little had been done to evaluate them.”

While the drivers of irregular migration are naturally difficult to measure, it is possible to conduct research in this area. For example, Australia’s Irregular Migration and Border Research Programme, which was introduced in 2012 to improve the evidence base for policy, has generated detailed and publicly available research on the decision-making of migrants to Australia, based on surveys and fieldwork in a range of countries of origin and transit (DHA, n.d.). Lessons could be sought from similar polices in other countries—such as the 2012 decision by the government of Spain to deny universal healthcare to most undocumented migrants.

Unintended impacts on legal residents

There was clearly an understanding of the possible unintended consequences of interior enforcement policies on older residents without documentation, although here again the evidence was not extensive. A Policy Equality Statement published in October 2015 examining the potential discriminatory impacts of further measures related to residential tenancies, driving and bank accounts also identified the risk that some legally resident groups may lack documentation, despite having legal status in the UK (Home Office, 2015b).[3]

The most detailed assessment of potential unintended consequences is found in the 2015 evaluation of the “Right to Rent” pilot. The Home Office evaluation did not find evidence that the ‘average’ ethnic minority renter would face difficulties proving their legal status, although it identified specific problems facing a small minority of older citizens without good documentation (Home Office, 2015a).

In an evaluation of the Right to Rent scheme, the Independent Chief Inspector of Borders and Immigration criticised the evaluation, stating that: “The Home Office did complete an evaluation of Phase 1 prior to RtR’s wider roll out, although the Home Office conceded that the decision to roll RtR out in England had, in effect, already been taken, subject to not finding that the scheme was causing “significant discriminatory behaviour”. […] Meanwhile, doubt was cast on the evaluation methodology, and it seemed that such evidence as there was of negative impacts (for example, 8 (out of 33) voluntary and charitable sector groups had reported that they had seen the exploitation of people who did not have the right to rent by landlords) was explained away.”

The report also highlighted data deficiencies as key problem in evaluating risks of unintended consequences, noting that: “In the absence of even any ‘soft’ indicators of impact on, for example, voluntary returns, the Home Office lays itself open to criticism about the breadth of new legislation and the costs versus benefits. It is also harder for it to answer concerns about the potential damage to communities and to individuals.”

4. Independent scrutiny of the impacts of migration policies

Regardless of the evidence-gathering that takes place in advance of implementation, evaluation will generally be needed afterwards to understand what effects a policy has actually had. Meaningful evaluation will often require a plan to collect additional data that are not already routinely available.

4.1 Evaluation of policy

Formal evaluations of major policy changes represent an important tool to identify undesired consequences of migration policies across the board. These are periodically conducted, including by external organisations such as the National Audit Office, the Independent Chief Inspector of Borders and Immigration, or external contractors commissioned by the government. However, there is no process for making sure that major policy changes are evaluated systematically or that evaluations address the full range of consequences a policy has.

As the recent MAC (2018) report noted: “There is insufficient attention given by the Government to monitoring or evaluating the impact of policy changes. We know little about the impact of the immigration skills charge, the health charge or changes to the Tier 2 system. We know almost nothing about the actual earnings of non-EEA migrants with a Tier 2 visa. This is because there is no monitoring or evaluation despite data existing which could be used for this purpose.”

Other examples of major changes that might have benefited from formal evaluation range from the introduction of the £18,600 minimum income requirement for non-EEA spouses to the restrictions on licensing of colleges sponsoring international students to the implementation of transitional labour market controls on Romanian and Bulgarian nationals between 2007 and 2014.

The problem of lack of evaluation is not specific to migration policy. Understandably, government does not always have an incentive to conduct evaluations, which may bring to light policy failures, and it is not necessarily reasonable to expect government departments to ‘mark their own homework’ by conducting evaluations internally (NAO, 2013). Independent evaluations are thus likely to play an important role in any genuine strategy to understand the impacts of policy.

4.2 Understanding users’ experience of the system

A key problem that emerged from the Windrush experience was that there was a gap between policymakers’ perceptions of how the immigration system worked, and users’ actual experience. This suggests that collecting information more systematically about problems users encounter as they navigate application processes would be helpful.

New Zealand, for example, conducts an ongoing ‘customer experience’ survey to help improve immigration processes for those who use them. The MAC report mentioned above (MAC, 2018) also pointed to a relative lack of feedback from the ultimate users of the work visa system.

It is notable that where there has been a dedicated effort to understand users’ experiences and needs—namely in the settled status process for EU citizens after Brexit—this has led to an application process that looks dramatically different from most of the rest of the immigration system. This raises the question how government might learn from apparently successful information-gathering processes such as this to improve users’ experience of other parts of the immigration system.

4.3 Collecting and using the necessary data

Scrutiny of how the immigration system is functioning is held back by a lack of relevant operational data. Home Office data have become significantly more detailed over the past 5 years, and the ‘migration transparency data’ include a growing range of statistics on areas such as processing times and costs. However, available data allow only limited scrutiny of how immigration-related decision-making and enforcement policies are implemented in practice, whether by stakeholders or formal evaluators.

For example, ICIBI inspection reports have frequently highlighted operational data limitations that make evaluation difficult, from a lack of information on the basis of asylum claims and the circumstances under which appeals are allowed, to concerns about the quality of data on the revocation of immigration status to the tracking of non-detained foreign national offenders (ICIBI, 2018b; ICIBI, 2018c; ICIBI, 2017b).

In recent years there have been significant improvements to Home Office migration statistics, with more information provided on detention, the outcomes of cohorts of asylum seekers, and migrants’ journeys through the immigration system. However, there are some areas in which little data is available to facilitate scrutiny of migration and enforcement-related operations. Examples of important data improvements include:

• Data on grants and refusals by detailed category of application. There is currently little information on numbers of people given an immigration status outside of the mainstream work and family routes (such as people on 10 and 20 year long-residence routes or people granted status outside of the immigration rules for different reasons). There is also limited information on refusals, e.g. in categories ranging from spouse visa extensions to refugee applications for ILR, which would make it easier to assess the impacts of current processes and requirements.
• Reasons for refusal of Home Office applications of different types, which would help to identify any systematic issues relating to particular types of applications (refusal reasons data is currently available for citizenship but not for other applications).
• Information on the characteristics of granted and refused applicants (e.g. age, gender, nationality), which would allow more detailed scrutiny of how policy changes affect different groups.
• Metrics for assessing the quality of decision-making, e.g. detailed information on refusals of status that are appealed and the outcomes at appeal (this may require linking Home Office and Ministry of Justice data; similar figures are available for annual cohorts of asylum applicants but not for other applicant categories); or information on enforcement proceedings initiated against people subsequently found to have leave to remain in the UK.
• More detailed data on the full distribution of processing times for different case types (rather than just share of straightforward cases met within service standards); and information on the experience of people with ‘non-straightfoward cases’ to which normal processing time-frames do not apply, including reasons for cases being classified as non-straightforward.

Such data would also help formal evaluations. For specific policy changes, proper evaluation will generally also require a plan, set out in advance, for how relevant data will be collected and made available. The National Audit Office’s (2013) analysis of evaluation practices across government, for example, recommended that: “When new policies are announced, departments should explain how they intend to evaluate reliably those policy impacts, and to use the findings in decision-making. This should include an explanation of the policy design choices they have made to facilitate robust evaluation.” Input from those who will be conducting independent evaluation would also help to ensure that a full analysis can be completed (e.g. powers for the ICIBI or other evaluating bodies to request that certain data are collected).

5. Conclusions and lessons for the future

The Government has a complex task managing migration policy and it is inevitable that some policy failures and operational errors will take place. However, the process of identifying and addressing unintended consequences could be facilitated by a combination of 1) more systematic evidence gathering before policies are implemented, 2) feedback from the actual users of the immigration system; 3) the production of policy-relevant data that enables scrutiny and evaluation; and 4) systematic independent evaluation of all major policy changes, with plans for data collection put in place well in advance.

A more systematic approach to evidence gathering and evaluation should help to ensure that unintended consequences like those experienced by some of the Windrush generation do not go unnoticed in the future.

A looming example is the process for documenting EEA citizens living in the UK after Brexit. The government is dedicating significant effort to developing a scheme that will include all eligible EU citizens and has been particularly careful in gathering feedback from users. However, it is not clear that the necessary data will be available to evaluate the scheme or know whether particular groups are ‘falling through the cracks.’ As the Migration Observatory (2018) has pointed out, this would likely require a combination of detailed applications data and—ideally—collection of survey data to assess whether significant numbers of EU citizens simply do not apply and thus risk losing their legal status.

References

• Department of Home Affairs (DHA). “Irregular migration and border research.” Website, www.homeaffairs.gov.au/about/reports-publications/research-statistics/research/live-in-australia/irregular-migration-border-research
• Gordon, I., K. Scanlon, T. Travers, C. Whitehead. “Economic Impact on the London and UK Economy of an Earned Regularisation of Irregular Migrants to the UK.” London School of Economics, London, 2009
• Home Office, 2015a. Evaluation of the Right to Rent Scheme: Full evaluation report of phase one. Research Report 83, October 2015
• Home Office, 2015b. Home Office Policy equality statement (PES), Immigration Bill 2015 – Access to Services
• ICIBI, 2017a. An inspection of the ‘hostile environment’ measures relating to driving licences and bank accounts January to July 2016
• ICIBI, 2017b. An inspection of the Home Office’s management of non-detained Foreign National Offenders
• ICIBI, 2018a. An inspection of the “Right to Rent” scheme
• ICIBI, 2018b. An inspection of the Home Office’s production and use of Country of Origin Information
• ICIBI, 2018c. An inspection of the review and removal of immigration, refugee and citizenship “status”
• Migration Advisory Committee (MAC) EEA Migration in the UK: Final Report. London: MAC
• Migration Observatory. Measuring Success: Will We Ever Know How Many Eligible EU Citizens Did Not Apply for Settled Status? July 2018
• National Audit Office. 2013. Evaluation in Government. London: NAO
• Woodbridge, J. “Sizing the Unauthorised (Illegal) Migrant Population in the United Kingdom in 2001.” Online Report 29/05, Home Office, London, 2005

 

Thanks to Joe Owen, Will Somerville and Colin Yeo for comments on a previous draft.

 

Notes

[1] For 2014 impact assessments see: Overarching Impact Assessment – Immigration Bill (IA No: HO0097); Impact Assessment; tackling illegal immigration in privately rented accommodation (IA No: HO0094); Regulating migrant access to health services in the UK (IA No: HO 0095); Immigration Act 2014: Access to Banking. Further relevant impact assessments were also produced with the 2016 Immigration Act, which are summarised and can be linked to from the Overarching Impact Assessment – Immigration Bill IA No: HO0214.

[2] The impact assessment on tackling illegal immigration in privately rented accommodation relies on an out-of-date estimate of the illegally resident population for 2007 (Gordon et al, 2009), itself derived from an earlier estimate based on 2001 data (Woodbridge, 2005)—13 years before the Immigration Bill.

[3] The PES notes “The scheme applies to older people regardless of their country of origin, but some non-UK born older people may have additional difficulties in providing original documentation. Some may have had their immigration records destroyed. Some will have originally come into the country under old legislation but may have difficulty in evidencing this. Some may be able to evidence it, but landlords might be unwilling to go to the trouble of verifying unfamiliar documentation. The government allows for the use of a wide range of documents and allows landlords to accept expired passports where there remains a likeness in the photograph to the tenant.”