- Alienation Online: An Analysis of Populist Facebook Pages In Brandenburg - October 24, 2017
- Shy Tories & Virtue Signalling: How Labour Surged Online - June 16, 2017
- The Limits Of Survey Data: What Questionnaires Can’t Tell Us - March 7, 2017
All research methodologies have their limitations, as many authors have pointed before (see for example Visser, Krosnick and Lavrakas, 2000). From the generalisabilty of data to the nitty-gritty of bias and question wording, every method has its flaws. In fact, the in-fighting between methodological approaches is one of social science’s worst kept secrets: the hostility between quantitative and qualitative data scholars knows almost no bounds (admittedly that’s ‘almost no bounds’ within the polite world of academic debate) and doesn’t look set to be resolved any time soon. That said, there are some methods that are better suited than others to certain types of studies. This article will examine the role of survey data in values studies and argue that it is a blunt tool for this kind of research and that qualitative study methods, particularly via deliberation, are more appropriate. This article will do so via an examination of a piece of 2016 research published by the German ministry for migrants and refugees (the BAMF) which explored both the demographics and the social values held by refugees that have arrived in Germany in the last three years. This article will argue that surveys are unfit to get at the issues that are most important to people.
The Good, The Bad & The Survey
Germany has been Europe’s leading figure as the refugee crisis has deepened worldwide following the collapse of government in Syria and the rise of ISIS. Today, there are 65.3 million displaced people from across the world and 21.3 million refugees (UNHCR, 2016), a number that surpasses even the number of refugees following the Second World War. The exact number of refugees living in Germany (official statistics typically count all migrants seeking protection as refugees, although there is some difference between the various legal statuses) is not entirely clear and the figure is unstable. And while this figure still lags behind the efforts made by countries like Turkey and Jordan, this represents the highest total number of refugees in a European country and matches the pro capita efforts of Sweden. Meanwhile, there are signs that Germany’s residents do not always welcome their new neighbours. For example, in 2016, there were almost 2,000 reported attacks on refugees and refugee homes (Antonio-Amedeo Stiftung, 2017) a similar trend was established by Benček and Strasheim (2016), and the rise of the far-right and anti-migrant party, the AfD in local elections last year points to unresolved resentment towards the newcomers.
In this context then, it makes sense for the BAMF (Bundesamt für Migration und Flüchtlinge), the ministry responsible for refugees and migrants in Germany, to respond to pressure in the media and from politicians to get a better overall picture about the kinds of people the refugees to Germany are. As such, their 2016 paper: “Survey of refugees – flight, arrival in Germany and first steps of integration”[1] details a host of information about newcomers in Germany. The study, which relied on questionnaires given by BAMF officials in a number of languages, and a face-to-face or online format (BAMF, 2016, 11), asked questions of 4,500 refugee respondents. For the most part, the study offers excellent insight into the demographic history of refugees to Germany and will be helpful for policymakers looking to ensure that efforts to help settles refugees are appropriately targeted. For example, the study detailed the relatively high level of education enjoyed by typical refugees to Germany (an average between 10 and 11 years of schooling) (ibid., 37) and some of the specific difficulties this group have in successfully navigating the job market (ibid., 46) and where this group turns to for help for this.
In addition to offering the most up-to-date information about refugees’ home countries and their path into Germany, the study is extremely helpful for politicians and scholars looking to enhance their understanding of logistical and practical issues facing migrants; for example, who has access to integration courses? How many unaccompanied children are in Germany? How many men and how many women fled to Germany? Here, the study is undoubtedly helpful. However, the latter stages of the report purport to examine the social values held by refugees, and it is this part of the study that this article takes issue with.
Respondents were asked to answer questions about their values. The topics included: the right form a government should take, the role of democracy, voting rights of women, the role of religion in the state, men and women’s equality in a marriage, and perceived difference between the values of refugees and Germans among others. While this article doesn’t take issue with the veracity of the findings reported in the article, it does argue that the methods used here are inappropriate for the task at hand. Consider first the questions relating to refugees’ attitudes toward democracy and government. The report found that 96% of refugee respondents agreed with the statement: “One should have a democratic system”[2] compared with 95% of the German control group (ibid., 52). This finding was picked up in the liberal media and heralded as a sign that refugees share central German social values. It is entirely possible that this is true. However, it isn’t difficult to see the ways in which this number might have been accidentally manufactured and should hence be treated with considerable caution.
To do so, one must first consider the circumstances of the interview or questionnaire. As a refugee in Germany, you are confronted with the authority of the BAMF regularly, and you are also likely aware that it is representatives from this organization that ultimately decide on you and your family’s status in Germany and whether you will have the right to stay or not. You are then asked for detailed information about your family history, your education and your participation in integration courses by a representative from this institution. Finally, the interviewer asks what your views are on democracy, women’s rights and religion. Is it too much of a jump to suggest that someone who has had to flee their home and take the extraordinarily dangerous trip to Europe is savvy enough to spot a potential trap here? In these circumstances, there is a tendency to give the answer the interviewer wants to hear. This interviewer bias effect is not a problem exclusive to surveys of refugees’ social values (Davis, 2013), however the power imbalance in these interactions exacerbates the effect. The argument advanced here is not that refugees do not hold a positive view of democracy, but that the trying to find out their views via a survey of this sort is flawed. In fact, the report doesn’t find any significant points of departure between Germans and refugees on any of the major values other than the difficulties presented by women earning more money than their husbands and its potential to cause marital difficulties (ibid., 54).
Asking Questions About Essential Contested Concepts
Beyond the serious power imbalance noted above, another key issue not addressed in the BAMF study is the question of contested concepts. Essential contested concepts, an idea first advanced by W.B. Gallie in 1956, are the big topics like art, beauty, fairness and trust. These big topics, which also include traditionally social scientific and political topics like democracy and equality, are defined as ‘essentially contested’ when the premise of the concept – for example ‘freedom’ – is widely accepted, but how best to realise freedom is disputed (Hart, 1961, 156). The BAMF survey uses these big topics without offering a definition to go with them. What do people mean when they say that ‘men and women should have equal rights[3]’ (BAMF, 2016, 52)? What does equality mean in this context? There are of course many different ways that ‘equality’ between men and women can be interpreted. For example, many conservative Catholic churches argue that men and women are ‘equal’ but different, and have clear family roles for men and women. Likewise, participants could equally mean to say that they believe that men and women should have equal, shared family responsibilities, there is no way to know this from this study. Hence, it is difficult to know how best to interpret these kinds of statistics without considerable context.
As part of the work undertaken by Social Science Works, the team are regularly confronted by these kinds of questions via deliberative workshops with Germans and refugees. In these workshops the team ask questions like “What is democracy?”, “What is freedom?”, “What is equality?”. In doing so, the aim of the workshops is to build a consensus together by formulating and reformulating possible definitions[4], finding common ground between conflicting perspectives and ultimately defining the concepts as a group. What is among the most striking things about these meetings is the initial reluctance of participants to volunteer answers – there is a real lack of certainty about what these kinds of words mean in practice, even among participants who, for example, have studied social and political sciences or work in politics. With the benefit of hindsight, workshop participants have acknowledged these problems in dealing with essentially contested concepts, participants have commented:
“Social Science Works has encouraged me to question my own views and views more critically and to develop a more precise concept for large and often hard to grasp terms such as “democracy”, “freedom” or “equality”. This experience has shown me how complicated it is for me – as someone who I really felt proficient in these questions – to formulate such ideas concretely.” (German participant from the 2016 series of workshops)
“The central starting point for the training was, for me, the common notion of understanding of democracy and freedom. In the intensive discussion, I realized that these terms, which seem self-evident, are anything but.” (German participant from the 2016 series of workshops).
In attempting to talk about these big issues, it become clear just how little consensus there is on these kinds of topics. The participants quoted here work and volunteer in the German social sector and hence confront these kinds of ideas implicitly on a daily basis. The level of uncertainty pointed at here, and from Social Science Works’ wider experience working with volunteers, social workers and refugees suggests that the lack of fluency in essentially contested concepts is a wider problem. In the context of the BAMF research then, it is clear that readers ought to take the chapter detailing the ‘values’ of refugees and Germans with a generous pinch of salt.
Building Consensus & Moving Forward
This article does not seek to suggest that there is no role for survey data in helping to answer questions relating to refugees in Germany. For the most part, the BAMF research offers excellent data on key questions relating to demographics and current social conditions. Hence, the study ought to make an excellent tool of policy makers seeking to better target their support of refugees. However, it is equally clear that to discuss essentially contested concepts like democracy and equality, a survey is a very blunt tool, and here the BAMF study fails to convince. The study seeks to make clear that the social and political values between Germans and refugees are similar and the differences are minimal. The experience in the deliberative workshops hosted by Social Science Works suggests that this is probably true, insofar as both groups find these concepts difficult to define and have to wrestle to make sense of them. This is not something articulated in the BAMF research, however.
Our collective lack of fluency in these topics, even among social and political scholars, has long roots best described another time. However, if we are to improve our abilities to discuss these kinds of topics and build collective ideas for social change and cohesion, there are much better places to begin than a questionnaire. If we are to build a collective understanding of our political structures and our social values, we need to address this lack of fluency by engaging in discussions with diverse groups and together building a coherent idea about social and political ideas.
Notes
[1] Original German: “Befragung von Geflüchteten – Flucht, Ankunft in Deutschland und erste Schritte der Integration“
[2] Original German: „Man sollte ein demokratisches System haben.“
[3] Original German: „Frauen haben die gleichen Rechte wie Männer“
[4] For a more detailed overview of the deliberative method in these workshops, see Blokland, 2016.
Literature
Amadeu Antonio Foundation (2016), Hate Speech Against Refugees, Amadeu Antonio Foundation, Berlin.
Benček, D. and Strasheim, J. (2016), Refugees Welcome? Introducing a New Dataset on Anti-Refugee Violence in Germany, 2014–2015, Working Paper No. 2032, University of Kiel.
Davis, R. E.; et al. (Feb 2010). Interviewer effects in public health surveys, Health Education Research, Oxford University Press, Oxford.
Hart, H.L.A., (1961), The Concept of Law, Oxford University Press, Oxford.
IAB-BAMF-SOEP (2016), Befragung von Geflüchteten – Flucht, Ankunft in Deutschland und erste Schritte der Integration, BAMF-Forschungsbericht 29, Nürnberg: Bundesamt für Migration und Flüchtlinge.
UNHCR (2016), Global Trends: Forced Displacement in 2015, UNHCR, New York.
Visser, P. S., Krosnick, J. A., & Lavrakas, P. (2000), Survey research, in H. T. Reis & C. M. Judd (Eds.), Handbook of research methods in social psychology, New York: Cambridge University Press.