The eu’s Legitimacy in the Eye of the Beholders

5.3 – Any Limitations? On Reliability and Validity

The reliability and validity of the research are also important to discuss. Reliability focuses on the accuracy of measurements and whether the research is repeatable. The qualitative nature of the data means the repeatability is low, yet the method triangulation helps overcome the problems associated with the accuracy of the findings. Also the use of multiple data sources per case – two newspapers per country – increases the accuracy. Thus reliability as repeatability might be low, but the accuracy is high, because of the use of method and data triangulation (Yin 2009: 116). This means that the overall conclusions in similar research will be in line with our findings, thus the reliability of the research is secured.

Validity has two forms: internal and external. Internal validity is concerned with the fact whether one is actually researching what one wants to be researching. The use of multiple methods – method triangulation – and different perspectives on the same data set – theory triangulation – are ways to overcome this limitation. The internal validity is thus safeguarded by these forms of triangulation.

The external validity or the so called generalisability is concerned with whether the results can be generalised to a greater group of cases? Investigating multiple cases, in this case three, is positively correlated to the external validity. On the other hand, the uniqueness of the countries might call the generalisability into question. Then again the inclusion of multi-level governance in the framework might prove insightful for research into the legitimacy of nation-states or other governmental actors inhabited in networks. Still, the uniqueness of the problem, the EU’s legitimacy deficit, makes the external validity low, and one should be very careful when making any universal statements based on this research.

Finally, awareness of the different pitfalls concerning reliability and validity is essential in overcoming them. Common sense is a good guide, and also Bruce Berg (1989: 124) advice to be honest seems an ethical and practical advice in this regard.

5.4 – The Operationalisation

The theoretical framework gives us several important concepts to operationalise for the analysis of the perception of legitimacy in public discourse. Subsequently, the codification process is outlined. Then the analytical framework is discussed, and finally three hypotheses are derived from the theoretical framework.

The codification process went systematically as follows: Each retrieved article was given a unique tag: the publishing date, the newspaper, and the country. Then it was closely read and all the observations on the EU’s legitimacy or lack thereof were summarised for further analysed later on. After reading the complete article, it was assessed on whether it is supportive of the European project as a whole and whether it takes the reality of multi-level governance into account.⁷⁷

The individual observations were analysed more thoroughly. Each short description of the statement was coded with its publishing date, newspaper and country in order to give it a unique tag. Then it was determined whether an observation is positive or negative about the EU’s current legitimacy. Thus if the observation advocates change then the judgment is negative. Thereafter the perspective on legitimacy was analysed based on the three facets of legitimacy. And finally, it was established whether an observation takes multi-level governance into account.⁷⁸

5.4.1 – The Analytical Framework

The operationalisation of the analytical framework is described in the following paragraphs. We will walk through the different elements starting with the facets. The first facet is the dimension of an observation, thus whether it uses a universalist or nationalist concept of legitimacy. The republicanisms are nowadays enshrined in mongrel concepts, so it is important to have a clear picture of them. Once again therefore the defining features of the republicanisms are:

Universalist Republicanism	Nationalist Republicanism
Humanist	Nationalist
De-Territorial	Territorial
People	Raison d’État
Democratic Institutions	Commercial Interests
Rule of Law	Militaristic

Figure 5.2: The Features of the Two Republicanisms
The dimensions are further operationalised in combination with the second facet. The second facet is the distinction between the components of legitimacy: input, throughput and output. The components are useful in assessing the influence of the idea of constitutional representative democracy, but also for further exposing the mixed heritage in main stream democratic theory. The operationalisation of the components divided within the dimensions looks as follows:

	Input	Throughput	Output
Nationalist	The Nation Sovereignty	Efficiency Expertise	Trade Military Power/Influence
Universalist	Demos Popular Mandate Popular Support	Transparency Legality	Peace Human Rights

Figure 5.3: The Operationalisation of the Dimensions and the Components
Now let us take a closer look at the boxes in ‘Figure 5.3’. In the nationalist input category falls Scharpf’s thick identity or national identity, this could display itself as loyalty to the nation-state. Also the idea of sovereignty falls into this category, both when used to defend against further loss of state powers or in defence of self-government of a nation. This category can also be applied to the European level, for example if people argue for the emergence of a European nation. The universalist input is concerned with the ‘thin’ identity or demos linked to popular mandate and support. This category includes support for the European institutions, but also the democratic ‘level’ of these institutions.

The throughput distinction is based on March and Olsen’s distinction between the logic of consequence and appropriateness (1989). Nowadays, democratic theory often seems to incorporate both logics and places them on equal footing.⁷⁹ This distinction serves us well. Nationalist throughput legitimacy relates to the demands of the logic of consequence. Important are economic demands, like effectiveness and in relation to this the best people will take the best decisions: expertise. The logic of appropriateness makes demands on a universalist normative basis. Transparency is demanded by the idea of democracy and legality by the rule of law.

Nationalist output focuses on the state’s interests. Their primary interests lie in the realm of real politics, like trade interests, strategic military interests and having power. The universalist output ultimately focuses on the humanist goal of peace for all people, but on a more realistic scale to safeguard and where possible spread peace and human rights.

The final facet of legitimacy to be operationalised is the model distinction. The models focus on the question whether the EU should be legitimised through the institutions of the Member States or its own institutions directly elected by the entire European population or at least by the populations. This distinction is operationalised as follows:

Direct Legitimacy	Indirect Legitimacy
European Parliament	National Heads of State
Referenda	Treaties
European Senate	National Parliaments
Elected President	Appointed Commissioners

Figure 5.4: The Operationalisation of the Models
‘Figure 5.4’ shows us that the current EU system of governance is mixed, because elements of both models are to more or lesser extent in place. This more or lesser extent is important for the analysis, because, for example, the European parliament already exists, but a complaint might be that it has too little power. Others might argue that the role of the national parliaments is too insignificant.

This facet is meant to analyse the role of these models on the perception of EU’s legitimacy, but also to analyse whether people can get to grips with the idea of multi-level governance. This brings us to the operationalisation of whether multi-level governance is taken into account. This analysis was added, because the model analysis is imperfect. The level of analysis – article or observation – is irrelevant for the operationalisation. Three primary features are established based on the definition of multi-level governance in the theoretical framework. Only if an article or observation takes all three features into account is it coded ‘positively’.

1. 
   Actors from multiple tiers of government are active in nested negation,
   
2. 
   They are interdependent, because of the Verflechtung of issues
   
3. 
   Finally, both governmental and non-governmental actors play a role
   

This framework for understanding legitimacy is at the same time parsimonious as comprehensive. Admittedly, all categorisations are artificial and it is certainly possible to analyse more facets or make further distinctions within the current facets. This though would lead to a far too complex framework for a useful analysis.

5.4.2 – Hypothetically Speaking

Before turning to the empirical findings three hypotheses are formulated on the basis of the theoretical framework, which will help us flesh out important issues. First, the European political elites are trying to tackle, what they have coined, the EU’s ‘democratic deficit’. The democratic deficit is operationalised as a lack of legitimacy based on universalist principles. This analysis might effect the focus of the discourse. The first hypothesis assumes the elites are right in their analysis and thus it goes as follows:
H1: The EU only has a democratic deficit.
The second theoretical chapter focussed on the influence of ideas of democracy and governance on the assessment of legitimacy. The two heritages of Hont have largely disappeared in mainstream and academic discourse on democracy. Mongrel concepts are often given a democratic or neutral interpretation even when they represent nationalist republican values. The most prominent idea of democracy is that of the constitutional representative democracy. Its legitimacy is traditionally strong in the realm of input and throughput legitimacy.

The other subject of chapter three was governance. Multi-level governance is in itself a new reality, which has not yet established a clear relationship to legitimacy. Many scholars therefore use the more familiar distinction of direct legitimacy based on the model of a nation-state and indirect legitimacy based on the model of an international regime. Neither of these models though fully matches the reality of the EU.

Van Nispen en Posseth (2007: 261) distinguish multi-level governance from multi-level government. The latter perspective represents the idea that the EU is just another level of government. Let us assume that the legitimacy of the EU is assessed in accordance to this perspective, thus in line with the mainstream principles of the constitutional representative democratic nation-state. In short, the EU’s legitimacy is assessed as though it is a nation-state, but is off course not explicitly referred to as such. In terms of conceptualisation of legitimacy, that is to say no distinction is made between universalist and nationalist dimensions and a focus on input, throughput, and direct legitimacy.

The second hypothesis will assume that the actual reason for the EU’s legitimacy deficit is a mismatch in perception between theoretical conceptualisation in accordance to the idea of multi-level government and the reality of multi-level governance. The hypothesis thus goes as follows:

H2: The reason for the EU’s legitimacy deficit is that it is perceived as a political order of multi-level government and thus its legitimacy is assessed as such leading to a mismatch with reality.
Finally, in chapter four, the public discourse and the mass media were analysed. The mass media play a prominent role in the public discourse in modern society. As we saw their role is assessed as flawed and could therefore contributed to the EU’s legitimacy deficit. The final hypothesis states that the mass media shortcomings contribute to the legitimacy deficit and thus reads as follows:
H3: The EU’s legitimacy deficit is increased by the mass media.