The European Quality of Government Index: A Critical Analysis

Measuring the quality, efficiency, and impartiality of public forms of governance in the EU is a perilous task that scholars have tried to address for decades. In order to do so, the European Quality of Government Index (EQI) was created to frame the standards of governance within the Union, and has been publishing reports ever since 2010 (QoG Institute, 2021). This essay will attempt to understand the EQI, research how it obtains its data and measurements, how it translates them into results, and what has been the evolution of this analytical tool. An overview of the institution behind the index will also be necessary, to understand their motivation behind this, and what they hope to get out of it. For the first time, the indicator will be mapped and its background will be described to assess what exactly it measures and who conducts the research. A thorough analysis of the issues it faces will allow this paper to precisely explain how the indicator works and what it allows to see. In a second time, the background behind the creation of the index will be mapped, highlighting why the EU chose to support it. The methodology of the index will be explained, to further our understanding and the thought process behind it. Finally, a critical analysis of the EQI will allow us to understand the problems it portrays, how accurate it is as a measurement tool, as well as underline what its shortcomings are, plus how to potentially fix them.

Creation of the Index

Back in the early 2000s, while academicians were debating how to estimate the quality of public institutions, and with which factors to measure it, the Department of Political Science at the University of Gothenburg, Sweden, decided to take action. The University created and founded an independent institute named the Quality of Governance institute (QOG) in 2004, led by Professors Bo Rothstein and Sören Holmberg. This institute, now made of a team of thirty researchers, scholars, data analysts, moderators, and students, is tasked with researching the causes and consequences of good governance (Charron et al., 2021). According to the researchers, the institute was created to answer three core interrelated problems that came forward when trying to investigate the relationship between decent quality of governance and quality of life. Indeed, its founders believed that the relationship between the two variables was not that a strong economy and social development of a region led to efficient political institutions, but rather that strong and efficient governance was needed to obtain a developed and healthy society (QoG Institute, 2021). Following this belief, the institution has focused on tackling three issues: A theoretical one, regarding the definition of corruption and good governance, but also involving a dilemma regarding public institutions and their existence towards corruption. This dilemma, coined by North as a ‘credible commitment’, lies in the fact that strong public institutions are needed to frame a society, and protect its individuals (1999). However, by strengthening its administration, an organisation also makes it more sensitive to corruption, as powerful governance is more likely to be able to use its public powers for private gains and get away with it.

In line with this issue, the definition of corruption which the institute has decided on is ‘the abuse of entrusted public power for private gains’, underlying the importance of trust in public servants to conduct their tasks accordingly. The second issue highlighted was a methodological one: the majority of the research on governance efficiency was based on single-case studies, which made it hard to generalise. To counteract this problem, the QOG had to make sure to gather relevant data, by conducting large-scale research across different regions and countries (QoG, 2021). Finally, an empirical problem arose: how could the institution make sure to assess valid and reliable data that would be able to measure what they were looking for, and most importantly, how could they gather enough data for their index? In order to fix this last issue, the QOG decided on conducting large-scale research in collaboration with survey centres and multiple institutions and universities, to create a world-leading and freely accessible database. This database would not only be useful to create the Equality of Government Index but would also be a stepping stone to further research. Indeed, as this indicator was created by an academic institution, it also had a scholarly goal in mind and aimed to improve the research in the field in the long term (Rodríguez-Pose, Garcilazo, 2015). The dataset indeed provides small-scale organisations and Ph.D. students the qualitative resources to conduct further research in government and corruption studies, who would not have had the occasion to do so otherwise. 

The EC’s objective

In 2010, the Quality of Governance Institute was mandated by the European Commission to conduct research into the levels of corruption and governance within the Union. This research, funded and supported by the EU, aimed at cross-examining the quality of governance in all the 238 (now 281) regions of the Union, by getting the perception of its citizens. Indeed, large-scale surveys were used to gather a representative opinion of European citizens, the trust they have in their institutions, their relation to corruption and bribery, and their overall opinion on the quality of governance (EC, 2021). These surveys were conducted in 2010, 2013, 2017, and 2021, and are scheduled to continue to be released every three to four years. The Commission highlighted the importance of the reports being very understandable and tasked the QOG to create maps, graphs, and interactive web tools displaying the data, making the EQI accessible to all. The goal of this tool was not only for the EU to have an accurate grasp on the citizens’ feelings on corruption but also for the EU inhabitants to be able to access this data and become more aware of the current political situation. This effort to politicise the issue of corruption is also a modest attempt for the EU to reduce it, as the anti-corruption fight relies strongly on shaming and peer pressure. Bringing up the debate on corruption is a way to combat it, as hearing and reading about it downplays the will to give into it (Annoni & Charron, 2017). 

Moreover, before delving into the methodology and data used for this work, it seems important to make a parenthesis to explain why exactly the EU desired to conduct such research. Multiple scholars have highlighted the fact that, for a nation to grow economically, regional development is key(Rodríguez-Pose & Garcilazo, 2015; Charron et al., 2015). By investing in multiple regions of a country, the quality of infrastructures, education, and social cohesion improves, leading to a nationwide growth of standards of living. A pluri-investing campaign led by the EU in all the regions, especially the most marginalised ones, will have positive repercussions on all the Member States. Nonetheless, a key aspect counting for the success of this development is the quality of governance. Indeed, regional and national governance must be efficient in order to make effective use of these funds and direct them towards necessary areas, to foster an improvement in the region. Moreover, a higher quality of governance implies lower public corruption levels, which often impedes the development of a region.

An interesting case study of the Czech Republic and Hungary between 2009 and 2012 perfectly highlights this issue, by correlating the development of those countries with the regional investments done simultaneously by the EU and the governments (Fazekas & King, 2019). However, the researchers also found out that investing in regions with poor governance and prominent levels of corruption has a counterproductive effect, where EU funds are misused, mismanaged, and more importantly, used for private gains. Even worse, due to overly formalistic compliance and EU funds overriding domestic accountability, the research has shown that EU funding in those regions increased public corruption by 34% (Ibid). This shows that, in order for the EU to be able to properly develop certain regions of the Union, it needs a scientifically accurate indicator of the public corruption levels to counter it by strengthening its governance before wasting precious funds on it. A counterproductive approach of blinded investments may only further governmental fraud, hence the need for the EQI to assess the regional levels of governance and set the ground for further funding. The analysis of regions over countries is also a core aspect of the research: Rodríguez-Pose dedicates an article to this decision, by underlining the reasons behind this choice from the EU (2020). The study of countries as a whole did not provide an accurate enough depiction of the reality, as strong discrepancies existed within the same state depending on the area. A narrowed study of regions allows more accurate results, depicting territorial inequalities with a better sense of realities, and giving clearer results for the analysis. Following this trend, research within social sciences has been increasingly focusing on regions over the recent years, highlighting the relevance of this belief (Charron et al. 2021). Wickberg follows up with this statement, by underlining the sophistication of “a measurement that takes into account sub-national rankings and interprets corruption differently (2021).”

Furthermore, the importance of knowledge in global politics has been underlined by scholars such as Stone, highlighting the need for studies of this kind to be available to all to reinforce the fight against corruption (2013). As knowledge networks grow in importance in shaping transnational governance, accurate measurement tools have become a necessity in modern-day politics. The author concludes that the rise of these indexes contributes to reshaping global policymaking, providing new opportunities and challenges for actors to address while involving public opinion in the debate, creating a framework of knowledge necessary to combat public corruption. 

Methodology and results

Now that the reasons behind the creation of the EQI have been discussed, a description of its methodology is needed. When creating their research design, the QOG took several variables into account, to fix the three issues aforementioned. The Commission desired a report that would cover opinions from all the regions of the EU, to get representative and relevant results (EC, 2021). Indeed, they had to construct a framework able to measure public sector corruption within the EU that would be statistically relevant, while taking into account the differences between member states. In order to do so, the institute created a questionnaire of about fifteen questions, that it asked inhabitants of each of the EU regions, divided in the NUTS 1 and 2 systems, in two separate rounds. The survey was conducted by a private agency over the phone for the first rounds of the study (2010, 2013), and progressively evolved into a hybrid methodology, half-phone half-internet based, for multiple reasons: When it comes to a sensitive topic such as corruption, respondents are more likely to be completely honest when answering by themselves on a computer, rather than while talking to someone else (Charron et al. 2021). The data gathered was more representative, as younger generations are less likely to own a phone line, and older ones are less likely to answer an internet survey. Finally, for practical reasons, the gathering and analysis of data directly typed into a computer led to a considerable gain of time and money. This allowed not only to improve the quality of the answers, but also their quantity, going from 78,000 to 129,000 respondents between 2017 and 2021 (QoG, 2021). Other notable improvements done over the years include an increase of the regions involved (Malta & Latvia were omitted in the first studies), leading to an improvement in representativeness. The increased number of languages available was also an amelioration, with the survey being available in rare regional languages such as Basque or Catalan. This process goes back to Rodríguez‐Pose’s findings that giving an increased voice to all regions, especially the marginalised ones, will have a positive impact on the whole country (2020).

The questionnaire itself started with the definition of corruption chosen by the QoG (‘the abuse of entrusted public power for private gain’), and then asked the responders to answer truthfully by giving a score between 0 and 10 to each statement, 0 being low/disagree and 10 high/agree. The questions asked are divided into three categories: corruption (among public actors), impartiality (of public agents), and quality (of public services). They involved questions such as ‘How would you rate the quality of the public health care system in your area?’ or ‘All citizens are treated equally in the public education system in my area,’ and end up being narrower, such as ‘Corruption is NOT present in elections in my area’ (Charron et al. 2021). This methodology allows for a gradual increase in respondents’ perception of corruption and creates a representative dataset to be obtained through the questionnaire. The data is then aggregated and standardized by region, to gather a relevant average, and obtain results relevant for both region and member states. The relevant results are then divided into the three pillars aforementioned – corruption, impartiality, and quality – in order to create the European Quality of Governance Index. While one could argue that a qualitative study only based on impressions could be insubstantial, it is nevertheless a relevant way of acquiring data. Indeed, perception matters because “agents base their actions on their views and impressions (Kaufmann et al., 2009).”

If citizens believe public agencies to be corrupted, they are less likely to trust them and support them, efficiently weakening governance, and leading to causality between the perception of corruption and quality of governance.  While the findings are already heavily discussed by the QoG itself, (see Charron et al., 2021), they show that the regional approach was needed for accurate results: strong discrepancies exist in-between regions of the same state. Overall, Western countries are more heterogenous, while strong differences exist in Southern and Eastern European countries, which are also more prone to corruption according to the index. While Northern countries have remained top performers in terms of anti-corruption, Southern regions, particularly Italy, Spain & Greece have known a decline in quality of governance (EC, 2021). 

Reflections and possible improvements

It seems important to highlight that, as an indicator made by scholars and not politicians, the EQI is constantly evolving and aiming for scientific accuracy, by self-reflecting on its ontology and methodology (Wickberg, 2021). Its measurements allow to understand a narrower view of corruption, aimed at public institutions and is a clearer and constantly evolving process. This ‘new indicator’ paves the way for a better understanding of corruption, by cross-analysing regional and transnational levels (Mungiu-Pippidi, 2016). In addition to this fact, the Quality of Governance institute itself reflects on its research every year and releases a self-report on possible improvements for future studies. Among the limitations highlighted, the 10-points scale was an important one. Indeed, the grade of ‘5’ was often used by respondents as a neutral answer or a way to express ‘I don’t know’. This slightly biased the results, and a straightforward way to overcome it would be to turn it into a 6-points scale, which leaves no neutral middle ground (Annoni & Charron, 2017).

The questions are also constantly evolving, the latest results highlighting for instance that a question about ‘other people using bribery to obtain advantages’ showed too much variability due to its speculative nature, and had to be removed. Overall, the middle ground between a representative survey and a short enough one to gather substantial answers is tedious to find, as it leads to certain categories lacking accuracy. For instance, the impartiality component does not account for differences in gender, ethnicity, or age, which are crucial factors when it comes to discrimination (Charron et al. 2021). On a broader level, critiques of corruption measurement indexes themselves can be made. In their work assessing other corruption indexes, Shukhova & Nisnevich drew overall negative conclusions (2015). While they were not directly measuring the EQI, they concluded from analysing the five most common corruption measuring tools that the more corrupt a country, the less accurate these tools are to efficiently measure the exact corruption level. Overall, the scholars concluded that indexes tend to underestimate corruption levels in these countries or regions, because of unaccounted variables and issues with the methodology and data gathering. While this conclusion cannot necessarily be applied to the European Quality of Governance Index, it is important to keep in mind that it should be taken with a grain of salt. 

Conclusion

The original approach taken by the European Quality of Governance Index in measuring public corruption has allowed for strong advances in framing and understanding it (Rodríguez‐Pose, 2020). The scholarly analysis and the wide and free available database that results from it paves the way for further academic research in the field, which will benefit EU governance in the long term. Moreover, this study allows the EU to improve its understanding of its citizens at a meso-level, via the regional approach it takes in analysing the responses (Charron et al. 2015). Finally, it is worth noting that the EQI improves every year, through self-assessment and critical analysis, and now possesses over 330 000 responses in total over the past ten years. Nonetheless, a limitation in this research is the lack of scholarly literature directly analysing the index, besides the one coming from the QoG itself. This issue should be addressed in the future by public policy scholars to further improve the quality of this index, and grasp a better understanding of public corruption. 

References

Annoni, & Charron, N. (2017). Measurement Assessment in Cross-Country Comparative Analysis: Rasch Modelling on a Measure of Institutional Quality. Social Indicators Research, 141(1), 31–60. Https://doi.org/10.1007/s11205-017-1821-z

Charron, N., Dijkstra, L., & Lapuente, V. (2015). Mapping the regional divide in Europe: A measure for assessing quality of government in 206 European regions. Social indicators research, 122(2), 315-346.

Charron, N., Lapuente, V. & Bauhr, M. (2021). Sub-national Quality of Government in EU Member States: Presenting the 2021 European Quality of Government Index and its relationship with Covid-19 indicators. University of Gothenburg: The qog Working Paper Series 2021:4.

European Commission (2021) European Quality of Government Index. retrieved from: https://ec.europa.eu/regional_policy/en/information/maps/quality_of_governance/

Fazekas, M., & King, L. P. (2019). Perils of development funding? The tale of EU Funds and grand corruption in Central and Eastern Europe. Regulation & Governance, 13(3), 405-430.

Kaufmann, D., Kraay, A., & Mastruzzi, M (2009). “Governance matters VIII: aggregate and individual governance indicators, 1996-2008”. In: World bank policy research working paper 4978.

Mungiu-Pippidi, A. (2016). For a new generation of objective indicators in governance and corruption studies. European Journal on Criminal Policy and Research, 22(3), 363- 367.

North, D. C. (1999). Institutions and credible commitment. Available at SSRN 6042. 

Rodríguez‐Pose, A. (2020). Institutions and the fortunes of territories. Regional Science Policy & Practice, 12(3), 371-386 

Rodríguez-Pose, A., & Garcilazo, E. (2015). Quality of government and the returns of investment: Examining the impact of cohesion expenditure in European regions. Regional Studies, 49(8), 1274-1290.

Shukhova, A., & Nisnevich, Y. (2017). Measurement of validity of corruption indices. Higher School of Economics Research Paper No. WP BRP, 42.

Stone, D. (2013). Knowledge Actors and Transnational Governance. London: Palgrave Macmillan. 

The QOG Institute (2021). The Quality of Government (QOG) Institute, Gothernburg University. Retrieved from: https://www.gu.se/en/quality-government

Wickberg, S. (2021). Understanding corruption in the twenty-first century: towards a new constructivist research agenda. French Politics Vol.19, 82–102.

Further Reading on E-International Relations