Indices (also called composite indicators), dashboards, or scorecards can be powerful tools for measuring phenomena that are not yet measured. They can help investors identify opportunities, policy makers to know where to improve and other stakeholders advocate for important topics.
There are however, several misconceptions about indices that we need to keep in mind.
1. Indices measure the issue holistically
There are a number of reasons why this may not be the case. First, data availability is limited. Collecting data is not always possible, either because it is costly (e.g. household surveys), because many phenomena are intangible, such as culture, or the infrastructure for measurement does not exist (e.g. sensors or stations to measure water quality). With digital technologies gaining ground we may be in the position to address more and more data gaps. Second, indices have to be comparable across countries, and very often do not report aspects that apply to a subset of countries.
For example Malaria incidence, which is not endemic in many countries.Thirdly, indices are highly simplified versions of reality and cannot provide all the context necessary to give a full picture about an issue in a country. For example measures of energy use have to be seen in a context of the industrial structure and level of development. The main value of indices is that they are practical tools to 1) identify strengths and weaknesses 2) benchmark against similar countries so as to identify practices for inspiration or emulation 3) track progress over time and 4) create an objective baseline and a common language to start a dialogue among stakeholders on how to improve.
2. Indices give the answers on how to improve
Most indices are diagnostic tools and do not provide solutions. Some indices measure outcomes, e.g. the Global Gender Gap Index, which measures the gap between men and women irrespectively of the income levels of countries or the Environmental Performance Index, which measures the distance from the ideal.Other indices are more focused on the inputs needed to achieve a certain outcome. For example, the European Innovation Scoreboard or the COVID Economic Recovery Index. Indices provide a snapshot of the performance of a country on a specific issue. Achieving improvements is a more complex affair.
All indices therefore need further research and a detailed analysis of the indicators to identify the dimensions of the challenge in the country. Then once, these dimensions are known, solutions need to be designed and the index can help identify which countries did well and provide inspiration for solutions. In most countries, identifying and implementing solutions needs collaboration among the different stakeholders, awareness raising, capacity building. Indices are only a starting point, albeit a useful one and it is important to understand how the index was built in order to know how it can be most useful.
3. A good index is mainly based on statistical data
Hard data is important in an index, but not necessarily better than qualitative data such as for example from surveys. The different types of data have their own strengths and weaknesses. Survey data often give voice to the most important stakeholders in an index project and provide a unique, proprietary data source. Of course, it is key that the survey is representative and well designed to allow for cross-country comparisons.
In the case of the Global Youth Index, a survey was used to give youth a voice in atopic that is complex. Such comparisons across countries are rate and survey data can fill important gaps. Other data sources that are qualitative include policy assessments which are based on desk research or on machine learning techniques. Other types of data used in indices are big-data based indicators. These include for example data from mobile network operators to complement existing data on mobile phone use.
If you need help in building an index or would like to know how to improve on it, get in touch.
