Zindex is a tool that allows you to compare procurement of different public buyers; it has been functioning in Czech Republic for 10 years. It was developed by the Czech think tank EconLab, and during its existence, this ranking with various indicators of procurement efficiency has significantly improved the conduct of tenders in Czech Republic. It had a measurable impact on the activity of over 50 buyers — cities, hospitals, ministries.

Zindex rates procurement organizers, highlights their main shortcomings and motivates them to improve — the best buyers are publicly awarded. The Zindex project received awards and recognition of the European Commission, Open Society Foundation and Open Contracting Partnership. So, what is the point of the ranking, and why has it been so successful?

You can read about procurement practices in Ukraine, Czech Republic, Hungary, and Poland at the link. It will help you better understand the field of procurement in Czech Republic and the realities in which this ranking operates.

Publication of the ranking

Each year, EconLab experts publish a detailed dashboard for each buyer (such as City of Prague ranking) and a chart of the best procurement organizers (such as Cities 2020 ranking).  Procuring entities are divided into several categories to avoid comparing apples and oranges. These include cities, hospitals, state institutions, regions, public enterprises, and so on. Additionally, in some categories the experts also distinguish big and small procuring entities.

If you clicked on the links above, you may have already noticed that only the best organizers have their place in the ranking — only 50% of all of them. The EconLab team publishes the unordered list of the others. Why is there no information about which of them is the worst? The developers of the ranking found that the impact is higher when focusing on the good practice—as buyer perception focuses more on constructive critique, rather than finger-pointing. This decision comes at a cost—the media coverage of the rating is much weaker. However, media awareness can be sacrificed when it comes to the real impact on the activity of procurement organizers. Overall, over 50% of the buyers declared that the results of Zindex.cz brought them valuable feedback (an increase by 21%). In addition, each year roughly 10% of rated buyers commit to making steps towards improvement of the procurement practices.

Buyer feedback and awarding

An important part of the rating process, however, happens before final publication of the evaluation. The experts send the preliminary results at least 2 months in advance and run a hotline for consulting on the results. Buyers can consult on the results obtained. Such an approach has multiple benefits, such as:

  1. Procurement organizers have a better understanding of their results —experts explain, discuss, comment on buyers’ procurement practices and give recommendations on their improvement, so that buyers can improve and purchase more efficiently.
  2. The EconLab team improves methodology‚ the think tank receives valuable feedback on shortcomings of the rating methodology.
  3. This process motivates procuring entities to change — experts recalculate some indicators if buyers fix errors during the feedback period. This is the most effective leverage, which improved buyers’ procurement practices by 20-30%.

After all feedback is incorporated, EconLab awards the best performing buyers with a statue of “Neplejtvák” (could be translated as non-wasting fish), which can be seen as an Oscar for public buyers.  The idea is to foster positive competition in doing public procurement following best practice. An additional goal is to disrupt the negative perception of tenders in the society. The world of a procurement official is grim, and positive feedback from an NGO is quite rare. Thus, the EconLab award positively motivates specialists in this field.

How is the rating done?

The heart of Zindex is of course the rating methodology itself — it is publicly available. In general, EconLab aims at measuring buyer’s compliance with best practice recommendations, particularly transparency, efficiency, and decreasing corruption risks in public procurement.  The think tank measures these using available data by calculating the following indicators:

  1. Public procurement as a share of total procurement transactions. Here, accounting data and procurement information are compared. What share of the procurement is actually published?
  2. Competitive contracting. What share of tenders goes through competitive procedures?
  3. Consistent conduct. How often a buyer cancels tenders or makes changes to them?
  4. Winner concentration. How often the same company gets awarded by the buyer?
  5. Level of competition at the bidding. What is the actual competition in tenders, and how many entrepreneurs take part in them?
  6. Pro-competitive tools. How often the buyer applies e-auction, design contest, etc.?
  7. Legal misconduct. Are there any cases of violations detected by the Office for the Protection of Competition?
  8. Buyer’s journal data quality. How clearly and timely does the buyer publish information about their tenders, their prices, and winners in the central register?
  9. Buyer profile data quality. Does the buyer provide additional information on their profile? How does the buyer publish their small tenders whose amount is below national thresholds?

As simple as some indicators may seem, in practice they are quite difficult to calculate. To get a credible ranking, the think tank needs to accommodate legal changes, specifics of multi-lot tenders, framework agreements, fix publication errors, and do thousands of other minor adjustments — everything for the sake of solid and fair results.

EconLab specialists helped implement a similar initiative in Slovakia. Back then they had to change about 25% of the ranking to adapt it to the legislation, data sources, and practices of another country. The volume of work turned out to be less than analysts expected, so, now they are thinking about further promotion of the tool among countries.

 

This publication was prepared based on the article of Jiří Skuhrovec, chairman of EconLab, within the project “Tackling corruption in public procurement in Ukraine and V4 countries with new approaches.”

The project is co-financed by the Governments of Czechia, Hungary, Poland and Slovakia through Visegrad Grants from International Visegrad Fund. The mission of the Fund is to advance ideas for sustainable regional cooperation in Central Europe.