Global Partnership Monitoring Framework:
Proposed improvements to the indicator to measure transparency
The IATI Secretariat is publishing this paper as a contribution to the current review of the Global Partnership transparency indicator and draws upon the experience of the pilot exercise which concluded in March 2014. The paper proposes improvements to the methodology to make the indicator more consistent and robust for the next iteration, and expansion to include a quality dimension.
The IATI Secretariat is investing substantial resources in assisting its members in improving the overall quality of data published to the IATI standard. Work on the indicator is integral to this commitment.
The IATI Secretariat supports the Indicator
IATI has from the outset emphasised the importance of combining donor accountability with meeting the needs of developing countries, as evidenced most recently by its survey on the needs of aid information management systems.
Paragraph 23(c) of the Busan Outcome document reflects the most important consensus ever reached on the transparency of development cooperation. It calls for the “electronic publication of timely, comprehensive and forward looking information on resources provided through development co-operation” to “meet the information needs of developing countries and non-state actors, consistent with national requirements.”
The Global Partnership transparency indicator is a good measure of this Busan commitment and reflects both the spirit and the letter of its intent. The IATI Secretariat supports the methodology of the indicator and believes that it can become a useful tool within the post-2015 development agenda. For this reason the Secretariat has invested substantial time and resources into the development of the Indicator and remains committed to continue this investment.
In the lead up to the First High-Level Meeting of the Global Partnership for Effective Development Cooperation held in Mexico in April 2014, the technical teams of the Joint Support Team, DCD-DAC and IATI had insufficient time to fully test the draft methodology and engage in a more comprehensive consultation with those being assessed and those who wished to make use of the indicator. There was an online consultation in autumn 2013 and as many views as possible were taken into account in that short period. Nevertheless the resulting indicator had limited buy-in and appreciation from many of its stakeholders, a situation compounded by unsatisfactory review procedures once scoring was shared.
The IATI Secretariat believes it is essential for the future of the indicator to resolve these procedural shortcomings by ensuring that such issues are comprehensively addressed at this point. This work can become a useful vehicle for the IATI Technical Team to assist publishers in meeting their December 2015 Busan commitments, and the team plans to devote a section of the IATI Dashboard - http://dashboard.iatistandard.org/ - to engage with all IATI publishers on the methodology and statistics that make up the indicator. Proactive engagement and buy-in by IATI Steering Committee members will ensure that the indicator becomes a useable data quality tool and not a source of dissatisfaction.
Based on feedback from stakeholders and lessons learned during the pilot of the indicator, the IATI Secretariat would like to propose a number of modifications to the methodology – outlined in detail in the attached table. Most of these are either corrections or minor improvements that do not alter the overall shape or construct of the indicator. Rather they are aimed at strengthening the logical consistency of the methodology. The table includes a “Discussion” column which explains the thinking behind each of the proposed changes.
This proposal also takes into account the section on future considerations in the indicator methodology paper. It includes consideration of the quality and usability of the information (which requires further technical work), enabling broader participation in the context of the multi-stakeholder approach of the Global Partnership, and incorporating an assessment of foundations, NGOs and other official providers that endorsed the Busan agreement.
Measuring Data Quality
During the initial drafting and consultation on the methodology, a number of organisations discussed how the quality of data should be measured. While there is currently no concrete proposal to address this point, it is included here as a marker to serve as a reminder that this is an area that still needs to be developed. Such work would build on ongoing work by OECD to analyse and strengthen the quality of reporting to CRS and by IATI to assess and improve data quality in the IATI registry. The challenge will be to develop a quantitative assessment of the quality of information across the component systems of the common standard for the next iteration of the indicator.
The IATI Secretariat proposes a revision to the basis on which the coverage ratio – activities published to IATI as a proportion of total activities - is calculated. It is suggested that there should be one consistent measure for all three dimensions of the indicator – timeliness, comprehensiveness and forward-looking – that is most relevant to the provision of timely data to help developing countries plan for and manage their aid inflows. It is also proposed that activity-level forward-looking data is best assessed by counting activities with relevant forward data, rather than the value of future budgets.
The IATI Secretariat proposes a revision to how fields that are not applicable are handled, for example loan information or expenditure for activities still in the planning stage. The Secretariat suggests the removal of nine fields that are not applicable in all circumstances, as their inclusion distorts the results.
Finally, this paper calls for the inclusion of three new fields – conditions, results and specific geo-referencing - reflecting priorities in development effectiveness, as identified in Accra and again at Busan.
This proposal concerns the methodology for scoring IATI data, with just two exceptions:
- forward-looking activity-level data, as mentioned above; and
- the calculation of the time lag for CRS reporters, to bring it closer to symmetry with the time lag calculation for IATI data.
All interested parties are invited to submit comments on this paper by 25 September 2014.
The Secretariat would particularly like to encourage members of the IATI Steering Committee to submit comments to inform the discussion of the next steps for the transparency indicator during the Steering Committee meeting on 15-16 October 2014. The results of that discussion will then feed in to further consideration by the Joint Support Team of improvements to the indicator, alongside comments from DAC members and the DCD Secretariat.