IATI registry log dates analysed to assess when updates have been made.
a. Updates reported for two of the last three months = “Monthly” = 4
b. Updates reported for one of the last three months = “Quarterly” = 3
c. Updates reported for any of the last six months = “Six-Monthly” = 2
d. Updates reported for any of the last twelve months = “Annual” = 1
A full update is assumed to have taken place when the observed most recent transaction date changes; i.e. it is assumed that all data are refreshed to that date. (Observations are recorded on a nightly basis and retrieved from a statistical archive for this analysis.)
1. For publishers of 1 year or more
a. Updates reported in 9 of the last 12 months = “Monthly” = 4
b. Updates reported in 3 of the last 4 quarters = “Quarterly” = 3
c. Updates reported in 2 of the last 2 six-months = “Six-Monthly” = 2
d. Updates reported in 1 of the last 12 months = “Annual” = 1
2. For publishers of 6 months or more
a. Updates reported in 4 of the last 6 months = “Monthly” = 4
b. Updates reported in 2 of the last 2 quarters = “Quarterly” = 3
d. Updates reported in 1 of the last 11 months = “Annual” = 1
3. For publishers of 3 months or more
a. Updates reported in 3 of the last 3 months = “Monthly” = 4
d. Updates reported in 1 of the last 5 months = “Annual” = 1
4. For publishers of less than 3 months
d. Updates reported in 1 of the last 2 months = “Annual” = 1
The current methodology is flawed for two reasons.
First, following an upgrade to the underlying software platform on which the IATI Registry is hosted it is no longer possible for a machine to reliably calculate when data were actually updated. Transactions are the most regularly updated elements in IATI and so monitoring transactions is a more reliable way of detecting real updates.
Second, the current logic is incorrect: for example, if an annual update is done within the last three months it is scored as “Quarterly”. The revised logic is more thorough in inspecting all activity over the past year for publishers that have been publishers for more than 1 year. For newer publishers there is additional logic which appears convoluted but is our best attempt to create machine logic over short time lines.
The following comment was received privately:
In response to the previous comment
The intention is to exclude invalid dates, so future transaction dates will not be considered. There does still remain the problem of the clock catching up with an unchanged, previously invalid future date. I don't think we can design a watertight system and hopefully the overall improvement in data quality tools will have flagged a publisher with invalid transaction dates.
We have chosen transaction as it is the element that changes most as an activity progresses. What we are therefore tracking is not simply that there has been some change to the data, but that the change reflects progress on the activity.
No, this is for any single update across a publisher's entire portfolio. I agree this has not been explained very well.
If no updates have been made in the past 12 months, is the score 0, or 1?