I thought it would be useful to bring this issue to the attention of the wider community. Poorly maintained IATI infrastructure is presenting a significant challenge to the use of IATI data at country level (currently here in Bangladesh).
A recurrent issue has come up with using the IATI Datastore. It is not updating data automatically – the last data available for the Netherlands is from early July (though their data is updated monthly). We previously faced the same problem with World Bank data. This means that a) new activities are not found and b) existing activities are not updated. This clearly presents a fundamental obstacle to using IATI data at country level.
After previously reporting this problem in early July, the data finally became available after two weeks (which is too slow, but OK if it doesn’t happen again), thanks to a bugfix by Andy Lulham . I reported yesterday that this issue has now appeared again for Netherlands data. My bug report was essentially disregarded and the issue closed, on the basis that the problem may lie with the Registry (which is not a reason for closing the issue: the Datastore is still not functioning correctly, as it is not returning the requested data).
I am extremely frustrated that such a basic, core service to IATI’s mission is so unreliable and that feedback on basic problems with using IATI data is not actioned speedily by the Secretariat, especially given the apparent focus now on data use.
Having been involved in IATI for about 7 years, it is really embarrassing to have to keep explaining to people here in Bangladesh that their data is fine, the government’s software is fine, but it is IATI infrastructure that is preventing their data from being automatically imported and updated. This is undermining confidence in the use of IATI at country level.
How do we fix this problem?
- Does the Secretariat need additional resources or just to prioritise existing resources better?
- Does the Secretariat even think people should be trying to use the Datastore or does it recommend an alternate option for accessing the data?
- Can the Secretariat have some kind of targets for uptime and resolution of major issues?
Suggestions welcome…
Apologies for getting into details here… Here’s the ticket (from June) regarding the apparent underlying registry issue: https://github.com/ViderumGlobal/ckanext-iati/issues/125
The issue was fixed on staging two weeks ago. Yesterday I tested the latest version. It looks good to me. Please could the ticket be reviewed by IATI Technical Team / Viderum be given the go-ahead to push this to production?
I think there are signs of improvement with Viderum. I’ve previously suggested that a more hands-on approach to managing (e.g. diving into the code a bit more) might be more productive, and would help the tech team have a better handle on what’s going on with CKAN stuff.
bill_anderson:IMO fixing bugs that are affecting real users right now should feature on this list. I don’t think it’s a given.
Bill Anderson , thanks for your response
This was the same reply I got from your colleagues and I have to say it is a very poor answer. The IATI Datastore is not delivering a basic service required for use at country level. Whether this is the result of failings in a service that the IATI Secretariat has outsourced to a third party or not, the Secretariat remains responsible. Further, it is really not clear that the Registry alone is responsible, and even if it is, the fact that OIPA works and the IATI Datastore doesn’t shows there are workarounds. Whether you choose to fix the Registry or the Datastore, both are within your power
I remain highly frustrated by the lack of prioritisation given to the Datastore, in preference for other tools like pyIATI where there has been no demonstrable user need or demand outside of the IATI tech team.
The Datastore does not need to be rebuilt. It just needs basic issues to be resolved in a timely manner rather than rather brusquely dismissed.