Using forward data | IATI Connect

As part of our efforts to encourage country-based staff (and partner country officials) to use IATI data and tools, I’ve spent more time lately exploring the Datastore and the D-Portal. I have not been able to access forward-looking data in a practical way, and wonder if I may be missing something, or may have identified a gap that should be filled.

More specifically:
1 - Is there a place to access the data published in the Organizational files? I’m thinking in particular of the country forward budgets (where donors provide estimates of their budget over the next 3 to 5 years). By access I mean a reasonably user-friendly interface, not a bunch of xml files. It’s fairly simple data that would look good as csv. I did check the D-Portal, but the forward data provided by country is the sum of the project budgets, not the data published in the Organization file.

2 - Is there a way to download data on planned disbursements for a group of projects? When using the Datastore Query Builder, budgets and past transactions are downloaded but not planned disbursements. In addition, planned disbursements don’t seem to be visible on the D-Portal project profiles - even if they were though, we cannot expect people to look up one project after another to compile this data.

Forward-looking information is consistently among the top asks from partner countries, and something donor country-based staff need access to. It seems odd that there would not be a reasonably simple way to access the data. If there is one, I’d be grateful for instructions on how to find it.

Comments (9)

mugabe • 7 years ago

i have been as well looking around for such information on D-portal and data store but i can’t find any! please whoever knows how to get these information can help us and no way to get them then this needs to be improved in further dialogues.

Andy Lulham • 7 years ago

Hi both,

YohannaLoucheur:

When using the Datastore Query Builder, budgets and past transactions are downloaded but not planned disbursements.

You’re talking about commitments here, right? I.e. transactions with transaction-type == 2? If so, these do seem to be included in the CSV output from the datastore (though you can’t filter for them specifically.)

YohannaLoucheur:

Is there a place to access the data published in the Organizational files? I’m thinking in particular of the country forward budgets (where donors provide estimates of their budget over the next 3 to 5 years). By access I mean a reasonably user-friendly interface, not a bunch of xml files.

This is a really good point – I don’t know of any tool for org file viz (though I don’t think it would be too difficult to build.) I’d be interested in having a go!

Yohanna Loucheur • 7 years ago

andylolz:

You’re talking about commitments here, right? I.e. transactions with transaction-type == 2? If so, these do seem to be included in the CSV output from the datastore (though you can’t filter for them specifically.)

Nope, talking about Planned Disbursements. These aren’t in the Transaction element.
http://iatistandard.org/202/activity-standard/iati-activities/iati-activity/planned-disbursement/

That being said, it would be neat to be able to filter by type of transaction…

Matt Bartlett • 7 years ago

Hi Yohanna

On organisation budgets - there is some data on this in d-portal if you choose a publisher from the list on front page, and look at a recipient country table eg here http://d-portal.org/ctrack.html?publisher=CA-3&tongue=eng#view=publisher_countries
The two green budget columns show data for the same year - the ‘{year} allocated’ column shows budget information from the publisher’s activity file(s), and the ‘{year} total’ column shows the budget data from the publisher’s organisation file. You can also scroll through subsequent years using the arrow on the ‘IATI Budget’ column.
(quick caveat, this data for some other publishers needs re-importing after an issue last week, but should be back in tomorrow!)
It’s per publisher, but is that a start on what you were looking for?

Yohanna Loucheur • 7 years ago

Thanks Matt! Have to admit I hadn’t quite noticed the arrows to scroll for future years… I’m not alone, judging by Levis and Andy’s responses.

James Coe • 7 years ago

I’d like to +1 Yohanna’s comment.

Yesterday I was trying to see the countries / thematic spends of certain donors. Using IATI, I found the only reliable way to do this was to open up the Registry and trawl through the organisation files.

Ultimately, the question I was asking was a pretty easy one – “who is spending what in country X?” The data was there but getting to it was manual and thus time-consuming and required a level of comfort with the Registry that we cannot realistically expect people to have.

Please can somebody let me know if I am missing something?

Dale Potter • 7 years ago

Just to chip in with regard to what is possible to obtain from the official IATI Datastore.

Unfortunately data from the planned-disbursement element is not available in Datastore CSV outputs. However, it is available in the XML and JSON technical formats, so a script could be written to loop over all data and obtain desired elements from the data. Whilst this requires some technical input, this would be a lot easier than trawling through every file from the Registry! An example of how to do this is here: https://github.com/dalepotter/iati-datastore-custom-csv

There is also a wider issue in that the Datastore contains no information from organisation files, such as information on the total-budget element.

We’ll definitely add both these ideas to our list of features for the new datastore given the response here. Others: please do continue to post if this information would be useful - either as a reply here or on the Datastore codebase issues that I have created for obtaining planned-disbursement data and getting data from organisation files.

Yohanna Loucheur • 7 years ago

Thank you Dale for following up on this and creating the issues on github.

I fully understand that resources are limited and issues need to be prioritized. As such, I would like to help us all focus on the issues most likely to make a difference to users.

dalepotter:

There is also a wider issue in that the Datastore contains no information from organisation files, such as information on the total-budget element.

The total-budget element is one of the the least interesting in the Org file (for one thing, it’s available in national budgets, annual reports etc). The country-budget is probably the most important - partner countries ask donors for this information to support their planning.

As such, I would suggest to change the example you provide on github from total-budget to country-budget, to help focus people’s mind on the most important issue from the start.

dalepotter:

An example of how to do this is here: https://github.com/dalepotter/iati-datastore-custom-csv

Thanks for developing this. It’s way beyond my technical expertise, but hopefully it will help others until this element can be added to the Datastore.

Dale Potter • 7 years ago

YohannaLoucheur:

The total-budget element is one of the the least interesting in the Org file (for one thing, it’s available in national budgets, annual reports etc). The country-budget is probably the most important - partner countries ask donors for this information to support their planning.

Thanks for the extra clarification Yohanna. I have amended the GitHub issue to cite the recipient-country-budget element.

Re the code to output a customised CSV file from the Datastore: If anyone thinks this functionality could be useful but difficult to understand, I’m happy to spend some time to talk through the code, or provide more documentation

Please log in or sign up to comment.