This might is an issue that some may have hit before. It’d be great to hear from others who regularly ingest IATI data into their projects, to understand if this is a headache, or not, or whether you have a workaround. Thinking of @Herman @markbrough @tdavis @siemvaessen @matmaxgeds and others…
With FTS, we want to import IATI activities, to construct Flows. A (obvious) vital cog to this is the IATI transaction.
We also know that IATI publishers will be adding transactions to their activities over time - which is expected behavior.
- On first import an activity has two transactions.
- When the activity is updated, a month later, two more transactions have been added by the publisher. The activity now has four transactions.
- When we import the updated activity, we know it’s some data we’ve encountered before, because the
activity-identifieris present. That’s helpful!
- However, the transactions have no identifier. Our import now gives us six transactions (the first two, plus the new four)!
I can picture people frowning at this! Of course, our system and data import routines could have some extra process to generate and store a hash of the data, for example - or even check the transactions dates and values etc etc. We understand that, but the point is: do we think it efficient to shift this overhead from the publisher to data users, each of whom would have to work out the problem and compute a workaround?
transaction element does actually include a @ref attribute, which could be use to declare an identifier for a transaction. For example:
<transaction ref="1"> <transaction ref="2"> <transaction ref="3"> <transaction ref="4">
Or, more verbosely:
<transaction ref="EX-AMPLE-ORG-ACTIVITY100-transaction1"> <transaction ref="EX-AMPLE-ORG-ACTIVITY100-transaction2"> <transaction ref="EX-AMPLE-ORG-ACTIVITY100-transaction3"> <transaction ref="EX-AMPLE-ORG-ACTIVITY100-transaction4">
<transaction ref="oranges"> <transaction ref="apples"> <transaction ref="pears"> <transaction ref="cashews">
(OK, that last example isn’t advisable, but @reidmporter might like it)
For our import to FTS, we know this solution actually works! When publishers include a @ref, it acts as an identifier we can check, to understand if this is a transaction we’ve seen before.
We are aware of scenarios such as transactions being edited, or data being added to a transaction (a receiver-org activity-identifier, for example), but would appreciate we try and stick to the main point here: should we strive to include a @ref that can act as a transaction identifier within any activity, to help data users?
It’d be great to hear from others.