At 12pm GMT+1 today (8th May 2019), the IATI technical team ran a maintenance upgrade on the IATI Discuss service. Unfortunately, we experienced some problems with this, leading to a backup being restored so that service could be resumed. Unfortunately, any posts and other changes that would have been made since this time and 2pm GMT+1 will have been lost - we sincerely apologise for any inconvenience this may have caused.
In technical terms, we have isolated the upgrade failure to an issue with an outdated Docker image and a memory leak within that. The memory leak is a common issue with Docker, and was preventing us from interacting with any micro-services that Discuss runs. Pruning stagnant Docker images unfortunately led to further problems, leading us to restore the backup as a last resort in order to get Discuss up and running again.
We will attempt the upgrade again on Monday 13th May at 2pm GMT as a planned maintenance task. In order to ensure a smooth upgrade, we will also carry out some further steps during this time, as defined below. This inevitably means that users will experience downtime.
Next steps include:
-
Increase the memory of our server
-
Explore the Docker memory issue, and ways that we can safely reclaim space
-
Solidify upgrade processes for the service
-
Define an upgrade schedule
-
Upgrade the Docker image
-
Upgrade the Discuss software
The technical team thank you for your patience on this matter.
Please log in or sign up to comment.