» Published on Thu, 20 Jun 2024 16:04:00 +0000

• Post-Mortem

  At 15:48 UTC on 20th June 2024 a mis-formulated database cleanup script erroneously cleared bespoke customer settings and data fields from live tables within the US Kallidus-Sapling platform. As per policy, Kallidus maintains active backups of all production data and initiated the restore process. All missing fields were confirmed to be reinstated at or before 16:27 UTC on the 21st June 2024.

  To ensure that there is no repetition of this incident, supplementary controls on routine database maintenance activities have been put in place which restrict how this activity is undertaken, alongside an additional level of scrutiny in the review of any scripts that may be ultimately deployed into the production environment.

  Please accept our apologies for any inconvenience this incident caused.

  » Updated Tue, 25 Jun 2024 09:13:00 +0000

• Resolved

  All data can now be viewed.

  Details of the cause and mitigation steps will be shared in due course on this page.

  Thank you for your patience whilst we resolved this issue, please accept our apologies for any inconvenience caused.

  » Updated Fri, 21 Jun 2024 16:29:00 +0000

• Update

  Profile data is continuing to be restored and customers will start to see data re-appearing in the user profile page; There is still a sub-set of data to be restored back to pages but we will update as soon as this is fully completed. This issue continues to only affect a small sub-set of customers specifically in our N. America region, and we continue resolve as swiftly as possible. Thank you for your continued patience whilst we continue to work to resolve this.

  » Updated Fri, 21 Jun 2024 15:40:00 +0000

• Update

  Profile data is now partially restored and customers will start to see data re-appearing in the user profile page from now; There is still a sub-set of data to be restored back to pages but we will update as soon as this is fully completed. This issue continues to only affect a small sub-set of customers specifically in our N. America region, and we continue resolve as swiftly as possible. Thank you for your continued patience whilst we continue to work to resolve this.

  » Updated Fri, 21 Jun 2024 14:32:00 +0000

• Update

  Our engineers are in the final stages of testing a potential solution for the incident and we hope to provide some more positive news on our next update. This issue continues to only affect a small sub-set of customers specifically in our N. America region, and we continue resolve as swiftly as possible. Thank you for your continued patience whilst we continue to work to resolve this.

  » Updated Fri, 21 Jun 2024 13:36:00 +0000

• Update

  Thank you for your continued patience whilst we work to resolve this issue affecting a subset of Sapling customers.

  We will continue to update this page with our progress.

  » Updated Fri, 21 Jun 2024 12:47:00 +0000

• Update

  Our engineers are continuing to work on the issue with profile data not displaying correctly. This issue continues to only affect a small sub-set of customers specifically in our N. America region, and we continue resolve as swiftly as possible. Thank you for your continued patience whilst we continue to work to resolve this.

  » Updated Fri, 21 Jun 2024 11:35:00 +0000

• Update

  Our engineers are continuing to work on the issue with profile data not displaying correctly. This issue continues to only affect a small sub-set of customers specifically in our N. America region, and we continue resolve as swiftly as possible. Thank you for your continued patience whilst we continue to work to resolve this.

  » Updated Fri, 21 Jun 2024 10:29:00 +0000

• Update

  Our engineers are continuing to work on the issue with profile data not displaying correctly.This issue continies to only affect a small sub-set of customers specifically in our N. America region, and we continue resolve as swiftly as possible. Thank you for your continued patience whilst we continue to work to resolve this.

  » Updated Fri, 21 Jun 2024 09:26:00 +0000

• Update

  Some residual issues with data not displaying as expected on profiles is ongoing. This issue affects a small sub-set of customers specifically in the N.America region, and we continue resolve as swiftly as possible. Thank you for your continued patience whilst we work to resolve this.

  » Updated Fri, 21 Jun 2024 08:38:00 +0000

• Update

  Some residual issues with data not displaying as expected on profiles is ongoing. This issue affects a small sub-set of customers specifically in the N.America region, and we continue resolve as swiftly as possible. Thank you for your continued patience whilst we work to resolve this.

  » Updated Fri, 21 Jun 2024 07:30:00 +0000

• Update

  We are investigating reports of some residual issues with data not displaying as expected on profiles. This issue affects a small sub-set of customers, and we are working to resolve as swiftly as possible.

  Thank you for your continued patience whilst we work to resolve this.

  » Updated Fri, 21 Jun 2024 06:28:00 +0000

• Resolved

  All data is now visible as expected and normal service has resumed. You may now modify profile templates again if you wish.

  Thank you for your patience whilst we worked on this, and please accept our apologies for any disruption this caused for you.

  Details of the root cause will be shared on this page in due course.

  » Updated Thu, 20 Jun 2024 19:57:00 +0000

• Update

  We are now in the process of restoring the ability to view data on profiles.

  We will continue to update this page with our progress, thank you for your ongoing patience.

  » Updated Thu, 20 Jun 2024 19:56:00 +0000

• Update

  Please accept our apologies for any inconvenience being caused by this issue.

  We have identified the issue and are working to resolve it as swiftly as possible. We will continue to update this page with our progress.

  » Updated Thu, 20 Jun 2024 18:53:00 +0000

• Update

  Thank you for your patience whilst we continue to work on resolving this issue.

  We are making progress and will continue to share updates on this page.

  Please accept our apologies for any inconvenience caused.

  » Updated Thu, 20 Jun 2024 18:03:00 +0000

• Update

  Thank you for your patience whilst we continue to investigate. We have identified an issue linked to data not being displayed on some profile templates, all data is still present in the database.

  We are working to resolve this issue as a priority now. In the meantime, please do not modify the profile templates. We will let you know once the issue is fixed, and you can then proceed with modifications.

  We will continue to keep this page updated with our progress. Please accept our apologies for any inconvenience caused.

  » Updated Thu, 20 Jun 2024 16:53:00 +0000

• Investigating

  We are investigating reports that some Sapling customers are experiencing issues accessing some pages, and some information is not presenting as normal.

  We will keep this page updated with our investigation progress. Thank you for your patience whilst we look into this.

  » Updated Thu, 20 Jun 2024 16:04:00 +0000