» Published on Thu, 10 Jan 2019 13:26:00 +0000

• Post-Mortem

  Between 13:19 and 15:57 GMT (10/11/2019) and between 19:19 (10/11/2019) and 07:08 GMT (11/11/2019) we identified an issue accessing all Kallidus Systems (excluding Recruit). The issue was caused by an underlying problem with our hosting provider experiencing an outage with their storage infrastructure which was not limited to Kallidus. This caused various services used by Kallidus to be unavailable. Unfortunately a hosting provider issue means we were in the position of waiting for our hosting provider to restore service.

  Our hosting provider has provided a preliminary root cause, which is subject to change as they continue to investigate. If there are any major changes we will send out an update:

  Summary of impact: Between 13:19 UTC on 10 Jan 2019 and approximately 05:30 UTC on 11 Jan 2019, a subset of customers leveraging Storage may have experienced intermittent service availability issues. Other resources with dependencies on the affected Storage scale unit may have also experienced impact related to this event.

  Preliminary root cause: Engineers determined that a number of factors, initially related to a software error, caused several nodes on a single storage scale unit to become temporarily unreachable. This, along with the increase in load on the scale unit caused by the initial issue, resulted in impact to customers with Storage resources located on this scale unit.

  Mitigation: To recover the nodes, the hosting provider team undertook a sequence of structured mitigation steps, including:
  performing a code update to mitigate a software error reducing background processes in the scale unit throttling and offloading traffic to allow the scale unit to gradually recover

  » Updated Mon, 14 Jan 2019 14:43:00 +0000

• Resolved

  Between 13:19 and 15:57 GMT we identified an issue accessing all Kallidus Systems (excluding Kallidus Recruit). We have implemented a workaround to mitigate our hosting provider's issue and normal service has resumed. We await and RCA from our provider which we will share in due course. Thank you for your patience during this time.

  » Updated Thu, 10 Jan 2019 16:09:00 +0000

• Update

  Starting at 13:19 GMT an issue with accessing all Kallidus systems (excluding Recruit) was identified. Our Hosting provider is investigating and service is partially unavailable. Additional information will be provided in 60 minutes or as events warrant. Thank you for your patience during this time.

  » Updated Thu, 10 Jan 2019 15:19:00 +0000

• Update

  Starting at 13:19 GMT an issue with accessing all Kallidus systems (excluding Recruit) was identified. Our Hosting provider is investigating and service is partially unavailable. Additional information will be provided in 60 minutes or as events warrant. Thank you for your patience during this time.

  » Updated Thu, 10 Jan 2019 14:16:00 +0000

• Update

  Engineers are aware of an investigating an issue with accessing all Kallidus systems (excluding Recruit). Additional information will be provided shortly.

  » Updated Thu, 10 Jan 2019 13:30:00 +0000

• Investigating

  Engineers are investigating an issue on Kallidus Learn & Classic LMS. Additional information will be provided shortly.

  » Updated Thu, 10 Jan 2019 13:26:00 +0000