Degraded Service - Kallidus Learn

» View Event Details | Created Wed, 03 Jan 2024 15:14:00 +0000

Resolved

RCA: Between 15:00 and 15:38 GMT on 03-01-2023, customers using the Accreditation Compliance, Sharing and Recommendations or Course Promotions features were reporting 'Hmm that didn't work' errors when trying to access the Kallidus Learn system. We had identified that there was an issue introduced during an earlier deployment of our software. After a brief investigation, we had reverted the earlier deployment to enable access as expected. Mitigation steps were taken and the team continues to monitor. Thank you for your patience while we resolved this issue.

Posted: Fri, 05 Jan 2024 11:21:00 +0000

Resolved

Engineers have identified the cause of the error and resolved the error. An RCA for this incident will be posted here in due course.

Apologies once again for the inconvenience caused.

Posted: Wed, 03 Jan 2024 15:55:00 +0000

Investigating

Engineers are currently investigating an issue with the Kallidus student Home page, showing a 'Hmm that didn't work' error to learners for a sub set of customers. We are aware of the issue and are working to resolve the matter as quickly as possible.

Apologies for the inconvenience this is causing at the moment.

Posted: Wed, 03 Jan 2024 15:14:00 +0000

Degraded Service: Learn Lite and Kallidus OTS content

» View Event Details | Created Mon, 08 Jan 2024 08:34:00 +0000

Post-Mortem

RCA: From 3pm 07/01/2024 until 9.40am 08/01/2024 users may have had problems launching Kallidus Off The Shelf courses. The root cause was identified as an unresponsive component that underlies our course delivery system. Upon restarting the server normal service was resumed. Active monitoring has been put in place to alert us to further instances of this problem, whilst we continue to investigate any other learning opportunities arising from the issue.

Posted: Tue, 09 Jan 2024 13:57:00 +0000

Resolved

Access to all Kallidus OTS courses has now been restored and the issue resolved.

Apologies for the inconvenience once more, while the team worked to resolve this today.

An RCA for this incident will be made available here in due course.

Posted: Mon, 08 Jan 2024 10:00:00 +0000

Update

Engineers are continuing to investigate access issues to our Off-the-Shelf content offering.

Posted: Mon, 08 Jan 2024 09:27:00 +0000

Investigating

Our engineers are currently investigating an issue with access to our Kallidus OTS content offering at the moment. We will keep this status page updated with progress toward resolving the matter.

Apologies for the inconvenience this is causing.

Posted: Mon, 08 Jan 2024 08:34:00 +0000

Degraded service: Kallidus Learn

» View Event Details | Created Tue, 09 Jan 2024 09:38:00 +0000

Post-Mortem

Summary: Between 09:20 and 10:18 GMT on 09-01-2024, users of Kallidus 360, Learn, Perform and Recruit may have experienced errors at login.

Preliminary root cause: Kallidus engineers discovered that a cloud resource, critical to the authentication process, had reached 100% of its provisioned capacity leading to errors elsewhere within the application. Resizing this resource resolved the issue. Kallidus apologises for any inconvenience caused by this disruption and thanks you for patience while we resolved the issue.

Posted: Tue, 16 Jan 2024 17:13:00 +0000

Resolved

Engineers have now resolved the issue with access to Kallidus Learn (may have also affected access to other Kallidus Suite products in some cases).

Please accept our apologies again for the issue encountered today.

An RCA for the incident will be made available here in due course.

Posted: Tue, 09 Jan 2024 10:27:00 +0000

Investigating

We are receiving reports of a subset of customers receiving 503 - service unavailable messages when accessing Learn. Our engineers are aware and currently investigating the issue.

Please accept our apologies for the inconvenience this is causing today.

Posted: Tue, 09 Jan 2024 09:38:00 +0000

Learn service disruption

» View Event Details | Created Thu, 18 Jan 2024 08:07:00 +0000

Post-Mortem

Summary: A change to the user authentication process in Learn caused issues with access, imports, and administration for some customers from 18/01/2024 to 23/01/2024. In some instances this also impacted other Kallidus applications using the same authentication. The issues were mitigated separately and a deep retrospective is underway.

Details: On 18/01/2024, a deployment introduced a change to the user authentication process in Learn, this in turn then used an incorrect configuration to direct users to their Identity Provider.

This caused some customers to experience issues with accessing Learn, including general login issues with SSO and username/password credentials, and issues with Learn administration. In some instances this also impacted other Kallidus applications using the same authentication. There were also knock-on effects to course imports, and user imports.

The majority of access issues were mitigated by 18/01/2024 18:00 GMT, and the issues with SSO and Learn administration were resolved on 23/01/2024 by 13:00 GMT.

The user imports, KUI tool, were also affected by this change, and access was restored on 22/01/2024 for affected customers.

A deep retrospective for this issue is already underway to identify areas for improvement and prevent similar incidents in the future.

Please accept our apologies for this disruption and any inconvenience caused.

Posted: Thu, 25 Jan 2024 16:25:00 +0000

Update

We are currently carrying out our retrospective activities following this disruption, with this in mind we will endeavor to share the details of root cause by 25th January.

Posted: Mon, 22 Jan 2024 08:46:00 +0000

Resolved

A fix has now been deployed to resolve today's issues, and normal service is beginning to resume.

We are taking further steps to mitigate and will share details of the root cause on this page in due course.

Please accept our sincerest apologies for the disruption this will have caused. Once again, thank you for your patience during this challenging day.

Posted: Thu, 18 Jan 2024 16:47:00 +0000

Update

Our Engineers are continuing to work to resolve the intermittent access and performance issues with Kallidus Learn and Perform. This remains our priority and we will continue to update this page with news as and when we have it.

Please accept our apologies for any disruption and inconvenience this is continuing to cause

Posted: Thu, 18 Jan 2024 15:48:00 +0000

Update

Engineers are continuing to investigate and work to resolve intermittent access and performance issues with Kallidus Learn and Perform. This is our priority and we will update this page with news as and when we have it. Please accept our apologies for any disruption and inconvenience this is continuing to cause

Posted: Thu, 18 Jan 2024 14:55:00 +0000

Update

We are continuing to investigate and work to resolve intermittent access and performance issues with Kallidus Learn and Perform.

This is our priority and we will update this page with news as and when we have it.

Please accept our apologies for any disruption and inconvenience this is continuing to cause

Posted: Thu, 18 Jan 2024 13:46:00 +0000

Update

Engineers are continuing to work to resolve issues affecting access for some users who use a separate URL to access learn, to bypass SSO. This is our priority and we will update this page with news as and when we have it. Please accept our apologies for any disruption and inconvenience this is continuing to cause.

Posted: Thu, 18 Jan 2024 12:47:00 +0000

Update

Engineers are continuing to work to resolve issues affecting access for some users who use a separate URL to access learn, to bypass SSO. This is our priority and we will update this page with news as and when we have it.

Please accept our apologies for any disruption and inconvenience this is continuing to cause.

Posted: Thu, 18 Jan 2024 11:50:00 +0000

Update

We are continuing to work to resolve issues affecting access for some users who use a separate URL to access learn, to bypass SSO.

This is our priority and we will update this page with news as and when we have it.

Please accept our apologies for any disruption and inconvenience this has caused.

Posted: Thu, 18 Jan 2024 10:50:00 +0000

Update

This issue has now been resolved for the majority of customers. Some customers who use a seperate URL to access Learn, to bypass SSO, will still be experiencing access issues.

We are continuing to work on this as our priority and will update this page with news as and when we have it.

Please accept our apologies for any disruption this has caused.

Posted: Thu, 18 Jan 2024 09:55:00 +0000

Update

We are continuing to investigate this issue as our priority.

Some customers may still be unable to access their Learn site, or the Administration side of Learn.

Please accept our apologies for this disruption, we will continue to keep this page updated as and when we have updates.

Posted: Thu, 18 Jan 2024 08:55:00 +0000

Investigating

We are currently investigating an issue following this morning's maintenance window. Some customers may see errors when attempting to access the administration area of Learn sites.

In most cases Learners should be able to continue with their use of the system.

Please accept our apologies for any inconvenience this causes.

We will update this page with further updates as soon as we have them.

Posted: Thu, 18 Jan 2024 08:07:00 +0000

Degraded Service - Kallidus Learn

» View Event Details | Created Mon, 22 Jan 2024 09:11:00 +0000

Post-Mortem

RCA: Summary: A change to the user authentication process in Learn caused issues with access, imports, and administration for some customers from 18/01/2024 to 23/01/2024. In some instances this also impacted other Kallidus applications using the same authentication. The issues were mitigated separately and a deep retrospective is underway. Details: On 18/01/2024, a deployment introduced a change to the user authentication process in Learn, this in turn then used an incorrect configuration to direct users to their Identity Provider. This caused some customers to experience issues with accessing Learn, including general login issues with SSO and username/password credentials, and issues with Learn administration. In some instances this also impacted other Kallidus applications using the same authentication. There were also knock-on effects to course imports, and user imports. The majority of access issues were mitigated by 18/01/2024 18:00 GMT, and the issues with SSO and Learn administration were resolved on 23/01/2024 by 13:00 GMT. The user imports, KUI tool, were also affected by this change, and access was restored on 22/01/2024 for affected customers. A deep retrospective for this issue is already underway to identify areas for improvement and prevent similar incidents in the future. Please accept our apologies for this disruption and any inconvenience caused.

Posted: Wed, 13 Mar 2024 10:16:00 +0000

Resolved

Engineers have now rolled out the fix in relation to logged-in access reverting to SSO logins. An RCA for this incident will be posted here in due course.

Please accept our apologies for any disruption this caused.

Posted: Wed, 24 Jan 2024 07:42:00 +0000

Update

Our engineers have identified the issue affecting a subset of our customers in relation to logged-in access reverting to SSO logins and are working on a plan to roll the fix out to all affected customers. Due to the nature of the proposed fix, we intend to do this safely and with the least amount of disruption to learners. This may mean a delay in deploying the fix until we can do this safely overnight. We will update this page if this plan changes.

Posted: Tue, 23 Jan 2024 09:09:00 +0000

Update

Our Engineers are continuing to investigate the issue with access via login reverting to SSO access.

Please accept our apologies for any disruption this is continuing to cause today.

Posted: Mon, 22 Jan 2024 14:21:00 +0000

Update

Engineers are continuing to investigate the issue with access via login reverting to SSO access. As soon as we have news to share on this we will update accordingly. Please accept our apologies for any disruption this is continuing to cause today.

Posted: Mon, 22 Jan 2024 12:12:00 +0000

Update

Engineers are continuing to investigate the issue with access via login reverting to SSO access.

As soon as we have news to share on this we will update accordingly.

Please accept our apologies for any disruption this is causing.

Posted: Mon, 22 Jan 2024 11:00:00 +0000

Investigating

Engineers are currently investigating issues with access to Learn sites for a subset of customers. Some customers who use a separate URL to access Learn to bypass Single Sign On, will still be experiencing access issues. We are continuing to work on this as our priority and will update this page with news as and when we have it.

Please accept our apologies for any disruption this is causing.

Posted: Mon, 22 Jan 2024 09:11:00 +0000