Degraded service: Sapling (North America)
» View Event Details | Created
Post-Mortem
RCA: On 28th April 2025, Kallidus engineers performed data correction for US-hosted customers to fix legacy inconsistencies, but this unexpectedly caused issues preventing some users from booking time off. After investigating the problem on 29th April, the team identified the cause and restored the system from a prior backup, placing it into maintenance mode briefly while recovery began. Manual data recovery spanned 15 days due to system usage conflicts, with all affected data fully restored by 14th May 2025.
To mitigate future risks, Kallidus will implement dry-run testing with peer review for multi-user data corrections, avoid bulk customer changes, and enhance rollback and recovery processes for faster issue resolution.
Posted: