Resolved
Resolved

We've now resolved the incident. Thank you for your patience.

Avatar for

All services are available and all metrics are green. We are double-checking the nightly loads and will then progress to close the incident.

Avatar for

Our current status is all services are available. We are still monitoring closely.
We see still a small backlog of Data Jobs being processed by the platform. This may delay single Data Jobs for a little bit more than usual.

Avatar for

Monitoring is continuing. All services are looking healthy.

Avatar for

All services are looking good. We'll keep monitoring for some time more.

Avatar for

We keep on monitoring the services. All services are looking good.

Avatar for

The services are fully available again. We are closely monitoring the infrastructure.

Avatar for

We are currently bringing the remaining frontend services back into their target state. Therefore the frontend may show slower performance or some requests failing.

Avatar for

The system is fully available. We are closely observing the processing of the backlog for the next hours.

Avatar for

The system is fully available and working through the backlog. We are closely monitoring and will be back soon with another update.

Avatar for
Recovering

Based on our monitoring, error rates and latency have now recovered. We will continue to observe the overall platform health and keep you updated.

Avatar for

We are still investigating closely all the issues and share another status update soon.

Avatar for

The performance of the application may still be degraded for some services. We are continuing our investigation of the issues.

Avatar for
Investigating

We are again seeing increased failure rates in one of the service. We will share another update once we have more details

We have adjusted database pool size parameters and are continuing monitoring. We are still seeing occasional long-running requests which we are investigating.

Avatar for

We have successfully scaled the components and are continuing to monitor the application metrics. We will provide another update in roughly one hour.

Avatar for
Recovering

We are currently scaling up various components throughout the platform. Some customers might experience slightly elevated error rates during this operation.

Avatar for

We have rolled out changes to the database connectivity layer for multiple core services. We will monitor whether this has the desired effect and will update you as soon as we have results.

Avatar for

We are continuing our investigation of the issues.

Avatar for

We have just restored the Process Analytics service. We are in contact with the cloud provider to further investigate the root cause of the database issues we encountered today. We will continue to update you as we make progress.

Avatar for

We don't have any further updates to share so far. We are still working on Process Analytics.

Avatar for

Action Engine is available again. We are still working on restoring Process Analytics.

Avatar for

We have resolved the overall performance issues and are continuing work on Process Analytics and Action Engine.

Avatar for

We are performing emergency maintenance on the Process Analytics and Action Engine services and have therefore taken them offline temporarily. We will provide another update in about 30 minutes.

Avatar for
Identified

We are still working on resolving the issues. We will provide another update in 30 minutes.

Avatar for
Recovering

We have applied another mitigation and are closely monitoring recovery.

Avatar for

We are still investigating the root cause of the issues.

Avatar for
Investigating

Unfortunately, the issues have resurfaced and we are continuing our investigation

Avatar for
Recovering

We have mitigated the underlying issue and are waiting for everything to fully recover.

Avatar for
Identified

We have identified a potential root cause and are applying mitigations to improve the performance and reduce the error rate.

Avatar for
Investigating

We are currently investigating degraded performance on the EU-3 realm.

Avatar for
Began at:

Affected components
  • EU-3