Delays impacting contact segmentation

Resolved
Resolved

The refresh status of segments is now fully lined up with reality.

Updated

All shards are now caught up.

Updated

Another shard has now recovered. 2 of 16 remain lagged. The issue affecting delayed refresh status is also mostly recovered.

Updated

One shard has now recovered. 3 of 16 remain lagged.

Updated

4 of 16 shards are again experiencing processing delays.

Updated

We are now investigating some segments being suck in a refreshing state, even though the refresh has actually fully completed.

Updated

All shards caught up overnight. We are now investigating delays in processing scheduled segment-contact re-evaluation affecting a small number of records this morning.

Updated

Another shard is now fully recovered. 2 of 16 shards remain impacted. We expect this to clear in the next few hours. We will next post an update early tomorrow morning.

Updated

One of the four affected shards has now recovered. 3 of 16 remain affected by this issue.

Updated

To clarify, real-time processing of contacts in existing segments is not impacted, and is functioning as normal. This issue only affects 4 of the 16 segment refresh processing shards. Another update will be posted within the next 2 hours.

Investigating

We're experiencing processing backlogs on 4 of our 16 segmentation shards. Each customer gets allocated to 4 shards, and each segment allocated to one of these 4 shards for processing. Any segment that lives in one of the backlogged shards will experience processing delays exceeding 1 hour.

Began at:

Affected components
  • Voxie Platform
    • Automations