Service Disruption Identified | July 18, 2024
CivicPlus monitoring has detected a possible performance issue that may be affecting your CivicPlus solution. CivicPlus Systems Engineers are investigating this alert and a timeline to resolution and next steps will be provided as soon as available.
Comments
9 comments
-
Official comment
CivicPlus Systems Engineers have applied a fix and have restored service to your CivicPlus Solution. An incident report will be posted here within 24 business hours.
-
CivicPlus Systems Engineers have identified a service disruption with Microsoft Azure Central US region that is causing users to be unable to sign-in to and/or load CivicPlus solutions. We will continue to update this post as more information is available.
You can also monitor the outage status directly through the Microsoft Azure Status Page.
1 -
CivicPlus Systems Engineers continue to monitor a service disruption with Microsoft Azure Central US region that is affecting CivicPlus solutions.
Microsoft has shared the following information:
Impact Statement: Starting at approximately 21:56 UTC on 18 Jul 2024, a subset of customers may experience issues with multiple Azure services in the Central US region including failures with service management operations and connectivity or availability of services.
Current Status: We are aware of this issue and are actively investigating. The next update will be provided in 60 minutes, or as events warrant.
This message was last updated at 23:17 UTC on 18 July 2024
An update will be added to this post when it is available. You can continue to monitor the outage status directly through the Microsoft Azure Status Page.
0 -
CivicPlus Systems Engineers continue to monitor a service disruption with Microsoft Azure Central US region that is affecting CivicPlus solutions.
Microsoft has shared the following additional information:
Impact Statement: Starting at 21:56 UTC on 18 Jul 2024, a subset of customers may experience issues with multiple Azure services in the Central US region including failures with service management operations and connectivity or availability of services.
Current Status: We are aware of this issue and have engaged multiple teams to investigate. As part of the investigation, we are reviewing previous deployments, and are running other workstreams to investigate for an underlying cause. The next update will be provided in 60 minutes, or as events warrant.
Customers with disaster recovery procedures set up can consider taking steps to failover their services to another region https://learn.microsoft.com/azure/architecture/resiliency/recovery-loss-azure-region
This message was last updated at 23:45 UTC on 18 July 2024
An update will be added to this post when it is available. You can continue to monitor the outage status directly through the Microsoft Azure Status Page.
0 -
CivicPlus Systems Engineers continue to monitor a service disruption with Microsoft Azure Central US region that is affecting CivicPlus solutions.
Microsoft has shared the following additional information:
Impact Statement: Starting at approximately 21:56 UTC on 18 Jul 2024, a subset of customers may experience issues with multiple Azure services in the Central US region including failures with service management operations and connectivity or availability of services.
Current Status: We are aware of this issue and have engaged multiple teams to investigate. As part of the investigation, we have reviewed previous deployments. We’ve determined the underlying cause and are currently applying mitigation through multiple workstreams. The next update will be provided in 60 minutes, or as events warrant.
Customers with disaster recovery procedures set up can consider trying to take steps to failover their services to another regions, and may consider using programmatic options for this if they experience issues.
https://learn.microsoft.com/azure/architecture/resiliency/recovery-loss-azure-region .
This message was last updated at 01:19 UTC on 19 July 2024
An update will be added to this post when it is available. You can continue to monitor the outage status directly through the Microsoft Azure Status Page.
0 -
CivicPlus Systems Engineers continue to monitor a service disruption with Microsoft Azure Central US region that is affecting CivicPlus solutions.
Microsoft has shared the following additional information:
We’ve determined the underlying cause and are currently applying mitigation. We will start to see incremental recovery in next 90 minutes.
An update will be added to this post when it is available. You can continue to monitor the outage status directly through the Microsoft Azure Status Page.
0 -
CivicPlus Systems Engineers continue to monitor a service disruption with Microsoft Azure Central US region that is affecting CivicPlus solutions.
Microsoft has shared the following additional information:
We are currently applying mitigation. Customers should see signs of recovery at this time as mitigation applies across resources in the region.
An update will be added to this post when it is available. You can continue to monitor the outage status directly through the Microsoft Azure Status Page.
0 -
CivicPlus Systems Engineers continue to work on resolving this issue as quickly as possible. An update will be added to this post when it is available.
0 -
CivicPlus Authentication Performance Issue - July 18, 2024
Incident Description
At 5:32 PM CDT, CivicPlus became aware of a Microsoft Azure outage that impacted CivicPlus's Platform services. The CivicPlus Product Engineering team began investigating the issue immediately and determined that the primary cause was an international Microsoft Azure outage. CivicPlus's Product teams confirmed that our Platform services had returned to normal functionality at approximately 7:32 AM CDT on July 19th, 2024, once the US Microsoft Azure outage was resolved.Cause
CivicPlus Platform services were affected by an international Microsoft Azure outage. CivicPlus Platform utilizes Microsoft Azure for hosting. You can read more about the incident here: https://azure.status.microsoft/en-us/status/history/.Resolution
According to Microsoft, Central US Microsoft Azure services regained functionality at approximately 7:15 AM CDT. CivicPlus Platform services returned to normal expected functionality shortly thereafter.Issue Status: Resolved - no further updates should be expected.
0
Please sign in to leave a comment.
Available Templates
No Change Update
CivicPlus Systems Engineers continue to work on resolving this issue as quickly as possible. An update will be added to this post when it is available.Timeline Identified
CivicPlus Systems Engineers have identified the issue and expect to have a resolution deployed by ESTIMATED TIMELINE (if not a specific date, include business hours / business days clarification).Fix Implemented
CivicPlus Systems Engineers have applied a fix and have restored service to your CivicPlus Solution. An incident report will be posted here within 24 business hours.