Sign-in Issue Identified | July 11, 2023
Completed
CivicPlus Systems Engineers have identified an issue related to a recent security enhancement rollout that may be effecting your ability to sign-in to your CivicPlus solution. Customer Support is standing by and able to quickly remediate the issue.
If you are experiencing issues with sign-in please submit a request through the CivicPlus Help Center.
Comments
3 comments
-
Official comment
Incident Report
On July 11, 2023 at 4:00pm CST, CivicPlus Engineers completed a release rollout to the CivicPlus Authentication platform which included technical security enhancements to improve the security posture of our single sign-on service, as part of our ongoing maintenance and security efforts. The security enhancements had been tested in multiple development environments across all CivicPlus products leveraging single sign-on over the course of the prior eight weeks.
At 4:06pm CST, CivicPlus Customer Support began receiving reports that users were receiving an error message when attempting to sign in to Web Central websites. Customer Support was able to identify and implement a fix to immediately resolve the issue for reporting customers. Once a support ticket was received, sign-in was restored within minutes.
At 4:09pm CST, CivicPlus Engineering was notified by Customer Support of the increase in volume of incoming reports of sign-in issues. Engineers immediately began a root cause analysis and identified a configuration cache was not automatically updating to the newer platform code, resulting in the sign-in error. Customer Support continued to resolve customer reports by manually clearing the configuration cache on each site while engineers worked on a broader resolution.
At 5:23pm CST, CivicPlus Engineers completed the development of an automated resolution to clear site caches across all sites in Pacific and Mountain time zones. The script was deliberately implemented to execute across an extended period of time to mitigate server overload which could have lead to site performance issues. The resolution automation completed at 6:27pm CST.
As a result of this incident, CivicPlus Engineers plan to review and address caching priorities and redundancies of configuration data related to shared services, such as single sign-on.
-
CivicPlus Engineers have applied a fix and we are validating that service has been restored to your CivicPlus Solution. We will continue monitoring the Solution through tomorrow morning to ensure the fix addresses the issue for all impacted customers. An update will be provided once we are confident this issue has been addressed.
0 -
CivicPlus Systems Engineers have applied a fix and we have verified that service has been restored to your CivicPlus Solution. An incident report will be posted here within 24 business hours.
0
Please sign in to leave a comment.
Available Templates
No Change Update
CivicPlus Systems Engineers continue to work on resolving this issue as quickly as possible. An update will be added to this post when it is available.Timeline Identified
CivicPlus Systems Engineers have identified the issue and expect to have a resolution deployed by ESTIMATED TIMELINE (if not a specific date, include business hours / business days clarification).Fix Implemented
CivicPlus Systems Engineers have applied a fix and have restored service to your CivicPlus Solution. An incident report will be posted here within 24 business hours.