A recent major day-long outage at a telecommunications company could have been fixed “within minutes” with automated change management tools, according to a security expert who also sees generative AI dramatically improving the accessibility and proactivity of such systems.
Recent “high profile failures in Australia have achieved global attention,” David Link, CEO of performance analytics specialist ScienceLogic, told iTnews as the federal government kicked off a post-incident review of the November 8 incident – which disconnected millions of individuals and left businesses unable to carry out basic functions.
Investigations involving nearly 400 technical staff ultimately revealed the cause as a router software upgrade – a problem that was echoed during a recent outage at Westpac that was also attributed to a “routine technology update”.
“We know that about 70 percent of outages are because something changed,” Link explained, spruiking the merits of the firm’s AIOps platform – which has proven popular across Australian critical infrastructure operators including utilities, telecommunications providers, government agencies, and other systems “that just can’t fail.”
Monitoring key systems for changes has become more complicated in a digital transformation era where core applications now span multiple cloud and on-premises systems – which, Link explained, is why effectively proactive monitoring those platforms requires extensive investment in change detection and alert automation.
“It’s never fun to be in the middle of that kind of storm,” he said, “but if you have the right tool and proactive procedures, you can fix the problem swiftly with great confidence. There was recently a fairly high-profile outage in Australia that our product could have helped prevent and restore service within minutes.”
Integration of generative AI into the platform has proven to be an invaluable complement to existing analytics-driven change detection, Link said, providing “the best of all worlds” by providing a unified view of machine-generated content – such as log files, performance analytics, and configuration data – as well as human-generated support and incident related information such as bug databases, chat notes, and records of service resolution workflows.
“Combine that with what’s happening in real time, and merge them together to get the best remediation recommendation and ultimately faster outage resolution outcome for that moment in time.”