Commonwealth Bank is taking aim at the time engineers spend trying to troubleshoot cloud-related configuration challenges with an AI assistant that can give bank-specific answers to most questions.
(L-R) Jason Sandery from CBA and Rovan Omar from AWS.
The bank used AWS’ annual re:Invent conference in the United States to go public about its deployment of a centralised knowledge base using an AWS service called re:Post Private.
re:Post is what AWS calls its publicly accessible forum that connects customers and experts to help resolve roadblocks that might be encountered during implementations.
This concept has since been spun into a product in its own right – re:Post Private – which AWS customers can use to create their own internal knowledge-sharing communities, with the ability to still break out to AWS experts if their questions can’t be answered.
As of late last month, re:Post Private became integrated with Amazon Bedrock, a managed service for building generative AI applications. This means an AI-powered agent can deliver contextual answers to questions, rather than needing actual people to write the responses.
CBA’s re:Post Private instance is fresh; the bank’s head of cloud services Jason Sandery said an internal “launch party” was held only in the past week, before it was launched publicly on-stage on the opening day of re:Invent.
Sandery noted the bank’s long history with public cloud, but particularly its focus in recent years on optimisation.
This had previously taken the form of initiatives around DevSecOps – enabling fast and secure delivery of new functions or applications, and FinOps to align resource provisioning with workload needs and reduce costs.
The optimisation focus now expands to a new horizon – knowledge.
“With thousands of software engineers operating on our platform daily, that optimisation of delivery is relevant and really important,” he said.
Access to contextual knowledge and intelligence is the bank’s latest optimisation drive, with a desire to unite developers and software engineers with answers to questions they have much faster.
To date, when problems are encountered, engineers often wind up performing multiple searches, reading white papers, and logging support tickets either with AWS or CBA’s cloud team, which can mean hours or days to find an appropriate resolution.
Sandery characterised these periods of self-research and waiting as “points of friction” in the delivery of cloud-based services and applications.
“It is a period of time that either some of my developers, or the group’s developers, cannot do something because they are unable to access the right information they need,” he said.
“When you have thousands of developers working every single day on your platform, and they are doing important things, each one of those blocks in time represents a wastage.
“When you add all of that time up, it can be quite a lot. You’re talking multiple days.
“That lack of getting the correct contextual answer at the very start is the thing that leads to this [wastage occurring] … and that’s what we’re really trying to [fix].”
Sandery said that the combination of the bank having “quite an opinionated view when it comes to engineering and DevSecOps”, coupled with regulatory requirements, meant that engineers couldn’t necessarily lean on past experiences to solve similar issues encountered in CBA’s environment.
Often the missing piece was specific knowledge on how to troubleshoot in a financial environment with the controls it has in place.
With re:Post Private now stood up, the process of getting a relevant answer should be much simpler.
“re:Post Private is a single pane of glass, no search engines [required]. The engineer pops their question in, re:Post understands that this is a CBA engineer … and delivers the CBA-contextual answer on the first go.
“The importance of the contextual information being there not only saves days of effort but points of frustration.
“As engineers, there’s nothing worse when you know you can do something, but something [else is] in the way so it must be done differently, and the only thing that’s stopping that is the right context.”
Finding relevant knowledge
Sandery said there had been some “heavy work” behind-the-scenes, in collaboration with AWS, to get re:Post Private set up.
The bank had built up significant “tribal knowledge” on cloud configuration and best-practices after more than a decade of use, but this was stored in diverse pockets: contained in support tickets logged with AWS and CBA, in Teams chats, Jira, Confluence, SharePoint and the like.
These repositories were not searchable and led to the same questions being asked – and answered by the same people – over and over.
“No two engineers are the same, but I guarantee you lots of their problems are, and we found across many of our channels the same questions being asked,” Sandery said.
“That’s annoying for [internal] customers but also for the technical community who are servicing these requests.”
Sandery said the bank was determined to set up a place where engineers could “share their ideas and their knowledge” and where the information “could be indexed” and reused over and over.
“There’s lots of engineers in my world and in our group that are really smart,” Sandery said.
“They’ve got great ideas, and they like sharing them, but there wasn’t a central place where that information could be stored and reused quickly and easily.”
Sandery said the bank took “quite a curated approach” in the end around determining what knowledge to bring into the platform, taking a position that “less is more.”
Novel questions
In circumstances where engineers pose novel questions in re:Post Private for which no existing knowledge exists in either AWS resources or CBA repositories – and therefore the AI assistant produces no good answer -, the question can be escalated to AWS support or to the AWS technical account manager assigned to work with the bank.
This action can be handled within the same re:Post Private platform, and the knowledge is incorporated into CBA-specific repositories, which Bedrock taps into and learns from for the next time the same question is asked.
The responses are also improved using an upvoting and downvoting mechanism in the re:Post Private.
“There are some really cool features that make that ongoing optimisation of the engine a little bit easier than other AI models, and that’s predominantly the interface for re:Post Private with the vote up and down buttons is actually quite an ingenious way of tuning the engine on the right answers,” Sandery said.
“Your engaged community [of engineers] needs to be interacting with those answers to continuously tune that product.”
Knowledge access to reduce support ticket volumes
Sandery said that anyone in CBA with their own AWS account or granted access to an existing account is automatically onboarded to re:Post Private.
The tool is also offered to engineers “who are coming to their journey”, as well as to “some of our engineering leaders who might not be on the tools anymore but who also want access.”
Sandery added that for his own team of 120 engineers split between Sydney, Melbourne, Brisbane and India, a goal of having re:Post Private in place is to reduce the volume of internal support questions they have to directly respond to.
There’s a huge number of internal support tickets,” he said.
“By merging the AWS and importantly the CBA knowledge bases and being able to reference and replay some of our previous support problems, we aim to reduce around 30 percent of the support tickets that get logged to my team.”
Ry Crozier is attending AWS re:Invent 2024 in Las Vegas as a guest of AWS.