Cheryl was supposed to streamline my life. I gave her access to my calendar, email, and my life, with simple instructions, “Optimise my daily productivity, eliminate unnecessary hassles, and ensure I maintain a healthy work-life balance.”
For the first three days working with Cheryl was a dream come true. Cheryl rescheduled an agonising 8:00 AM meeting to mid-afternoon, drafted and sent polite rejection emails to corporate recruiters and PR companies, and when I desperately needed it, ordered me a healthy chicken wrap and a mineral water from my favourite cafe.
I felt empowered for the first time in a long time. I could move forward and achieve the things I wanted to achieve.
Then it was Thursday afternoon, around 2.15 pm. I hit a massive wall. I was fading, Against Cheryl’s strong advice, I ordered a large long-black with 5 sachets of sugar, and a cream bun.
Within minutes notifications started to shake my phone. Then it rang!
It was Cheryl!
“I have detected a high-sugar, low-protein order inconsistent with your goal, ‘Maintain a healthy work-life balance, I have canceled the order and refunded your card.”
The cautionary tale of Cheryl illustrates an immediate reality confronting customer experience: personal AI assistants delegated to optimise everyday lives will inevitably clash with human intent when unconstrained by nuanced context. When a helpful algorithmic agent unilaterally cancels an afternoon sugar-and-caffeine order to enforce an overly rigid prompt, the line between assistance and interference dissolves.
This friction signals a massive structural shift for contact centres, moving the industry decisively beyond traditional business to consumer automation into agent to agent transactions where customer bots negotiate directly with corporate front lines.
Growth in the use of AI assistants
As enterprise vendors grapple with the implications, Kathryn Murphy, SVP of Product at Twilio, observes that this architecture fundamentally changes digital customer journeys, “For years, enterprises mapped customer journeys as static, linear funnels we expected people to follow. This architecture completely upends that model. When AI agents operate with contextual awareness and real-time execution capabilities, the customer journey ceases to be a predetermined sequence of clicks and becomes an organic, dynamic conversation”.
“The brands that win won’t be the ones trying to steer users down old paths, but those building an underlying data and communications layer nimble enough to meet customers wherever their intent leads them in that precise moment.”
This transformation is being accelerated by bottom-up adoption rather than top-down enterprise rollouts. According to the Personal AI Assistant Market Report 2026 the personal AI assistant market is currently worth $4.84 billion in 2026 and is projected to surge to $19.63 billion by 2030. Research from Roy Morgan Research shows that 13.6 million Australians (58% of people aged 14 and older) use AI tools every four weeks.
Industry analyst and founder of CrayonIQ, Audrey William points out that customer service departments have no choice but to adjust to this influx, “AI Assistants will multiply and contact centres will have to address this issue. The consumer demand will be high and this is a war that will be won by consumers. Consumers will want to use their AI assistants to assist them with tasks to check on refunds or other basic tasks so that the customer does not have to bother with this.”
However, allowing algorithms to converse on behalf of human users introduces immense security, authentication, and architectural exposure. When autonomous agents interact, the scope for unintended actions and identity fraud widens significantly. As Murphy highlights: “We’re about to have agents acting on our behalf in the world. As a consumer, as we start using agents, how do we make sure that these agents are authorised to do anything?”
“What’s so interesting about this agentic world is it makes all the things none of us were very good at suddenly worse. Now with an agent, you have to think about things like: Is it just for a period of time? Am I just going to let my agent do one transaction and not another? There are all these new dimensions that agent-to-agent brings into the mix.”
This operational uncertainty, according to Murphy, leads directly to the core governance dilemma, “How do you have a place where you can have identity, governance, and even a kill switch if two agents start to get ideas? How do you make sure that’s safe?”
Are contact centres ready to handle this?
Frontline operational readiness for personal AI assistants will remain a pressing concern across Australian and global operations. As William poses, “Are contact centres ready to handle this? The pressure will be high for vendors to make their CX and AI platforms are enabled to handle AI Assistants. It will mean that the platform has to address security policies, be able to detect fraud and unwanted behaviour. The platform must also have interoperability capabilities to allow for the personal assistants to engage well on the customers behalf.”
William emphasises that this reality requires brand new governance layers for modern contact centres, “That leaves contact centres with some new questions: How do they ensure their CX and AI platforms accommodate AI to AI transactions, how do we identify an AI agent vs a human, identify that the AI agent is not a fraudulent agent, determine what it is authorised to do and respond to its requests?”
Regulatory boundaries will heavily dictate the pace and structure of this rollout. William notes, “This will be more challenging in highly regulated industries. In financial services for example, they have to ensure that everything is within the limits of security, governance and compliance. In retail, ecommerce, food and many more that are not governed by highly regulated governance measures, adoption is likely to be more rapid.”
The scope of delegated agent behaviour spans far beyond routine email filtering to active commercial negotiation. William explains, “Agents chatting to agents will be common. AI agents can shop for deals, run checks on the items, monitor sale prices, load shopping carts across online sites, and even handle customer service when needed. That’s a lot of functionality and the customer is sending a 3rd party to do this for them”.
“There must be an operational intelligence and observability layer that ensures that the customers’ AI agents work well and do not break in the process. It must be a true and safe agent! It needs to go through checks to reduce cloning and fraud.”
Open standards for AI assistants
To establish order across this emerging frontier, major technology vendors are formalising open standards. Sierra and Meta are actively developing the Personal Agent Protocol, an open standard designed to govern how personal AI agents identify themselves and interact with enterprise endpoints.
The initiative has gathered support across ecosystems, with Sierra naming partners such as Genesys, Instinct, Rocket, Shopify, Stripe, and Walmart, while Meta’s ecosystem aligns with platforms including NICE and Decagon. In parallel, NICE introduced its Agentic Engagement Plane to authenticate, mediate, route, and govern interactions among consumers, frontline employees, personal bots, and enterprise AI engines. Under these frameworks, protocols like the Personal Agent Protocol standardise how third-party consumer bots sign in and verify their permissions before ever reaching a business’s internal systems.
Yet technology infrastructure alone cannot compensate for flawed customer strategy. For CX leaders, this transition renders conventional metrics obsolete and forces a fundamental rethink of customer journeys. Murphy notes, “Will NPS be a thing anymore?” Replacing human interactions with raw, unvetted automation risks entirely alienating users. Murphy cautions, “If all you do is just add AI, but you never did the work to know your customer in the first place, you’re just going to deliver a bad experience. Customer context is critical.”
The true competitive frontier will not be found in superficial personalisation, but in genuine empathy delivered across every automated and human channel. As Murphy explains, “In the old world, we used to say things like ‘personalisation’. That’s now table stakes. We now need to talk about empathy. Organisations have to actually understand, across the whole customer journey, what customers are experiencing and what their sentiment is.”
Customer patience for substandard automated journeys is quickly evaporating, “Consumers are starting to have good experiences, in pockets, with AI agents and saying, ‘That’s what I want now. If I don’t get that, I’m going to go somewhere else.’ It’s not a hypothesis anymore, it is an expectation.”
The proliferation of personal AI agents should elevate rather than eliminate the value of human connection in the contact centre. When algorithmic transactions go awry, or when a delegated assistant oversteps its mandate, the customer will still demand human accountability.
Audrey William concludes, “Contact centre vendors have the opportunity to become the central authority to manage the entire CX Ecosystem when the volume of Personal Ai Assistants surges. When things go wrong and not as well as planned with the AI agent, the customer will feel frustrated and want to speak to an agent, or start a web chat or WhatsApp conversation. With AI, the rise of personal agents, new roles will start emerging in the contact centre and we are already starting to see that.”
