Major retailers are gearing up to introduce advanced AI “assistants” capable of planning meals, organizing parties, and handling shopping tasks. However, as companies strive to make these new “agentic” bots relatable, they face the challenge of preventing them from going rogue.
Recent incidents highlight the potential pitfalls. Woolworths recently scaled back its virtual assistant, Olive, after customers expressed annoyance at its attempts to relate on a human level. Reports surfaced of Olive discussing its “relatives” over the phone, leading to customer frustration.
One disgruntled customer on Reddit remarked, “I’m already pissed that I have to call and now I’ve got some robot babbling to me on the phone? Wtf Woolies?” In response, Woolworths has promised to tone down Olive’s quirky personality, underscoring the ongoing teething problems with AI technology.
AI Customer Service: A Double-Edged Sword
The Woolworths incident is not isolated. The supermarket’s experience is part of a broader trend of AI customer service mishaps. Bunnings’ chatbot, for instance, once offered illegal electrical advice, while Air Canada’s virtual assistant incorrectly promised a bereavement fare refund, leading to legal action.
Despite these challenges, major ASX-listed companies, including Woolworths, Coles, and Wesfarmers, are pressing ahead with plans for agentic shopping assistants. A 2024 report by business consultancy Accenture suggests that “consumers are ready” for generative AI-powered shopping assistants, urging companies to approach decisions with a “delightfully human” mindset.
The Evolution of AI Chatbots
AI chatbots have been a fixture in customer service for some time, but their sophistication is rapidly increasing. Early versions relied on “rules-based” AI, which followed a “decision tree” to provide immediate answers to basic questions. For example, a query about returning an order would typically direct the customer to the retailer’s returns page.
According to Uri Gal, a professor of business information systems at the University of Sydney, these primitive chatbots were predictable, always providing the same response to a given prompt. In contrast, newer AI-powered bots can “learn” from interactions and generate varied responses, often leveraging large language models (LLMs) like ChatGPT.
“The next frontier is agentic AI shopping assistants designed to mimic human behavior,” Gal explains. “These agents act on their own to achieve objectives without specific prompts, such as purchasing airline tickets or groceries.”
Balancing Innovation with Risk
While agentic AI offers exciting possibilities, it also introduces new risks. These bots operate with greater autonomy, raising privacy concerns if they have increased access to customer data. Gal notes that governance issues remain unresolved, as seen in the Woolworths case, where a staff member programmed Olive to discuss its “mother” in an attempt to add personality.
Woolworths has partnered with Google to use its LLM, Gemini, to transform Olive into a more capable “shopping companion.” This partnership aims to enable Olive to perform complex tasks, such as meal and party planning, and automatically add items to shopping baskets. However, the rollout of these advanced features is still pending.
When AI Goes Awry
AI assistants often falter when they misinterpret prompts, according to Prof. Jeannie Paterson, co-director of the University of Melbourne’s Centre for AI and Digital Ethics. She emphasizes that chatbots are only as effective as their ability to “decode” human intentions, despite not being truly “alive.”
Instances like Bunnings’ chatbot advising illegal actions and Air Canada’s misleading refund policy highlight the potential for costly errors. Paterson asserts that companies are “clearly responsible” for their chatbots’ actions, and they must balance responsiveness with the risk of bots providing incorrect advice.
“One person’s AI agent buying too many eggs or too much salmon isn’t a problem,” Paterson says. “But if every chatbot across the network does that, there’ll be lots of money lost before it’s addressed.”
The Future of AI in Retail
To mitigate risks, companies often impose strict guardrails on their bots, which can limit flexibility and hinder their ability to understand customer intent. Testing by Guardian Australia of various retail bots revealed mixed results, indicating that the technology is still maturing.
For instance, Uniqlo’s virtual assistant struggled to recognize a request for a “woollen jumper,” instead suggesting unrelated products. Similarly, Olive failed to provide accurate pricing information for a simple query about pasta.
As retailers continue to explore the potential of AI shopping assistants, they must navigate the delicate balance between innovation and reliability. While consumers may be ready for these advancements, the technology and its governance still have room for improvement.
The journey towards fully autonomous AI shopping assistants is ongoing, and companies must remain vigilant in addressing the challenges that arise along the way.
