Designed to deceive: How AI is changing the dark patterns of digital commerce
JIRAPORN SURACHARTKUMTONKUN, XI WEN CHAN, ANDREAS CHAI AND ALASTAIR SMITH |
Online shopping has never been easier. A few clicks can take us from browsing to buying, often before we have had time to consider whether we really want — or can afford — what we are purchasing.
But increasingly sophisticated digital design can actually make that decision harder than you might think.
Known as dark patterns, manipulative interface designs are deliberately structured to steer people towards choices they might not otherwise make. They include hidden cancellation buttons, pre-ticked boxes, misleading scarcity messages, unexpected costs and frictionless payment processes.
The problem is widespread. Research has identified dark patterns on more than 10 per cent of thousands of e-commerce websites, while Australian research found that 83 per cent of consumers had experienced negative effects from manipulative digital designs. A quarter reported sharing more personal information than intended, while one in five spent more than they had planned.
For younger Australians, the risks can be particularly significant.
Why young consumers are vulnerable
Early adulthood is often the first time people independently manage subscriptions, digital payments and buy-now-pay-later services. At the same time, they are among the most active users of digital platforms.
This combination of high digital engagement and limited financial experience can create a significant vulnerability. Urgency cues, default settings and frictionless payment systems can encourage people to prioritise an immediate reward over longer-term consequences.
Artificial intelligence is now making these techniques more sophisticated.
AI supercharges digital manipulation
Generative AI can produce convincing product descriptions, images and testimonials at a fraction of the previous cost. Automated systems can also continually test different designs, wording and pricing cues to determine what is most likely to generate a response.
This means digital interfaces are no longer necessarily static. They can be continually adjusted according to users’ behaviour.
The same technologies can also make fraudulent operations harder to detect. Australian regulators have warned about so-called “ghost stores” — websites designed to look like legitimate retailers but operating as scams. AI-generated images and automated content can allow operators to rapidly create convincing storefronts and rebrand when they are detected.
The distinction between aggressive marketing, manipulation and outright fraud is therefore becoming increasingly difficult to police.
Is Australia’s regulatory framework keeping pace?
Australia currently relies largely on broad provisions of the Australian Consumer Law to address misleading or unconscionable conduct. In late 2024, Treasury proposed a general prohibition on unfair trading practices, alongside specific measures targeting subscription traps, drip pricing and mandatory account creation. These reforms have not yet been enacted.
International approaches provide potential models. The European Union’s Digital Services Act prohibits certain manipulative interfaces, while the UK has introduced powers targeting harmful online choice architecture. In the United States, efforts to make subscription cancellation as easy as sign-up have also highlighted the growing focus on design itself.
For Australia, this suggests a shift may be needed from responding to consumer harm after it occurs towards examining potentially manipulative designs before they are deployed.
Using AI to fight AI-enabled manipulation
Technology could also form part of the solution.
Researchers have developed systems capable of analysing website structures, identifying visual features such as countdown timers and detecting manipulative language. Other tools can simulate interactions within mobile apps to identify problematic patterns in real time.
For Australian regulators, these technologies could support more continuous monitoring of high-risk digital environments and help identify emerging patterns before they affect large numbers of consumers.
AI-enabled dark patterns sit at the intersection of consumer protection, financial crime and youth vulnerability. As artificial intelligence makes manipulation cheaper, faster and more targeted, Australia faces a growing governance challenge: ensuring that digital environments designed to influence our choices do not quietly undermine our ability to make those choices freely.
The question is no longer simply whether consumers are being persuaded online, but how much of that persuasion is being engineered — and whether our regulatory systems can keep pace.
Dr Jiraporn (Nui) Surachartkumtonkun, Associate Professor Xi Wen (Carys) Chan, Professor Andreas Chai and Alastair Smith are members of the Academy of Excellence in Financial Crime Investigation and Compliance. Andreas is Director of the Academy, while he and Nui are also members of the Griffith Asia Institute.
This article draws on research presented in the forthcoming book chapter, “Designed to Deceive: AI-Enabled Dark Patterns, Youth Vulnerability, and the Governance of Australia’s Digital Economy”, to be published in the Handbook of Public Governance and Artificial Intelligence by De Gruyter.