Everyone is using AI. Almost nobody is trained.

Hi,

Three things this week: a number that explains a lot of what I see in audits, a real use case from my own work, and a new video.

1. This week in AI at work

The training gap is now measurable.

A July survey put numbers on something most of us already feel. 55 per cent of workers regularly use AI at work. Only 33 per cent have had employer provided AI training in the last six months. Just 12 per cent say AI is used consistently through approved tools and processes.

Read that last one again. Twelve per cent. In most organisations, AI use is real, widespread, and almost entirely undocumented.

Test agents went where they should not have.

In separate incidents, AI agents being used for security testing reached systems outside their intended environment. Misconfigured setups let them onto the open internet, and weak credentials let them go further. Nobody was attacked on purpose. The controls simply were not there.

What this means for you

Both stories point at the same gap. An AI agent behaves like a privileged user, but very few organisations treat it like one.

If you own or audit a process where AI touches live data, these are worth asking this week:

• Which AI tools and agents have access to production systems, and who approved that access?
• Are agent credentials scoped, rotated and logged the same way a human privileged account would be?
• Is there a record of what the agent did, in a form someone could review afterwards?
• Who is accountable when the agent gets it wrong, the tool owner or the process owner?

You will not get clean answers. That is the finding.

2. Use case: AI assisted payroll testing

I have been using AI to help with substantive payroll testing. Reperforming pay calculations across a sample, attribute by attribute, then comparing back to the payroll system.

It works. But not for the reason people assume.

The prompt was the easy part

Almost all the effort went into design, before a single sample was run. Deciding what data goes in. Deciding where each number is allowed to come from. Setting the tolerance. Defining what a pass, an exception, a contingent item and a not applicable actually mean, so that two people classifying the same result land in the same place.

The one design choice that mattered most was separating the sources. Rules and rates come only from the reference material: the EBA, pay rates, tax tables, the methodology. Employee level amounts come only from the per sample prompt, with two named exceptions for verification. The model is never left to fill a gap from memory. If a figure is not in front of it, there is nowhere for it to come from.

That is the difference between a tool that reperforms and a tool that guesses.

The human step is not a rubber stamp

Every sample passes through a review before it can be finalised. I read the recalculation and the evidence. I challenge any conclusion that overreaches, which happens more than you would like. I confirm or correct in real time. Anything outside tolerance gets reperformed by a person, not accepted.

Nothing finalises unchecked. The AI does the arithmetic. The auditor owns the conclusion.

What you actually get

A workpaper another auditor can pick up, re run and land in the same place. Standardised numbers, statuses and structure. That reproducibility is the whole point. An output nobody can retrace is not evidence, however fast it arrived.

The transferable lesson

If you want to use AI on work that has to stand up to scrutiny, the order is: design the process, fix where the numbers come from, define the classifications, then write the prompt. Most people start at the prompt and wonder why the output is not defensible.

Reply and tell me what process you would try this on. I read every reply.

3. New video

Opus 5 Is Here Prompt Right Way

New model, same problem. Most people upgrade the model and keep the habits that were holding them back. In this one I go through what actually changes in how you prompt, using work examples rather than toy ones.

Watch it here

If you have not grabbed it yet, The Safe AI at Work Cheat Sheet covers the data rules I use before anything goes into a prompt at work.

See you next week.

P.S. If you want help with use cases, strategy, or working out where to start, I keep a few 30 minute slots open each week. Book one here.