The problem: the practice system answers the wrong question
Authorised accounting firms in Norway are subject to quality control by Regnskap Norge, the industry body. The inspection is document-based: the firm completes a control form and submits the documentation the inspector asks for.
The form covers ten control areas across two levels. The firm level asks about the practice's own procedures: risk assessment, contingency plan, anti-money-laundering routine, quality management system, continuing education. The engagement level asks about each individual client: engagement agreement, authorisations, business understanding, quality assurance of the trial balance, annual accounts and tax return with submission receipt.
Before the inspection, someone has to answer every point. For every engagement. In a firm with a hundred clients that's weeks of work, and it's the kind of work where you find what you're looking for and miss what you didn't think of.
The obvious move is to ask the practice system. PowerOffice Quality, where Norwegian accounting firms keep their quality documentation, knows which tasks are ticked off as complete, by whom and when.
That's where the gap sits. A ticked-off task says someone believed they were finished. The inspector asks something else: does the document exist, and does its content meet the requirement?
The first version of our system checked completion status in PowerOffice. The client told us that wasn't good enough, and they were right. It became the absolute constraint the rest of the system is built around: a ticked-off task in Quality cannot satisfy a control point. Not partially, not as supporting evidence, not in combination with anything else.
Version two reads the documents instead.
What the system does
It takes the documentation for one control subject, either a client or the firm itself, and answers one question per control point: does the documentation the requirement asks for exist, is it valid on the inspection date, and does its content hold up against the legal requirement?
The answer is one of five: satisfied, deviation, deficiency, requires manual review, or not applicable.
Out comes a deviation report with legal basis, quotation and page number for each finding, a portfolio overview across all engagements, and the structured findings as a file. The report you read is a projection of those structured findings, never the other way round. That's why you can ask why a point was flagged and get the evidence reference back, rather than a guess based on the report text.
How it works
Intake sees a set of files for one subject. A ZIP, a folder, individual files or a direct lookup against Quality all go in the same way and are treated alike.
The first version took ZIPs the client downloaded themselves. That was deliberate: we didn't want to build an integration before we knew the inspection logic held. The system now pulls documents straight from the quality module. It's read-only access, and the system never writes anything back.
The first set we received was a flat dump. No folders, no metadata, and roughly half the filenames were random character strings: one such file turned out to be a VAT receipt from the Norwegian Tax Administration. Every file also carried the same date, namely the day someone downloaded them from the practice system.
That settled two design decisions. Documents are identified by content, not filename. And dates are read out of the text inside the document, not the file system.
From there it runs in fixed steps. Text and dates are extracted. Documents are classified against a closed type list, first with signature rules and then with a language model for the remainder. Control points are bound to the documents that matter. Then the deterministic rules run: exists, correct period, age, deadline, consistency, scope.
Those rules are the gate. The language model runs only after they have passed, and only on points that require a judgment about content.
Everything is evaluated against a stated inspection date, never against today's date. That makes runs reproducible, and lets the firm simulate the inspection before it happens.
What the system is not permitted to do
It's easy to write in a prompt that the language model should be careful. It's another thing to make it impossible for it not to be. These aren't guidelines in a document. They're built into the code and locked by more than 800 automated tests.
It can never upgrade a finding. The language model can confirm or downgrade, never turn a deficiency into satisfied. The system can convict, never acquit. The asymmetry is deliberate: a machine that can acquit is a machine that can be wrong in the expensive direction.
It cannot assert anything without evidence. Every finding points to a named rule, or to a quotation with a page number from a specific document. An assessment without a valid quotation is automatically downgraded to requires manual review.
It doesn't claim absence until everything has been read. If there are documents the system couldn't read, it refuses to claim anything is missing, and the run exits with an error code. An inspection over a half-read set should not look like a clean bill of health.
What we've run it against
The system was built together with an authorised accounting firm facing document-based inspection this autumn. This isn't a pilot with nothing at stake. They had a real need for the product, and the reports have to hold up in front of a real inspector in a matter of weeks.
The engagement level was first run against the documentation for a single client: 59 documents across three ZIP files. The inspection has since been run across the entire engagement portfolio, around 4,400 documents pulled straight from the practice system.
That's where the integration stops being a technical detail. A person can download 59 files by hand. 4,400 documents is not a workflow, it's an excuse not to bother.
Three findings from those runs say the most about what separates document control from checkbox control:
A document belonging to an entirely different client sat filed in the wrong client folder. The consistency check on organisation numbers didn't catch it, because the document didn't contain a single nine-digit number. The check was extended to look for company names that don't belong, and then it surfaced.
Bank reconciliations covered the wrong financial year. The file was named correctly. The contents were not.
And in a number of engagements, Quality itself has no signature date on the engagement agreement. That's not a deviation, and the report says so: a date in a practice system doesn't satisfy the requirement, and an empty field doesn't prove the agreement is missing. But these are the places the practice system knows nothing, and where an inspector would look first.
The firm level runs too, but currently reports a deficiency on everything, because the firm's own procedural documentation hasn't been submitted yet. In practice that report works as the collection list.
How we know the safeguards hold
We ran a structured stress test against our own engine. Not a review of whether the code looked tidy, but an attempt to break the guarantees above. It found eleven weaknesses.
Some were the system crashing instead of reporting, for instance when a file was locked by a sync client. Some could make the report untrue without anything looking wrong: documents in subfolders were skipped, so the engine could report a deficiency for something that had been there all along. And one was a genuine weakness in the quotation check, where character normalisation meant a quotation could be matched in a document that said something else.
All eleven are fixed, each with a test that keeps the fault closed.
The fixes then went through two independent critical reviews. They found six new problems, and four of those had been introduced by the fixes themselves.
That last part is the most useful thing we've learned. Fixing introduces faults. A system that doesn't audit its own fixes doesn't have control, it has optimism.
One clarification we insist on, internally too: the number of passing tests is not a measure of professional accuracy. It's a measure that faults we have found don't come back. The two are not the same, and anyone who conflates them is selling you something.
Status, and what isn't built
We're saying this plainly, because a system that oversells itself is exactly the failure the product exists to prevent.
56 control points are built, across nine of the form's ten areas. The one area not covered can't be inspected at all: it asks what legal form the firm has and what its revenue was, meaning information the firm supplies rather than requirements that can be breached. Two individual points also remain, and both are listed with a reason.
The report says this itself. At the top sits a caveat naming the areas it doesn't cover and why. The caveat shrinks on its own as areas are built out.
We also publish the weaknesses we know about, in the documentation the client receives. Scanned documents without a text layer aren't read. Years buried deep inside a spreadsheet get missed. Both lead to requires manual review, never to an approval.
A clean report never means you're ready. It means no deviations were found in what was actually checked. A tool that lets you believe you're finished is worse than no tool.
The engine doesn't know it's doing accounting
The system consists of two parts held strictly apart.
The engine takes in files, extracts text and dates, recognises document types, runs rules, demands evidence and writes the report. The rules it can run are called exists, period, deadline, consistency and scope. None of them are industry-specific.
The control points are the other part. They live as text files, one per control area, with the legal basis and the inspector's instruction reproduced verbatim from the form. They are data, not code. They're what knows this is about engagement agreements and tax returns.
We built it this way because the requirement was that new inspection years could be added without code changes. Regulation changes more often than software should. A new inspection year is a new folder.
The same goes for a new industry.
Where else this belongs
Construction is the nearest example. A contractor holds declarations of conformity, certificates with expiry dates, health and safety plans, deviation handling and handover documentation. The questions are identical: does the document exist, does it cover this project, is the certificate still valid on the date that matters, and does the paperwork belong to the subcontractor it claims to.
The same pattern shows up in transport, food production, healthcare and staffing. The common factor isn't the industry. It's that someone with authority arrives and asks for documentation against a list, and the list is long enough that manual review becomes the bottleneck.
We haven't delivered this in construction yet. What we have is an engine where the domain knowledge sits in text files someone can read and correct, rather than being cast into the code. The difference between accounting and construction is that folder, not the machinery underneath.
That also means a new industry is honest work, not a switch we flip. The control points have to be written by someone who knows the requirements, and they have to be tested against real documents. That's the part that takes time, and the part worth doing properly.
What this means for you
The point isn't that AI reads documents. Plenty of tools do that.
The point is that the difference between an assertion and a finding is a quotation you can look up, and that this difference has to be built into the system rather than promised by the vendor.
If you work in an industry where someone arrives and asks for documentation against a list, it's worth a conversation. Your control points are different. The questions underneath are the same.
AI doesn't start with technology. It starts with structure.
Book 30 minutes and we'll show you a real report →
Read also: Who's Responsible for AI in Your Business? Here's Why You Need an AI Committee