Enhancing Data Accuracy with GenAI
Presenters : Nestene Botha
Overview
Generative AI tools can take much of the manual effort out of getting figures from invoices, bank statements, ledgers and financial statements. They cannot, however, be relied on to produce a correct number. A language model predicts likely text, so an accurate figure and an inaccurate one come from exactly the same process, and nothing in the output tells them apart. Current research indicates this is a structural feature of how the models are trained, not a defect that the next release will fix.
This session sets out where generative AI belongs in a numeric workflow and where it creates risk. The reliable pattern is extract, calculate, explain. The model moves data out of documents and describes the result, while the calculation is done in a spreadsheet, practice software or a code tool whose working can be read. The session covers how to design extraction and reconciliation steps so that errors become visible: tables with source columns, item counts, a three-list reconciliation that must add back to the source totals, and an instruction that turns silent gaps into stated exceptions.
It then sets out a four-step verification routine, how review is divided in a small practice, and the working-paper evidence to keep on file, drawing on guidance from the IAASB, IRBA and PCAOB that the tool changes the procedure but not the evidence required. It closes with accountability: SARS's own auto-assessment figures and its Tax Practitioner Connect guidance, together with the IESBA Code, the SAICA Code and POPIA section 8, show that responsibility for the figure stays with the practitioner.
Topics covered
- Why a language model gets numbers wrong: prediction rather than lookup, hallucination as a structural feature of training, the limits of published hallucination rates, a five-minute test to run on any tool, and the three failure modes (invented, imported and incomplete)
- Where GenAI belongs: separating language and structure tasks from source-of-truth tasks, the extract–calculate–explain pattern, how to check whether a tool actually ran code or only generated a result, and supplying the period, entity, currency and basis explicitly
- Extraction and reconciliation: asking for tables with a source column instead of summaries, the risks of scanned documents, what careful extraction achieves in practice, date, currency and format errors, a structured three-list reconciliation, asking the tool to challenge its own output, and sampling
- The verification routine: four checks in order, who checks what in a small practice, the working-paper evidence to keep, standard-setter guidance from the IAASB, IRBA and PCAOB, and how to describe AI-assisted work to clients accurately
- Accountability: SARS auto-assessment as a worked example of completeness risk, SARS's stated shift towards practitioners validating pre-populated data, two decision questions to ask before using AI on a figure, and professional duties under the IESBA Code, the SAICA Code and POPIA section 8
Learning outcomes
Practitioners will be able to:
- Explain to a colleague or client why a language model produces confident but incorrect figures
- Distinguish the tasks in a numeric workflow where generative AI adds value from those where it creates risk
- Design an extraction or reconciliation step whose output can be checked against the source
- Apply and document a verification routine over AI-assisted numeric work
- Explain where accountability for an AI-assisted figure sits, and why it does not move
Who should attend
Tax practitioners, accountants, bookkeepers and auditors who use, or are considering, generative AI tools for extracting, reconciling or reporting financial and tax data.