Numbered, Therefore True?

In 2017, Australia’s Commonwealth Ombudsman made a deliberately narrow finding about a new online debt system. When people supplied their actual earnings for each fortnight, the system could calculate a debt accurately. The trouble lay in the gaps.

Where the information was missing or incomplete, the system filled those gaps from tax records. Those records usually showed how much an employer had paid someone over a longer period. Social security entitlement, however, depended on the income attributable under the law to each particular fortnight. The software spread the aggregate amount evenly across the relevant fortnights and calculated from there.

The operations could be executed exactly. That did not make the liability accurate; it made the result exact relative to an assumed income pattern.

A proxy was promoted

Data matching had been used before Robodebt. A discrepancy prompted a compliance officer to investigate, seek payslips and, where necessary, obtain payroll records from an employer. Averaging was available as a last resort when the actual income could not be established.

Robodebt changed the job of the discrepancy. The Ombudsman’s 2017 report records that the department stopped gathering third-party evidence as part of the online process. Individuals now had to reconstruct and enter their own fortnightly earnings. If they did not respond, or could not fill every gap, an average could become the basis of the debt.

This organisational change arrived in the form of a calculation. A figure that had once opened an inquiry could now support a debt demand. In practice, the individual had to do the work of disproving the distribution inferred from the department’s aggregate data. The Commonwealth’s legal obligation to establish a properly founded debt did not move with that work.

Pseudo-precision often enters in just this way. An observation at one level is converted into a more specific claim through an assumption that is no longer visible in the output. The tax record supported the total income. It did not, by itself, show when that income had been earned. For a salaried person on steady pay, an even distribution might be reasonable. Casual, sessional and intermittent work were a different matter.

What 229 cases did and did not show

Before the scheme’s large-scale rollout, the department compared averaging with manual calculations in 229 cases. The Royal Commission’s final report records in Volume 1, page 43 that averaging produced a higher debt in 55.9 per cent of them and a lower debt in 39.3 per cent. Just over 95 per cent of the averaged amounts differed from the manual result.

That is not a national error rate. The report gives no basis for treating the 229 cases as a representative sample of the whole programme, and some differences favoured the recipient. The comparison established something more limited: in that set, spreading aggregate income evenly almost never reproduced the debt obtained from the actual fortnightly record. Turning “just over 95 per cent” into “95 per cent of all Robodebts were wrong” would commit the very error at issue.

The decisions did not invalidate every conceivable use of averaging. Their narrower finding was that averaging alone was insufficient here to establish both the existence and the amount of a debt. Whether it could carry either claim in another setting would depend on the evidence and the governing law.

The legal boundary became clear in Prygodicz v Commonwealth of Australia (No 2). The declaration in Annexure C was carefully bounded. It covered decisions where the benefit depended on income in each fortnight, the Commonwealth assumed income above the amount reported, and that assumption came solely from evenly apportioning PAYG data for a longer period — with neither evidence of constant fortnightly earnings nor other support for the assumption.

The court also recorded the scale of the broader averaging-based programme. At least A$1.763 billion had been asserted against about 433,000 people; roughly A$751 million had been recovered from around 381,000 of them. Those figures included debts determined wholly or partly through averaging. They should not be read as the headcount of the narrower set of decisions described in Annexure C.

The point is not that a database lied. Several institutions took part in converting a useful lead into an enforceable claim: tax data supplied the total, software apportioned it, policy assigned that output a role, communications placed the response burden on the recipient, and debt collection followed. The number acquired authority from the process around it.

Ours included

It would be easy to leave Robodebt in a file marked “bad government algorithm.” That would spare this school of thought its own test. Intelligence Architecture also names layers, numbers steps and packages distinctions into playbooks. Those devices make difficult work discussable. They can also make a provisional idea look measured.

One of this site’s foundation texts actually uses a four-part sequence: sense, interpret, decide, act. It presents them as a practical model for analysis, not a linear process or a universal law. That limit is not a disclaimer tucked around the idea; it is the condition for using it. If the four distinctions neither change the inquiry nor locate a concrete break or lead to a different action, four has added order, not knowledge.

The same discipline applies to maturity levels and readiness scores. Combining several answers into one value may aid navigation. Before that value is allowed to describe an organisation, the aggregation rule, missing evidence, borderline cases and permitted use have to remain visible. Otherwise, sound observations are being asked to support a conclusion at a level they never measured.

Every framework therefore begins with an awkward question: What does the evidence establish before the framework adds its categories, weights and boundaries?

Robodebt did not fail because someone calculated. An unsupported distribution was given the evidential status of observed income, and its output was given the operational status of a debt. Our own numbers deserve scrutiny for the same reason. They clarify only if the reader can still see where observation ends.