Major concerns with Overseer

We take issue with the Overseer chief executive Dr Caroline Read, who accuses us of misleading statements (read her piece here). 

More compulsory and widespread use of OverseerFM as a regulatory tool is among sweeping changes to freshwater management proposed by the Government.  We believe New Zealanders must understand clearly the risks involved with this.

It’s difficult to believe Dr Read has fully read and appreciated the council’s position. Because after accusing us of being misleading, she confirms our core criticism: that OverseerFM is a nutrient budgeting model and not a real-time nitrogen loss calculator and is not designed to deliver “an absolute pass/fail against an absolute value”. But that’s exactly what’s happening now in some places, with more widespread misuse proposed.

We agree that when used properly, OverseerFM can give a farmer insight into improving farm practices aimed at reducing losses. But even then, its limitations and uncertainty should be clearly identified and widely understood. This is especially so in Taranaki, with its relatively high rainfalls and rainfall variability.

We are far from alone in our concerns:

The Parliamentary Commissioner for the Environment called for an urgent and fully independent review of OverseerFM before any widespread adoption as a regulatory tool. 

The PCE says:

Measuring the scale and environmental effects of nutrient loss from an individual property, as distinct from those of the wider catchment, is ‘problematic in the extreme’. It cannot be done.

OverseerFM is a model that uses algorithms to simplify and summarise highly complex natural processes and standardise complex local variability. All models operate with a level of uncertainty. The PCE says its uncertainty and level of transparency is not acceptable in a regulatory setting. Those being regulated need to feel confident they’re getting fair treatment.

OverseerFM can’t measure actual nutrient levels or losses, nor what happens to the nutrients after that, nor what happens in the surrounding and receiving environments.

Even for farming systems for which it has been calibrated, Overseer-calculated results for losses of nitrogen can be 25% to 30% inaccurate – but even this does not take into account any errors with measurements or uncertainty from data inputs, providing only part of the picture of quantifying uncertainty. Outside of calibrated areas (including Taranaki), the ‘uncertainty is likely to exceed 50%, but could be much higher still’. 

Dr Julie Everett-Hincks, legal and scientific researcher at the University of Otago, says that ‘Overseer would not likely withstand legal challenge, but more importantly, is it right to burden farmers with regulatory compliance when the tool used cannot reasonably measure nutrient losses? In its current form and governance structure, Overseer is not fit to be a regulatory tool’. 

Professor Troy Baisden, BOPRC chair in Lake and Freshwater Science at the University of Waikato: “When used to enforce regulation, Overseer lacks the openness and transparency needed for scientists to review model results or develop improvements.” 

Former IPCC working group director Martin Manning, Massey University Professor Emeritus of Industrial Mathematics, Graeme Wake, Massey agricultural senior scientist Tony Pleasants, and a retired associate professor of mathematics, John Gamlen:  Spending millions fixing Overseer’s problems will not be enough unless the underlying mathematical model is ditched and replaced by more sophisticated modelling that can reflect interactions between different biological processes. “New Zealand is at serious risk of losing its credibility in agricultural and environmental management with the public and from our international colleagues … the science does not stand up to peer scrutiny.”

TRC has a long-standing policy of supporting the use of OverseerFM as a benchmarking tool for the individual farm, as an assistant in reviewing and informing farm management practices, and as a means of education. 

We’re a successful and experienced water management regulator. When making rules, we seek to ensure they’re soundly based, fit for the purpose of solving real problems, and able to be monitored and enforced. We want to avoid unjust and perverse outcomes – behaving professionally demands nothing less.

So, we share the view of many experts and authorities that OverseerFM is unfit for use as a regulatory tool. 

• Basil Chamberlain is chief executive of the Taranaki Regional Council

OverseerFM