learn

Factsheet: Can I Trust This Data?

Cawthron Institute have played a key role in LAWA by assessing how data is collected and analysed against best practice standards. How each site measures up to standards is explained in the Can I trust this data?  link for the River Quality and Lake Quality topics.

About Cawthron

Cawthron Institute is New Zealand's largest independent science organisation, offering a broad spectrum of services to help protect the environment and support sustainable development.

Based in Nelson, Cawthron works with regional councils, unitary authorities, government departments, major industries, private companies, and other research organisations throughout New Zealand and around the world. 

Cawthron is a diverse organisation employing almost 300 scientists, laboratory technicians, researchers and specialist staff from more than 20 different countries.

Cawthron’s scientists have unique expertise in aquaculture research, marine and freshwater resource management, food safety and quality, algal technologies, biosecurity and analytical testing. Their ground-breaking science is supported by substantial testing and research laboratories, state-of-the-art technology and a purpose-built aquaculture park.

They also run an extensive community and education programme to help foster the next generation of scientists.

Since establishment more than 100 years ago, Cawthron has strived to support New Zealand through applied science that works for the economy and environment.

What is Cawthron’s role in LAWA?

Cawthron’s role in LAWA is diverse. Primarily, their scientists make sure that you can trust the data presented on LAWA.  As an independent scientific institute, they also help to assure that the data is presented without any bias. Cawthron has worked alongside regional council and unitary authority staff to verify the processes and methods used for data collection, laboratory analysis of samples collected and the statistical analysis and interpretation of the results presented.

If you want to find out how robust the information for a particular site is, simply click on the Can I trust this data? link and you will be able to read about the level of confidence you should have in the data. We say that the data received the 'Cawthron tick' and you can find ticks in the River Quality and Lakes topics on LAWA.  

The colour of the tick describes how robust the information is. For example, some sites will have a tick that is green (high quality data and analysis appropriate) and some sites will have an orange tick (conclusions should be made with caution). All information shown on LAWA has either a green or an orange tick - any data that might have received a red tick is not presented on LAWA.

 

Cawthron tick green High quality data and analysis appropriate
Cawthron tick orange Conclusions should be made with caution

 

 

RIVER WATER QUALITY

This Cawthron tick assesses DATA COLLECTION, QUALITY CODING and TREND ANALYSIS for water quality indicators.

 

DATA COLLECTION

Water sample collection and field measurements of water quality indicators need to be done following appropriate protocols. Subsequent laboratory analyses of water samples also need to use appropriate analytical methods with suitable detection limits.  The National Environmental Monitoring Standards - Water Quality (2019) provides detailed guidance on sampling, measurement, and management of water quality data. These standards help to ensure that consistent approaches are applied across all of Aotearoa New Zealand so data is robust and can be compared among different regions. The standards address all processes associated with:

  • field work preparation
  • sampling point selection
  • sampling equipment preparation and use
  • field observations and measurements
  • water sample collection methods
  • water sample handling and transport
  • laboratory processing and test methods, reports, quality checks and accreditation
  • quality assurance procedures
  • data quality coding, and
  • data archiving.

 

Most water quality data is now collected and processed according to these national standards, so the latest data is very robust. However, earlier data may have been collected using alternative approaches, which may have varied to some extent across different parts of the country. The Cawthron Tick considers the length and measurement frequency of the dataset used for analyses and any variability in methods that may have occurred.

 

QUALITY CODING

The quality coding of data is important to determine how much faith should be placed on a particular data point. National Environmental Monitoring Standards (NEMS) recommend that all environmental data should be quality coded following the NEMS protocol (NEMS Quality Code Schema document) to allow for national comparison of environmental data. Some councils already have some quality coding procedures in place, but others are in the process of reviewing their existing procedures or introducing new quality coding procedures following NEMS recommendations. The Cawthron Tick for Quality Coding will be reviewed once all councils have adopted NEMS Quality Coding protocols.

 

TREND ANALYSIS

To see if water quality has improved or deteriorated at a site, LAWA also displays trends for the last five, ten and fifteen years. The ability to detect trends is dependent on the length and frequency of the sampling record. For example, occasional samples collected over a short period can provide limited evidence that a meaningful change is occurring at a site, over and above expected daily/seasonal variability. Longer more frequent sampling records can provide much better evidence. Monthly sampling is recommended for robust water quality trend analysis. The Cawthron Tick assesses sampling frequency for each site and gives the appropriate tick colour. If you want to find out more about how trends are calculated have a look at our trends factsheet.

 

MACROINVERTEBRATES

The 2 overarching areas the Cawthron tick looks at for the macroinvertebrate tick are SAMPLE COLLECTION & PROCESSING and QUALITY CODING and TREND ANALYSIS.

 

SAMPLE COLLECTION & PROCESSING

To evaluate the sample collection process, the Cawthron Institute assesses sampling frequency, sampling protocols and stand-down periods after floods following guidance in the National Environment Monitoring Standards - Macroinvertebrates (2022).

As macroinvertebrate density and abundance depend on the season (with higher abundance during spring and summer), sampling frequency and timing is very important. Because of this, councils sample sites regularly in the same season and under the same flow conditions each year.  See LAWA’s macroinvertebrate factsheet which explains the importance of timing in more detail.

Samples need to be collected following standardised protocols to assure that all councils collect their data in the same way. The semi-quantitative kick-net standard protocol is recommended for state of the environment monitoring (NEMS Macroinvertebrates 2022). Using this method, all habitat types present should be sampled in proportion to their occurrence at the site.

Macroinvertebrates are prone to being washed downstream and out of the river system during floods. Therefore, councils need to have a “stand-down” period of at least 2 weeks before sampling following a flood that has moved more than 25% of the riverbed in stony/gravel rivers or scoured more than 50% of the riverbed in soft-sediment rivers. This gives the animals a chance to recover and recolonise their habitats. Macroinvertebrate abundance and diversity data may be compromised if samples are collected within the first two weeks after a flood.

To evaluate sample processing, the Cawthron Institute assesses whether councils are following current national guidance. Once macroinvertebrates have been collected, they are sent to a laboratory for sample processing. For state of the environment samples, the ‘200+ fixed count with scan for missed taxa’ protocol is preferred (NEMS Macroinvertebrates 2022). External quality control procedures are required for each taxonomic laboratory to make sure that they have the right knowledge, skills and equipment to accurately identify and count the wide variety of freshwater macroinvertebrates.

 

QUALITY CONTROL

The quality coding of data is important to determine how much faith should be placed on a particular data point. National Environmental Monitoring Standards (NEMS) recommend that all environmental data should be quality coded following the NEMS protocol ('NEMS Quality Code Schema document') to allow for national comparison of environmental data. Some councils already have some quality coding procedures in place, but others are in the process of reviewing their existing procedures or introducing new quality coding procedures following NEMS recommendations. The Cawthron Tick for Quality Coding will be reviewed once all councils have adopted NEMS Quality Coding protocols

 

TREND ANALYSIS

To see if macroinvertebrate communities have improved or deteriorated at a site, LAWA also displays trends for the last ten and fifteen years. The ability to detect trends is dependent on the length and frequency of the sampling record. For example, occasional samples collected over a short period can provide limited evidence that a meaningful change is occurring at a site, over and above expected daily/seasonal variability. Longer sampling records can provide much better evidence. If you want to find out more about how trends are calculated have a look at our trend factsheet.

 

LAKES

The three main areas the Cawthron Institute assesses relating to lake water quality monitoring are SAMPLE COLLECTION & PROCESSING, QUALITY ASSURANCE and TREND ANALYSIS.

 

SAMPLE COLLECTION & PROCESSING

Water sample collection and field measurements of water quality indicators need to be done following appropriate protocols. Subsequent laboratory analyses of water samples also need to use appropriate analytical methods with suitable detection limits.

The National Environmental Monitoring Standards – Water Quality (2019) provides detailed guidance on sampling, measurement, and management of water quality data. These standards help to ensure that consistent approaches are applied across all of Aotearoa New Zealand so data is robust and can be compared among different regions. The standards address all processes associated with:

  • field work preparation
  • sampling point selection
  • sampling equipment preparation and use
  • field observations and measurements
  • water sample collection methods
  • water sample handling and transport
  • laboratory processing and test methods, reports, quality checks and accreditation
  • quality assurance procedures
  • data quality coding, and
  • data archiving.

 

Most water quality data is now collected and processed according to these national standards, so the latest data is very robust. However, earlier data may have been collected using alternative approaches, which may have varied to some extent across different parts of the country. The Cawthron Tick considers the length and measurement frequency of the dataset used for analyses and any variability in methods that may have occurred.

 

QUALITY CODING

The quality coding of data is important to determine how much faith should be placed on a particular data point. National Environmental Monitoring Standards (NEMS) recommend that all environmental data should be quality coded following the NEMS protocol ('NEMS Quality Code Schema document') to allow for national comparison of environmental data. Some councils already have some quality coding procedures in place, but others are in the process of reviewing their existing procedures or introducing new quality coding procedures following NEMS recommendations. The Cawthron Tick for Quality Coding will be reviewed once all councils have adopted NEMS Quality Coding protocols.

 

TREND ANALYSIS

LAWA displays any regional trends for the last five, ten and fifteen years. The ability to detect trends depends on the length and frequency of the sampling record. For example, occasional samples collected over a short period can provide only limited evidence that a meaningful change is occurring at a site, over and above expected daily/seasonal variability. Longer more frequent sampling records can provide much better evidence. Monthly sampling is recommended for robust water quality trend analysis. The Cawthron Tick assesses sampling frequency for each site and gives the appropriate tick colour. If you want to find out more about how trends are calculated have a look at our trend factsheet.