Cawthron Institute have played a key role in LAWA by assessing how data is collected and analysed against best practice standards. How each site measures up to standards is explained in the Can I trust this data? link for the River Quality and Lake topics.
Cawthron Institute is New Zealand's largest independent science organisation, offering a broad spectrum of services to help protect the environment and support sustainable development.
Based in Nelson, Cawthron works with regional councils, government departments, major industries, private companies, and other research organisations throughout New Zealand and around the world.
Cawthron is a diverse organisation employing almost 200 scientists, laboratory technicians, researchers and specialist staff from more than 20 different countries.
Cawthron’s scientists have unique expertise in aquaculture research, marine and freshwater resource management, food safety and quality, algal technologies, biosecurity and analytical testing. Their ground-breaking science is supported by substantial testing and research laboratories, state-of-the-art technology and a purpose-built aquaculture park.
They also run an extensive community and education programme to help foster the next generation of scientists.
Since establishment more than 90 years ago, Cawthron has strived to support New Zealand through applied science that works for the economy and environment.
What is Cawthron’s role in LAWA?
Cawthron’s role in LAWA is diverse. Primarily, their scientists make sure that you can trust the data presented on LAWA. As an independent scientific institute, they also help to assure that the data is presented without any bias. Cawthron has worked alongside regional councils to verify the processes and methods used for data collection, laboratory analysis of samples collected and the statistical analysis and interpretation of the results presented.
If you want to find out how robust the information for a particular site is, simply click on the Can I trust this data? link and you will be able to read about the level of confidence you should have in the data. We say that the data received the 'Cawthron tick' and you can find ticks in the River Quality and Lakes topics on LAWA.
The colour of the tick describes how robust the information is. For example, some sites will have a tick that is green (high quality data and analysis appropriate) and some sites will have an orange tick (conclusions should be made with caution). All information shown on LAWA has either a green or an orange tick - any data that might have received a red tick is not presented on LAWA.
|High quality data and analysis appropriate|
|Conclusions should be made with caution|
Scientific Indicator Tick
This Cawthron tick assesses DATA COLLECTION, STATE & TREND ANALYSIS for water quality indicators.
Data collection needs to be done thoroughly and using appropriate protocols to obtain robust, standardised data. The Cawthron Institute assesses:
- Whether council staff are adequately trained to collect water samples
- All councils have internal field sampling protocols in place which outline field sampling programmes relevant to each council. These programmes ensure that:
* All water quality parameters recommended in national guidelines are sampled
* Suitable sampling practice is followed (i.e., sampling occurs at the same site, ideally mid-stream)
* Appropriate sampling frequency (i.e., monthly, at approximately the same time of day) and sampling conditions (i.e., during all flows)
* Adequate sampling equipment and its regular calibration (i.e., prior to each sampling run for pH and dissolved oxygen probes, monthly for temperature probes)
* Adequate recording of sample results and site location (i.e., recording of metadata on field sheets, photographs)
* Adequate storage (i.e., chilled storage bins) and transport of samples (i.e., within 24 hours) to laboratories
- All councils use International Accreditation New Zealand (IANZ) laboratories (www.ianz.govt.nz) which use approved methodologies and detection limits. There may be minor differences in laboratory analysis methods between different laboratories. Regional Councils and unitary authorities are currently working towards standardising laboratory analysis techniques based on national recommendations. This will increase the robustness of the freshwater quality data shown on LAWA.
- Adequate field data entry processes are in place for each council before the data are sent to LAWA’s data processing team.
STATE & TREND ANALYSIS
LAWA displays the current regional and national water quality (=state) of our rivers and streams to give you an idea about the effects of different pressures (i.e., urbanisation, farming, etc.) on the condition of our freshwater resource right now. Find out more here on how the state of water quality is calculated.
To see if water quality has improved or deteriorated at a site, LAWA also displays trends for the last five and ten years. The ability to detect trends is based on the length and frequency of the sampling record. For example, occasional samples collected over a short period can provide limited evidence that a meaningful change is occurring at a site, over and above expected daily/seasonal variability. Longer more frequent sampling records provide much better evidence. Monthly sampling is recommended for robust water quality trend analysis. The Cawthron tick assesses sampling frequency for each site and gives the appropriate tick colour. If you want to find out more about how trends are calculated have a look at our Trend factsheet.
Cawthron also assesses the method used for trend analyses. Ideally, a ‘flow adjustment’ process during trend analysis is used to compensate for differences in flow on the sampling days across the data record. For example, if water quality sampling happens during high flow on one day and during low flow on another day at a particular site, differences in the data are caused by different flows rather than consistent changes at the site. Unfortunately continuous flow data is not available for the majority of water quality monitoring sites. Therefore, trend analyses reported on LAWA are not flow adjusted which is why Cawthron gives trend analyses an orange tick.
The 2 overarching areas the Cawthron tick looks at for the macroinvertebrate tick are SAMPLE COLLECTION & PROCESSING and QUALITY CONTROL.
SAMPLE COLLECTION & PROCESSING
For the ‘Sample Collection Tick’, the Cawthron Institute assesses sampling frequency, sampling protocols, sample habitat, and stand-down periods after floods.
As macroinvertebrate density and abundance depend on the season (with higher abundance during spring and summer), sampling frequency and timing is very important. Because of this, councils sample sites regularly at the same time(s) and under the same flow conditions each year. See LAWA’s macroinvertebrate factsheet which explains the importance of timing in more detail.
Samples need to be collected following certain sampling protocols to assure that all councils collect their data in the same way. There are three accepted and widely used sampling protocols and you can find out more here. Although the Cawthron Tick is based on national recommended guidelines (i.e., NEMaR), it acknowledges the existing variations in sampling protocols in its tick colours. This means, for example, that councils which follow NEMaR recommendations score the same colour tick as councils that follow other accepted protocols (e.g., C1 as recommended by Stark et al. 2001). Councils are currently in the process of developing National Standards for macroinvertebrate sampling protocols.
Similar to sampling frequency and sampling protocols, all councils should use the same sampling habitat (i.e. where in the river is the sample taken – in riffles, runs or pools, or all three). Our macroinvertebrate factsheet explains this aspect in more detail.
Macroinvertebrates are very small, which means they are prone to being washed downstream and out of the river system during floods. Therefore, councils need to have a “stand-down” period for at least 2 weeks before they go back to a river for sampling following a flood greater than 3 times the median flow. This gives the animals a chance to recover and recolonise their habitats. Macroinvertebrate abundance and diversity data may be compromised if samples are collected within the first two weeks after a flood.
For the “Sample Processing Tick”, the Cawthron Institute assessed whether councils are following current best practice. Once macroinvertebrates have been collected, they get transported to a laboratory for sample processing. There are three accepted protocols which are widely used by councils to process their samples (i.e., P1, P2, P3 – Stark et al. 2001). See here for more information on these sampling protocols. Proposed national guidelines and leading literature are inconsistent in their recommendations about which protocol to use and councils are currently in the process of developing National Standards for macroinvertebrate sample processing.
To really be sure that the data collected and processed by councils is of high quality, quality control measures need to be applied in the field and the laboratory. Similar to sample collection and sample processing, three widely used and accepted quality control protocols are available (QC1, QC2, QC3). Proposed national guidelines and leading literature are, however, inconsistent in their recommendations about which quality control protocol to use and councils are currently in the process of developing National Standards for quality controlling macroinvertebrate data.
MACROINVERTEBRATES - Comprehensive Cawthron Tick text
SAMPLING - FREQUENCY
SAMPLING - PROTOCOLS
SAMPLING - HABITAT
SAMPLING - STAND DOWN PERIOD
QUALITY CONTROL - FIELD
QUALITY CONTROL – LABORATORY
The three main areas the Cawthron Institute assesses relating to lake water quality monitoring are SAMPLE COLLECTION & PROCESSING, QUALITY ASSURANCE and TREND ANALYSIS.
SAMPLE COLLECTION & PROCESSING
National guidelines for the collection and processing of lake water quality data have not been developed for New Zealand yet. This means, that there is some variability in the way councils collect and process their data. Until national standards for sample collection and processing have been developed, there is no single agreed way to collect and process lake water quality data.
Although the Cawthron Tick is based on national recommended guidelines (i.e., NEMaR ), it acknowledges the existing variations in sampling protocols in its tick colours. This means, for example, that councils which follow NEMaR recommendations score the same colour tick as councils that follow other accepted protocols (e.g., Burns et al 2000).
How do we know if the data LAWA shows can be trusted or not? One part of our review looks at processes behind the data collection and data analysis, which we refer to as Quality Assurance or QA. There are many different sub-categories that fall under QA, such as 1) data processing and 2) quality coding. For some of these sub-categories, national standards already exist (see the NEMS website for NEMS Quality Coding Standards). Once national standards are developed, data will be collected and processed in the same way so all data throughout New Zealand will be comparable. However, there are no National Standards, yet for the ‘Quality Assurance’ aspect.
Regional Councils and the Ministry for the Environment are currently assessing the range of Quality Assurance and Quality Management Systems that are in place. Once this information is collected, a National Standard for Quality Assurance will be developed to provide confidence that national datasets can be compared.
1) Data Processing
All councils should have robust data processing measures in place. National guidelines for data processing are currently being developed. Until then, the Cawthron Tick acknowledges councils that have some form of data processing measures set up. These include specific data entry procedures and protocols, high levels of training for staff involved with data entry and appropriate archiving of processed data.
2) Quality coding
The quality coding of data is important to determine how much faith should be placed on a particular data point. National Environmental Monitoring Standards (NEMS) recommend that all environmental data should be quality coded following the NEMS protocol ('NEMS Quality Code Schema document') to allow for national comparison of environmental data. Some councils already have some quality coding procedures in place, but others are in the process of reviewing their existing procedures or introducing new quality coding procedures following NEMS recommendations. The Cawthron Tick for Quality Coding will be reviewed once all councils have adopted NEMS Quality Coding protocols.
LAWA displays any regional trends for the last five and ten years. The ability to detect trends is based on the length and frequency of the sampling record. For example, occasional samples collected over a short period can provide limited evidence that a meaningful change is occurring at a site, over and above expected daily/seasonal variability. Longer more frequent sampling records provide much better evidence. Monthly sampling is recommended for robust water quality trend analysis. The Cawthron tick assesses sampling frequency for each site and gives the appropriate tick colour. If you want to find out more about how trends are calculated have a look at our Trend factsheet.
LAKE - Comprehensive Cawthron Tick text
STATE – SAMPLING
STATE – QUALITY ASSURANCE - DATA PROCESSING
STATE – QUALITY ASSURANCE – QUALITY CODING
TREND – SAMPLING FREQUENCY