Validation workflow

Use the validation workflow to show that a calibrated PEMS agrees with a reference method. The workflow runs a relative accuracy test audit (RATA). It compares the PEMS values with reference measurements at different load levels, and it gives an accepted or not accepted result for each tag.

Before you start

• You must have the SuperAdmin role.
• The emission source must have a completed calibration. A validation always belongs to one calibration.
• You must have a completed measurement workflow with the reference measurements of the test.
• You must know which RATA type applies: initial, quarterly or yearly.

Start the workflow

1. In the menu, open Calibration workflows. The Workflow Manager opens.
2. Click the Validation tab.
3. Click Start Validation.
4. In Create Validation, select the calibration in Select Calibration, and the Valid from date.
5. Submit the form. The new validation workflow opens.

Steps

The workflow page shows the Validation Workflow overview and the list of steps. Click a step to open it. Each step has a button that validates the step and opens the next step. Click Previous to go back one step.

Step 1: Create Validation

This step is complete when the validation is created.

Step 2: Select Measurement Dataset

1. In Measurement Dataset, select the measurement of the test.
2. Validate the step.

Step 3: Select Validation dataset

1. In Request data, select the process values to plot, and click Request.
2. On the graph, remove the periods that are not part of a test run.
3. Validate the step. To start this step again, click Reset validation dataset.

Step 4: Run Model

1. Select the pipeline steps to run.
2. Click Run Model. The system calculates the PEMS values for the validation dataset.

Step 5: Create Operation Levels

1. In Create Operation Levels, set the Minimum Range Width, the Min. Values per Range and the Min. Consecutive Values.
2. Click Create Operation Levels. The Operation Levels table shows each load level with its bottom and top value.
3. Optional: change a level, then click Update Operation Levels. Click View Runs to see the test runs in a level.
4. Validate the step.

Step 6: Compute Validation Results

1. In Standard Tags, select the tags to audit.
2. In RATA Type, select the type of report: initial, quarterly or yearly PS16.
3. In Diluent Standard Tags, select the diluent tags, for example O₂ or CO₂. The system audits a diluent tag with an absolute difference, in percent, not with a relative accuracy.
4. Compute the results.

Result

The results page shows the Relative Accuracy Test Audit for each audited tag:

• Audit summary: the emission source, the report type, the runs and the failing assessments.
• Performance Results: the assessment for each load level and for the whole validation, with the Acceptance Criteria and the result.
• Audited series: the test runs that the audit used, with the start time, end time and load level of each run.

Click Download workbook to get the results as a spreadsheet, with one sheet for each tag.

The acceptance criterion depends on the RATA type and on the mean PEMS value. The initial and yearly types use the relative accuracy (RA). The quarterly type uses the relative accuracy audit (RAA).

The mean PEMS value is the mean of the PEMS values of one audited tag, in the unit of that tag. The system does not convert the value before it compares it with the table. The limits in the table are for values in ppm. The system applies the table to each load level and to the whole validation. Diluent tags do not use this table. They use an absolute difference, in percent.

Mean PEMS value (ppm)	Initial and yearly (RA)	Quarterly (RAA)
More than 100	RA of 10 % of the reference method or less	RAA of 10 % of the reference method or less
More than 20, up to 100	RA of 20 % or less	RAA of 20 % or less
More than 10, up to 20	RA of 20 % or less	Absolute difference of 2 ppm or less
10 or less	Absolute difference of 2 ppm or less	Absolute difference of 2 ppm or less

The initial type also has an F-test for each load level and for the whole validation, and a correlation analysis. The correlation coefficient must be 0.8 or more.

For a low value, the result shows an absolute difference in ppm, not a percentage. This is correct.

Common problems

Problem	What to do
"Unable to create operation levels for selected data."	The dataset has too few values for the settings. Make the Minimum Range Width or the Min. Values per Range smaller, or select more data in step 3.
"The computation returned no audited tag."	Make sure that each selected standard tag has both a PEMS series and a reference series in the validation dataset.
"Failed to load RATA types"	Reload the page. If the error stays, contact support.
The results have no PEMS values.	Complete step 4, Run Model, before you compute the results.
An error says that the step is not allowed.	The workflow is not at that step. Complete the steps before it first.