How to submit air quality monitoring reports with precision and compliance

The official Ambient Air Quality Data Submitter's Guide is for people who submit air quality monitoring reports to the province. It also explains how to update monitoring station info. 

Reporting on air quality

You'll learn what's expected from those who submit data and maintain station info.

Though the Guide tells you how to submit air quality data, it doesn't say what to submit. Other rules and approvals determine what you need to submit.

Online training manuals explain how to submit this data using the Electronic Transfer System (ETS).  They include manuals for setting up your account, logging in, getting help, using the submission website, managing roles, submitting files, saving your work, checking submission status, and dealing with errors. These manuals are on this website https://training.energy.gov.ab.ca/Pages/Air.aspx under "Courses."

A variety of accounts exist in the Electronic Transfer System (ETS), which manages air quality data.  Every company can only have one Site Administrator, and a Backup Site Administrator.  Area Operators that weren't registered as companies before have created special accounts for you.

You have to fill out some forms to become a Site Administrator, which you can find on their website. Site Administrators create other accounts in the system. These accounts will also have a special role called "Coordinator," and Coordinators can assign different roles to other accounts.

User accounts in the Electronic Transfer System (ETS) have different roles.  Usually the "Site Administrator" account is for the company, and it can do pretty much everything in ETS, but it's best if they just create other accounts and assign the "Coordinator" role.

Industry accounts have roles like "Coordinator," "Station Manager," "Reviewer," "Submitter," and "Viewer." Coordinators are like the main managers, and they can also take on other roles if needed. Managers can update station details, Reviewers can approve data submissions, Submitters can upload data, and Viewers can just look at it.

XML Schema Guidance

XML Schema is like a set of rules for organizing and submitting air quality monitoring reports.  You can find this schema on the ETS Support and Online Learning page.  In some parts of the schema, it might say certain information is optional, but that doesn't mean you can skip it.  Your file won't fail if it's missing, but you should include everything.

There's also an Excel document called Reference Tables V2.1 on the same page.  You can use these tables to get specific codes for your schema.  Check them out on the Alberta Online Learning Portal.

Future-oriented data

Section 3 includes a table that explains the different fields in an XML Schema. Here's what they mean:

- The Data field is like a container for all the data you want to submit, and you need it all the time.
- The next field, schemaVersion tells the system what version of the schema you're using.
- Under General Comments, You can add comments about the data set if something happened. Don't go over 1500 characters.
- For every measurement, you need an interval, which contains the start and end times.
- The station shows where the air data was captured, and each station has a unique ID.
- Every measurement needs an ID, which is unique to the station.
- A mobile station (like a drone or vehicle) takes readings based on longitude, latitude, and elevation. Only use them for mobile stations.
- The Program Code specifies why the monitoring was done, and you need it for every measurement.
- The Project ID, given by the department, identifies special projects.
- The Approval ID is required for data from facilities with approvals.
- The Contravention Number is used when data exceeds certain limits, and it's provided when needed.
- A Valid Variable Combination (VVC) contains the actual measured value and what was monitored.
- The VvcCode is like a code that explains what was monitored.
When available, this is the measured value.
- Air samples that aren't a part of continuous monitoring are simply labelled as samples.
- Each lab sample has a unique Sample Reference number.
- Date Analyzed, LabId, Lab Accredited: These three separate fields mark when and where the sample was analyzed.
- Method Detection Limit: The lowest level the method can detect.
- For measuring sample volumes, use Volume and Volume Unit.
- If a pump was used, the period duration is how long it took for the sample to be collected.
- Dates for shipping and receiving samples are given next, followed by:
- Precipitation Gauge Depth and bucket diameter.
- Flags are Codes that describe data quality or why data is missing.
- The last space is for Comments:  Limit 1500 characters for explanations or comments on measurements. 

You can use these fields to organize and describe your electronic air quality monitoring reports.

Here's what to expect when you submit air quality monitoring reports:

1. You can choose the unit of measurement for your data submission based on what makes sense.
2. You should report all times in Mountain Standard Time (UTC -7).
3. There's a website with a list of Program Codes and their meanings.
4. When you pick the values you're submitting, you also choose a Time Code to show how often those values happen (like hourly, daily, weekly, etc.). It explains when to use each one.

You can use flags to explain when something unusual happens in your data or to note required action. When a reading is missing a value, you should use a flag to explain why it's missing, or your submission might get rejected. There are some flags that also require you to leave a comment.

Keeping track of data

These expectations help make sure your air quality monitoring reports are clear, organized, and understandable.

Here's a quick summary of the Discovery of Errors section:

The air quality monitoring reports need to be resubmitted if you find any mistakes, miss any data that should have been submitted, use incorrect flags, or come across any other issues. Resubmissions don't need to be notified via email anymore.

You should mark every resubmission with a version number, like V01 (for the first), V02 (for the second), etc.  You can keep track of different versions of your data this way.

In order to resubmit data, include a version number in the filename to indicate it's an amendment.  The original and amended versions are stored in the data system, but only the latest version is reported.

The comments section of the XML data file should explain any errors, omissions, or issues in the data, describe what went wrong, and say what you did to fix it.

The Air Monitoring Directive (AMD) for airsheds and the EPEA Approval for industry usually specify data submission timelines.  Within 30 days of discovering an error, data must be corrected and resubmitted.

Data completeness/operational time: Industrial submitters need to include a single data point for each continuous parameter, showing the percentage of analyzer time. As part of the completeness requirement, this percentage is reported in the XML data file.

By following these guidelines, submitters can help the province make sure data is accurate and errors are fixed right away.

Summary of the Validations section

Validations are checks the Air Data System does on files. Basically, there are three types of validations:

A primary validation (ETS validations) - is done right after submission. It doesn't look at the data itself, but checks other details such as:

- See that you include the company name.
- Making sure you upload the right file type (e.g., XML).
- File names should be valid and unique.
- Maintaining file integrity (no corruption).
- User authorization for Area Operators and Industrials.
- Checking that the XML file follows the right schema.

Data Validation (Errors) - These validations happen in the Enterprise Data Warehouse. Their focus is on data accuracy and consistency. Checks include:
- Validating approvals, stations, flags, VVCs, labs, program codes, and project IDs.
- Validating that the data for continuous collection type VVCs follow specific rules, like having the right number of readings per month or using the right flags.
- Ensure data values are within expected ranges, like wind direction not exceeding 360 degrees.
- Detecting duplicate measurements: Flagging measurements taken at the same station at the same time.
- Confirming the file name matches the data inside, including approval numbers, station names, and dates.
- Performing these validations is crucial for maintaining data accuracy and integrity, as well as identifying and preventing errors.

Data Validation (Warnings) - checks submitted air quality monitoring reports for potential problems. Checks like these result in warnings, not failures. Warnings alert the submitter to potential problems without necessarily indicating anything wrong. The submitter can acknowledge a warning and keep going. WARNINGS can also point out things the submitter needs to fix.

This stage includes measurement validations like:
- Checking if the submission is a resubmission.
- Checking if 90% or more of continuous data readings have values, which might indicate trouble.
- Making sure lab data below the method detection limit (MDL) is flagged correctly.
- Validating zero/span checks and calibrations were done correctly and flagged for continuous data.
- Checking if the interval for passive data is within 30 (± 4) days.
- Checking AAAQO exceedances and contraventions for Core and Regulatory Program Codes. Warnings about AAAQO exceedances for other program codes.
- Detecting "jammed instrument" situations, when a reading stays the same (and non-zero) for five consecutive readings.

While most validations are done, some are still in development and might be updated later.  The provincial government intends to update this guide when things change.

Requirements for metadata

Stations:

Station names should be meaningful and reflect their location. Numbers aren't allowed in acronyms. Stations with common names can have alternate names.

Geo-coordinate accuracy: Latitude and Longitude should be accurate to 5 decimals, and elevation should be accurate to 1 meter.

Data collection date: When the station started collecting data.

Type:

1. Permanent Stations that fixed for a long time.
2. Moveable Stations (PAML) can be moved intermittently, but need a new name and ID.
3. For aircraft measurements, on a mobile platform (MAML) with changing coordinates.
   

Every XML reading needs geolocation.

Data management is the key to a decent environment

Maintaining the station:  First of all, Naming

For Submitters: Station names are assigned by facilities and must remain consistent once ID is assigned
For Operators: You can change the names, but be careful as it can confuse people.

Secondly, Relocating

Stations that move should be terminated and a new one created. Moves up to 200 meters are okay if data isn't affected significantly or if Mobile Stations move regularly.

Thirdly, Termination

Update the station's termination date (last day of data collection) when it's no longer used. If the station becomes active again, you can remove the termination date.

Valid Variable Combinations (VVCs) at each station:
A VVC defines combinations of Parameter, Collection Type, Time Code, Method, and Unit. These relationships can be maintained by Station Manager users. Validation of VVCs will be enabled to make sure data aligns.

An Equipment section will be developed.

New reference data request:
If you need reference data (like VVCs), email AMDFeedback@gov.ab.ca. Reference data will be added by internal Air Data Administrators.

What to do if you need help

You can reach out to these people if you need help:
Setup and support for ETS accounts:

1. Send an email to ETSAccountSetup@gov.ab.ca Call 780-644-2300
2. Technical Support (for XML errors, admin module problems, etc.):  Send an email to ETS@gov.ab.ca
3. Questions about industry / regulatory reporting:  Send an email to Air.Reporting@gov.ab.ca
4. Reporting questions for airsheds and areas:  Send an email to Air.Data@gov.ab.ca
5. General AMD Reporting Questions / VVC Requests:  Send feedback to AMDFeedback@gov.ab.ca

They're there to help you with reporting and technical stuff. If you have any questions or need help, don't hesitate to get in touch.

The "Ambient Air Quality Data Submitter's Guide" explains how to update monitoring station information.  Here's what data submitters and station information managers should do.

Air quality data submission requires accuracy, consistency, and adherence to established procedures and standards.  Assistive and supportive services are also available.

Do you have concerns about air pollution in your area??

Perhaps modelling air pollution will provide the answers to your question.

That is what I do on a full-time basis.  Find out if it is necessary for your project.

Have your Say...

on the StuffintheAir         facebook page

Other topics listed in these guides:

The Stuff in the Air Site Map

And, 

See the newsletter chronicle. 

Thank you to my research and writing assistants, ChatGPT and WordTune, as well as Wombo and others for the images.

GPT-4, OpenAI's large-scale language generation model (and others provided by Google and Meta), helped generate this text.  As soon as draft language is generated, the author reviews, edits, and revises it to their own liking and is responsible for the content.