How the Tag Historian System Works

This page provides an overview on how the Tag Historian system evaluates tag changes, stores them, and later retrieves those values.

Deciding When to Store

The diagram above demonstrates the evaluation cycle for determining if the system needs to collect a sample starting with the sample mode type and working through timer settings and deadband change sufficiency.

When configuring history on a tag, these are some of the several checks that occur. These checks ultimately determine if the system should collect a sample. Each check is handled by something on the backend of Ignition called an "Actor". The notable actors here are Sample Mode, Min/Max Timer, and Deadband, which each correlate to similarly named properties on the tag's configuration.

Sample Mode

First the system needs to detect a value change on the tag, which is handled by the Sample Mode actor.

• If the tag is using a Periodic or Tag Group mode, then the storage evaluation occurs at the same rate specified by the mode's rate.
• If the tag is using an On Change mode, then storage evaluation occurs every time the tag changes value.
• In addition, there is a periodic timer for On Change modes running that checks if an On Change tag has remained static for a period of time longer than the configured maximum time between samples. If this occurs then the system immediately collects a sample, regardless of the other actors. The periodic timer evaluates all tags that share a tag provider, history provider, and rate simultaneously. So two tags that share a tag provider and history provider, but have different sample rates, will be evaluated separately.

Min/Max Timer

The next actor is the Min/Max Timer. Where applicable, if the mode's minimum time between samples has not yet been reached, then the new value is discarded. Otherwise, if the maximum time has been exceeded, then the system will collect a sample. The Max Time between samples will bypass the deadband, but not the sample rate. If the value is in between any minimum and maximum allowable sample periods, then the value is handed off to the next actor.

When using a Tag Group sample mode, there are two locations where a Max Time can be defined: On the tag's history settings, and on the Tag Group's history settings. The Tag Group's settings override the settings on the Tag, except when the Tag Group is using it's default values: in other words, neither the Max Time Between Samples or Max Time Units settings on the Tag Group have been changed.

Deadband

Next the Deadband actor examines the newly changed value in comparison to the previous value. The deadband will ultimately determine if the new value is sufficiently different enough than the previous value. If so, then a sample is collected.

Storing a Value

Once a sample is collected, it's added to a History Set, which is a collection of samples to store from a particular Tag Group/Scan Class (values using a "periodic" sample mode are all bundled together under an "exempt" group).

After being bundled, the History Set is processed by the Store and Forward System.

Memory Buffer

The History Set first reaches the Memory Buffer. If the Local Cache (in the Disk Store) is empty, then we know there are no other sets to process, and our new set is sent off to a Database Sink.

If the Local Cache has records, then the memory buffer will direct the newly acquired History Set to the Disk Store.

Disk Store

The Disk Store stores records on disk, in one of two areas.

The Local Cache is used to store sets that need to be sent to the Database Sink. Sets stay here until the Store and Forward system's Write Time or Write Size are met.

The Quarantine stores records that failed to be sent to the Database, typically due to an error reported by the database.

Database Sink

A Database Sink is responsible for several things:

• Bundle samples from a History Set into a SQL transaction
• Execute queries against the database.
• Keep tag id values in memory, so the sink doesn't have to constantly query the database for identifying features when a new record from an existing tag comes in.
• Samples that failed to be inserted into the database will be retried as individual queries before being moved to the Quarantine.

Retrieving a Value

Many features in Ignition can request data from the historian system. Requests must include some key attributes:

• Start datetime.
• End datetime.
• Tag paths.
• Sample size, determining how many datapoints should be returned.
• Aggregation mode, used in cases where many points are shown in a relatively small time frame.
• Return format, either wide or tall.

Request Origin

Tag History requests from Perspective and Vision Component Bindings can be cached for future use. Because of this the history system takes note of where the request originated from. If it was from a Tag History component binding then the system will check the appropriate cache. Other origins, such as the system.tag.queryTagHistory function don't have a cache, and instead immediately query from the database.

Of note, component bindings that opt out of using the cache (enabling the Vision "Bypass Tag History Cache" setting, or by disabling the Perspective "Cache & Share" setting) will skip the cache and query their results from the database.

Cache

Requests originating from a component binding are compared to existing caches. If data within the cache matches the request exactly, then the request is sent to the Processor.

If the cache is missing some of the requested data points, then they're retrieved from the database.

Query Data from Database

Here the system will write and execute a query to retrieve records from the database, based upon the start datetime and end datetime. The Tag Historian normally partitions data across multiple tables, so this step typically involves several SELECT queries to discover which partitions need to be consulted.

If pre-processed partitions are enabled, the system will determine if it should use those processed tables at this step.

Processor

The Processor has several responsibilities. Ultimately it needs to create a result set to return to the originator, but it also needs to process the data in the following ways:

• If some of the records from the query were retrieved from the cache, then the Processor is responsible for combining the cached records with the newly queried records.
• The Processor is responsible for placing records into time slices, as well as aggregating or interpolating the data as necessary. For example, if a query is requested to return 10 data points over a 10 minute time period, the processor would create 10 time slices that span one minute each. If there were multiple values collected within any of those slices, the values are aggregated using the selected aggregation method. If there aren't any records within that slice, interpolation will create a value.

Store in Cache

If the request originated from a system that has a cache, the result set created by the Processor is cached at this step. Regardless, after this step the result set is handed back to the object that originally made the request.

Storage Format

A listing of tables used by the Tag Historian can be found on the Ignition Database Table Reference page.