Using Store and Forward
The current status of the Store and Forward system can be viewed from the Platform > System > Store & Forward page of the Gateway Web Interface.
Store and Forward Page​
The Store and Forward Engines page displays a list of the Store and Forward engines, as well as metrics for the store rate, forward rate, and number of datapoints currently in each Store and Forward system. If the targeted database connection becomes faulted, the datapoints wait in the Store and Forward system until the database connection is restored.
Each database connection gets its own Store and Forward engine, so the page will display one engine for each database connection. The basic stats display how much data is being pushed through the system to databases and if any datapoints are being dropped.
Store and Forward Connections​
| Attribute | Description |
|---|---|
| Store Rate | Throughput for the number of datapoints per second stored within the cache. |
| Forward Rate | Throughput for the number of datapoints per second stored to remote storage, database or Gateway. |
| Pending Count | Number of pending cached datapoints that have yet to be forwarded and stored to the remote storage. |
| Quarantined Count | Number of datapoints quarantined. |
Store and Forward Engines​
| Attribute | Description |
|---|---|
| Engine Name | Name of the Store and Forward engine. |
| Status | Current state of the Store and Forward engine. |
| Store Rate | Number of datapoints that go through the Store and Forward engine per second. |
| Forward Rate | Number of datapoints to be forwarded on to the database per second. |
| Pending | Number of pending cached datapoints waiting to be forwarded. |
| Quarantined | Lists all quarantined items in the engine. Includes the number of occurrences, a description of the where the items originated from, and the reason why the datapoint was placed into the quarantine. Provides an opportunity to retry, export, or delete the items. |
| Dropped | Number of dropped datapoints. A datapoint is considered dropped if it can not be added to one of the buffers, such as when a buffer is full and the engine can no longer accept new datapoints. |
Monitoring the Store and Forward Engine​
Go to the Platform > System section of the Gateway.
Click Store & Forward. The Store & Forward Engines page will be displayed showing each Store and Forward engine along with the current throughput and different datapoints.
You can click on View Details listed under the three dots menu to view Memory, Disk, and Forward Rates. The Details panel also provides in-depth information on the data pipelines and quarantine management of the engine.
Store and Forward Details​
Status​
| Name | Description |
|---|---|
| Memory Rate | The number of datapoints entering the Memory Buffer per second. |
| Disk Rate | The number of datapoints entering the Disk Buffer per second. |
| Forward Rate | The rate in milliseconds at which data will be considered eligible for forwarding to any attached data sink. |
Data Pipelines​
| Name | Description |
|---|---|
| System | The system using this Store and Forward engine for its database connection. |
| Flavor | The pipeline type. |
| Memory | The buffer data count measured in number of datapoints. |
| Memory Rate | The metrics of the memory buffer measured in datapoints per second. |
| Disk | The database storage measured in datapoints per second. |
| Disk Rate | The disk cache measured in datapoints per second. |
| Quarantined | The number of quarantined datapoints. |
| Forward Rate | The rate in milliseconds at which data will be considered eligible for forwarding to any attached data sink. |
Quarantine Management​
| Name | Description |
|---|---|
| System | The system using this Store and Forward engine for its database connection. |
| Flavor | The datatype of the quarantined datapoints. |
| Reason | The reason for the datapoints being quarantined. |
| Quarantined | The number of datapoints currently quarantined. |
Managing Quarantined Data​
The quarantined items at the bottom of the Details panel will have some buttons that allow you to control the data. A quarantined item can be retried, where it will be placed back in the Store and Forward system to go through properly once the original reason for quarantine has been fixed. Alternatively, it can either be deleted so that it is no longer taking up space in the Store and Forward system, or exported to your local machine where you can save it to try again later.
If you exported the file, you can import the file again when you resolve the issue that caused the data to be quarantined in the first place. Refer to the Controlling the Quarantine Data page for more information.