Outgoing vs. Incoming Connections
When using the Gateway Network, you will be working with two type of connections.
- Outgoing Connections: To establish communications, create an outgoing connection on the local machine. The outgoing connection always begins the connection process to a remote machine. After the outgoing connection is created, the local machine will attempt to use the connection to establish communications with the remote machine.
- Incoming Connections: On the remote machine, an incoming connection will automatically be created when the new connection attempt is detected. For connections where security settings require manual approval, you will need to approve the incoming connection before it can be used. If no security controls have been set, the incoming connection will automatically accept the connection from the local machine and begin sharing data.
Connections and Servers
Every machine on the Gateway Network is known as a Server. When you establish a connection to a remote machine, the remote Server sends data about itself and also sends data about any other Servers known to that machine. For example, assume your local machine is GatewayA. The remote machine is known as GatewayB. GatewayB also knows about another remote machine named GatewayC. As soon as your local GatewayA establishes a connection with GatewayB, GatewayB also sends information about the existence of GatewayC.
Modules such as the Enterprise Administration Module (EAM) are aware of this relationship and allow communication between GatewayA and GatewayC, even though there is no direct connection from the local machine to GatewayC.
If you are cloning Gateways to then be connected via Gateway Network, it is important to notice that there is a Gateway unique identifier in %IgnitionInstallationDirectory%/data/.uuid. No two Gateways connected via Gateway Network should share a .uuid. Generally, Gateways are cloned by restoring the same Gateway backup on multiple servers. Since Gateway backups carry their .uuid with them, restoring the same Gateway on multiple servers will result in multiple Gateways having the same .uuid. To get around this, you must stop your Ignition service, delete %IgnitionInstallationDirectory%/data/.uuid, then start your Ignition service so that a new, unique .uuid is generated. Doing this before connecting two cloned Gateways will prevent any .uuid collisions.
Which Server Should I Configure the Outgoing Connection On?
In regards to connecting multiple Gateways over the Gateway Network, there is little difference between an Outgoing and Incoming connection: these terms simply indicate which server the connection was configured on, and are mostly ignored by the rest of Ignition. Thus, assuming GatewayA and GatewayB, configuring an outgoing connection from A to B is equivalent to configuring an outgoing connection from B to A. When connecting two Gateways, only a single connection is required between them.
The Gateway Network General Settings set the basic rules for the system. By default, these settings are lenient to allow for easy setup, but can be set for security.
Uncheck this checkbox to disable using the Gateway Network.
If true, only connections that use SSL to encrypt traffic will be allowed. This setting only applies to incoming connections. Default is true.
Require Two Way Auth
Enforces two-way SSL authentication. If true, you will need to install the remote machine's certificate on this machine, in addition to manual approval of this machine's certificate on the remote machine.
If you check this setting, you will need to provide the remote machine's certificate. To do this, manually export a certificate from the remote machine's metro KeyStore, located in <installdir>/webserver/metro-keystore. Default KeyStore password is metro , and the alias is metro-key . Then place the certificate on the local machine in
On Ignition versions 8.1.14 and greater, the certificate is stored at
The maximum number of threads that will be used to upload messages. Applies to outgoing connections. Default is 5.
Send Buffer Limit
The number of outstanding messages that can be waiting for acknowledgement at a time. Default is 5.
The maximum number of threads that will be used to download messages. Applies to outgoing connections. Default is 5.
Processing Queue Limit
Number of received messages that can be held until they are processed by the local system. When this capacity is exceeded, new messages are rejected and errors are reported to the remote Gateway. Applies to incoming connections.
|Websocket Idle Timeout|
The maximum number of milliseconds that a websocket is allowed to remain idle before it is closed. This value should always be set higher than outgoing connection ping rates to avoid premature connection termination.
By default, the security level for incoming connections is set to “Unrestricted”, meaning that every remote machine that attempts to connect to the local machine will be accepted without question. You have several options to control security from the Gateway Network settings.
Allow Incoming Connections
If false, only outward connections defined on this gateway will be allowed. Uncheck this checkbox to disable all remote machines from being able to establish an incoming connection. To establish any connections with remote machines, you will need to create outgoing connections from this machine. Default is true.
Dictates what connections are allowed. Options as follows:
Connections with a Gateway Name in this list are automatically allowed if the Connection Policy is set to SpecifiedList. Separate Gateway names with a comma.
If enabled, this Gateway will be allowed to act as a proxy, and forward requests between Gateways that do not have direct connections. Default is false.
This setting was replaced in 8.1.14 with Allowed Proxy Hops.
|Allowed Proxy Hops|
The maximum number of proxy hops which could be used to reach the destination Gateway. Any number less than or equal to zero is equivalent to no proxy hops allowed.
Gateway Network Connection Example
To establish a basic communication link between two Gateways, first log into the Gateway where you want to establish the outgoing connection. For this example, we use an SSL connection.
- On the Gateway Webpage, navigate to Config -> Networking -> Gateway Network.
- Click on the Outgoing Connections tab. Click the Create new Outgoing Gateway Connection link.
In the Host field, enter the network address of the remote server.
In the Port field, enter the SSL port used by the remote server. By default, this is set to 8060 (which is defined /data/gateway.xml).Note: This port is different from the default SSL port an Ignition Gateway would use when communicating to a client (default port 8043).
Check the Use SSL checkbox.
- Use the default settings in the Ping section and Timeouts section of the page.
Click the Create New Outgoing Gateway Connection button at the bottom of the page.
You'll see a confirmation message that the connection was created.
At this point, your Gateway transmitted its certificate to the connected Gateway, but the incoming connection is not yet allowed. The Gateway’s connection will not show up under the Incoming Connections tab until after the certificate has been approved.
Log into the other Gateway. Navigate to Config -> Networking -> Gateway Network.
Click on the Incoming Connections tab. The first Gateway’s certificate should be present. The certificate Common Name field holds the network address of the machine that transmitted the certificate. The Serial field holds a numeric string that is automatically generated when the certificate is created, and is unique to every certificate.
- Click the approve button to accept the certificate. You'll see a confirmation message. Click the Confirm button.
Outgoing and incoming connections can be deleted for cases when the connection no longer exists on the other side.
- To delete a connection, navigate to Config -> Networking -> Gateway Network.
- Click on either the Outgoing Connections tab or the Incoming Connections tab.
Click More, and then select Delete next to the connection.
Note: For incoming connections, if a remote machine is still connected to the local machine with an outgoing connection, a new incoming connection will be created after deletion. For these cases, you must log into the remote Gateway and delete the outgoing connection. Then you can delete the local incoming connection.
Certificates and SSL
When a remote machine establishes an incoming connection, its Gateway server name is transmitted and appears in the Server Name field under Gateway Network -> Incoming Connections . However, no identity authentication is performed when the connection is created. The local system accepts the remote system id without question. To perform identity authentication on a connection, you must use Secure Socket Layer (SSL) and certificates. By default, SSL is enabled.
Client certificates are the certificates of peer Gateways that the current Gateway trusts when it is making outgoing connections. Client certificates live under:
When the Gateway makes an outgoing connection to a peer Gateway whose certificate is not yet trusted, the peer certificate (or its certificate chain if one is configured) is copied into:
This model allows users to configure the Gateway Network client to trust the peer Gateways on outgoing connections by moving the certificate on the file system from
$GATEWAY_HOME/data/gateway-network/client/security/pki/trusted/certs/. This file system change will be picked up immediately by the Gateway and the connection will be trusted when it attempts to reconnect again.
The Gateway Network config UI's Incoming Connections tab was made compatible with this new model so that incoming connection certificates may continue to be approved, denied, or deleted there.
Server certificates are the certificates of peer Gateways that the current Gateway trusts when it is handling incoming connections. Server certificates live under:
Requiring a Certificate
To require all incoming Gateways to use SSL, navigate to Config -> Networking -> Gateway Network .
Select the General Settings tab, and check the Require SSL checkbox.
Click the Save Changes button.
Denying a Certificate
You can deny a certificate under the Certificates tab by clicking the deny link to the right of the certificate. The connection that has been using that certificate will no longer be allowed to connect. You can delete certificates that are no longer in use. Keep in mind that if you delete a certificate, and a remote machine is still using that certificate, it will reappear on the Certificates page. In this case, you must navigate to the remote Gateway and delete its outgoing connection. Then you can permanently delete the certificate from the Certificates page.
Gateway Network Diagnostics
The Diagnostics tab on the Gateway Network Settings page gives you insight to the Gateway and and remote server response times.
- To test the response time of a remote server, select the server name from the Server dropdown list.
- Click the Submit button.
- The results will be displayed indicating if the call to the remote server was successful, what the response time was, and if there were any errors.
Gateway Network Queue Management
The Gateway Network system shares information across Gateways using a configurable number of send and receive threads. Gateway Network Queues are named to reflect their purpose and enable Ignition to prioritize which subsystem should have access to send or receive a thread at any given time. The Queue Management tab allows users to further direct how a queue should behave.
Clicking modify for a queue type accesses the Queue Settings page where the following information is available to view or edit.
|Queue Name||Name of the queue you are modifying (read only).|
|Description||Description for the queue you are modifying (read only).|
This setting is configured by the queue and is unchangeable. If true, the queue will not dispatch another task until the current active task has completed. When a queue uses synchronous delivery, the maximum number of allowed active tasks is fixed at 1 and cannot be changed. Default is false.
|Max Active||The maximum number of active tasks allowed at a time. A task is considered active when it has been dispatched to the Gateway Network connection. You can set a limit to ensure that the Gateway Network connection will not become overloaded. Set this value to -1 to not enforce a limit on active tasks. Default is -1.|
Determines the queue's priority in relation to other queues. A lower priority may result in messages in this queue taking longer to send, but can help prevent a Gateway Network connection from being overloaded.
Adjust Queue Message Capacity
Overloading a Gateway’s queue active messages can starve the send or receive threads for Gateway Network connections. Starving the send or receive threads for the connection can potentially prevent other subsystem messages from getting through. To solve this issue, edit the Max Active field to set a limit that will allow the Gateway Network connection to continue functioning correctly.
Since this is a known problem with Tag History Queues, we will use an example with a GatewayA and GatewayB, where GatewayA requests tag history data from GatewayB to display on a chart. A user has noticed other items related to the Gateway Network connection, such as remote tags, are no longer displaying properly. They use the Gateway Network Diagnostics tab to confirm the network is not the issue, which indicates a slow database may be causing a cascading effect where other messages are delayed by all the Tag History Query calls.
This cascading effect happens because the Gateway Network asynchronously dispatches some functions. One Gateway thread handles the initial call, and a different Gateway thread dispatches the call’s results. In our example, GatewayA is unaware that GatewayB is taking a long time to run the queries and will continue to dispatch more tag history requests. When the limit of send and receive threads available to a Gateway Network connection is reached, other messages can no longer get through. This can be verified as the issue by checking GatewayB’s current Incoming/Outgoing Tasks and Results Response Time.
Adjusting queue message capacity controls the flow of requests to prevent thread starvation across the network. Select Modify for the Tag History Queue to access Queue Settings and adjust capacity. Make sure the Gateway responsible for the thread requests is where the queue capacity is updated. In this example, GatewayA Tag History Queue Max Active count was set to 2 and saved by clicking Create New Queue Override Settings.
The Gateway Network Status page Remote Gateway Details now shows pending items since the Tag History Active upper limit has been met. Note that pending tasks are ones that have not yet been picked up for dispatch by the Gateway Network connection. These will be discarded if not dispatched within 1 minute. Be aware these requests being deleted may show more errors on components like the GatewayA chart, but all other subsystem messages can now get through without issue.