The AC500 High Availability system is designed for the demand of automation systems that require a higher availability, which is realized by redundant devices and communications. The redundancy concept reduces the risk of losing production due to failure of parts of the automation system and thereby minimizes scheduled idle times.
For instance, control can be taken over by the secondary station automatically if the primary station fails.
AC500 High Availability system implements redundancy based on standard AC500 PLCs:
-
PLC
-
Field communication
-
SCADA communication
General differences in high availability / redundancy systems are in which way and how fast the switchover between redundancies happens.
-
Cold standby: A replacement system is there but not up and running - Process has (to allow) to completely stop for switchover – e.g. outputs may go to zero.
-
Warm standby: Both CPU may be running (= warm) but e.g. communication need to be started/stopped for switch-over - Process needs to tolerate longer freeze times e.g. on outputs - e.g. several seconds.
-
AC500 High Availability systems are "hot-standby":
-
Redundant CPUs and all communications are always up and running (hot)
-
Continuous failure detection in both CPU´s and mutual exchange of status
-
Continuous synchronization of critical/historical data from primary to secondary
-
Automatic switch-over in very short time in case of any failure in primary CPU
-
Details of AC500 HA operation along the figure above:
-
PLC redundancy: The two PLCs (A and B) are running in parallel and calculating and reading.
One is “primary” = active, which means also writing data to field devices.
The other one is “secondary” (= stand-by), also calculating but only reading data from field and receiving synchronization (or short = sync) data from the primary.
-
Synchronisation data are critical internal variables with e.g. historical content, which will be transmitted from primary to secondary CPU over the sync connection, so that secondary always has the latest data and can take over immediately. Automatically synced are the historic data of the special HA library function blocks (like counters, timers, integral controllers, …), additional Data e.g. of events and diagnosis can be synced by the user with sync blocks. The sync connection also transmits a “lifecom1” signal (back and forth) containing diagnosis data of each CPU, so that both CPU know the status of the other CPU. If secondary CPU receives no “lifecom1” anymore it assumes that primary CPU has a failure and takes over primary status. If the sync connection is broken both CPUs would try to adopt primary status, therefore, a parallel separate connection “lifecom2” is used to differential a “sync link” failure from an “other PLC” failure. The “lifecom2” should be routed via a different physical communication path than the data sync/lifecom1, e.g. the Field or SCADA network.
-
The field I/O connection is performed via the Ethernet protocol ModbusTCP - connecting the CI52x devices (CI521 or CI522).
For high availability/redundancy of the field or SCADA network, proven Ethernet network redundancy mechanisms are used. (In AC500 this is assumed to be realized by at least 2 (to avoid a single point of failure) external, managed switches), which has the advantage to be able to use AC500 HA with any faster redundancy mechanism / protocol.
-
For the I/O communication with CI52x modules two variants exist.
For smaller systems, the CI52x modules can be directly daisy chained (as in previous figure above) if MRP (Media Redundancy Protocol) or DLR (Device Level Ring) is used. CI52x are not actively participating in ring recovery however, a special FW allows fast ring detection and very short freeze times. Larger systems with e.g. many IO and clusters typically anyway connect to the network via a dedicated managed switch.
-
SCADA connection is redundant by nature of the two Ethernet ports and can be extended with further redundancy level as well by managed switches. SCADA itself can also switch the primary PLC to ensure communication to the active PLC in case of a simple connection and a connection failure. If the redundancy mechanism of the OPC DA server is not used, SCADA level itself must be able to handle and differentiate primary and secondary PLC and IP addresses based on the HA-status bits. For CP600 a script exists to do the same for Modbus or AC500 communication protocol.
In most PLC applications the critical components to fail are, beneath PLC, typically the power supply or communication components such as wires or switches. Therefore a SPOF (Single Point Of Failure) has to be avoided by adding redundant devices or redundancy functions wherever a failure likelihood is high and failures are not tolerable.
HA core functionality typically can tolerate only a single failure in the different levels. Then, a repair of the failed part is highly advised to achieve and ensure redundancy again. As shown in the above figure, the I/O-network cabling already provides a second independent redundancy layer e.g. for cable failure by its redundancy mechanism (e.g. ring), which can keep up communication without switching the PLCs: There a second failure in the PLC level could be tolerated as long as both connecting, managed switched still work, but it is highly advised to repair immediately anyway.
The AC500 High Availability system itself only takes care of the first fault. For example, in case of a second fault the primary PLC remains primary PLC until the second fault occurs. This results in no further switchovers (manual switchovers included).
Due to the efficient data sync mechanism, which allows data sync over normal and shared ethernet networks, with a well-planned communication network, the PLCs can operate geographically separated (by many 10th of kilometers). So even in catastrophic events with full mechanical destruction still one PLC will be available to control the process or infrastructure.
The secondary PLC or single CI52x modules can be exchanged in a running system without interruption of the primary PLC or the process. (Check document in “Examples” directory of Automation Builder if HA package was installed.)
Libraries
In order to achieve high availability, the CODESYS application must be enhanced with HA function blocks, from the HA-Modbus TCP library and the CI52x library. If the bulk data manager tool (BDM) is used for configuring the System and I/O modules - this is done automatically for the basic initial configuration step by code creation resulting in a prepared user specific “template” application (see below).
-
HA-Modbus TCP library contains HA control and HA utility function blocks
-
HA control function blocks manage the core HA functionality by collecting diagnosis and switching if necessary.
-
HA utility function blocks provide standard functions in the application program with internal sync for integral data e.g. timers, counters, PI control.
-
-
CI52x library contains a function block to configure and communicate to the communication interface modules and ensures that only the primary PLC writes to the outputs. The inputs are read by both PLCs.
-
For both PLCs the same application must be used/downloaded.
Bulk data manager tool (BDM)
For configuration of the CI52x Modbus TCPs, a separate Bulk Data Manager tool (BDM) is provided. Especially in larger systems usage of BDM is recommended to comfortably engineer HA and create CI52x related configuration and variable data in one place:
-
Configuration and parameters of the used I/O modules
-
Program code creation for variable naming, configuration, communication and all basic HA functionality
The BDM tool can serve SCADA programming and documentation as well in an efficient manner.