Upgrading legacy control systems has become a critical priority for engineering professionals due to inefficiencies, cybersecurity risks and regulatory compliance gaps.
Learning objectives
- Understand the challenges and risks associated with legacy control systems.
- Elevate the benefits of upgrading to modern control systems.
- Develop a strategic plan for upgrading legacy control systems.
Control system insights
- Legacy building control systems, while once cutting-edge, now present significant challenges such as inefficiency, security risks and high maintenance costs, making modernization essential for improving reliability, compliance and long-term performance.
- Upgrading a control system requires careful planning, technology evaluation and execution and the benefits — including enhanced efficiency, cybersecurity and scalability — far outweigh the risks of maintaining outdated infrastructure.
- This modernization process, though complex, offers significant benefits such as improved performance, enhanced security and long-term cost savings.
In a rapidly evolving technological landscape, maintaining outdated control systems can pose significant challenges and risks. Legacy systems often suffer from inefficiency, security vulnerabilities and lack of support. These limitations can lead to reduced reliability and resilience, operational disruptions, increased maintenance costs and compliance issues. Upgrading these systems is crucial to ensure operational continuity, competitiveness, security, reliability, compliance and to enhance efficiency.
Understanding legacy control systems
There are countless control systems used worldwide, typically constructed using the latest technology available at the time of installation. However, what is considered “current” only remains so for a limited period (see Figure 1). As technology continues to evolve rapidly, control systems owners must monitor their systems and available technology to determine the right time to upgrade.
Legacy control systems are outdated systems that continue to be used despite the availability of newer technologies. Examples include old supervisory control and data acquisition (SCADA) systems in manufacturing; outdated heating, ventilation and air conditioning control systems in buildings; and obsolete traffic management systems.
Legacy systems often suffer from inefficiency, security vulnerabilities, failing components, lack of support reducing the reliability and resilience of the system (see Figure 2). Moreover, these systems may not be compatible with modern technologies and maintaining these can lead to operational disruptions, higher costs, compliance challenges and increased vulnerability to cyber-attacks. These risks highlight the critical need to upgrade to modern systems.
Although the benefits of upgrading legacy control systems may seem obvious, it is important to overcome the counterarguments. Careful consideration should be given to a business case analysis before deciding to upgrade.
Here are four ways to help justify the cost of the control system upgrade.
Improved efficiency and performance
Modern control systems enhance operational efficiency by automating processes, reducing manual intervention and optimizing resource use. For example, upgrading from a legacy programmable logic controller (PLC) to a modern distributed control system can significantly improve process control and data acquisition. As control systems age, the cost and impact of failure increases exponentially.
Modern technologies offer more robust security features, including encryption, access control and real-time monitoring, which help protect against cyber threats. For instance, modern control systems generally support zero-trust cybersecurity concepts including multifactor identification, continuous monitoring and validation, least privilege access and encryption.
Better compliance with regulations
Upgraded systems are designed to meet current regulatory standards, ensuring compliance and reducing the risk of legal penalties. In the electric utility industry, modern control systems can help industries comply with standards such as the North American Electric Reliability Corporation Critical Infrastructure Protection (NERC CIP) and ISO/IEC 27001.
While upgrading involves an initial cost investment, the long-term financial benefits include reduced maintenance expenses, improved productivity and lower risk of costly disruptions. Although not strictly tangible, the long-term financial benefit is typically related to cost avoidance (see Figure 3).
Planning the upgrade
There is no “one-size-fits-all” approach to upgrading legacy control systems. It is important to spend time defining the scope, schedule and resources that will encompass an upgrade.
Begin by evaluating the existing system and identifying its limitations, assessing its performance and determining the need for upgrades. The evaluation should include conducting a thorough audit of the current infrastructure and identifying critical areas for improvement (see Figure 4).
As part of the evaluation, clear objectives for the upgrade should be defined, such as improving efficiency, enhancing security and ensuring compliance. These objectives should be aligned with the overall strategic goals of the organization. These goals can be defined in collaboration with the client stakeholders in a workshop format. One of the most important considerations is whether the changes will be made to a live process. If the work is going to be completed on a live process, considerations such as manpower, safety and manual control should be planned out.
Budgeting for the upgrade involves estimating costs, allocating resources and securing funding. This includes considering both the direct costs (hardware, software, installation) and indirect costs (training, downtime) associated with the upgrade.
Lastly, a timeline with key milestones should be established to ensure the upgrade process stays on track. Setting realistic deadlines for each phase of the project and regularly reviewing progress to address any issues promptly, is a preferred practice.
Evaluating, engineering and implementing control systems
A comparison of different modern control systems should be conducted based on features, performance and compatibility with existing infrastructure. For example, comparing PLCs from different world-class vendors can help identify the best fit for specific needs. Determining a control system’s compatibility with current infrastructure is crucial to avoid operational disruptions during an upgrade. Evaluating the interoperability of new systems with existing equipment and communication protocols must be considered.
Furthermore, scalability and future-proof solutions should be chosen to accommodate future growth and technological advancements. For example, selecting a control system that supports modular expansion and integration with internet of things devices can provide long-term benefits.
Selecting reliable vendors and forming strategic partnerships are essential for a successful upgrade. Vendors should be evaluated based on their track record, support services and ability to meet specific project requirements.
Engineering and design: When upgrading a SCADA system, it is essential for the engineering team to collaborate with client stakeholders to clearly define the scope of work. This collaborative effort ensures that all parties have a shared understanding of the project’s extent.
Once the scope is established, the engineering team can develop comprehensive design documents to guide the construction team in safely executing the upgrade. These documents may include a basis of design report, demolition drawings, construction drawings and specifications. If required by the plant owners, it is crucial that the design documents incorporate provisions to maintain plant operations during the upgrade process.
Additionally, the design should outline testing and commissioning procedures and clearly define the criteria for project completion. Upon finalizing the design, it can be issued as a request for proposal to solicit qualified firms to carry out the upgrade..
Phased versus “big bang” approach: The phased approach involves upgrading systems gradually, while the “big bang” approach involves a complete overhaul at once. Each has its advantages and disadvantages; a phased approach may reduce risk yet take longer, while a big bang approach can be undertaken at a faster pace, however, can potentially be more prone to problems.
Strategies for managing and mitigating risks involve thorough planning, testing and contingency measures. Conducting risk assessments and developing mitigation plans for potential issues should be part of this process.
Training and testing: Training and supporting staff during the transition is crucial to ensure they can effectively use the new systems. This includes providing comprehensive training programs and ongoing support to address any challenges. Equally important, thorough testing and quality assurance are necessary to confirm the upgraded systems function correctly and meet performance standards. It is imperative to conduct various tests, such as functional testing, performance testing and user acceptance testing to ensure a successful installation has been completed.
Water treatment plant upgrade
Although the upgrade team has successfully modernized numerous control systems, the challenges faced differ with each project. This example focuses on a recent upgrade for a large water treatment plant in the Pacific Northwest (see Figure 5).
A control system upgrade had been under consideration for almost 10 years, as numerous components of the legacy control system were failing. In 2022, the decision was made by the team to upgrade the control system and the SCADA network. The current system comprises five pairs of redundant processors and one standalone processor without redundancy. These processors were communicating via a copper network that was more than 24 years old.
During the design phase, the project development parameters for the upgrade included the following requirements:
- This is a live plant that could not stop operations during the upgrade
- Minimize expenses and fast-track the upgrade
- Communication with the owner, operator and the engineers was critical
- The luxury of time did not exist
To facilitate these requirements, two assumptions were made that impacted the design process:
- The contractor was going to be installing a fiber optic network to replace the existing copper network. There was no justified reason for re-use or in-kind replacement of the copper network.
- The contractor would be replacing the existing, outdated processors and input/output (I/O) with the latest processors and I/O available from the same manufacturer. This approach enables the team to use its expertise with the existing codebase and efficiently employ the manufacturer-provided software tool to convert legacy code into a format compatible with the new processors.
These two assumptions significantly minimized design and programming efforts. The team envisioned a phased approach where the network was upgraded, then the processors and finally, the I/O. During the process, the engineering team discovered a migration box that would allow contractors to quickly swap out the processors and the I/O at the same time. This would cut the anticipated system down time from hours to minutes.
A supplier offered significant assistance in this endeavor. The supplier had comprehensive maps that outlined the steps to commence from the existing system to a fully upgraded, modern system. By contacting the manufacturer or manufacturer’s representatives for assistance, during the planning work for the upgrade, it can provide the team with beneficial insights for a successful upgrade.
Once the design was completed, the process of preparing for the installation began. The initial focus of the network upgrade involved pulling fiber optic cables between each PLC cabinet and server rack. Fiber optic patch panels and network switches were then installed within each PLC cabinet, to support the network and establish a ring topology. We then ordered the various components required for the upgrade. The supply chain was still feeling the effects of the COVID-19 pandemic, such as the lead time on all the electronics devices was significant. We kept everyone informed and minimized the impact by working on other tasks while waiting for parts to arrive.
Programming, installation of the control system
While the network upgrade was underway, the software engineers initiated the programming effort. Their priority was to obtain the software necessary for transitioning from the old PLC logic to the new PLC platform.
The translation went relatively smoothly; however, some differences between the old and the new systems presented unforeseen challenges for the team. The primary issue that arose was in the way that the old and the new systems manage floating-point values. The new system’s floating-point values are required to start on even numbered registers whereas the old system had no such constraint. The original code had some floating-point values that started on even numbered registers and some that started on odd numbered registers. Ultimately, we found that the translation tool saved significant effort in rewriting code, even with the floating-point register constraint.
As the installation of the processors began, the contractor recommended installing the processors and the I/O simultaneously rather than the planned phased approach. This would allow the team to compress the schedule and regain some of the lost time caused by equipment lead times. After discussion with the various stakeholders, everyone agreed that this would be a beneficial change.
Two significant issues were identified that required some unplanned time and effort:
- All of the devices on the network were initially installed with their default rapid spanning tree protocol (RSTP) settings, which caused the network switches to assign device priority based on MAC ID. There were devices on the network that should not have been prioritized, so RSTP was drawing all network traffic onto devices that could not handle the traffic which created network slowdowns and failures. Once the switches were configured to be the priority on the network, the network started operating with no limitations.
- The concept code contained numerous complex calculations managed by enhanced math blocks, which effectively obscured the intent and process behind calculating the resulting values. Because the original code was written 25 years earlier, there were no reliable resources that could help explain these calculations. The programming team had to go through every calculation to understand each of them and ensure that the rewritten calculations in the new program were yielding the correct result.
Commissioning the control system
As the project transitioned into the commissioning phase, the team adjusted the construction sequence to replace both the processor and the I/O racks simultaneously in each panel. To leverage the engineers’ increasing knowledge and experience as the installation and configuration proceeded, the project team opted to work from least complex to most complex process area. The team planned to do one enclosure panel and then address any issues identified, applying learned discoveries to subsequent processor conversions.
A test bed in the control room allowed the team to test the code for each processor on the SCADA network before installation in the enclosure. This allowed experts to observe the variables and verify that each I/O was being processed correctly without impacting the various processes.
Use of the test bed exposed an issue; the new processors were running on the new network, which effectively isolated them from the old processors that were running on the copper network. There were values that were shared between the distinct, temporarily isolated processors that were part of the process control decision tree. Plans then included an intermediary PLC that would “talk” to both networks and share the required data during the installation and commissioning periods.
During each panel’s installation, all the processes associated with that PLC needed to run in manual operation. This required coordination with the operations group to ensure appropriate staffing. The team executed the installation and commissioning of each PLC, progressing sequentially until the project was completed. As each PLC was completed, testing scripts were run to ensure that the data was accurate and displayed and historized correctly.
Finally, the system was monitored and any issues that popped up for a defined break-in period were addressed. During this period, the team was on-site regularly to address punch list items and provide issue resolution. Despite having a few hiccups along the way, this project was a success. The water treatment plant now has a modern control system that is more reliable, robust and easier to support.
Why update a legacy control system?
In the face of rapid technological advancements, organizations are recognizing the need to upgrade their legacy control systems and the critical importance of modernizing these systems to enhance operational efficiency, security and regulatory compliance. Legacy control systems, while once state-of-the-art, now pose significant challenges, including inefficiency, security vulnerabilities and high maintenance costs.
Upgrading legacy control systems offers numerous benefits, including improved performance, enhanced security and long-term cost savings. The upgrade process is complex and time-consuming, requiring careful planning from the initial system assessment to final implementation. Establishing clear objectives early on is essential for selecting the right technology.
Compatibility, scalability and future-proofing are critical factors that must guide this selection. Once a solution is chosen, a well-structured design and installation strategy — whether phased or a “big bang” approach — must be followed to help organizations make informed decisions and ensure ongoing success.
