Achieve IT/OT Convergence Goals with Git

In the industrial automation space, information technology (IT) teams and operational technology (OT) teams have been living in different universes. Although they’ve worked in the same orbits and their paths occasionally cross, they’ve seldom had meaningful collaboration.

Understandably, it’s tougher to communicate when you don’t speak the same language.

In the manufacturing world, the OT teams are controls or automation engineers responsible for making sure machines on the factory floor do what they are programmed to do while IT oversees computers, servers, storage, networking, and security. These two professions had little reason to collaborate until the onset of smart manufacturing and the Industrial Internet of Things (IIoT).

This began to change with connected sensors sharing real-time data about a machine’s performance, enabling companies to make better maintenance and business decisions. Suddenly, IT infrastructure became more relevant on the manufacturing floor as teams needed to store their collected data. Furthermore, concerns such as cybersecurity put OT software increasingly within the purview of IT organizations.

Despite this new opportunity for collaboration, many industrial companies are still managing two separate IT and OT technology stacks. As the amount of software driving industrial operations has increased, the desire to have IT manage the infrastructure for both IT and OT has emerged for many organizations. The goal for this alignment is to drive greater operational efficiency and ownership across the organization.

Ideally, this approach takes the burden of managing software, data infrastructure, and security off controls or automation engineers, allowing them to focus on ensuring the factory floor runs smoothly.

It’s easy to see how situations can quickly get out of hand. For example, we recently worked with a 1,000+ employee firm working with seven different toolchains across their IT and OT teams. The company lacked a source of truth for where their files are backed up, and managing multiple toolchains became expensive from a licensing and operational perspective. IT completely lacked visibility into what OT teams were doing. The easiest and quickest way to get everyone in an organization to speak the same language is to use Git (global information tracker) as your underlying, unifying infrastructure.

Copia was founded to deliver modern development tools for industrial automation. One of its greatest features is it enables Git-based version control to be meaningfully applied to programmable logic controller (PLC) programming. Through the benefits of IT/OT convergence, Copia bridges the gap between existing industrial technologies and modern software toolchains. It enables IT teams to abstract away OT toolchains, focusing on supporting infrastructure with which they are familiar.

Eliminating industrial automation’s black box

One major obstacle to IT/OT collaboration is IT software engineers typically use common text-based programming languages while controls engineers program using visual languages (ladder logic and function block diagrams). The contents of these proprietary binary files can only be seen by opening them in specific applications, often only accessible via the controls engineer’s workstation.

So, it’s not surprising that IT has nearly zero real-time visibility into the work of OT. While IT has predominantly standardized its source control around Git, OT has generally followed no standardized processes or best practices. OT typically handles version control using a hodgepodge of manually updated archive folders (copy/paste/rename) and various vendor products, such as SharePoint, SVN, OEM software, etc.; ad-hoc methodologies more prone to human error.

In the industrial OT world, information has been extremely siloed. Programming goes directly from the engineer’s desktop to the machine on the shop floor; nobody else has visibility into it. A Git-based approach ensures data is stored in a central location, changes are tracked, and access permissions can be controlled, giving IT teams leverage to accomplish organizational objectives.

What keeps IT people up at night are problems such as security and maintaining company infrastructure, including data storage. Copia puts OT’s complex toolchains behind a Git-based toolset that IT’s already proficient at and often has the infrastructure to leverage which, in many cases, other teams within the company already rely on to streamline other in-house development tasks. Copia removes the mystery from the equation making it seamless for IT to oversee the infrastructure supporting OT’s work. Meanwhile, OT can simply execute without worrying about security or if they’ll have access to their mission-critical file backups.

A terrifying aerospace lesson

Siloed, manual, and ad-hoc version control is a disaster waiting to happen. Horror stories from the field are common. I was recently consulted on a shocking situation in the aerospace industry, where all the code for a company’s manufacturing environment was stored on a single engineer’s laptop. You can probably guess what I’m going to tell you next.

That’s right: the laptop went missing. A technician lost it. Manufacturing code for a mission-critical military jet component just disappeared.

Losing programming code for a factory manufacturing household appliances would have horrible business implications. But in this aerospace example, it could have become a national security crisis.

Of course, not all IT/OT issues have Pentagon-level stakes, but this same scenario could have a huge impact on a company’s bottom line. From a security standpoint, it underscores the way OT teams currently maintain their own infrastructure for data storage is a mismatch of company objectives. While the controls team in this organization was exceptional, data storage and security weren’t anywhere near their primary concern.

Not so surprisingly, I hear similar stories in other industries. An engineer works on a project and doesn’t back up the latest version of the code, then quits the company. If the team gets the laptop back they must then search for the engineer’s work, if it wasn’t maliciously deleted.

But this problem transcends disgruntled employees. Job turnover is a common occurrence so, if you’re working with a client supporting machines installed 10 years ago, there’s likely been a lot of staff turnover. If you’re scrambling to find information stored on an individual engineer’s computer, you’re doomed if they’re no longer around.

Broadly, this all boils down to an organizational concern. OT teams are supposed to keep the plant running. IT teams are supposed to manage information security and create processes around data storage and maintenance. Let’s explore the benefits of having IT and OT standardize their infrastructure around the same Git-based platform.

Collaboration benefits

Standardizing around Git using Copia enables IT teams to better leverage existing infrastructure, reduce maintenance overhead, standardize where data storage occurs, enable auditability and traceability, and improve security.

Git-based source control enabled with Copia brings IT and OT teams on the same page by:

• Reducing the need to manage an entirely separate IT and OT stack by standardizing around Git
• Simplifying the OT toolchain; instead of managing multiple different OT industrial toolchains to solve the same problems, IT teams now only need to manage one
• Implementing coherent SOP that aligns with how software teams make changes; controls and software teams can speak the same language for better collaboration
• Standardizing file data storage by integrating Copia into other leading Git providers (GitHub, Gitlab, Azure); making it easier for everyone in the organization to audit, trace, and quickly find correct data when needed
• Improving security by aligning current OT storage solutions with IT best practices

Breaking the silos between IT and OT makes excellent business sense. It provides the ability to use existing resources to effectively execute company objectives. Git-based source control offers an easy path to alignment and should be considered more broadly by industrial companies. IT’s familiarity with Git, coupled with its ability to instantly improve access control and traceability, makes it a low-risk solution to implement, and Copia’s purpose-built tool sets enable OT teams to improve the development workflows.

Copia Automation

About the author: Adam Gluck is founder and CEO of Copia Automation. He can be reached at adam@copia.io. Copia Automation is a partner member of the Control System Integrators Association (CSIA).