Updating diagnostics for Boeing’s 737 MAX

Base2, an engineering and software development company that specializes in heavily regulated industries, has built a diagnostic software application used on the Boeing 737 MAX that functions similarly to a car’s check engine light.

Updated sensor technology on the 737 MAX disrupted the way diagnostics and testing for manufacturing and maintenance is conducted. The 737 features new engines and components, along with updated sensor technology that made obsolete traditional wing diagnostics and testing methods used during manufacturing and maintenance – methods that had been in use since the 737’s introduction in the 1960s.

The 737 MAX predecessor, the Next Generation (NG), added an Onboard Network System (ONS) that can connect to the sensors. Additional software can be installed on the ONS, so engineers created a software solution to calculate faults that would run on the existing ONS.

Called the Onboard Maintenance Function (OMF), it diagnoses issues when components are installed during manufacturing. For regular maintenance flight check-ups, it will diagnose fault conditions at the gate. OMF improves gate-turn efficiency, which improves on-time departure rates. With more than 3,000 737 MAX on order – valued at more than $300 billion – more efficient manufacturing is important to Boeing.

As Base2 Development Lead Donevan Dolby explains, maintenance crews have to perform diagnostic checks during the manufacturing process and on aircraft when in commercial use, during each gate turn. On older 737s, technicians need to physically climb into the maintenance bay for manual checks of the sensors, which just like your car, light up with check engine message codes. Technicians find and read codes from multiple servers, then look up code meanings in a manual. The new 737 MAX has limited hardware space so this server function needed to be moved. With OMF, technicians enter the flight deck and can view all sensor maintenance messages/diagnostics at once from a single location.

OMF began with more than 1,700 requirements from Boeing and took two and a half years to develop. Built using more than 34,000 lines of Clojure code, it is one of the largest Clojure code bases to date. Dolby recently discussed the development with Aerospace Manufacturing and Design Magazine (AM&D).

AM&D: What prompted simplification of diagnostics to a simple check engine light, go/no-go indicator?

Donevan Dolby: Many operators have asked that the 737 MAX include access to additional airplane data, and that it be securely available to flight, cabin, and maintenance teams during flight or while on the ground.

OMF is similar but more comprehensive than a go/no-go indicator or check engine light. There are multiple screens and steps that the technicians use to view the active messages and error codes.

All 737 predecessors to the MAX required technicians to look at all the sensors in the maintenance bay of the aircraft to read the codes and look them up in manuals. Some of the 737 MAX’s new hardware components no longer have these displays. A centralized window for viewing all the data was needed to access information from the sensors that no longer had displays and to replace a manual process.

The OMF solution pulls all this sensor data into a central application that can be used on the flight deck or a maintenance device such as a laptop or tablet.

The OMF solution’s automated, central location on the flight deck speeds checkpoints for maintenance technicians to improve gate turn for airlines. It also allows the assembly crew to examine active faults, improving time to market for Boeing.

AM&D: How does it work?

DD: The ONS platform provides connectivity to the sensors that are monitoring aircraft components such as engine, avionics, doors, and radios. The OMF is not flight-critical, so if a logic equation provided by Boeing is incorrect, it does not endanger the passengers and crew.

It’s a rules-based means of determining error conditions in an automated fashion. The OMF system monitors and reports on the health and efficiency of the aircraft and gives mechanics an immediate window into the aircraft systems.

AM&D: Are there parallels to automobile diagnostics devices (OBDII) that a technician plugs into an electronic control unit?

DD: I like to think of it like the OBDII on an automobile, which has approximately 3,500 codes, but there are more than 6,000 codes that can be reported on for the 737 MAX. The OMF does more than just report on codes. It takes specific conditions and sequences, combines rules and events, manages dependencies, and aggregates fault conditions.

AM&D: How is the software different?

DD: The most unique element of this project was that we used a functional programming language, Clojure, to write the software and were the first ones to use this language on commercial aircraft. Clojure is a relatively new software language that allowed us to write rules and code capable of handling massive amounts of data under significant hardware limitations. We estimate that if we had used Java to write the OMF, it would have resulted in more than half a million lines of code, which would have been significantly more difficult to test and debug. The OMF was written in just 32,000 lines of Clojure code. It is also the first time Clojure has been used on aircraft software.

AM&D: What are other potential benefits?

DD: We hope that airliners will see a cost and time savings through this software solution.

Test flights and feedback from Boeing have reported that the OMF solution will save time for airliners at gate turn, but we won’t have quantifiable results until after Boeing’s first delivery late this year.

Our solution will also give airlines a critical and comprehensive window into the health of their aircraft, allowing them to provide more preventative maintenance and monitoring.

For Boeing, the OMF can allow them to quickly test hardware components during assembly. With more than 3,000 737 MAX orders waiting to be fulfilled and more than $300 billion in revenue on the line, any savings in time to delivery will also save the OEM money and allow the company to deliver to its customers faster.

AM&D: How many sensors can it connect?

DD: There are hundreds of sensors now connected by this solution. Previously, the sensors were disconnected.

AM&D: Any difference in how OMF works for OEM versus during MRO? What training is required?

DD: The OMF software functions the same during manufacturing and maintenance. However, during installation, they are running diagnostics as the components are installed. During operations the system is used for monitoring running systems.

Boeing provides maintenance training with simulated airplane systems for airlines. On the airplane OMF, it works the same regardless of who is doing the maintenance – MRO facility or airline.

AM&D: Where is the software loaded? How is it accessed? How does it verify when a trouble code has been fixed?

DD: The OMF runs on the Boeing aircraft server, the ONS platform, that was introduced on the 737NG. Access is rendered through laptop or tablet device or on the flight deck display. Examining the maintenance messages can be done from the flight deck displays, or a laptop/tablet connected via WiFi to the ONS network.

When an active trouble code has been resolved, a not-active message appears in the place of an active error code or message.

AM&D: What does Boeing have to say about their technicians’ experience?

DD: Boeing spokesperson Jessica Kowal says, “On our test airplane, we have been impressed with the responsiveness of the OMF system on both the manufacturing and maintenance interfaces. When the airplane is in service, even a small reduction in maintenance turnaround time would produce impressive benefits for airline customers.”

AM&D: What is Clojure code?

DD: Clojure is a functional programming (FP) language that runs on the Java virtual machine. Clojure is not proprietary; it is open source and builds on Java, allowing it to take advantage of all the libraries built for Java.

FP languages are quite different from imperative, object-oriented languages such as C++ or Java. The main differences are that FP languages such as Clojure use something called immutable state, meaning that functions don’t change data objects; instead, they create new instances. In imperative languages, where objects can be changed, it introduces the need to handle concurrency – if the threads/processes are trying to change the same object, they need to take turns. This is problematic from a development and debugging standpoint.

Clojure uses LISP syntax that allows for very concise code, typically an order of magnitude less code than Java or C++. This means less code to test and debug and easier to get to more than 90% code coverage with unit tests.

AM&D: Will the software need to be updated regularly?

DD: The Clojure community is very active, releasing new versions of the language and libraries fairly frequently, but that does not in turn mean that the OMF code base needs to be updated. It would be a very expensive project to update versions of OMF running on the 737 MAX as there is a long process for verifying and approving the application for release to the aircraft.

Base2 Solutions

www.base2s.com

About the author: Eric Brothers is AMD’s senior editor and can be reached at ebrothers@gie.net or 216.393.0228.