Direct answer #
Brownfield warehouse automation integration does not require a full operational shutdown when the migration is architected as a series of reversible increments governed by explicit interface contracts. The Pearl Gateway editorial position is that the safest path is a phased coexistence model: legacy control systems continue to own the physical process while new automation is introduced in isolated zones, with rollback points defined before each cutover. This approach relies on disciplined change management, not on a single “big bang” event. The technical foundation includes a dual-command arbitration layer, a data-dictionary contract for every exchanged signal, and a pre-defined rollback trigger matrix. This article details the architecture, the contract templates, the worked example of a zone cutover, and the explicit boundaries where a full shutdown remains the only defensible choice.
Key takeaways #
- Reversible increments are the core principle: Each integration step must be independently deployable and independently undoable, with a rollback point established before the step begins.
- Interface contracts are the binding element: Every signal exchanged between legacy and new systems must be documented in a machine-readable data dictionary with units, scaling, deadbands, and failure semantics.
- Controlled coexistence is a design goal, not an accident: The legacy system retains authority until a formal handover, and the new system operates in a shadow or advisory mode during the transition.
- Rollback points are technical artifacts, not paperwork: A rollback point includes a verified system state snapshot, a tested restoration procedure, and a trigger matrix that defines who can call a rollback and under what conditions.
- Safety boundaries are non-negotiable: Lockout/tagout procedures per 29 CFR 1910.147 [S4] apply to any physical intervention, and no software architecture can substitute for the physical isolation of hazardous energy.
- Timeouts and retries must be contractually defined: Every inter-system message requires an explicit timeout, retry count, and failure action, with values validated against the slowest permitted physical response.
- A full shutdown remains necessary for specific conditions: When the integration touches safety-rated logic, when the legacy system cannot be restored from a verified snapshot, or when the physical layout requires the conveyor to be stopped for mechanical modification, a controlled shutdown is the only defensible option.
Phased architecture overview: the three-horizon model #
The Pearl Gateway editorial recommendation for brownfield integration is a three-horizon model, adapted from the systems engineering principle of incremental commitment described in the NASA Systems Engineering Handbook [S1]. Horizon 1 is the observation phase, where the new system is installed in parallel but has no control authority. Horizon 2 is the arbitration phase, where the new system can issue commands but only through a dual-command arbitration layer that requires both the legacy and new systems to agree. Horizon 3 is the authority phase, where the new system has full control over a defined zone, and the legacy system is reduced to a monitoring role.
This model is deliberately conservative. It assumes that the legacy system is correct until proven otherwise, and that every new capability must earn its authority through demonstrated reliability in a lower-authority mode. The three horizons are not necessarily sequential across the entire facility; different zones can be at different horizons simultaneously. For example, Zone A might be in Horizon 3 while Zone B is still in Horizon 1, provided the interface contracts explicitly define the boundary between zones.
The key architectural decision is the arbitration layer, which sits between the legacy PLC, the new WCS, and the physical equipment. This layer is not a new controller; it is a deterministic routing and voting mechanism. In Horizon 1, the arbitration layer passes legacy commands through unchanged and logs new-system commands as “would-be” actions. In Horizon 2, the arbitration layer requires both systems to issue a command within a defined time window before it passes the command to the equipment. In Horizon 3, the arbitration layer passes new-system commands through and logs legacy commands as “would-be” actions.
This architecture requires that the legacy PLC program be modified only at the I/O mapping level, not at the logic level. The legacy PLC continues to execute its original logic, but its outputs are routed through the arbitration layer rather than directly to the equipment. This is a reversible change: the original I/O mapping can be restored by a single configuration change, not by a program rewrite.
Interface contract definition: the data dictionary as the binding document #
The interface contract is the single most important technical artifact in a phased brownfield integration. It is not a memorandum of understanding; it is a machine-readable data dictionary that defines every signal exchanged between the legacy system, the new system, and the arbitration layer. The contract must be versioned, and any change to the contract requires a formal change review before it is implemented.
The minimum content for each signal in the data dictionary is: signal name, data type, physical unit, scaling factor, offset, deadband, minimum and maximum expected values, timeout value, retry count, and failure action. The failure action must be one of a small set of enumerated values: “hold last value,” “go to safe state,” “ignore,” or “escalate to operator.” The choice of failure action is a site-specific decision that must be documented in the contract, and it must be consistent with the physical safety analysis for the equipment.
For example, a photo-eye sensor signal would be defined as: name = “PE-101-STATUS,” data type = “BOOL,” unit = “n/a (true = beam broken, false = beam clear),” scaling = “1:1,” deadband = “n/a,” timeout = “500 ms (illustrative assumption),” retry count = “3 (illustrative assumption),” failure action = “hold last value.” The timeout and retry count are illustrative assumptions because they depend on the specific sensor and PLC scan rate; they must be validated against the physical installation before the contract is finalized.
The data dictionary also defines the command semantics, which are more complex than status semantics. A command signal must define not only the value but also the validity window, the acknowledgment requirement, and the abort behavior. For example, a conveyor start command might be defined as: name = “CONV-201-START,” data type = “BOOL,” unit = “n/a (true = start, false = stop),” validity window = “100 ms (illustrative assumption),” ack required = “true,” ack timeout = “1,000 ms (illustrative assumption),” abort behavior = “stop and escalate.”
The contract must also define the heartbeat and watchdog parameters for each communication link. The Pearl Gateway article on heartbeat and watchdog logic [internal link] describes common failure modes, including missed heartbeats, stale data, and asymmetric timeouts. The contract must specify the heartbeat interval, the missed-heartbeat threshold, and the action taken when the threshold is exceeded. These values are site-specific and must be validated against the actual network latency and PLC scan times.
Reversible increment design: the unit of change #
A reversible increment is the smallest unit of change that can be deployed, tested, and rolled back independently. The Pearl Gateway editorial recommendation is that an increment should be limited to a single zone, a single equipment type, or a single control function. The increment must have a defined start condition, a defined end condition, and a defined rollback condition.
The start condition is the set of preconditions that must be true before the increment is deployed. These preconditions include: the interface contract is versioned and approved, the rollback point is verified, the operator training is complete, and the safety analysis for the zone is current. The end condition is the set of acceptance criteria that must be met before the increment is considered complete. These criteria are typically a combination of throughput metrics, error rates, and operator feedback.
The rollback condition is the trigger that initiates a return to the previous state. The rollback condition must be defined in terms of observable, measurable events, not subjective impressions. For example, a rollback trigger might be “the number of unplanned stops in the zone exceeds 3 in a 60-minute period (illustrative assumption)” or “the average cycle time exceeds 120% of the baseline for 30 consecutive minutes (illustrative assumption).”
Each increment must have a rollback point that includes three artifacts: a verified system state snapshot, a tested restoration procedure, and a trigger matrix. The system state snapshot is a capture of all configuration files, PLC programs, parameter sets, and database records that will be changed by the increment. The restoration procedure is a step-by-step sequence that returns the system to the snapshot state, including the exact commands to be executed and the expected results. The trigger matrix defines who can call a rollback, under what conditions, and with what authority.
The rollback point must be verified before the increment is deployed. Verification means that the restoration procedure has been tested in a non-production environment, or at minimum, that the snapshot has been validated for completeness and integrity. An untested rollback procedure is not a rollback point; it is a hope.
Dual-command arbitration: the voting mechanism #
The dual-command arbitration layer is the technical mechanism that enables controlled coexistence. It is a deterministic routing element that sits between the controllers and the physical equipment. The arbitration layer receives commands from both the legacy system and the new system, applies the voting rules for the current horizon, and passes the resulting command to the equipment.
In Horizon 1 (observation), the arbitration layer passes legacy commands through unchanged. New-system commands are received, logged, and discarded. The log records the command, the timestamp, the intended target, and the “would-be” action. This log is the evidence base for validating the new system’s logic before it is given any authority.
In Horizon 2 (arbitration), the arbitration layer requires both systems to issue a command within a defined time window. The time window is a contract parameter, typically set to 2–3 times the maximum expected command latency (illustrative assumption). If both commands arrive within the window and they agree, the command is passed to the equipment. If they disagree, the arbitration layer defaults to the legacy command and logs the disagreement. If only one command arrives, the arbitration layer defaults to the legacy command and logs the missing command.
In Horizon 3 (authority), the arbitration layer passes new-system commands through unchanged. Legacy commands are received, logged, and discarded. The legacy system continues to run its logic, but its outputs are no longer connected to the equipment. This is the point of no return for the increment, and it should only be entered after the new system has demonstrated acceptable performance in Horizon 2.
The arbitration layer itself must be fail-safe. If the arbitration layer loses power or communication, it must default to a state that is safe for the equipment. The default state is typically “pass legacy commands through” because the legacy system is the known quantity. This default must be implemented in hardware or in a separate safety-rated logic path, not in the same software that implements the voting rules.
Zone boundary definition: physical and logical separation #
The zone is the fundamental unit of physical separation in a phased integration. A zone is a contiguous set of equipment that shares a common control boundary and a common safety boundary. The zone boundary must be defined in both physical and logical terms.
Physically, a zone boundary is the point where a product or pallet can be held in a safe state while the control authority changes. This is typically a buffer conveyor, a stop gate, or a turntable that can hold product without blocking upstream or downstream equipment. The physical boundary must have a defined capacity, a defined holding time, and a defined restart procedure.
Logically, a zone boundary is the point where the interface contract changes. All signals that cross the zone boundary must be explicitly defined in the data dictionary, and the arbitration layer must be configured to handle the boundary signals according to the horizon for each zone. A signal that crosses a boundary between a Horizon 3 zone and a Horizon 1 zone must be handled differently than a signal that crosses a boundary between two Horizon 3 zones.
The Pearl Gateway editorial recommendation is to define zones based on the physical material flow, not on the organizational structure or the software architecture. A zone should be a set of equipment that can be isolated with a single lockout point, that has a single upstream and downstream interface, and that can be operated in a manual mode without affecting adjacent zones.
Zone boundaries must also be defined in the restart authorization procedure. The Pearl Gateway article on restart authorization [internal link] describes the conditions under which equipment can be restarted after a stop. In a phased integration, the restart authorization must consider the horizon of each zone. A zone in Horizon 2 requires both systems to agree on the restart command, while a zone in Horizon 3 requires only the new system to issue the restart command. The restart authorization procedure must be documented in the interface contract and must be consistent with the safety analysis for the zone.
Rollback point mechanics: snapshot, restore, and trigger matrix #
A rollback point is a technical artifact that enables a return to a known-good state. It is not a backup in the traditional sense; it is a complete description of the system state at a specific moment, plus the procedure to restore that state, plus the conditions under which the restoration is authorized.
The system state snapshot must include: the legacy PLC program (including all function blocks, data tables, and I/O mappings), the new WCS configuration (including all zone definitions, routing tables, and parameter sets), the arbitration layer configuration (including all horizon settings and voting rules), the interface contract version, and the database records that are affected by the increment. The snapshot must be stored in a location that is accessible during a rollback, which means it cannot be stored only on the system being rolled back.
The restoration procedure must be a step-by-step sequence with exact commands and expected results. For example, a restoration procedure might be: (1) stop the new WCS service, (2) restore the legacy PLC program from the snapshot, (3) restore the arbitration layer configuration to the “pass legacy” default, (4) verify that the legacy PLC is in RUN mode, (5) verify that all I/O points are reporting valid values, (6) resume the legacy WCS service, (7) verify that the zone is operating in manual mode. Each step must have a pass/fail criterion, and the procedure must define what to do if a step fails.
The trigger matrix defines the conditions under which a rollback is initiated. The matrix must include: the trigger event, the data source that detects the event, the threshold value, the person or role authorized to call the rollback, and the time limit for making the decision. For example, a trigger might be “unplanned stop rate exceeds 5 stops per hour for 2 consecutive hours (illustrative assumption),” detected by the WCS monitoring service, threshold = 5 stops/hour, authorized caller = shift supervisor, time limit = 15 minutes after the threshold is crossed.
The trigger matrix must be reviewed and approved before the increment is deployed. It must be consistent with the operational reality of the facility. A trigger that is too sensitive will cause unnecessary rollbacks; a trigger that is not sensitive enough will allow a failing increment to continue past the point where a rollback is feasible.
Timeout and retry contracts: defining inter-system communication failure semantics #
Every inter-system communication link in a phased integration must have an explicit timeout and retry contract. This contract is part of the interface data dictionary and defines what happens when a message is not acknowledged within a specified time.
The timeout is the maximum time allowed between the transmission of a message and the receipt of an acknowledgment. The timeout value must be derived from the physical characteristics of the link: the network latency, the PLC scan time, the sensor response time, and the actuator response time. The Pearl Gateway editorial recommendation is to set the timeout to 3 times the sum of the expected latencies (illustrative assumption), rounded up to the nearest 100 ms. This provides margin for normal variation without allowing a failed link to go undetected for an extended period.
The retry count is the number of times a message is retransmitted before the link is declared failed. The retry count must be balanced against the timeout: a high retry count with a long timeout can delay the detection of a failed link, while a low retry count with a short timeout can cause false positives on a busy network. The Pearl Gateway editorial recommendation is a retry count of 3 (illustrative assumption) for control messages and 5 (illustrative assumption) for status messages, with the understanding that these values must be validated against the actual network conditions.
The failure action is what happens when the timeout and retry count are both exceeded. The failure action must be defined in the interface contract and must be consistent with the safety analysis for the equipment. The available failure actions are: “hold last value,” “go to safe state,” “ignore,” and “escalate to operator.”
The following table summarizes the recommended timeout and retry contract parameters for different message types. All values are illustrative assumptions and must be validated against the specific installation.
| Message type | Typical content | Timeout (ms) | Retry count | Failure action |
|---|---|---|---|---|
| Control command | Start, stop, speed setpoint | 500 (illustrative) | 3 (illustrative) | Go to safe state |
| Status update | Position, speed, sensor state | 1,000 (illustrative) | 5 (illustrative) | Hold last value |
| Heartbeat | Link health signal | 2,000 (illustrative) | 2 (illustrative) | Escalate to operator |
| Configuration change | Parameter update, zone reconfiguration | 5,000 (illustrative) | 1 (illustrative) | Abort and escalate |
The timeout and retry contract must be tested before the increment is deployed. The test should simulate a failed link by disconnecting the communication cable or by blocking the message at the network level, and the system should be observed to confirm that the failure action is executed correctly.
Coexistence operating modes: shadow, advisory, and authority #
Controlled coexistence is achieved through a set of operating modes that define the relationship between the legacy and new systems. The Pearl Gateway editorial recommendation is to define three operating modes that map to the three horizons: shadow mode, advisory mode, and authority mode.
In shadow mode, the new system runs its full logic but has no connection to the physical equipment. It receives the same inputs as the legacy system and computes the same outputs, but its outputs are logged and compared to the legacy system’s outputs. The comparison is the primary validation tool. If the new system’s outputs consistently match the legacy system’s outputs, the new system’s logic is considered correct. If they diverge, the divergence must be investigated before the new system is given any authority.
In advisory mode, the new system can issue commands, but the commands are only executed if the legacy system issues the same command within the arbitration window. This mode is used to validate the new system’s command timing and to identify any discrepancies in the interpretation of the physical process. Advisory mode is the most complex mode because it requires the arbitration layer to be fully operational and the timeout and retry contracts to be validated.
In authority mode, the new system has full control over the zone. The legacy system continues to run, but its outputs are not connected to the equipment. The legacy system’s outputs are still logged for comparison, and any significant divergence between the legacy and new systems is flagged for investigation. Authority mode is the final step in the increment, and it should only be entered after the new system has demonstrated acceptable performance in shadow and advisory modes.
The transition between modes must be a controlled event. The transition is initiated by a formal command from the integration team, and it must be documented in the change log. The transition must also be reversible: the system must be able to return to the previous mode if the new mode does not perform as expected. The rollback point for the increment must include the configuration for all three modes, so that a rollback can return the system to any previous mode, not just to the original state.
Data dictionary contract table: signal definition example #
The data dictionary is the binding contract for every signal exchanged between systems. The following table shows a representative subset of a data dictionary for a conveyor zone with a divert gate. All values are illustrative assumptions and must be validated against the specific installation.
| Signal name | Direction | Data type | Unit / semantics | Scaling | Deadband | Timeout (ms) | Retry count | Failure action |
|---|---|---|---|---|---|---|---|---|
| CONV-101-SPEED | Legacy → New | REAL | m/min | 0.1 per count | 0.5 m/min | 1,000 (illustrative) | 5 (illustrative) | Hold last value |
| CONV-101-START | New → Legacy | BOOL | true = start | 1:1 | n/a | 500 (illustrative) | 3 (illustrative) | Go to safe state |
| DIVERT-101-POS | Legacy → New | INT | 0 = straight, 1 = divert | 1:1 | n/a | 500 (illustrative) | 3 (illustrative) | Hold last value |
| PE-101-STATUS | Legacy → New | BOOL | true = beam broken | 1:1 | n/a | 500 (illustrative) | 3 (illustrative) | Hold last value |
| ZONE-101-MODE | Arbiter → Both | INT | 0 = shadow, 1 = advisory, 2 = authority | 1:1 | n/a | 2,000 (illustrative) | 2 (illustrative) | Escalate to operator |
| ZONE-101-HB | Both → Arbiter | BOOL | true = heartbeat alive | 1:1 | n/a | 2,000 (illustrative) | 2 (illustrative) | Escalate to operator |
This table is a template, not a complete contract. A real data dictionary will have hundreds or thousands of signals, and each signal must be reviewed and approved by both the legacy system owner and the new system owner. The review process is the primary mechanism for catching misunderstandings about units, scaling, and failure semantics before they cause operational problems.
Equipment control module inspection: pre-integration baseline #
Before any integration work begins, the existing equipment control modules must be inspected to establish a baseline. The Pearl Gateway article on equipment control modules [internal link] describes the inspection points and early warning signs that should be documented. This baseline is essential for two reasons: it identifies existing issues that could be incorrectly attributed to the integration, and it provides the reference point for the rollback snapshot.
The inspection should cover: the physical condition of the control panel, the wiring terminations, the fuse and breaker states, the PLC I/O module status, the communication module status, and the operator interface. The inspection should also include a functional test of each piece of equipment in manual mode, with the results documented in the baseline report.
The baseline report must be stored with the rollback snapshot. If a rollback is required, the restoration procedure should return the equipment to the baseline state, not to an idealized state that never existed. The baseline report is also the reference for the acceptance criteria: the integration is considered successful if the equipment performs at or above the baseline level, not if it performs at some theoretical maximum.
The inspection should also identify any equipment that is already in a degraded state. If a conveyor has a worn bearing or a sensor has an intermittent connection, these issues should be documented and addressed before the integration begins. Integrating new control software onto equipment with existing mechanical or electrical issues will produce confusing results that are difficult to diagnose.
Worked example #
This worked example demonstrates the calculation of a zone cutover from Horizon 1 (shadow) to Horizon 3 (authority) for a single conveyor zone. The example uses illustrative assumptions for all input values.
Inputs #
- Zone: CONV-101, a single conveyor with a divert gate.
- Baseline throughput: 120 units/hour (illustrative assumption).
- Baseline unplanned stop rate: 1 stop per 8-hour shift (illustrative assumption).
- Shadow mode duration: 5 working days (illustrative assumption).
- Advisory mode duration: 3 working days (illustrative assumption).
- Acceptance criteria for shadow mode: 95% of new-system commands match legacy commands (illustrative assumption).
- Acceptance criteria for advisory mode: 100% of arbitration windows resolve without disagreement (illustrative assumption).
- Rollback trigger: unplanned stop rate exceeds 5 stops per hour for 2 consecutive hours (illustrative assumption).
Intermediate calculations #
Shadow mode validation: The new system logs 10,000 commands during shadow mode (illustrative assumption). The comparison shows 9,600 matches and 400 mismatches. The match rate is 9,600 / 10,000 = 96.0%. This exceeds the 95% acceptance criterion, so the system passes shadow mode.
Advisory mode validation: The arbitration layer processes 2,500 command pairs during advisory mode (illustrative assumption). All 2,500 pairs resolve without disagreement. The disagreement rate is 0 / 2,500 = 0.0%. This meets the 100% acceptance criterion, so the system passes advisory mode.
Throughput comparison: The baseline throughput is 120 units/hour. During the final day of advisory mode, the measured throughput is 118 units/hour (illustrative assumption). The throughput ratio is 118 / 120 = 98.3%. This is within the acceptable range of 95%–105% of baseline (illustrative assumption), so the throughput criterion is met.
Result #
The zone is approved for transition to Horizon 3 (authority mode). The transition is scheduled for the next planned maintenance window. The rollback point is verified, including the system state snapshot, the restoration procedure, and the trigger matrix.
Sensitivity #
The most sensitive parameter in this example is the shadow mode match rate. If the match rate were 94% instead of 96%, the zone would not pass shadow mode, and the integration would be delayed. The match rate is sensitive to the interpretation of the “match” criterion: does a match require identical values, or does it allow for a tolerance band? The contract must define the tolerance explicitly. For this example, a match is defined as the new-system command being within 5% of the legacy command for analog values, and identical for discrete values (illustrative assumption).
The second most sensitive parameter is the advisory mode disagreement rate. A single disagreement might be acceptable if it is explained by a known timing difference, but the contract must define what constitutes an “explained” disagreement. For this example, a disagreement is considered explained if the legacy command arrived more than 200 ms after the new-system command (illustrative assumption), indicating a timing skew rather than a logic difference.
Limitations #
This worked example assumes that the zone is a simple conveyor with a divert gate. A real zone will have more complex interactions, including multiple sensors, multiple actuators, and interlocking with adjacent zones. The example also assumes that the baseline data is accurate and representative. If the baseline was measured during a period of unusual operation, the acceptance criteria may be inappropriate. Finally, the example does not address the safety analysis, which is a separate and mandatory activity that must be completed before any transition to authority mode.
Safety boundaries and lockout/tagout requirements #
No software architecture can substitute for the physical isolation of hazardous energy. The OSHA lockout/tagout standard, 29 CFR 1910.147 [S4], requires that energy-isolating devices be physically locked and tagged before any servicing or maintenance activity. This requirement applies to any physical intervention on the equipment, including the installation of new sensors, the modification of wiring, or the replacement of actuators.
The Pearl Gateway editorial position is that the phased integration architecture does not change the lockout/tagout requirements. Even if the new system is in shadow mode and has no control authority, the physical installation of new hardware requires a lockout/tagout procedure. The integration team must coordinate with the facility’s lockout/tagout program and must ensure that the equipment is in a safe state before any physical work begins.
The Pearl Gateway article on lockout planning context [internal link] describes the common failure modes and diagnostic evidence related to lockout procedures. The article emphasizes that lockout is not a one-time event; it is a process that must be planned, executed, and verified for each intervention. The integration team must document the lockout points for each zone and must verify that the lockout is effective before beginning work.
The safety boundaries for the integration must be defined in the interface contract. The contract must specify which signals are safety-related and which are not. Safety-related signals must not be routed through the arbitration layer; they must remain on the safety-rated path. The arbitration layer is for operational control, not for safety functions. If the integration requires changes to safety-related logic, the integration must stop, and a full shutdown and safety revalidation must be performed.
WCS communication protocols: OPC UA and MQTT in the integration context #
The communication between the new WCS, the arbitration layer, and the legacy PLC can be implemented using standard industrial protocols. The Pearl Gateway editorial recommendation is to use OPC UA for the primary control path and MQTT for the monitoring and telemetry path, with the understanding that the specific protocol choice is a site-specific decision.
OPC UA is defined by the OPC Foundation in a multi-part specification [S3]. The specification covers the address space model, the services, the information model, and the security model. OPC UA is well-suited for the control path because it provides a rich information model that can represent the zone hierarchy, the equipment state, and the command semantics. The OPC UA information model can be used to implement the data dictionary contract directly, with each signal represented as a node in the address space.
MQTT is defined by the OASIS MQTT Version 5.0 specification [S5]. MQTT is a publish-subscribe protocol that is well-suited for the monitoring path because it supports many-to-many communication with minimal overhead. The MQTT specification defines the message format, the quality of service levels, and the session semantics. MQTT can be used to publish the heartbeat signals, the status updates, and the “would-be” command logs from the arbitration layer.
The choice between OPC UA and MQTT is not either/or; they can be used together. OPC UA provides the deterministic request-response and subscription model for control, while MQTT provides the lightweight publish-subscribe model for telemetry. The interface contract must specify which protocol is used for each signal, and the timeout and retry contracts must be consistent with the protocol’s semantics.
The NIST Guide to Operational Technology Security [S2] provides guidance on securing these communication paths. The guide emphasizes the importance of network segmentation, access control, and monitoring for OT systems. The integration team must ensure that the new communication paths are secured according to the facility’s OT security policy, and that the new system does not introduce unsecured communication channels into the OT network.
Failure mode coding for integration diagnostics #
The integration will introduce new failure modes that must be diagnosed and resolved. The Pearl Gateway article on failure mode coding [internal link] describes a systematic approach to classifying failures based on their symptoms, causes, and evidence. This approach should be applied to the integration to create a diagnostic framework that distinguishes between failures in the legacy system, failures in the new system, and failures in the integration layer.
The failure mode code should include: the affected zone, the affected equipment, the failure symptom, the probable cause, the diagnostic evidence, and the recommended action. For example, a failure might be coded as: zone = CONV-101, equipment = divert gate, symptom = gate does not divert, probable cause = arbitration layer disagreement, evidence = arbitration log shows legacy command = divert, new command = straight, recommended action = investigate new-system logic for gate control.
The failure mode coding framework must be established before the integration begins, and the integration team must be trained on the coding system. The coding system is the basis for the rollback trigger matrix: a rollback trigger is a specific failure mode code or a pattern of codes that indicates the integration is not performing acceptably.
The failure mode coding also supports the post-integration review. After the integration is complete, the failure mode codes are analyzed to identify patterns that indicate systemic issues. For example, if a large number of failures are coded as “arbitration layer timeout,” this indicates that the timeout values in the interface contract are too short and need to be revised.
Reliability-centered maintenance in the integration context #
The integration of new control software is an opportunity to review the maintenance strategy for the affected equipment. The Pearl Gateway article on reliability-centered maintenance [internal link] describes the operating principles and system boundaries for a maintenance approach that is based on the criticality of the equipment and the failure modes that are most likely to occur.
The integration team should work with the maintenance team to identify the equipment that is most critical to the integration’s success. This is typically the equipment that has the highest throughput impact, the longest repair time, or the highest safety risk. The maintenance strategy for this equipment should be reviewed and updated as part of the integration.
The integration also introduces new maintenance tasks, such as the verification of the arbitration layer configuration, the validation of the timeout and retry contracts, and the testing of the rollback procedures. These tasks should be added to the preventive maintenance schedule, with the frequency based on the criticality of the equipment and the observed failure rates.
The reliability-centered maintenance review should also consider the impact of the integration on the existing maintenance procedures. For example, if the integration changes the way that a conveyor is started and stopped, the maintenance procedure for that conveyor must be updated to reflect the new control path. The maintenance team must be trained on the new procedures before the integration is deployed.
Read-rate monitoring for integration validation #
For facilities that use barcode or RFID scanning, the read-rate is a critical performance metric that should be monitored during the integration. The Pearl Gateway article on read-rate monitoring [internal link] describes the operating principles and system boundaries for this monitoring. The read-rate is the percentage of scans that are successfully decoded, and it is a direct indicator of the health of the scanning system and the quality of the labels or tags.
During the integration, the read-rate should be monitored continuously and compared to the baseline. A significant drop in read-rate could indicate that the new system is not correctly interpreting the scan data, or that the integration has introduced a timing issue that affects the scanning process. The read-rate data should be included in the rollback trigger matrix: a drop in read-rate below a defined threshold for a defined period is a rollback trigger.
The read-rate monitoring should also be used to validate the new system’s routing logic. If the new system is making routing decisions based on scan data, the read-rate data can be used to verify that the routing decisions are correct. A mismatch between the scan data and the routing decision is a failure that must be investigated.
Machine-vision inspection in the integration context #
If the facility uses machine-vision inspection, the integration must account for the vision system’s data flow. The Pearl Gateway article on machine-vision inspection [internal link] describes the common failure modes and diagnostic evidence for these systems. The vision system is typically a source of data for the control system, and the integration must ensure that the vision data is correctly routed to both the legacy and new systems.
The vision system’s inspection results are often used to make divert decisions. During the integration, the divert decisions made by the new system must be compared to the decisions made by the legacy system, using the same vision data. This comparison is part of the shadow mode validation. If the new system makes different divert decisions than the legacy system, the difference must be investigated before the new system is given authority.
The vision system also has its own failure modes, such as missed inspections, false positives, and communication timeouts. The integration must define how these failures are handled in the context of the new system. The failure action for a vision system timeout must be defined in the interface contract, and it must be consistent with the safety analysis for the divert decision.
Operator training and human-machine interface design #
The integration changes the operator’s interaction with the equipment. The operator interface must be updated to reflect the new control modes, and the operators must be trained on the new interface and the new procedures. The Pearl Gateway editorial recommendation is to provide the operator with a clear indication of the current horizon for each zone, so that the operator knows whether the legacy system, the new system, or both are in control.
The operator interface should also provide a clear indication of the arbitration layer status. If the arbitration layer is in a disagreement state, the operator must be alerted. The operator must be trained on the rollback trigger matrix and must know who to contact if a rollback trigger is detected.
The operator training must be completed before the increment is deployed. The training should include: the new interface layout, the meaning of the horizon indicators, the procedure for manual operation in each horizon, and the procedure for reporting anomalies. The training must be documented, and the operator’s understanding must be verified before the increment is deployed.
Change management and governance structure #
The phased integration requires a formal change management process. The NASA Systems Engineering Handbook [S1] emphasizes the importance of a structured approach to change management, including the identification of the change, the assessment of the impact, the approval of the change, and the verification of the change after implementation.
The governance structure for the integration should include: an integration lead, a legacy system owner, a new system owner, a safety representative, and an operations representative. The integration lead is responsible for the overall plan and the execution of the increments. The legacy system owner is responsible for the legacy PLC and the legacy WCS. The new system owner is responsible for the new WCS and the arbitration layer. The safety representative is responsible for the safety analysis and the lockout/tagout procedures. The operations representative is responsible for the operator training and the operational acceptance.
The change management process must include a formal review of each increment before it is deployed. The review must confirm that: the interface contract is versioned and approved, the rollback point is verified, the operator training is complete, the safety analysis is current, and the acceptance criteria are defined. The review must be documented, and the approval must be recorded.
When this guidance does not apply #
This guidance does not apply when the integration touches safety-rated logic. If the new system is intended to replace or modify any safety function, such as an emergency stop circuit, a light curtain, or a safety-rated speed limit, the phased integration approach described here is not appropriate. Safety functions must be revalidated as a complete system, and this typically requires a full shutdown and a formal safety assessment.
This guidance does not apply when the legacy system cannot be restored from a verified snapshot. If the legacy PLC program is not backed up, or if the backup is known to be incomplete, the rollback point cannot be established, and the integration should not proceed. The integration team must first work with the facility to establish a complete and verified backup of the legacy system.
This guidance does not apply when the physical layout requires the conveyor to be stopped for mechanical modification. If the integration requires the installation of new mechanical equipment, such as a new divert gate or a new sensor mount, the physical work must be performed under a full shutdown and lockout/tagout. The phased integration architecture applies to the control software, not to the physical installation.
This guidance does not apply when the facility does not have the organizational capacity to support the change management process. The phased integration requires a dedicated integration lead, a formal review process, and a commitment to the rollback trigger matrix. If the facility cannot provide these resources, a full shutdown and a conventional “big bang” cutover may be the only feasible approach, despite the higher risk.
Sources and standards #
- NASA — NASA Systems Engineering Handbook. In “Brownfield Warehouse Automation Integration: A Phased Architecture Without a Full Shutdown”, source [S1] supports the attributed terminology or boundary; the warehouse-specific synthesis remains Pearl Gateway editorial analysis.
- NIST — Guide to Operational Technology Security, SP 800-82 Rev. 3. In “Brownfield Warehouse Automation Integration: A Phased Architecture Without a Full Shutdown”, source [S2] supports the attributed terminology or boundary; the warehouse-specific synthesis remains Pearl Gateway editorial analysis.
- OPC Foundation — OPC UA Online Reference. In “Brownfield Warehouse Automation Integration: A Phased Architecture Without a Full Shutdown”, source [S3] supports the attributed terminology or boundary; the warehouse-specific synthesis remains Pearl Gateway editorial analysis.
- OSHA — The Control of Hazardous Energy (Lockout/Tagout), 29 CFR 1910.147. In “Brownfield Warehouse Automation Integration: A Phased Architecture Without a Full Shutdown”, source [S4] supports the attributed terminology or boundary; the warehouse-specific synthesis remains Pearl Gateway editorial analysis.
- OASIS — MQTT Version 5.0 Specification. In “Brownfield Warehouse Automation Integration: A Phased Architecture Without a Full Shutdown”, source [S5] supports the attributed terminology or boundary; the warehouse-specific synthesis remains Pearl Gateway editorial analysis.
Revision and editorial note #
The Pearl Gateway Editorial Team prepared “Brownfield Warehouse Automation Integration: A Phased Architecture Without a Full Shutdown” from the five linked source records. The published guide remains educational and requires site evidence before application.