Direct answer #
Observability for warehouse automation is the practice of correlating three telemetry classes—logs, metrics, and traces—with physical-flow confirmation from sensors, photoeyes, and equipment state machines. The goal is not to replace safety logic but to explain why a system behaved as it did. Trace context propagation, standardized by the W3C Trace Context specification [S1], provides the correlation IDs that link a software request to its physical consequence. OPC UA provides the structured information model for equipment state and alarms [S2], while MQTT 5.0 offers a lightweight, publish-subscribe transport for telemetry [S3]. This article defines the data model, the correlation strategy, and the architectural boundaries that keep observability data out of the safety path. It is an editorial recommendation from Pearl Gateway, not a compliance mandate.
Key takeaways #
- Correlation IDs are the backbone: W3C Trace Context [S1] defines standard headers (
traceparentandtracestate) that propagate a unique request identity across services. In warehouse automation, this same ID must be attached to PLC tags, MQTT messages, and log entries to link a software transaction to a physical event. - Telemetry is not safety: Observability data—logs, metrics, traces, and even physical confirmation signals—must never be used as the sole input to safety functions. NIST SP 800-82 Rev. 3 [S4] emphasizes that OT security and reliability require clear boundaries; Pearl Gateway extends this principle to the safety domain.
- OPC UA provides the semantic backbone: The OPC UA information model [S2] standardizes how equipment state, alarms, and historical data are represented. Using OPC UA nodes as the canonical source for equipment state avoids the ambiguity of raw PLC tag names.
- MQTT 5.0 is a suitable telemetry transport: The OASIS MQTT 5.0 specification [S3] provides features such as message expiry, user properties, and reason codes that are directly useful for warehouse telemetry pipelines, especially when the network is intermittent or bandwidth-constrained.
- Timestamps need a discipline: Every telemetry record must carry a timestamp with a defined time source, timezone, and clock synchronization method. Without this, correlation across PLCs, edge gateways, and cloud services is meaningless.
- Physical confirmation closes the loop: A trace that ends at a PLC write is incomplete. The trace must extend to the photoeye or sensor confirmation that the physical action occurred, with a defined timeout and retry policy.
- Architecture is a choice, not a standard: The specific topology—edge gateway, MQTT broker, OPC UA server, or cloud collector—is a site-specific engineering decision. Pearl Gateway provides editorial guidance, not a prescriptive standard.
Observability defined for warehouse automation #
Observability is the ability to infer the internal state of a system from its external outputs. In warehouse automation, the system is a collection of mechanical, electrical, and software components: conveyors, sorters, strappers, PLCs, edge gateways, and a warehouse control system (WCS). The external outputs are logs, metrics, traces, and physical signals such as photoeye state changes or motor current draw.
This article distinguishes observability from monitoring. Monitoring tells you that something is wrong; observability tells you why. For example, a monitoring dashboard might show that a tilt-tray sorter has a 12% jam rate. Observability answers the question: is the jam caused by a misaligned side guide, a late PLC command, a slow MQTT message, or a mechanical wear condition? The answer requires correlating the trace of the software command, the metric of the motor current, the log of the PLC state change, and the physical confirmation from the tray position sensor.
The scope of this article is limited to telemetry and correlation. It does not cover safety-rated control, which is governed by separate engineering disciplines. It also does not cover cybersecurity controls in depth, although NIST SP 800-82 Rev. 3 [S4] provides relevant guidance for OT environments. The focus is on the data model and the correlation strategy that makes observability actionable.
The three pillars: logs, metrics, and traces #
The industry commonly refers to logs, metrics, and traces as the three pillars of observability. Each pillar answers a different question, and none is sufficient alone.
| Pillar | Primary question | Typical warehouse example | Unit / format | Retention (illustrative assumption) |
|---|---|---|---|---|
| Logs | What happened? | PLC alarm “Photoeye PE-104 blocked” | Structured text (JSON, key-value) | 30 days hot, 1 year cold |
| Metrics | How much / how often? | Conveyor speed, jam count, throughput per hour | Numeric with unit (m/s, counts/min, units/hr) | 13 months for trend analysis |
| Traces | What is the full path of a single request? | WCS command “divert parcel #12345” from order to sorter chute | Span tree with duration and status | 7 days for active debugging |
Logs are the most granular but the least structured. A log entry might say “PLC-07: Motor M3 overload relay tripped.” That is useful, but it does not tell you which parcel was being conveyed, what the WCS commanded, or whether the photoeye confirmed the parcel’s position. Metrics aggregate behavior over time. A metric might show that Motor M3 trips at a rate of 0.4 times per hour, but it does not tell you which specific event caused the trip. Traces provide the end-to-end view of a single transaction. A trace for parcel #12345 would show the WCS receiving the order, sending the divert command to the PLC, the PLC executing the divert, and the photoeye confirming the parcel left the main line.
The correlation challenge is that these three pillars are produced by different systems with different clocks, different identifiers, and different retention policies. The remainder of this article addresses how to unify them.
Trace context and correlation IDs: the W3C standard #
The W3C Trace Context Recommendation [S1] defines a standard for propagating context information across distributed services. It specifies two HTTP headers: traceparent and tracestate. The traceparent header carries the trace ID, the parent span ID, and a flags field. The tracestate header carries vendor-specific data.
In a warehouse automation context, the “services” are not just microservices. They include the WCS, the PLC, the edge gateway, and the MQTT broker. The trace ID must be generated at the point of order intake or parcel induction and propagated through every subsequent step. When the WCS sends a command to the PLC over EtherNet/IP, the trace ID must be embedded in the command payload or mapped to a PLC tag. When the PLC publishes a telemetry message to MQTT, the trace ID must be included as a user property [S3]. When the edge gateway writes a log entry, the trace ID must be in the log record.
The W3C standard [S1] is HTTP-centric, but the principle of a globally unique trace ID and a parent-child span hierarchy is transport-agnostic. Pearl Gateway recommends implementing the same trace ID format—a 16-byte trace ID and an 8-byte span ID, both hex-encoded—in MQTT user properties and PLC tag structures. This is an editorial recommendation, not a requirement of the W3C standard, which does not address PLC or MQTT transports.
The key benefit is that a single trace ID can be used to query all telemetry sources. Given a trace ID, an engineer can find the MQTT messages, the PLC logs, the OPC UA historical data, and the WCS application logs that share that ID. Without a standard format, each system would use its own identifier, and correlation would require manual mapping.
OPC UA as the semantic model for equipment state #
OPC UA is not just a protocol; it is an information modeling framework. The OPC UA Online Reference [S2] lists the specification parts, including Part 3 (Address Space Model), Part 5 (Information Model), Part 8 (Data Access), Part 9 (Alarms & Conditions), and Part 13 (Aggregates). These parts define how to represent equipment state, alarms, and historical data in a structured way.
For warehouse automation, the relevant OPC UA concepts are:
- Nodes and references: Every piece of equipment, every sensor, and every state variable is a node in the address space. References define relationships, such as “Motor M3 is part of Conveyor C2.”
- Data Access (Part 8): Defines how to read and write variable values, including timestamps and status codes.
- Alarms & Conditions (Part 9): Defines how to represent alarm states, acknowledgments, and severity.
- Historical Access (Part 11): Defines how to query historical data, which is essential for post-incident analysis.
- Aggregates (Part 13): Defines how to compute aggregated values such as averages, minimums, and maximums over time intervals.
Pearl Gateway recommends using OPC UA as the canonical semantic model for equipment state. This means that the PLC’s raw tag “M3_Overload” should be mapped to an OPC UA node with a well-defined name, data type, and engineering unit. The OPC UA node is what the observability platform queries, not the raw PLC tag. This decoupling allows the underlying PLC to be replaced without breaking the observability layer.
The OPC UA information model also supports inheritance and composition. For example, a “Tilt-Tray Sorter” node might have child nodes for each tray, each induction point, and each discharge chute. This hierarchy mirrors the physical system and makes it easier to navigate from a high-level alarm to the specific component that caused it.
MQTT 5.0 as the telemetry transport #
MQTT 5.0, standardized by OASIS [S3], is a publish-subscribe messaging protocol designed for constrained networks and devices. It is well-suited for warehouse telemetry because it supports many-to-many communication, has low overhead, and can operate over unreliable networks.
The MQTT 5.0 specification [S3] introduces several features that are directly relevant to warehouse observability:
| MQTT 5.0 feature | Definition | Warehouse observability use case |
|---|---|---|
| User properties | Arbitrary key-value pairs included in the message header | Carry the W3C trace ID, the OPC UA node ID, and the equipment name |
| Message expiry interval | Time in seconds after which the message is deleted if not delivered | Discard stale telemetry from a sensor that has not reported in time |
| Reason codes | Numeric codes indicating the result of an operation | Distinguish a successful publish from a quota exceeded or authorization failure |
| Topic aliases | Short integer to replace a long topic string | Reduce bandwidth on a congested industrial network |
| Payload format indicator | Flag indicating whether the payload is UTF-8 or binary | Distinguish JSON telemetry from binary PLC data |
Pearl Gateway recommends using MQTT 5.0 for telemetry transport from edge gateways to the observability platform. The trace ID should be carried in a user property named trace_id, and the OPC UA node ID should be carried in a user property named node_id. This allows the observability platform to correlate the MQTT message with the OPC UA historical data and the trace context.
It is important to note that MQTT is a transport, not a semantic model. The payload of an MQTT message might be JSON, but the meaning of the fields must be defined by the OPC UA information model or a separate schema. MQTT does not enforce data types or units; that is the responsibility of the publisher and subscriber.
Timestamp discipline and clock synchronization #
Correlation is impossible without trustworthy timestamps. A trace event on the WCS might be recorded at 10:00:00.123 according to the WCS server clock, while the corresponding PLC event is recorded at 10:00:00.100 according to the PLC clock. If the clocks differ by more than the event interval, the correlation will be wrong.
Pearl Gateway recommends the following timestamp discipline:
- Single time source: All devices should synchronize to a common time source, typically a GPS-disciplined NTP server or a PTP grandmaster clock.
- Explicit timezone: All timestamps should be stored in UTC, not local time. Local time should only be applied at the presentation layer.
- Resolution: PLCs typically have millisecond resolution, while software systems have microsecond or nanosecond resolution. The observability platform must handle mixed resolutions without truncation.
- Clock quality: Each telemetry record should include a clock quality field, indicating whether the timestamp is from a synchronized clock, an unsynchronized clock, or an estimated time.
For example, consider a photoeye that detects a parcel at position PE-104. The PLC records the event with a timestamp of 2025-06-01T10:00:00.123Z. The WCS receives the MQTT message about this event at 2025-06-01T10:00:00.456Z. The difference of 333 milliseconds includes the PLC scan time, the MQTT publish latency, and the network transit time. Without synchronized clocks, this difference cannot be decomposed.
NIST SP 800-82 Rev. 3 [S4] does not specifically address clock synchronization, but it does emphasize the importance of reliable and secure OT operations. Clock synchronization is a prerequisite for reliable observability, and it should be implemented as part of the network infrastructure, not as an afterthought.
Physical-flow confirmation: closing the observability loop #
A trace that ends at a PLC write is incomplete. In warehouse automation, the physical world is the ultimate source of truth. A parcel either moved or it did not. A sorter tray either tilted or it did not. A strapper either applied the strap or it did not.
Physical-flow confirmation is the process of verifying that a commanded action actually occurred. This is typically done with sensors: photoeyes, proximity switches, encoders, or load cells. The confirmation signal must be part of the trace.
The correlation loop has the following steps:
- Command: The WCS sends a command to the PLC, e.g., “divert parcel #12345 at chute 7.” The trace ID is attached.
- Execution: The PLC executes the command, e.g., fires the divert arm. The PLC logs the execution with the trace ID.
- Physical action: The divert arm moves. A proximity switch confirms the arm reached its extended position.
- Physical confirmation: A photoeye downstream of the divert confirms that the parcel left the main line. The PLC records this with the trace ID.
- Completion: The PLC publishes a completion message to MQTT with the trace ID and the confirmation status.
Each step has a defined timeout. For example, the PLC might wait 500 milliseconds (illustrative assumption) for the proximity switch to confirm the arm extended. If the confirmation does not arrive within the timeout, the PLC logs a failure and publishes an alarm. The timeout value must be based on the physical characteristics of the equipment, not on a generic default.
It is critical to note that physical confirmation is observability data, not safety data. A photoeye that confirms a parcel’s position can be used for observability and for non-safety control logic, but it must not be used as the sole input to a safety function. Safety functions require dedicated, safety-rated sensors and logic, as defined by applicable safety standards. This article does not cover those standards.
Correlation ID mapping across protocols #
The W3C Trace Context standard [S1] defines the format for HTTP headers. In a warehouse automation system, the trace ID must be mapped to other protocols: EtherNet/IP, OPC UA, MQTT, and possibly proprietary PLC protocols. This mapping is an engineering decision, not a standard requirement.
Pearl Gateway recommends the following mapping strategy:
| Protocol / transport | Where to carry the trace ID | Format (illustrative assumption) |
|---|---|---|
| HTTP (WCS to edge gateway) | traceparent header [S1] |
00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01 |
| MQTT 5.0 | User property trace_id [S3] |
4bf92f3577b34da6a3ce929d0e0e4736 (16-byte hex) |
| OPC UA | Node property or method argument [S2] | String node property TraceId |
| EtherNet/IP | PLC tag (e.g., TraceID_Array[16]) |
16-byte array of USINT |
| Log file | Structured field trace_id |
Same 16-byte hex string |
The mapping must be implemented consistently across all systems. If the WCS generates a trace ID and sends it to the PLC via EtherNet/IP, the PLC must use that same ID when it publishes to MQTT. If the PLC cannot store a 16-byte array, an alternative is to use a 64-bit integer trace ID, but this increases the risk of collision. The W3C standard [S1] specifies a 16-byte trace ID, and Pearl Gateway recommends adhering to that length.
The mapping should be documented in a data dictionary that is version-controlled and reviewed whenever the system changes. A trace ID that is dropped at a protocol boundary is worse than no trace ID, because it creates a false sense of correlation.
Metrics and aggregates for warehouse flows #
Metrics are numeric measurements over time. In warehouse automation, the most useful metrics are throughput, rate, utilization, and quality. These metrics should be defined with explicit units and aggregation methods.
OPC UA Part 13 (Aggregates) [S2] defines standard aggregation functions such as Average, Minimum, Maximum, Count, and TimeWeightedAverage. These are useful for computing metrics from raw telemetry.
Pearl Gateway recommends the following metric definitions:
| Metric name | Definition | Unit | Aggregation method | Example value (illustrative) |
|---|---|---|---|---|
| Throughput | Number of parcels processed per hour | parcels/hour | Count over 1-hour window, then rate | 1,200 parcels/hour |
| Jam rate | Number of jams per 1,000 parcels | jams/1,000 parcels | Count of jam events / (total parcels / 1,000) | 3.5 jams/1,000 parcels |
| Conveyor speed | Surface speed of the conveyor belt | m/s | Time-weighted average over 1 minute | 1.8 m/s |
| Motor current | Electrical current draw of a motor | amperes (A) | Instantaneous, sampled at 10 Hz | 12.4 A |
| Divert accuracy | Percentage of diverts that result in the parcel reaching the correct chute | % | Count of correct diverts / total diverts | 99.2% |
| Photoeye blockage duration | Time a photoeye remains blocked | seconds (s) | Duration from block to unblock | 2.3 s |
Metrics should be computed from raw telemetry using a consistent aggregation window. For example, throughput should be computed over a fixed 1-hour window, not over a rolling window that changes with the time of day. The aggregation window should be documented and consistent across all dashboards.
Metrics are useful for trend analysis and anomaly detection. A sudden increase in jam rate from 3.5 to 8.0 jams/1,000 parcels is a signal that something changed. The trace and log data are then used to identify the root cause.
Logs: structured and context-enriched #
Logs are the most common form of telemetry, but they are often the least useful because they lack structure and context. A log entry that says “Error 0x1234” is nearly useless. A log entry that says {"timestamp":"2025-06-01T10:00:00.123Z","level":"ERROR","source":"PLC-07","trace_id":"4bf92f3577b34da6a3ce929d0e0e4736","message":"Motor M3 overload relay tripped","motor_current_A":18.2,"conveyor_speed_mps":0.0} is immediately actionable.
Pearl Gateway recommends the following fields for every log entry:
- timestamp: UTC, ISO 8601 format, with millisecond or microsecond resolution.
- level: DEBUG, INFO, WARN, ERROR, or FATAL.
- source: The device or service that produced the log, e.g., “PLC-07” or “WCS-01”.
- trace_id: The W3C trace ID [S1] for correlation.
- span_id: The specific span within the trace.
- message: A human-readable description.
- context: Key-value pairs with additional data, such as motor current, conveyor speed, or parcel ID.
Logs should be written in a structured format, such as JSON or key-value pairs. This allows the observability platform to index and query them without parsing unstructured text. The log schema should be versioned so that changes are tracked.
It is important to avoid logging sensitive data, such as operator credentials or customer personal information. The log schema should be reviewed to ensure it only contains operational data.
Traces: span hierarchy and duration #
A trace is a tree of spans. Each span represents a single unit of work, such as “send command to PLC” or “wait for photoeye confirmation.” The root span represents the entire transaction, such as “process parcel #12345.”
The W3C Trace Context standard [S1] defines the format for propagating the trace ID and span ID, but it does not define the span hierarchy or the semantics of spans. That is the responsibility of the tracing instrumentation.
For warehouse automation, Pearl Gateway recommends the following span hierarchy for a typical divert operation:
- Root span:
process_parcel— from order receipt to chute confirmation. - Child span 1:
send_divert_command— from WCS to PLC. - Child span 2:
execute_divert— PLC logic to fire the divert arm. - Child span 3:
confirm_physical_action— wait for proximity switch and photoeye. - Child span 4:
publish_completion— PLC publishes to MQTT.
Each span has a start time, an end time, a duration, and a status (OK, ERROR, or TIMEOUT). The duration of each span is a key diagnostic metric. If confirm_physical_action takes 800 milliseconds (illustrative assumption) when the expected duration is 300 milliseconds, that indicates a mechanical or sensor issue.
Traces should be sampled to control storage costs. A common strategy is to record 100% of error traces and a percentage of successful traces, such as 10% (illustrative assumption). The sampling decision should be based on the volume of traffic and the storage budget.
Worked example #
This example demonstrates how to correlate a trace, a metric, and a physical confirmation to diagnose a jam on a belt-to-roller transfer.
Inputs #
| Input | Value | Unit | Source / label |
|---|---|---|---|
| Parcel ID | P-98765 | — | Illustrative assumption |
| Trace ID | 4bf92f3577b34da6a3ce929d0e0e4736 |
— | W3C format [S1] |
| Command send time | 10:00:00.000 | UTC | Illustrative assumption |
| PLC execution start | 10:00:00.050 | UTC | Illustrative assumption |
| Photoeye PE-104 block time | 10:00:00.350 | UTC | Illustrative assumption |
| Photoeye PE-104 unblock time | 10:00:02.150 | UTC | Illustrative assumption |
| Expected photoeye block duration | 0.8 | s | Illustrative assumption |
| Conveyor belt speed | 1.5 | m/s | Illustrative assumption |
| Parcel length | 0.6 | m | Illustrative assumption |
Intermediate calculations #
Step 1: Command-to-execution latency.
Latency = PLC execution start − Command send time = 10:00:00.050 − 10:00:00.000 = 0.050 s = 50 ms.
Step 2: Physical confirmation delay.
Delay = Photoeye block time − PLC execution start = 10:00:00.350 − 10:00:00.050 = 0.300 s = 300 ms.
Step 3: Expected photoeye block duration.
Expected duration = Parcel length / Conveyor belt speed = 0.6 m / 1.5 m/s = 0.4 s.
Step 4: Actual photoeye block duration.
Actual duration = Photoeye unblock time − Photoeye block time = 10:00:02.150 − 10:00:00.350 = 1.800 s.
Step 5: Excess blockage time.
Excess = Actual duration − Expected duration = 1.800 s − 0.4 s = 1.4 s.
Result #
The photoeye was blocked for 1.8 seconds, which is 1.4 seconds longer than the expected 0.4 seconds. The parcel either stalled on the transfer or was followed by another parcel that kept the photoeye blocked. The trace shows that the command-to-execution latency (50 ms) and the physical confirmation delay (300 ms) are within normal ranges, so the issue is not in the control path. The excess blockage time points to a mechanical issue, such as a worn belt on the transfer or a misaligned side guide.
Sensitivity #
The result is sensitive to the expected photoeye block duration, which depends on the parcel length and conveyor speed. If the parcel length is actually 0.9 m instead of 0.6 m, the expected duration is 0.9 / 1.5 = 0.6 s, and the excess is 1.2 s. If the conveyor speed is actually 1.2 m/s instead of 1.5 m/s, the expected duration is 0.6 / 1.2 = 0.5 s, and the excess is 1.3 s. In all cases, the excess is significant, so the conclusion is robust.
Limitations #
This example assumes that the photoeye is the only confirmation sensor and that the parcel length is known. In practice, the parcel length might vary, and the photoeye might be triggered by a gap between parcels. The example also assumes that the clocks are synchronized; if the PLC clock and the WCS clock differ by more than 50 ms, the latency calculation is invalid. This example is for observability diagnosis only and must not be used for safety decisions.
Architecture recommendations: edge gateway, broker, and collector #
The architecture for warehouse observability depends on the site’s network topology, the number of devices, and the available bandwidth. Pearl Gateway provides the following editorial recommendations, which are not requirements of any cited standard.
Edge gateway: An edge gateway sits between the PLC network and the enterprise network. It runs an OPC UA client to read equipment state from the PLCs, an MQTT client to publish telemetry, and a trace collector to receive spans from the WCS. The edge gateway should buffer data locally so that telemetry is not lost during a network outage.
MQTT broker: The MQTT broker [S3] is the central hub for telemetry. It receives messages from edge gateways and forwards them to subscribers, such as the observability platform and the WCS. The broker should be configured with appropriate message expiry intervals and quality of service (QoS) levels. For telemetry, QoS 1 (at least once) is typically sufficient; QoS 2 (exactly once) is rarely needed and adds overhead.
Observability platform: The observability platform stores logs, metrics, and traces in a queryable format. It should support correlation by trace ID, timestamp range, and equipment name. The platform should also provide dashboards for monitoring and alerting.
Time series database: Metrics are best stored in a time series database that supports downsampling and retention policies. The retention period should be based on the site’s requirements; 13 months is a common choice for trend analysis, but this is an illustrative assumption.
The architecture should be designed so that the observability platform can be replaced without affecting the control system. The PLCs should not depend on the observability platform for any control or safety function.
Network diagnostics and telemetry quality #
Observability is only as good as the network that carries it. A dropped MQTT message or a delayed OPC UA read can corrupt the correlation. Network diagnostics are therefore a prerequisite for observability.
Pearl Gateway’s article on Managed Switch Diagnostics describes common failure modes such as port errors, CRC errors, and buffer overflows. These diagnostics should be collected as metrics and correlated with telemetry quality. For example, if the MQTT publish latency increases at the same time as the switch reports a high error rate, the root cause is likely the network, not the application.
The following network metrics should be monitored:
- Port utilization: Percentage of bandwidth used on each switch port.
- Error rate: CRC errors, alignment errors, and collisions per second.
- Discard rate: Frames discarded due to buffer overflow or filtering.
- MQTT publish latency: Time from publish to broker acknowledgment.
- OPC UA read latency: Time from request to response.
These metrics should be correlated with the trace data. If a trace shows a 500 ms gap between the PLC execution and the MQTT publish, the network metrics should be checked to see if the gap is due to network congestion or a slow application.
Equipment-specific observability signals #
Different types of warehouse equipment produce different telemetry signals. Pearl Gateway’s internal documentation provides guidance for specific equipment types, and the following summary highlights the key observability signals.
Strapping machines: The Strapping Machine Interfaces article describes inspection points such as strap tension, seal quality, and cycle time. These should be captured as metrics, with traces for each strapping cycle.
Belt-to-roller transfers: The Belt-to-Roller Transfers article covers commissioning checks such as belt tracking and roller alignment. During operation, the key observability signals are photoeye blockage duration, motor current, and transfer time.
Conveyor side guides: The Conveyor Side Guides article describes data signals such as guide position and wear indicators. These should be monitored as metrics, with alerts when the guide position drifts from the setpoint.
Tilt-tray sorters: The Tilt-Tray Sorters article describes data signals for tray tilt, induction, and discharge. The key observability signals are tray position, tilt time, and discharge confirmation.
Bomb-bay sorters: The Bomb-Bay Sorters article covers selection criteria and application boundaries. For observability, the key signals are the opening and closing time of the bomb-bay doors and the confirmation that the parcel dropped to the correct chute.
Capacity constraints: The Capacity Constraint Analysis article describes how to identify bottlenecks. Observability data should be used to measure the utilization of each equipment segment and to identify where queues are forming.
Each equipment type should have a defined set of observability signals, with units, thresholds, and correlation IDs. The thresholds should be based on the equipment manufacturer’s specifications and the site’s operational experience, not on generic defaults.
EtherNet/IP and PLC integration for observability #
EtherNet/IP is a common protocol for PLC communication in warehouse automation. Pearl Gateway’s article on EtherNet/IP Connections describes the operating principles and system boundaries. For observability, the key challenge is to extract telemetry from the PLC without interfering with the control logic.
Pearl Gateway recommends the following approach:
- Separate telemetry tags: Use dedicated PLC tags for telemetry data, such as
TraceID_Array,LastCommandTime, andLastPhotoeyeBlockTime. These tags are written by the control logic but read only by the observability system. - Read via OPC UA: Use an OPC UA server to expose the telemetry tags to the edge gateway. This avoids direct EtherNet/IP reads from the observability platform, which could interfere with the PLC’s scan cycle.
- Event-based logging: Configure the PLC to log events (e.g., photoeye block, motor start, alarm) to a local buffer. The edge gateway reads the buffer periodically and publishes the events to MQTT.
- Do not add logic: The observability system must not add logic to the PLC program. The PLC program should be identical whether or not the observability system is connected.
The EtherNet/IP connection should be configured with appropriate timeouts and retries. If the observability system loses connection to the PLC, it should buffer data locally and reconnect with a backoff strategy. The PLC should not be affected by the loss of the observability connection.
Safety boundaries: telemetry is not safety logic #
The most important boundary in warehouse automation is between safety and non-safety functions. Observability data—logs, metrics, traces, and physical confirmation signals—is non-safety data. It must never be used as the sole input to a safety function.
NIST SP 800-82 Rev. 3 [S4] provides guidance for OT security, but it does not define safety requirements. Safety is governed by separate standards, such as IEC 61508 and ISO 13849, which are not cited in this article. Pearl Gateway does not provide safety engineering services, and this article does not constitute safety guidance.
The practical implications are:
- Do not use photoeye confirmation for safety: A photoeye that confirms a parcel’s position can be used for observability and for non-safety control, but it must not be used as the sole input to a safety function, such as stopping a conveyor to prevent a collision.
- Do not use telemetry for safety decisions: A metric that shows a motor is overheating is a warning, not a safety trip. The safety trip must be implemented with a dedicated safety-rated thermal overload relay.
- Do not use trace data for safety: A trace that shows a command was sent is not proof that the physical action occurred safely. Physical confirmation is observability data, not safety data.
- Document the boundary: The system design should clearly document which signals are safety-rated and which are not. This documentation should be reviewed by qualified safety engineers.
This boundary is not a repetitive boilerplate; it is a fundamental engineering principle. Mixing observability and safety can lead to unsafe conditions, such as a safety function that is disabled because a telemetry message was lost.
When this guidance does not apply #
This guidance does not apply in the following situations:
- Safety-rated systems: Any system that performs a safety function, as defined by applicable safety standards, is outside the scope of this article. Observability data must not be used for safety decisions.
- Standalone machines: A single, standalone machine with no network connectivity and no WCS integration does not benefit from distributed tracing. A simpler approach, such as local logging and manual inspection, may be sufficient.
- Real-time control loops: Control loops that require deterministic, sub-millisecond response times are not suitable for observability architectures that introduce network latency. The observability system must be passive and non-interfering.
- Legacy systems without trace support: If the PLC or WCS cannot support trace ID propagation, the correlation strategy described in this article cannot be fully implemented. A partial implementation, such as timestamp-based correlation, may be the only option.
- Very small sites: A site with a single conveyor and a handful of sensors may not justify the complexity of an MQTT broker, an OPC UA server, and a trace collector. A simpler data logging approach may be more cost-effective.
- Cybersecurity-specific analysis: This article does not address cybersecurity controls, such as
Sources and standards #
- W3C — Trace Context Recommendation. In “Observability for Warehouse Automation: Logs, Metrics, Traces and Physical-Flow Correlation”, source [S1] supports the attributed terminology or boundary; the warehouse-specific synthesis remains Pearl Gateway editorial analysis.
- OPC Foundation — OPC UA Online Reference. In “Observability for Warehouse Automation: Logs, Metrics, Traces and Physical-Flow Correlation”, source [S2] supports the attributed terminology or boundary; the warehouse-specific synthesis remains Pearl Gateway editorial analysis.
- OASIS — MQTT Version 5.0 Specification. In “Observability for Warehouse Automation: Logs, Metrics, Traces and Physical-Flow Correlation”, source [S3] supports the attributed terminology or boundary; the warehouse-specific synthesis remains Pearl Gateway editorial analysis.
- NIST — Guide to Operational Technology Security, SP 800-82 Rev. 3. In “Observability for Warehouse Automation: Logs, Metrics, Traces and Physical-Flow Correlation”, source [S4] supports the attributed terminology or boundary; the warehouse-specific synthesis remains Pearl Gateway editorial analysis.
- NASA — NASA Systems Engineering Handbook. In “Observability for Warehouse Automation: Logs, Metrics, Traces and Physical-Flow Correlation”, source [S5] supports the attributed terminology or boundary; the warehouse-specific synthesis remains Pearl Gateway editorial analysis.
Revision and editorial note #
The Pearl Gateway Editorial Team prepared “Observability for Warehouse Automation: Logs, Metrics, Traces and Physical-Flow Correlation” from the five linked source records. The published guide remains educational and requires site evidence before application.