Direct answer #
Performance acceptance testing for automated material handling systems must separate throughput, technical availability, operational availability, and accuracy into distinct denominators. Combining these metrics into a single “efficiency” score obscures the root cause of underperformance and makes contractual acceptance ambiguous. Throughput is a rate measured in units per hour under a defined load profile. Technical availability is the ratio of scheduled time minus downtime to scheduled time, excluding operator-induced stops. Operational availability adds external delays such as missing work, changeovers, and staffing gaps. Accuracy is the ratio of correct transactions to total attempts, with a clearly defined numerator for what constitutes a correct outcome. Each metric requires its own sampling plan, confidence interval, and acceptance threshold. This article defines these denominators, provides statistical methods for confidence, and presents a worked example with explicit assumptions.
Key takeaways #
- Separate denominators prevent blame ambiguity: Throughput (units/hour), technical availability (%), operational availability (%), and accuracy (%) each measure a different failure domain. Combining them hides whether the conveyor, the operator, or the control logic caused a shortfall.
- Throughput is load-profile dependent: A rate measured with a favorable SKU mix and no changeovers is not comparable to a rate measured under worst-case sequencing. The test protocol must fix the load profile.
- Technical availability excludes operator and external delays: It answers “did the equipment run when commanded?” Operational availability answers “was the system available when the business needed it?” The two denominators answer different questions.
- Accuracy requires a transaction-level definition: A “correct” outcome must be defined per transaction type—barcode read, divert confirmation, weight match, or seal verification—before testing begins.
- Statistical confidence requires sample size planning: A short test with few cycles cannot prove a 99% availability claim. Confidence intervals and binomial or Poisson statistics must be applied to the observed data.
- Acceptance thresholds must be stated before the test: The pass/fail boundary, the confidence level, and the test duration are contractual parameters, not post-test discoveries.
Definitions and denominators: the four metrics that must not be merged #
Performance acceptance testing in automated warehousing typically involves four distinct metrics. Each has a unique denominator, a unique unit, and a unique failure mode. Merging them into a single composite score is a common editorial and engineering error that makes root-cause analysis impossible.
Throughput is the quantity of units processed per unit of time, typically expressed as units per hour (u/h) or cases per hour (c/h). The denominator is elapsed time under a defined load profile. Throughput is not a percentage; it is a rate. A system that processes 1,200 units in a 2-hour window has a throughput of 600 u/h. The load profile—SKU mix, carton dimensions, weight, and order of presentation—must be fixed in the test protocol because throughput varies with these parameters.
Technical availability is the ratio of time the system is capable of performing its function to the scheduled operating time, expressed as a percentage. The denominator is scheduled time. The numerator is scheduled time minus downtime attributable to equipment failure, control system faults, and maintenance interventions. Operator-induced stops, missing work, and external utility failures are excluded from the technical availability denominator. This metric answers: “Did the equipment run when commanded?”
Operational availability includes all delays that affect the business, not just equipment failures. The denominator is the same scheduled time, but the numerator subtracts equipment downtime plus external delays such as waiting for work, changeovers, operator breaks, and staffing shortages. Operational availability is always lower than or equal to technical availability. The gap between the two metrics quantifies the impact of operational discipline and external dependencies.
Accuracy is the ratio of correct transactions to total transaction attempts, expressed as a percentage. The denominator is the number of attempts. The numerator is the number of attempts that produced the correct outcome. A “correct outcome” must be defined per transaction type before testing. For a barcode read, correct means the read value matches the expected value. For a divert, correct means the unit reached the intended lane. For a weight check, correct means the measured weight is within tolerance.
These four metrics are not interchangeable. A system can have 99% technical availability and 60% operational availability because operators are not feeding it. The same system can have high throughput and low accuracy because the vision system misreads labels. The denominators are different, and the corrective actions are different.
Throughput: the rate denominator and its load-profile dependency #
Throughput is the most visible performance metric in a warehouse automation project, and it is the most frequently misrepresented. The denominator is time, and the numerator is units processed. But the time window and the load profile must be specified with precision.
The test protocol must define the following parameters:
- Test window: The start and end time of the measurement, excluding ramp-up and ramp-down periods. A 2-hour steady-state window is a common illustrative assumption, but the actual window must be stated in the protocol.
- Load profile: The SKU mix, carton dimensions, weight range, and presentation order. A profile with 80% small cartons and 20% large cartons will produce a different throughput than the reverse.
- Changeover policy: Whether changeovers are included in the throughput window. If changeovers are excluded, the throughput number represents pure processing rate. If included, it represents a blended rate.
- Reject handling: Whether rejected units are counted in the numerator. A unit that is processed but rejected for a quality issue is still a processed unit for throughput purposes, but it is not a correct unit for accuracy purposes.
The throughput equation is:
T = N / t
Where:
- T = throughput, expressed in units per hour (u/h)
- N = number of units processed during the test window, expressed in units (u)
- t = duration of the test window, expressed in hours (h)
For example, if a system processes 1,800 units in a 3-hour steady-state window, the throughput is 600 u/h. This is an illustrative calculation, not a field result.
The load profile is the critical assumption. A throughput claim without a load profile is meaningless because the system’s capacity is a function of the work presented to it. The test protocol must specify the load profile in the same detail as the acceptance threshold. The Controlled Equipment Access: Capacity Planning and Bottleneck Analysis article provides additional context on how load profiles interact with bottleneck identification.
Technical availability: the equipment-only denominator #
Technical availability measures the equipment’s ability to run when commanded. The denominator is scheduled operating time. The numerator is scheduled time minus downtime attributable to the equipment itself.
The technical availability equation is:
Atech = (tsched − tdown) / tsched × 100%
Where:
- Atech = technical availability, expressed as a percentage (%)
- tsched = scheduled operating time, expressed in hours (h)
- tdown = downtime attributable to equipment failure, control faults, and maintenance, expressed in hours (h)
Downtime categories that count against technical availability include:
- Conveyor motor failure
- PLC or control system fault
- Sensor failure or misalignment
- Network switch failure
- Preventive maintenance performed during scheduled time
Downtime categories that do not count against technical availability include:
- Waiting for work (no units available to process)
- Operator breaks and shift changes
- Changeovers requested by operations
- External utility loss (power outage, network outage outside the system boundary)
The distinction between technical and operational availability is not a semantic exercise. It determines who is responsible for the shortfall. If technical availability is 98% but operational availability is 80%, the equipment is not the problem—the operations team is. The acceptance test must report both metrics so that the responsible party is identifiable.
The Jam Recovery Governance: Commissioning and Acceptance Checklist provides a structured approach to categorizing downtime events during commissioning, which is directly applicable to availability measurement.
Operational availability: the business denominator #
Operational availability measures the system’s ability to support the business, including all delays that affect throughput. The denominator is the same scheduled time used for technical availability, but the numerator subtracts a broader set of delays.
The operational availability equation is:
Aop = (tsched − tdown − text) / tsched × 100%
Where:
- Aop = operational availability, expressed as a percentage (%)
- tsched = scheduled operating time, expressed in hours (h)
- tdown = equipment downtime, expressed in hours (h)
- text = external delay time, expressed in hours (h)
External delay categories include:
- Waiting for work (empty induction queue)
- Operator breaks, training, or meetings
- Changeovers requested by operations
- Missing or incorrect work orders
- Staffing shortages
Operational availability is the metric that matters to the warehouse manager because it reflects the system’s contribution to the business. But it is not a fair measure of equipment performance. A system can be technically perfect and operationally poor because the upstream process does not feed it.
The acceptance test should report both availability metrics side by side. The gap between them is a diagnostic signal. A large gap indicates an operational or integration issue, not an equipment issue. The Interface Retry Design: Commissioning and Acceptance Checklist is relevant here because interface retries often appear as external delays when upstream systems fail to deliver data or work.
Accuracy: the transaction-level denominator #
Accuracy is the ratio of correct transactions to total attempts. The denominator is the number of attempts, not time. The numerator is the number of attempts that produced the correct outcome.
The accuracy equation is:
Acc = C / Atotal × 100%
Where:
- Acc = accuracy, expressed as a percentage (%)
- C = number of correct transactions, expressed in transactions (txn)
- Atotal = total number of transaction attempts, expressed in transactions (txn)
The definition of “correct” must be established before testing. For each transaction type, the test protocol must specify:
- Barcode read: The read value must match the expected value exactly. A misread, a no-read, or a partial read is incorrect.
- Divert confirmation: The unit must arrive at the intended destination. A unit that arrives at the wrong lane is incorrect, even if it was successfully diverted.
- Weight verification: The measured weight must be within the specified tolerance of the expected weight. The tolerance must be stated in the protocol.
- Seal verification: The seal presence and integrity check must produce the correct pass/fail result.
Accuracy is often confused with throughput. A system can process 600 u/h with 95% accuracy, meaning 570 correct units and 30 incorrect units per hour. The throughput metric counts all 600 units. The accuracy metric counts only the 570 correct ones. Both numbers are needed to understand system performance.
The Retroreflective Photoeyes: Commissioning and Acceptance Checklist and the Presence Detection: Data Signals and Condition Monitoring article are relevant to accuracy testing because sensor-level correctness directly affects transaction-level accuracy.
Statistical confidence: why a short test cannot prove a high claim #
Acceptance testing is a statistical exercise. The test observes a sample of the system’s behavior and uses that sample to make an inference about the system’s true performance. The confidence in that inference depends on the sample size and the observed results.
For availability metrics, the relevant statistical model is often the binomial distribution. Each scheduled minute is either available or not. For accuracy metrics, each transaction is either correct or not. The binomial distribution describes the probability of observing a given number of successes in a fixed number of trials.
The confidence interval for a proportion is given by the Wilson score interval, which is more robust than the normal approximation for small samples or proportions near 0 or 1:
p ± z × √[(p(1−p)) / n + z² / (4n²)] / (1 + z² / n)
Where:
- p = observed proportion (e.g., availability or accuracy), dimensionless
- z = z-score for the desired confidence level (1.96 for 95% confidence, illustrative assumption)
- n = sample size (number of minutes for availability, number of transactions for accuracy), expressed in the appropriate unit
The key insight is that the confidence interval narrows as the sample size increases. A 2-hour test with 120 observed minutes cannot distinguish between 95% and 99% availability with high confidence. A 1-week test with 10,080 observed minutes can.
For throughput, the relevant statistical model is the Poisson distribution, which describes the count of events in a fixed time window. The confidence interval for a Poisson rate is:
λ ± z × √(λ / t)
Where:
- λ = observed rate, expressed in units per hour (u/h)
- z = z-score for the desired confidence level (1.96 for 95% confidence, illustrative assumption)
- t = observation time, expressed in hours (h)
The Managed Switch Diagnostics: Selection Criteria and Application Boundaries article discusses the importance of data quality for statistical analysis, which is directly relevant to confidence interval calculation.
Sample size planning: how many observations are enough #
Sample size planning is the process of determining how many observations are needed to achieve a desired confidence interval width. The required sample size depends on the expected proportion, the desired confidence level, and the acceptable margin of error.
For a proportion, the required sample size is approximately:
n = (z² × p(1−p)) / e²
Where:
- n = required sample size, expressed in the appropriate unit (minutes or transactions)
- z = z-score for the desired confidence level (1.96 for 95% confidence, illustrative assumption)
- p = expected proportion (e.g., 0.99 for 99% availability), dimensionless
- e = desired margin of error (e.g., 0.005 for ±0.5%), dimensionless
For example, to estimate an availability proportion of 0.99 with a margin of error of ±0.5% at 95% confidence:
n = (1.96² × 0.99 × 0.01) / 0.005² = (3.8416 × 0.0099) / 0.000025 = 0.03803 / 0.000025 = 1,521 minutes
This is an illustrative calculation. It shows that approximately 25.4 hours of observation are needed to achieve this precision, assuming the true availability is 99%. If the true availability is lower, the required sample size is different.
For accuracy, the same formula applies with the expected accuracy proportion. If the expected accuracy is 99.5% and the desired margin of error is ±0.2% at 95% confidence:
n = (1.96² × 0.995 × 0.005) / 0.002² = (3.8416 × 0.004975) / 0.000004 = 0.01911 / 0.000004 = 4,778 transactions
This is an illustrative calculation. It shows that nearly 5,000 transactions must be observed to achieve this precision. The test protocol must specify the sample size before testing begins, not after.
The Network Segmentation: Capacity Planning and Bottleneck Analysis article provides context on how data collection infrastructure affects the ability to gather large samples.
Acceptance thresholds: setting pass/fail boundaries before testing #
Acceptance thresholds are the pass/fail boundaries for each metric. They must be specified in the test protocol before testing begins. Post-test threshold setting is a common source of disputes because it allows either party to move the goalposts.
The threshold for each metric should include:
- The target value: The performance level the system is expected to achieve. For example, 98% technical availability.
- The confidence level: The statistical confidence required for the estimate. For example, 95% confidence that the true availability is at least 98%.
- The test duration: The minimum observation period. For example, 72 hours of continuous operation.
- The load profile: The work mix and presentation order during the test.
Table 1 summarizes the threshold parameters for each metric.
| Metric | Unit | Target value (illustrative) | Confidence level (illustrative) | Minimum test duration (illustrative) |
|---|---|---|---|---|
| Throughput | u/h | 600 u/h | 95% | 3 h steady-state |
| Technical availability | % | 98% | 95% | 72 h |
| Operational availability | % | 90% | 95% | 72 h |
| Accuracy | % | 99.5% | 95% | 5,000 transactions |
Table 1: Illustrative acceptance threshold parameters. All values are assumptions for demonstration; actual thresholds must be negotiated and stated in the contract.
The thresholds are not independent. A system that fails the accuracy threshold may still pass the throughput threshold because the two metrics measure different things. The acceptance criteria should state whether all metrics must pass independently or whether a weighted composite is allowed. The Pearl Gateway editorial recommendation is to require independent pass for each metric, because a composite score can hide a critical failure in one domain.
Test protocol design: sequencing, data collection, and failure handling #
The test protocol is the procedural document that governs the acceptance test. It must specify the sequence of tests, the data collection method, and the handling of failures during the test.
The protocol should specify the following elements:
- Test sequence: The order in which the four metrics are measured. A common sequence is: accuracy first (because it requires the least time), then throughput, then availability. The sequence must be stated in the protocol.
- Data collection: The method for recording each transaction and each downtime event. Automated data collection from the control system is preferred over manual logging because it eliminates transcription errors. The Managed Switch Diagnostics article discusses the importance of reliable data collection infrastructure.
- Failure handling: The procedure for responding to equipment failures during the test. If a failure occurs, the test clock may be paused or the failure may be counted as downtime. The protocol must specify which approach is used.
- Ramp-up and ramp-down: The periods at the beginning and end of the test during which the system is stabilizing. These periods are typically excluded from the measurement window.
- Operator roles: The staffing plan during the test. The number of operators and their responsibilities must be specified to ensure the test is repeatable.
The protocol must also specify how the load profile is generated. If the load profile is generated by an upstream system, the interface must be tested as part of the acceptance. The Interface Retry Design article is relevant here because interface failures can appear as throughput or availability shortfalls.
Worked example #
This section presents a complete worked example of performance acceptance testing for a hypothetical case conveyor system. All inputs are illustrative assumptions.
Inputs #
- Test window: 8 hours (480 minutes) of scheduled operation, illustrative assumption
- Steady-state throughput window: 3 hours (180 minutes) within the 8-hour window, illustrative assumption
- Load profile: 70% small cartons (≤ 40 cm), 30% large cartons (> 40 cm), illustrative assumption
- Units processed in throughput window: 1,080 units, illustrative assumption
- Equipment downtime: 24 minutes, illustrative assumption
- External delay: 45 minutes, illustrative assumption
- Total transactions: 4,000, illustrative assumption
- Correct transactions: 3,980, illustrative assumption
- Confidence level: 95% (z = 1.96), illustrative assumption
Intermediate calculations #
Throughput:
T = N / t = 1,080 u / 3 h = 360 u/h
Technical availability:
Atech = (480 min − 24 min) / 480 min × 100% = 456 / 480 × 100% = 95.0%
Operational availability:
Aop = (480 min − 24 min − 45 min) / 480 min × 100% = 411 / 480 × 100% = 85.6%
Accuracy:
Acc = 3,980 / 4,000 × 100% = 99.5%
Result #
The system achieved a throughput of 360 u/h, technical availability of 95.0%, operational availability of 85.6%, and accuracy of 99.5%. These are illustrative results for demonstration.
Confidence intervals #
Technical availability confidence interval:
Using the Wilson score interval with p = 0.95, n = 480, z = 1.96:
Lower bound = 0.926, Upper bound = 0.967 (illustrative calculation)
The 95% confidence interval for technical availability is 92.6% to 96.7%.
Accuracy confidence interval:
Using the Wilson score interval with p = 0.995, n = 4,000, z = 1.96:
Lower bound = 0.993, Upper bound = 0.997 (illustrative calculation)
The 95% confidence interval for accuracy is 99.3% to 99.7%.
Sensitivity #
The results are sensitive to the test duration and the number of transactions. If the throughput window were reduced to 2 hours, the throughput estimate would be based on fewer units and the confidence interval would widen. If the accuracy sample were reduced to 1,000 transactions, the confidence interval would widen to approximately 99.0% to 99.8% (illustrative calculation).
The technical availability estimate is particularly sensitive to the test duration. With only 480 minutes of observation, a single 24-minute failure reduces availability by 5 percentage points. With 10,080 minutes (1 week) of observation, the same 24-minute failure reduces availability by only 0.24 percentage points.
Limitations #
This worked example assumes a steady-state load profile with no changeovers during the throughput window. Real operations include changeovers, which reduce throughput. The example also assumes that all downtime events are correctly classified as equipment or external. Misclassification of downtime events will bias the availability estimates. The example does not account for seasonal variation in load profile or operator learning curves.
Downtime classification: the discipline that makes availability meaningful #
Availability metrics are only as good as the downtime classification that feeds them. Every minute of scheduled time must be classified into exactly one category: operating, equipment downtime, or external delay. The classification must be performed in real time or immediately after the event, not reconstructed from memory at the end of the test.
The classification rules must be specified in the test protocol. The following categories are recommended:
- Operating: The system is processing units or is ready to process units but no work is available. The latter is an external delay, not operating time.
- Equipment downtime: The system is not capable of processing units due to a failure of the equipment, control system, or network infrastructure within the system boundary.
- External delay: The system is capable of processing units but is not processing them due to factors outside the system boundary, such as missing work, operator unavailability, or upstream system failure.
The boundary of the system must be defined in the protocol. For example, if the system includes the conveyor, the sortation unit, and the control system, then a failure of any of these components is equipment downtime. If the system excludes the upstream WMS, then a WMS failure that stops the conveyor is an external delay.
The Jam Recovery Governance article provides a structured approach to classifying and responding to jam events, which is a common source of downtime in conveyor systems.
Accuracy transaction definitions: specifying the correct outcome #
The accuracy metric requires a precise definition of “correct” for each transaction type. The definition must be written in the test protocol and agreed upon by all parties before testing begins.
Table 2 provides illustrative transaction definitions for common automation functions.
| Transaction type | Unit of measure | Definition of correct outcome | Definition of incorrect outcome |
|---|---|---|---|
| Barcode read | Read attempt | Read value exactly matches expected value | No read, misread, or partial read |
| Divert operation | Divert attempt | Unit arrives at intended destination lane | Unit arrives at wrong lane or does not divert |
| Weight verification | Weigh attempt | Measured weight within ±0.5 kg of expected (illustrative tolerance) | Measured weight outside tolerance |
| Seal verification | Seal check attempt | Seal presence and integrity correctly identified | False pass or false fail |
| Label application | Label attempt | Label applied in correct position with correct content | Label missing, misaligned, or incorrect |
Table 2: Illustrative transaction definitions for accuracy testing. All tolerances and definitions are assumptions for demonstration.
The definitions must be testable. A definition that requires subjective judgment, such as “label is readable,” is not acceptable. The definition must specify measurable criteria, such as “label content matches the expected value and label position is within ±5 mm of the specified location” (illustrative assumption).
The accuracy metric should be calculated separately for each transaction type. A system can have 99.9% accuracy on barcode reads and 95% accuracy on diverts. The overall accuracy is a weighted average, but the per-type accuracy is more diagnostic.
Data collection infrastructure: the foundation of credible metrics #
Performance acceptance testing depends on reliable data collection. The data collection infrastructure must capture every transaction, every downtime event, and every unit processed. Manual data collection is acceptable for small tests but is not scalable to the sample sizes required for high-confidence estimates.
The data collection infrastructure should include:
- Transaction logging: Every barcode read, divert, weigh, and seal check must be logged with a timestamp and a result. The log must be tamper-evident and time-synchronized.
- Downtime logging: Every downtime event must be logged with a start time, end time, and classification. The classification must be performed by a trained observer or by the control system.
- Throughput counting: Every unit processed must be counted. The counting method must be independent of the transaction logging to allow cross-verification.
- Time synchronization: All data sources must be synchronized to a common time reference. A 1-second discrepancy between the conveyor PLC and the WMS can cause misclassification of events.
The Network Segmentation article discusses how network architecture affects the reliability of data collection. The Managed Switch Diagnostics article provides guidance on selecting network infrastructure that supports reliable data collection.
Common pitfalls in performance acceptance testing #
Several recurring errors undermine the credibility of performance acceptance tests. These pitfalls are identified from engineering practice and are not attributed to any specific standard.
Pitfall 1: Merging metrics into a composite score. A composite score that combines throughput, availability, and accuracy into a single number obscures the root cause of underperformance. A system that fails accuracy but passes throughput will appear to pass a composite threshold, hiding a critical defect.
Pitfall 2: Testing under an unrepresentative load profile. A test that uses only small cartons or only one SKU will produce a throughput number that is not achievable in real operations. The load profile must represent the expected operational mix.
Pitfall 3: Excluding downtime events from the availability denominator. Some test protocols exclude certain downtime categories, such as jam recovery or sensor cleaning, from the availability calculation. This practice inflates the availability number and hides recurring problems.
Pitfall 4: Insufficient sample size. A 2-hour test cannot prove a 99% availability claim. The sample size must be calculated before testing and must be sufficient to achieve the desired confidence interval width.
Pitfall 5: Ambiguous accuracy definitions. If the definition of “correct” is not specified before testing, disputes are inevitable. The definition must be written, measurable, and agreed upon by all parties.
Pitfall 6: Ignoring the gap between technical and operational availability. A large gap indicates an operational or integration problem. Ignoring the gap and focusing only on technical availability will not improve business outcomes.
Safety and site-specific decision boundaries #
Performance acceptance testing must not compromise safety. The test protocol must comply with all applicable safety regulations, including machine guarding requirements. The OSHA General Requirements for Machine Guarding (29 CFR 1910.212) [S4] establishes the minimum safety requirements for machines, including conveyors and automated equipment. The test protocol must ensure that all guards are in place, all safety interlocks are functional, and all emergency stop devices are accessible before testing begins.
Site-specific decision boundaries include:
- Lockout/tagout procedures: The test protocol must specify when and how lockout/tagout is applied. Testing that requires access to guarded areas must follow the site’s lockout/tagout procedure.
- Personnel exclusion zones: The test protocol must define the areas where personnel are not permitted during automated operation. These zones must be clearly marked and enforced.
- Emergency stop testing: The emergency stop function must be tested before the performance test begins. The test must verify that the emergency stop halts all motion within the specified time.
- Speed limits: The test protocol must specify the maximum operating speed. Testing at speeds above the design limit is not acceptable.
Safety is not a performance metric. A system that achieves all performance targets but has a safety defect does not pass acceptance. The safety review is a gate that must be passed before performance testing begins.
When this guidance does not apply #
This guidance applies to performance acceptance testing of automated material handling systems where the four metrics—throughput, technical availability, operational availability, and accuracy—can be measured independently. There are situations where this guidance does not apply.
Prototype or pilot systems: For a prototype system that is still being debugged, the statistical rigor described here is premature. The goal of prototype testing is to identify and fix defects, not to prove statistical performance. The sample size requirements and confidence intervals described here should be applied only after the system is stable.
Systems with no automated data collection: If the system cannot log transactions and downtime events automatically, the sample sizes required for high-confidence estimates are impractical. Manual data collection is feasible for small samples but not for the thousands of observations required for 95% confidence at tight margins.
Regulatory compliance testing: If the acceptance test is required by a regulatory body with specific test methods, those methods take precedence over this guidance. This article does not supersede any regulatory requirement.
Safety validation: This guidance does not cover safety validation. Safety functions must be validated according to applicable safety standards and regulations, not according to the statistical methods described here. The OSHA machine guarding standard [S4] is an example of a regulatory requirement that is not addressed by this guidance.
Systems with continuous process characteristics: For systems that process a continuous flow rather than discrete units, such as bulk material handling, the transaction-based accuracy metric does not apply. The accuracy metric described here assumes discrete, countable transactions.
Legacy system benchmarking: If the goal is to benchmark an existing system rather than to accept a new system, the statistical rigor described here may be excessive. A simpler measurement approach may be sufficient for benchmarking purposes.
Revision and editorial note #
This article was prepared by the Pearl Gateway Editorial Team. It was reviewed against the sources listed in the “Sources and standards” section, and all attributed external facts are drawn from those sources. The statistical formulas are derived from the NIST Engineering Statistics Handbook [S1] and are presented with clearly labeled illustrative assumptions. The warehouse-specific architecture choices and editorial recommendations are the opinion of the Pearl Gateway Editorial Team and are not presented as requirements imposed by any cited standard. This article remains educational and does not constitute engineering advice for a specific installation. Site-specific decisions must be made by qualified engineers familiar with the actual system and its operational context.
Sources and standards #
- NIST — Engineering Statistics Handbook. In “Performance Acceptance Testing: Throughput, Availability, Accuracy and Statistical Confidence”, source [S1] supports the attributed terminology or boundary; the warehouse-specific synthesis remains Pearl Gateway editorial analysis.
- NASA — NASA Systems Engineering Handbook. In “Performance Acceptance Testing: Throughput, Availability, Accuracy and Statistical Confidence”, source [S2] supports the attributed terminology or boundary; the warehouse-specific synthesis remains Pearl Gateway editorial analysis.
- MHI — Automated Storage and Retrieval Systems Fundamentals. In “Performance Acceptance Testing: Throughput, Availability, Accuracy and Statistical Confidence”, source [S3] supports the attributed terminology or boundary; the warehouse-specific synthesis remains Pearl Gateway editorial analysis.
- OSHA — General Requirements for Machine Guarding, 29 CFR 1910.212. In “Performance Acceptance Testing: Throughput, Availability, Accuracy and Statistical Confidence”, source [S4] supports the attributed terminology or boundary; the warehouse-specific synthesis remains Pearl Gateway editorial analysis.
- OPC Foundation — OPC UA Online Reference. In “Performance Acceptance Testing: Throughput, Availability, Accuracy and Statistical Confidence”, source [S5] supports the attributed terminology or boundary; the warehouse-specific synthesis remains Pearl Gateway editorial analysis.
Revision and editorial note #
The Pearl Gateway Editorial Team prepared “Performance Acceptance Testing: Throughput, Availability, Accuracy and Statistical Confidence” from the five linked source records. The published guide remains educational and requires site evidence before application.