Article

Drone safety cases need repeatable test conditions, not another successful demo

A drone completing one difficult flight can be useful evidence that a concept works. It is much weaker evidence that the same system will keep working when wind, navigation quality, communications latency, payload, software version or operator workload changes.

That distinction is becoming more important as operators move from supervised trials toward routine beyond-visual-line-of-sight operations. The new National Autonomy and Drone Innovation Centre at Westcott in the UK is interesting for that reason. Satellite Applications Catapult says the facility will add controlled position, navigation and timing tests, satellite and terrestrial communications testing, a remote-operations control room, repeatable indoor wind, robotic handling and a motion platform for moving-vessel and vehicle scenarios. The first equipment is due to enter service in autumn 2026, with the full centre planned for the second half of 2027. Satellite Applications Catapult

The facility does not prove any aircraft safe by itself. What it can provide is something the industry needs more of: a way to turn "it flew successfully" into measurements of where a system works, where it stops working, and how confidently those limits are known.

A successful demonstration does not define an operating envelope

A public demonstration tends to answer a simple question: did the mission finish?

A safety case needs harder answers. How much crosswind can the aircraft tolerate before landing error grows beyond the allowed area? How long can the command link degrade before the vehicle changes mode? What happens when GNSS quality falls gradually rather than disappearing cleanly? Does a detect-and-avoid system keep the same alerting performance when targets approach from different geometries? Can the remote pilot recognize a failure before the automation reaches its own limit?

Those questions are about boundaries and repeatability.

The FAA describes its UAS Test Site Program in similar terms. Its stated objective is to support verification of the safety of public and civil UAS, operations and navigation procedures before integration into the National Airspace System. The nine designated sites work on detect and avoid, command and control, airworthiness, BVLOS, UTM, multiple-aircraft operations and evaluation of proposed standards and procedures. FAA UAS Test Site Program

The point is not simply to accumulate flight hours. Useful test hours are tied to a question, a configuration and a measurable pass or failure condition.

Controlled variables make failure data useful

Field trials are necessary because the real environment always contains interactions a laboratory missed. They are also poor places to isolate causes.

If a multirotor drifts during a windy outdoor test, the log may contain wind, GNSS multipath, control saturation, changing payload motion and operator intervention at the same time. Repeating the flight tomorrow does not recreate the same conditions.

NIST's work on standard test methods for emergency-response robots shows the alternative. It decomposes missions into elemental capabilities such as maneuvering, sensing, endurance, radio communication, autonomy and safety, then defines repeatable apparatuses, procedures and metrics. Variables are controlled first to establish a baseline and introduced incrementally to measure how performance degrades. NIST Performance of Emergency Response Robots

That approach matters for drones because an operational limit is rarely one number. "Maximum wind 12 m/s" is less useful than a curve showing position error, power consumption, control margin and landing accuracy as wind speed and direction change. A repeatable wind generator such as the Windshaper planned for Westcott can produce that kind of evidence more efficiently than waiting for comparable weather outside.

The same principle applies to pilot proficiency. NIST's small-UAS work uses repeatable tasks and scoring so system capability and operator performance can be measured rather than described impressionistically. NIST UAS Standard Test Methods

Navigation resilience should be tested as degradation, not as a binary outage

GNSS failures are often discussed as if the receiver either has a valid position or has no position. Real failures can be less cooperative.

A receiver may retain a solution while accuracy degrades. Corrections may disappear. Multipath can bias the position. Spoofed measurements can look internally plausible. Different estimators may disagree for seconds before a fault monitor crosses a threshold.

Westcott's planned PNT facilities are intended to let developers test when satellite navigation signals are unavailable, disrupted or manipulated. That is more useful than simply switching GPS off because it lets a team measure detection thresholds and transition behavior.

A strong test matrix should record at least the navigation source, interference or degradation profile, estimator state, integrity flags, control-mode transitions and resulting path error. The important result is not that the aircraft eventually entered a fallback mode. It is how far the state estimate was allowed to diverge before that happened and whether the fallback remained inside the approved containment volume.

The same configuration discipline applies to every result. A PNT test is only portable if the receiver firmware, antenna, autopilot build, estimator parameters and relevant payload configuration are recorded. Change one of them and the evidence may need to be repeated.

Communications testing needs latency distributions and failure transitions

BVLOS communications are another area where nominal coverage can hide the engineering problem.

A link can remain technically connected while latency becomes too large for a particular control function. Cellular and satellite paths can fail differently. Switching between them can create short interruptions. Video may be unusable while telemetry remains healthy, or the reverse.

The planned Satellite Communications Integration & Test Lab at Westcott is meant to validate autonomous systems across satellite, terrestrial and hybrid networks. A useful test program would therefore measure more than average throughput. It should capture latency distribution, packet loss, outage duration, handover time, geographic or RF conditions, and the vehicle state reached when each path becomes unavailable.

Those measurements then need to be connected to operational behavior. A 500 ms delay may be acceptable for slow telemetry and unacceptable for another function. A two-second outage may be irrelevant if the aircraft continues a bounded mission autonomously, or critical if the operation depends on immediate human intervention.

Testing the network and testing the aircraft separately can miss that interaction. The real safety property is the closed loop formed by aircraft, communications service, remote-control software, automation and operator.

Detect-and-avoid systems need repeatable encounter geometry

Detect and avoid has the same evidence problem.

ASTM F3705-25 is a guide for testing DAA systems and describes methodologies and data collection intended to demonstrate performance characteristics against DAA requirements. It treats detection, alerting and avoidance as a connected chain rather than a single sensor specification. ASTM F3705-25

That distinction matters because a sensor can detect a target without the total system producing a useful avoidance action. Encounter angle, closure rate, target size, clutter, alert timing, tracking stability and autopilot response all affect the result.

A convincing DAA campaign therefore needs repeated encounter geometries and explicit metrics. It should show not just successful avoidance events but missed detections, late alerts, unnecessary maneuvers and the conditions around them.

This is where a test facility earns its value. Recreating the same difficult geometry several times is tedious in ordinary operations. It is exactly what is required if an engineer wants to know whether a software change actually improved the system.

Regulators care about assurance as well as claimed performance

The latest EASA UAS rules make this distinction explicit through SORA 2.5.

SORA describes robustness as the combination of integrity and assurance. Integrity is how effective a mitigation or safety objective is. Assurance is how certain the applicant can be that the claimed integrity has actually been achieved. Higher robustness demands stronger evidence and, at the highest level, verification acceptable to the competent authority or a designated entity. EASA Easy Access Rules for UAS

That is why repeatable testing is more than an engineering convenience. It can be part of the chain that connects a safety claim to evidence.

Suppose an operator claims that a vehicle remains controllable after loss of its primary navigation source. A demonstration can illustrate the behavior. A stronger case defines the relevant failure conditions, repeats them, measures the outcome, records configuration and uncertainty, and shows that the results cover the intended operation.

The gap between those two forms of evidence is where many promising systems slow down.

What builders and operators should ask for

A useful test report should make it possible to reproduce the claim. For a drone, subsystem or remote-operations stack, the most important questions are practical:

  • What exact hardware, firmware, parameters and payload were tested?
  • What environmental variables were controlled, and which were deliberately changed?
  • What was the pass or failure criterion before the test began?
  • How many repetitions were run?
  • What did the failures look like, not only the successful runs?
  • Were latency, position error, control margin and energy use recorded as distributions or only averages?
  • Which parts of the evidence depend on a particular communications or navigation provider?
  • What changed between test campaigns?
  • Which result defines an operating limit, and which result is merely a demonstration point?

This level of detail is less impressive in a launch video. It is much more useful when an operator has to decide whether a system belongs in a real safety case.

Shared test infrastructure will not remove the need for field evidence

Controlled facilities have limits.

A Windshaper cannot reproduce every atmospheric interaction around buildings. A PNT simulator cannot recreate every urban RF environment. A motion platform can approximate vessel movement without reproducing sea spray, corrosion and a tired deck crew. Laboratory communications tests cannot guarantee every commercial network behaves the same way at every site.

The correct relationship is therefore not laboratory instead of field testing. It is laboratory before, between and after field testing.

Controlled tests can identify breakpoints cheaply and repeatably. Field campaigns then test whether those breakpoints still describe the system in the operational environment. When a field failure appears, the team can bring the problem back into a controlled setting, reproduce it, change one variable and test the fix.

That loop is much more valuable than a sequence of increasingly ambitious demonstrations where every flight changes several things at once.

The useful product of a test centre is evidence

The Westcott investment is notable because its planned equipment is aimed at failure boundaries: disturbed navigation, hybrid communications, repeatable wind, remote operations and moving platforms. That is a more mature objective than providing another place where a drone can fly.

The FAA's test-site program, NIST's repeatable robot methods, ASTM's DAA testing guidance and EASA's emphasis on assurance all point in the same direction. Scaling drone operations depends less on proving that a vehicle can complete an impressive mission once and more on showing, with configuration-controlled evidence, how reliably it behaves across the conditions that matter.

A good demonstration answers "can it work?" A good test program answers the question regulators, operators and engineers eventually need: "where does it stop working, and how do we know?"