Self-driving cars should be taken seriously now that Mercedes is talking seriously about it, is a summary view of the Fairfax newspaper published comments of NSW Roads Minister Duncan Gay.
Here in lies the problem.
Lives are at stake. It doesn’t seem enough to go by reputation alone.
Consider a stream of traffic crossing an intersection. Crossing the streams may not be healthy. One rogue car travelling orthogonal to the flow flies into the fray causing fear, fatality and forensic overtime. It’s all effed up. The litigious blame game car manufacturers are so frightened of begins.
There are surprisingly similar parallels to HFT here for me. A rogue algo or HFT system takes out a busy security, exchange or market. The blame game also necessarily begins.
Reputable enough shouldn’t be a sufficient condition. The value of the damage, whether it be a single life or the economic confidence in a market system, is significant. Clearly the risk to life is a more present danger than that of economic liberty but the metaphor works as a thought experiment. Perhaps advocates of self-driving cars could learn from the experiences in market systems, including the failures that have already taken place just as market participants can learn from the nascent thinking around self-driving cars.
The SEC15c3-5 market rules prescribe risk control systems to be in place to disallow certain unpleasant behaviours. Those risk systems are not necessarily the independent systems you may expect though. There is no real detailed specification of their manifestation nor independence. Perhaps there should be some independence of systems mandated to stop rogue orders even if just internal independence. Kill switches triggered by drop-copy powered independent risk are like crumple zones and air bags that minimise the damage but only if the accident impact is constrained enough. Running into a wall at 200km/h ain’t gonna be pretty but travelling at 200km/h on an autobahn is safe enough with the right vehicle and circumstances. The system has to be well designed and tested as a poorly designed kill switch would be next to useless. No point the air bag coming out a minute after the accident. Controlling the unaccounted for trades within kill switch kill time, the accident impact window, is obviously important. Auditable, accountable and independent it must be.
An immediate death system for traders if they turn violently in an uncontrolled manner with their high speed trading and blow out their risk would certainly create an interesting attention focus. Perhaps too much of a parallel to the driving analogy but not doubt an increase in accountability, perhaps by diversifying the penalty, may assist. Traders being paid to take an call option on their future profit is certainly a dangerous enough game.
Car manufacturers are fearful of the costs of damage, via litigation, of getting things wrong. This is a mixed blessing as it stunts innovation but encourages safety. It's hard to get the balance right. Perhaps market systems could learn from this. What if every trading participant had to have a certificate of currency from an insurance firm for a bazillion dollars to cover the rogue electrons or photons? Conceivably there may be good old fashioned self interest at work making the markets function as a safer place. Smart insurers could make such insurance inexpensive by insisting on and auditing control systems. Unfortunately, there is always the black swan risk in face of the inevitable poisson boredom between accidents where complacency festers amongst insurers in both oversight and premia. Perhaps the private sector, with the right incentives to not get it wrong and the added diversity and creativity could provide an advance on mono-culture regulation. This may be an advance in accountability or it could be just more bureaucracy. We need to do the impossible and get the balance right but having the knobs available to tune the instrument would be a start.
Collision avoidance and mitigation in an automated car is somewhat analogous to a concurrently run risk powered kill switch. Some independence of collision mitigation system or systems in the car's automated driving framework would be useful so that they don’t stuff up at the same time.
In my car, I’d like not just separate sensors but n-versioned systems where the versions are truly independent to automate crash prevention. I’m happy enough to trust a single vendor, such as Mercedes or Google, with my driving system and get one efficient voice, but I want rigorous and independent safety oversight to protect my family in that car. N-versioned safety should be prescribed, similar to space, air or medical systems where independent thinkers build to the same specification and majority voting on the decisions takes place as it did on the space shuttle. When my life is at stake, I'd like a third independent opinion. It doesn't have to be onerous if we amortise the cost over many vehicles.
History has shown that formal methods, such as program proofs, and diversity through techniques such as n-versioning, can build more reliable systems, though they have never really scaled to the reliability heights that were expected. Errors in proofs and code are often correlated. Independent thinkers often make similar errors in building systems. There are simply not the pure multipliers of independence of error you’d hope for. There is also the reliance on components in the systems that do not have the same rigorous methodology. A provably correct system is no solace if your batteries may overheat or your o-rings malfunction. Systems and components matter. Engineering still counts. If it were easy to solve, there wouldn't be a problem. The bottom line still remains that safer is better regardless of such shortcomings.
If the roads were intelligent and could disable or control traffic with accident prevention mechanisms, that could be a good safety overlay. Imagine big brother radio signals controlling your car's safety as an independent arbiter. Cue safety barriers popping up out of the road to prevent unintended crossings of the flow. I'm not a great driver, being easily distracted behind the wheel, but I worry, perhaps wrongly, more about other drivers that I have no control over.
This does seem a bit expensive for the road system. However, for exchanges, the economics are different and it should be just good housekeeping. Sure the cars / market participants need to do the right thing, but if the cost effectiveness works for the infrastructure to have additional safety, why not? Rogue begone.
Multiple independent systems and the correct financial incentives to invest in risk mitigation are required. It seems we're not quite there yet. I think both self-driving car hopefuls and parties responsible for market structure could learn a bit from each other.
Fake safety is dangerous, just as uncontrolled speed may be. Try saying, "whale oil beef hooked" too fast and you'll understand.
All the best in trading safety. Double entendre intended.
 Whilst the FairFax original articles have been modified and I thought my memory deceived me, the original lives on in the AAP records.
 Fake irish lyrics