As was the Healthcare (or “Obamacare”) Website, the adoption and testing of “autonomous vehicles” (I’m tempted to say “Obama Car”) is overwhelmingly a “software testing” project. The analogy between the two, and the lack of solid “buck stops here” ownership is very concerning as I view the “Federal Automated Vehicles Policy”.
In a nutshell, the Federal DOT and NHTSA are (kind of) claiming overall ownership of this project while also admitting they can only make “Best Practice” suggestions. In fact, there appears to be even less direct control over this project than existed for the healthcare website. The size and complexities of this new computer code will no doubt dwarf that created for the healthcare website, and the consequences here are much graver.
The existence of different proprietary systems, and questions as to how (or “if”) they will talk to each other, is one analogous issue. Another is that this new “Highly Automated Vehicle” project (also) involves “retrofitting” software (in this case to a massive and diverse infrastructure that was not designed with these vehicles in mind). Add to this the fact that our roads, bridges, signs, and laws are closely guarded turf under the control of 50 different state jurisdictions. In fact, the NHTSA and Federal DOT have emphatically quoted this jurisdictional issue to me when explaining why they could not address, or even comment on a number of road safety dangers I brought to their attention. So, has something suddenly and magically changed such that these discussions – “prohibited by law” – are now acceptable? Those interested should read my essays located at www.edmundmiller.wordpress.com (In the case of New Jersey’s “Stop and Stay Stopped” crosswalk law, subsequent rises in pedestrian deaths – contrary to concurrent drops in all other types of auto deaths during some of these same years – seemed to confirm my fears, but that is a whole other story!)
People should keep in mind here that the REAL safety testing of automated vehicles will only occur after they are set free on our public streets. This will be what software testers call a “beta test”. There is no similarity whatsoever between the NHTSA’s oversight of straightforward, non varying “crash” tests, and the new responsibilities it has been assigned. For auto manufacturers, let alone the NHTSA, it will be logistically, perhaps even “cognitively” impossible to come even close to the level of testing that would be required (ahead of time) in order to verify safety under the endless number of real world scenarios these cars will encounter. Not only would the costs be prohibitively expensive, but the use of live, walking and talking test subjects (simulating pedestrians for example) would be unethical at full speed operation. So get ready people – we are all the “stakeholders” here!
From what I can see, the NHTSA is not yet requiring the establishment of an easily accessible, always available system by which the general population can report their dangerous or questionable encounters with these vehicles. I thought the whole purpose of a “beta test” was to provide a massive increase in “tester” manpower. By not providing this avenue for feedback, it defeats the whole purpose of a “beta” test!
And, concerning the preliminary testing that will take place, a reliance on the “self-reporting” of results by manufacturers should keep all of our eyebrows raised. Are we really to believe that every time a Tesla driver needs to retake control of his car from the automated system (where a potentially fatal crash would have occurred) this is being tabulated as a “likely fatal incident” in terms of its’ theoretical “fully autonomous” operation? Of course Tesla points out here that their cars are not yet intended to be “fully autonomous”, but the illustration still applies.
I fully appreciate the awkward position the Federal DOT and NHTSA have been forced into by the President’s push to promote this automation. The NHTSA – traditionally the watchdog of physical and design defects after the fact – is now (kind of) claiming ownership of many aspects of the upfront planning phase of this massively impactful, hugely complicated, and loosely defined project. The NHTSA is effectively establishing for itself a future “conflict of interest”. Secretary Fox even said “What we’re doing here is building safety in at the ground floor” when announcing the public release of the FAV Policy. It should be noted that the NHTSA is (kind of) claiming this ownership at the same time it has not been given the resources, expertise, or even the mandate to take proper control. Am I the only person to see an analogy here between this current situation and the inadequate resources in the hands of FEMA prior to their (criticized) responses to a number of subsequent disasters? When bad things happen – and they will – automakers will be able to point to the NHTSA’s (sort of) claims of ownership over the early phases of this project. At the same time, manufacturers will also likely claim immunity due to the lack of specificity established upfront. It is easy to imagine the potential for instances in which the NHTSA might be tempted to cover something up in order to avoid receiving flak from the public. The NHTSA, of all organizations, should have a strong understanding of the environmental conditions leading to poor quality (and “recalls” involving negligence). It is foolish to assume their own employees are somehow immune to these dynamics of self-preservation.
I have also noticed a failure to use unambiguous language, as would be required in the design stage of any software project. This is visible in the language used by the agency as it promotes “this technology”. There is in fact no single or easily encapsulated “technology” here. There are numerous physical technologies (that will no doubt change over time) and an even larger ongoing commitment to producing tons of “new and improved” computer code. If the claim is that “computer code” itself is a “new” thing – this is news (or “olds”) to me. Encompassing everything into one verbally convenient phrase such as “this technology” serves no real purpose. Computer coders cannot code, fix, or be held accountable for “this technology”.
Another ambiguous reference occurs on Page #10 of the FAV Policy. The text toggles “primary responsibility” between the “human operator” and the “automated system”. “Automated systems” are not cognizant beings, don’t “bleed”, and do not pay with their lives when things go wrong. This may sound academic, but confounding these concepts – even when primarily an issue of semantics – creates further “wiggle room” for those car manufacturers (or computer programmers) when things go wrong in the future. This becomes instantly obvious in a legal sense. I am not a lawyer but I am pretty sure the courts would actually hold the driver partially negligible – despite the NHTSA’s claims that the “automated system” was responsible – should an accident or death occur where the driver had previous knowledge that the automated system was not performing up to expectations. I am curious as to just how literal we are to take these descriptions?
We are already seeing auto manufacturers running wild with their proprietary claims surrounding the promise of their own future autonomous vehicles. No doubt much of this is due to their fear of seeming technologically inferior or “behind the curve”. They apparently have no fear that the NHTSA will call them out when it comes to these statements. Elon Musk recently claimed that “half a million lives” would have been saved worldwide had everyone been driving Teslas with the activated “autopilot” feature. He then told people to “Just do the math!” Well, I not only did the math, I also applied some basic scientific considerations such as “sample size”. With this it becomes instantly apparent that his claim (at this point in time) is ludicrous! Again, see “‘Accountability’ and ‘Countability’ – Misdirection in the ‘Autopilot’ Safety Debate” located at www.edmundmiller.wordpress.com for more on this.
There also seems to be a very important (likely high volume and deadly) mistake in the logic applied by the NHTSA when discussing the automation levels of these cars. There is no reason for the NHTSA, or anyone else for that matter, to assume that a human driver – even when fully attentive – will be able to react in time to every mistake made by an automated system! One needs only to imagine themselves in the following situation. If a driver is concentrating intensely on the road ahead and a passenger (out of nowhere) suddenly jerks the steering wheel to the side for no reason, it will spark all sorts of reflexes and reactions within the driver’s mind as he/she attempts to make sense of what just happened. The brain’s response might be “Don’t adjust the wheel because there must have been a good reason why my passenger did this”. Or it could be the exact opposite reaction, thus creating an over compensation in steering. These episodes will always occur – by definition – as complete surprises. There is absolutely no way for drivers to safely practice, or anticipate these realities ahead of time! It is preposterous for the NHTSA to be validating this “assumption of ultimate responsibility” (over mistakes made by the automated system) by applying it to this project! This “clause” is being used as a “catch all” by those involved as a way of avoiding a more complex and realistic discussion surrounding true causal factors.
My overall recommendation to the Federal DOT and NHTSA is that – considering their very limited degree of true ownership over this project – they absolutely must wield every possible element of control they have at their disposal during these early stages. At the very least, the following actions should be taken:
#1) The NHTSA should mandate and monitor the use of a single (overall) “Final Stage Test Plan” (created and updated within a single software application) that is shared, viewed, accessed, and updated by all of the car manufacturers. This single (overall) “Final Stage Test Plan” will list all of the real world scenarios (each one representing a single “test case”) that the cars of each particular “automation level” will need to navigate safely. This particular stage of testing – by definition here – must be conducted using a completely assembled car at speed, with all systems activated (individual component testing to be handled separately). These scenarios (“test cases”) should be reviewed ahead of time for completeness, shared among all manufacturers, and of course then tested by each manufacturer under their own proprietary systems. As new “tricky and dangerous” scenarios are discovered, these new “test cases” must be added to the original test plan (instantly viewable by all – as before). Each manufacturer must assign ownership of the testing of each individual test case to a single tester who will be responsible for literally “signing off” (as in actual “signature”!) when a vehicle passes a particular test. Keep in mind that this “Final Stage Test Plan” only describes the real world scenarios these cars must navigate safely (applicable to all manufacturers) and does not require the recording or revealing of any proprietary information. The potentially proprietary discussions related to the handling of problems (or “bugs”) will be controlled separately under each companies’ individual “project tracking” system as described below.
#2) Though not established by the NHTSA, each manufacturer should of course have their own “project tracking” system to track problems and their resolutions as they occur. This would of course follow standard software practices (assigning a unique identifier to each issue; stating whose hands a particular issue is in at any given moment in time; cross referencing the applicable test case if relevant; and so on).
#3) As these vehicles “go live” on our roadways (the “beta test”), there must be the establishment of an easy to use method by which the general population (bigtime “stakeholders”) can report any and all dangerous encounters. Of course this will lead to duplicate entries as some of the same problems repeat themselves. Therefore, on the same webpage, the NHTSA needs to continually update a “known problems” section enabling citizens to quickly log a “this happened to me also” entry. This will not only save everyone time and trouble, but it will also add important emphasis to particular dangers.
#4) The NHTSA must eliminate completely the notion that drivers can be held ultimately responsible (under any “level” of automation) for mistakes made by an automated system.
A few additional observations on the “FEDERAL AUTOMATED VEHICLES POLICY”
Page 9: SAE Levels “2” and “3” are poorly defined
Page 38: The DOT anticipates increased responsibilities similar to “licensing” of the non-human driver in the future. This is surprising as my local (state) inspection station doesn’t even have the resources to check my tire “tread wear” anymore. Perhaps I am missing something here?
Page 44: The FAV Policy glosses over the issue of liability and insurance coverage as related to the complexities and differences that will occur between the states. This is no small issue. Let’s not forget that in the case of accidents and deaths, all parties will have it in their interest to go after the same entity – that being the “auto manufacturers”. Therefore, it is clear that the first order of business for the manufacturers will be the seeking of legal immunity.
Page 59: As already noted, these automated cars – required to make decisions under an endless array of real world scenarios – will really only be tested once they are released in the real world. This fact greatly hinders the concept of granting “exemptions” based only on limited prior (non-real world) testing. Something to think about!
Page 72: Referring again to the complexities involved in testing the endless scenarios that would need to be handled by an automated car – the NHTSA is kidding itself if it thinks it will have the manpower and resources to adequately test – by itself – even one such vehicle before release to the real world.