Uber was in the headlines again last week, this time because on of their driverless cars was involved in an accident which killed a cyclist. Loss of life is always a tragedy and I don’t want to diminish the significance of this death, however, accidents like this are likely to happen as we develop AI and we should be able to agree on situations where projects must be shut down and times when they can continue.
We saw footage released showing the moments leading up to the collision. If we take the footage to be an honest, accurate and unaltered representation of events it appears that the car had very little opportunity to avoid this happening, with the cyclist crossing the road away from a designated crossing, unlit and shortly after a corner.
It’s hard to watch the footage without imaging yourself in that situation, and it’s hard to see how a human driver could have avoided the incident. There would only have been a split second to react. It’s quite possible that both human and machine would have produced the same result – Yet humans continue to be allowed to drive, and Uber is shutting down its self-driving vehicle programme.
So – Following a human death, how can we decide when our projects must be axed and when they can continue?
Intentional vs Incidental vs accidental
I would like to propose three categories of machine caused death. Under two of the three circumstances (intentional and incidental) I suggest that the programmes must be shut down. Under the 3rd (accidental) the project may continue, depending on a benchmark I will set out shortly.
Intentional death caused by AI will result from the likes of ‘lethal autonomous weapons’. I would propose that these should be banned under all circumstances from ever being created. As Max Tegmark described in Life 3.0, AI has the potential to be either the greatest tool ever created for humanity, or the most destructive – The latter being killbots. We want AI to go in the first direction like Chemistry or biology, which became useful to humanity rather than becoming chemical and biological weapons respectively – We have international treaties to ban them. Nuclear had the potential to be simply a power source to help humanity, but has ended up with a dual purpose – Generating energy to power our homes and incredibly destructive weapons.
Here are a few of the most poignant issues possible;
- With AI the risk is potentially higher than nuclear weapons. A machine with the coded right to take human life could do so with an efficiency orders of magnitude higher than any human could – Infecting our healthcare systems, power, or even launching our own nuclear weapons against ourselves.
- As a race we are yet to create our first piece of bug-free software, and until we do we do, we run the risk of this extremely fast automated system killing people we had never aimed for it to. And even if we regain control of the device within days, hours or minutes, the damage done could be thousands of times greater than any human could have achieved in that time.
- Using a machine only adds in a layer of ethical abstraction that allows us to commit atrocities (Automating Inequality, Virginia Eubanks).
An incidental death can be categorised as death which happens as the result of another action, but not as the primary motivation. This would include any action where and automated system was sure, or attributed a high probability to the possibility of a person being seriously injured or killed as the result of its primary goal or the steps taken to achieve it. We may imagine machines allowing this to happen ‘for the greater good’, as an acceptable step towards its primary goal. This should also be avoided and a cause to shut off and prevent any AI projects which allow this to happen.
- It’s a short distance between this and lethal autonomous weapons. An AI is highly unlikely to be human in the way it thinks and acts. Unlike humans which are carbon based lifeforms, evolved over thousands of years, an AI will be silicon based and evolve quickly over years, months or days. The chances of it feeling emotions, if it does at all… like guilt, empathy, love like a human is improbably. If it is given the flexibility to allow human death, its idea of an atrocity may be very different to ours, and due to its speed and accuracy even the fastest reactions in stopping this type of AI may be far too late to prevent a disaster.
This is the only area where I believe death with an AI is involved may be forgiven – And even in this case not in all circumstances. I would describe an accidental death caused by an AI as one where in-spite of reasonable steps being taken to collect and analyse available data and accident happened, which resulted in death or injury, that was believed only to have a low level of probability and became unavoidable. Here we may see this through the eyes of the Uber driverless vehicle;
- ‘An accidental death’ – The car should never be allowed to sacrifice human life where it is aware of a significant risk (we will discuss the ‘significant risk’ shortly), opting instead to stop entirely in the safest possible manner.
- ‘Reasonable steps’ – These should be defined through establishing a reasonable level of risk, above 0% which is tolerable to us. More on this below.
- ‘Collect and analyse data’ – I think this is where the Uber project went wrong. Better sensors or processing hardware and software may have made this accident preventable.
An AI designed to tolerate only accidental death should not set the preservation of human life as its primary objective. Clearly defined final objectives for AI seemingly have unintended results – With a matrix like human farm being possible to maximise human life but sacrificing pleasure. Maximising pleasure similarly could result in the AI dedicating its resources to generating a new drug to make humans permanently happy, or putting us all in an ideal simulated world. Indirect normativity (Nick Bostrom, SuperIntelligence) seems to be a more appealing proposition, instead teaching an AI to;
- Drive a car to its destination
- Take any reasonable steps to avoid human harm or death while fulfilling step A
- Do the intended meaning of this statement
But what if a driverless car finds itself in a situation where death is unavoidable, where it’s just choosing between one person or another dying?
If an AI designed only to tolerate accidental death finds itself in a situation where it’s only decision is between one life and another, even if inaction would result in a death, it may still be compliant with this rule. We should instead measure this type of AI from an earlier moment, that the actions leading up to this situation should have been taken to minimise risk of death. As new data becomes available which show no further options are possible which avoid death or injury the accident has already happened and a separate decision making system may come into force to decide what action to take.
A reasonable level of risk?
To enable AI to happen at all we need to establish the reasonable level for risk in these systems. A Bayesian AI would always attribute a greater than 0% chance of anything happening, including harm or death of a human, in any action or inaction that it takes. For example, if a robot were to make contact with a human, holding no weapons, travelling slowly and covered in bubble wrap, the chance of it transferring bacteria or viruses which have a small chance of causing harm is higher than 0%. If we are to set the risk appetite for our AI at 0% it’s only option will be to shut itself down as quickly and safely as possible. We must have a minimum accepted level for AI caused harm to progress, and I think we can reach some consensus for this.
With the example of the Uber self-driving car we may assume the equivalent number of deaths caused by human and machine. The machine was unable to avoid the death of a human, and if the evidence presented is an accurate reflection of the circumstances it seems likely a human too would have been unable to avoid the death. The reaction for this has been strongly anti-automation, so we can tell that a 1-to-1 exchange between human and machine deaths is not the right level – That we would prefer for a human to be responsible for a death if the number of casualties is not reduced by using a machine.
If we are to change this number to 2-to-1 this begins to look different. If we could half the number of deaths caused by driving or any other human activity automation begins to look a lot more appealing to a far greater number of people. If we extend this to a 99% reduction in deaths and injuries the vast majority of people will lean towards AI over human actors.
Where this number stands exactly I am not certain. It’s also unlikely that the ratio would remain static as growing trust in AI may lead us either direction. Indirect normativity may be our best option again in this instance, accounting for the moving standard which we would hold it to.
Setting a tolerance rate for error at 0% for anything is asking for failure. No matter how safe or fool proof a plan may seem there will always be at least a tiny possibility of error. AI can’t solve this, but it might be able to do a better job than us. If our goal is to protect and improve human life… maybe AI can help us along the way.