Why You’ll Never Delete Anything in the Age of Artificial Intelligence
Posted on Sunday, June 30, 2019
Machine Learning (ML) and Artificial Intelligence (AI) are radically changing the way businesses work. ML is able to process vast datasets to create algorithms that will increase operational efficiency. AI can take those algorithms to make automated decisions, reducing the need for manual intervention in the process.
This blessing is also a curse, however. Much of the information being fed into the ML engine, particularly IoT sensor data, is transient and of little interest minutes after it has been generated and processed and an automated decision made. In the wider scheme of business, you would think it was safe to delete these details– especially as they quickly begin to fill your available disk capacity.
Unfortunately, it’s not that simple.
Proving your decision-making processes
For all intents and purposes, the microdetails used to train and power ML and AI are useless once processed. But you cannot dispose of it.
Consider an autonomous car, for instance. As the car makes a right turn, thousands of sensors provide measurements that the AI processes and uses to complete the maneuver. As soon as the corner is turned, those measurements are no longer relevant.
But what happens if the car collides with a pedestrian as it makes the turn? Police accident investigators will want to identify the cause of the accident which will require the car operator to demonstrate the AI decision-making process going into the maneuver– including all the sensor data generated in the run-up to the accident.
Investigators will also need to be satisfied that the accident was a one-off event. With autonomous machine learning, it is possible that faulty sensor data has skewed the decision-making algorithms over time. If this is the case, the vehicle may have been unsafe for quite some time.
Not just cars
Autonomous cars provide an obvious example of why you cannot delete data used in ML/AI scenarios. But the truth is any automated decision is open to challenge– and your business will need to be able to provide data to explain the outcome of those processes.
Which means that CTOs are not going to see any end to increasing storage demands any time soon.