Video Don’t worry if the lighting is a bit off in your photos – artificially intelligent software can fix that.
Computer scientists from Nvidia, Aalto University in Finland, and the Massachusetts Institute of Technology in the US, have trained a neural network to restore images marred by flecks of noise. Computer vision algorithms are already automatically used to improve snaps taken on smartphones like the Pixel 2 or the iPhone X, but this takes things further.
With this new technique, the training process is slightly different from how the likes of Google and Apple train their phone software to clean up pictures.
Instead of feeding neural networks a pair of images, where one is high quality and the other one is blurry, this latest model – nicknamed noise2noise – can learn how to clean images without needing to see high-resolution examples.
“We apply basic statistical reasoning to signal reconstruction by machine learning — learning to map corrupted observations to clean signals — with a simple and powerful conclusion: under certain common circumstances, it is possible to learn to restore signals without ever observing clean ones,” according to the paper’s abstract.
The theoretical basis for why this works is a little tricky to understand. More traditional techniques use pairs of low-resolution and high-resolution images, learn to minimize the loss function by estimating the difference in pixel values between both pictures.
There is a wide range of values that the pixels can take to recreate a crisper image, and over time the neural network learns to average out these values. The same idea can be applied when training on pairs of spoiled images, if the gap between the pixel values in both images is relatively similar to the ones between a clean and blurry image.
“This implies that we can, in principle, corrupt the training targets of a neural network with zero-mean noise without changing what the network learns,” the paper said.
Training with technology
The team trained their noise2noise model on 50,000 images taken from the ImageNet dataset and added a random distribution of of noise to each image. The system has to estimate the magnitude of the noise in the photo and remove it.
It was tested on three datasets with images of buildings, people, and medical resonance imaging scans. Here’s a video with some of the results.