In my test of the new in DxO Photolab 8 included AI-based denoising DeepPrime XDS2, I questioned, based on the incredibly detailed results, how many of the resulting details are really present in the subject and how many are just faked. I would like to investigate this in more detail here.
AI denoising
Artificial intelligence (AI) has now also revolutionized image processing (incidentally, the lead image in this article was also created using ChatGPT 😉). Particularly impressive for us photographers are the results of AI-based denoising processes, which make it possible to take very usable pictures even with previously incredibly high ISO values.
Since the availability of AI-based denoising methods, I no longer have any problems working with 5-digit ISO values. The results that can be achieved in this way still amaze me, however. Below I show again a picture of our tomcat Tom in direct comparison of the original and the denoised version. As it was at that time still too dark despite ISO 12,800, I had to push the picture in Lightroom by one stop. So it actually corresponds to a photo taken at ISO 25,600!
On the left you can see the original and on the right the version processed with DxO’s DeepPRIME XD2s, in my opinion the best AI process currently available ( enlarged by 200%):
That’s really very impressive, isn’t it?
Further comparisons - also with other AI denoising methods - can be found in my article DeepPRIME XD2s in DxO PhotoLab 8. In principle, much of the following also applies to the other currently available AI-based methods.
How does it work?
In contrast to the previously used algorithm-based denoising mechanisms in conventional image processing programs, the way AI works is completely different. This involves training a complex neural network with a vast number of images.
In principle, you can shoot the same subject with low and high ISO values and train the AI with the image pairs so that it learns that the denoised high-ISO version comes closest to the version with the low ISO value. As you can see, this works very well. The more different images are available for training, the more computing power is used for training, the better the processes become.
Nowadays, AI denoising reveals details that are not visible to the naked eye in the original even by the best of intentions. The question therefore arises whether the details visible in the processed image are really there, or if the AI is just generating structures that it believes are appropriate.
In that sense:
This is a cat with lots of hair. I can see some of them. So I replace the noise in between with more parallel hairs. But I’m not allowed to do that in the eye….
However, this usually seems to work very well. But is the result
Fact or fake?
Current text-based AI systems are known to hallucinate when faced with uncertainty. If they don’t know something, they invent a plausible-appearing answer - it’s almost human 😉. Wikipedia has an interesting and detailed article on this topic.
It can also be shown in image processing that the AI sometimes only hallucinates. This is particularly visible with small geometric shapes that we know well: Letters. I have already noticed several times, that the AI denoising mechanisms have particular difficulties with small letters.
I created a real “endurance test” for this. I printed out a text with different font sizes and took pictures of it with my Canon EOS R5 at ISO values 100 and 51200 from a distance of approx. 3m (by the way, ChatGPT created the meaningless text for me 😉).
Here you can see a 400% enlarged section in the Adobe Lightroom comparison - on the left taken with ISO 100, on the right with ISO 51200 - both without denoising:
While I can read the text on the left up to the third paragraph without any problems, I even have difficulty reading the first paragraph on the right.
But what is exciting is what DeepPRIME XD2s makes of it. Here is another comparison of the ISO 100 image (left) with the ISO 51,200 image processed with DeepPRIME XD2s (right):
Here you can see - in my opinion - very well what the AI does. It creates contours around the original letters that seem plausible to it, but which have little to do with reality. At first glance, the image on the right appears much sharper and less noisy than before, but it still contains significantly less real information than the original on the left.
The loss of information due to denoising can be seen even better in a direct comparison of the denoised version (right) with the non-denoised version (left) at ISO 51200:
I personally find it almost easier to decipher the text in the unprocessed, very noisy version on the left. So in this example, the AI denoising only achieves the illusion of image enhancement - admittedly, with ISO 51200 this is also a very extreme example.
With the usual everyday motifs, however, this loss of real detail is normally not noticeable. The artificially generated enhancements with AI usually blend in very unobtrusively. Nevertheless, some of the visible image enhancements are only fictitious - in other words, more fake than fact.
But there is one area where DxO DeepPRIME XD2s actually gets more real detail out of my RAW file: at low ISO values
Low ISO values
This can be seen if you also process the ISO 100 image (left) with DeepPRIME XD2s (right):
In this comparison, DeepPRIME XD2s can actually extract more useful information from the RAW file than Lightroom. In the edited version on the right, the text looks much more sharply contoured and I can now still read the text in the fourth paragraph clearly. There is also significantly less color fringing around the letters.
Apparently, DxO can actually handle the Bayer Matrix of the sensor better than Adobe in Lightroom. DxO’s approach of applying the AI algorithm directly to the raw sensor data before the de-Bayer algorithm probably helps here. Since not all color information is available for each sensor pixel, the color values of neighboring sensor pixels are averaged to determine the color of each individual pixel. The DxO AI actually seems to manage this better than the algorithm integrated in Lightroom.
This makes the use of DxO DeepPRIME a worthwhile option, even for RAW files that are not noisy, in order to get the last bit of quality out of them.
And now?
With all the AI hype, I sometimes ask myself whether this is still photography. The term photography is composed of the ancient Greek φῶς phōs, (“light”) and the ancient Greek γράφειν gráphein (“to draw”) and therefore means “to draw with light”. A photographer’s mantra is also “It’s just the light - the light makes the picture”. However, light seems to be becoming less and less important.
Of course, cameras have always distorted reality. Three-dimensional objects become two-dimensional. In digital cameras, pixel colors are interpolated by the color filters of the neighboring pixels in the Bayer matrix. Many algorithms automatically process the raw data from the sensor, and the image processing program does the rest. But these algorithms are comprehensibly defined and reproducible. With AI it is different, AI is a “black box”.
I’m quite ambivalent about this myself. The topic extends far beyond denoising. In the post-processing of photos, image areas can be replaced with AI based generated fills, unwanted parts of the image can be removed, skies can be replaced and you can even create photorealistic images based solely on text input.
As a self-confessed nerd, I have of course already tried all this out by myself and I admit - I am very impressed. But I’ve also lost some of the magic and joy of the “craft of photography”. Mastering the technology is becoming less and less important.
When I think of the excited feeling when an image slowly materialized in the red light of the photo lab on the white photo paper in the developer bath - but I digress…
What do you think of this development? I am very curious about comments…
