A team of researchers led by Professor XIE Chengjun and Associate Professor Zhang Jie at Hefei Institutes of Physical Science (HFIPS), Chinese Academy of Sciences (CAS), unveiled Fourier-Image Signal Processing (ISP), a novel deep-learning based framework for RAW-to-sRGB image conversion.
This approach was accepted for publication in the 2024 proceedings of the Association for the Advancement of Artificial Intelligence (AAAI).
Converting RAW images to standard Red Green Blue (sRGB) images enhances the visual appeal and usability of smartphone photography. However, current methods struggle with color and spatial structure accuracy, especially with resolution and image type variations. Combining color mapping and spatial structure produces suboptimal results, due to the complex interplay between style and structure within the images.
To overcome these challenges, the team has developed a novel framework called Fourier-ISP. Inspired by the Image Signal Processing pipeline, this approach separates style and structure of the image within the frequency domain.
"It enabled independent optimization," said ZHANG Jie, member of the team.
Fourier-ISP consists of three subnetworks: one for refining the structural details, another for learning accurate colors, and a third for blending these elements seamlessly. This decoupling of style and structure enables enhanced performance in image conversion, producing sharper and more accurate color and structural details.
Extensive evaluations across varied datasets confirm that Fourier-ISP realizes state-of-the-art results in qualitative and quantitative assessments, surpassing existing methods in precision and detail reproduction. It demonstrates robust transferability and effectiveness in handling both structural and style information, ensuring enhanced color reproduction and texture preservation. Notably, Fourier-ISP achieved an impressive Peak Signal to Noise Ratio (PSNR) improvement of 0.17dB in Zurich-area collected RAW and RGB paired images dataset (ZRR-dataset).
This framework introduces a novel insight into the field of image processing, showcasing the potential of style-structure decoupling in achieving high-fidelity image conversion, particularly in mobile photography, according to the team.
Figure 1 Fourier-ISP Framework. (Image by ZHANG Jie)
Figure 2 The results image from ZRRdataset. The last row showcases the color histogram of the image. (Image by ZHANG Jie)
