Recently, a team of researchers led by Professor XIE Chengjun and Associate Professor Zhang Jie at the Hefei Institutes of Physical Science (HFIPS), Chinese Academy of Sciences, developed a novel deep-learning based technique for satellite imagery.
Their method, called the Frequency-Adaptive Mixture of Experts Network (FAME-Net), has been accepted for publication in the 2024 proceedings of the Association for the Advancement of Artificial Intelligence (AAAI).
High-resolution multispectral (HRMS) imagery is essential for agriculture, mapping, and environmental protection. However, directly obtaining such images faces technological limitations. To overcome this challenge, pan-sharpening techniques combine high-resolution panchromatic (PAN) and low-resolution multispectral (LRMS) images. Recent deep learning advancements have improved spectral and spatial details in pan-sharpening. Yet, neural networks still struggle with frequency bias and adapting to diverse remote sensing content.
Addressing the limitations, the team proposed the Frequency-Adaptive Mixture of Experts Network (FAME-Net). This method, drawing from discrete cosine transform and the Mixture of Expert (MoE) concepts, uses a frequency mask predictor for adaptive high and low-frequency masking. Different expert networks process these frequency-specific features, enabling focused attention on different frequency ranges. FAME-Net dynamically adapts its masks for various image contents, integrating multiple expert outputs through a gating mechanism.
In comparative analyses with existing state-of-the-art methods, FAME-Net not only demonstrates superior performance in preserving spectral quality and enhancing spatial resolution but also showcases excellent performance in full-resolution remote sensing imagery.
This research provides fresh insights into the field of image processing, demonstrating the effectiveness of integrating dynamic network structures and frequency domain information.
Figure 1 The overall structure of FAME-Net. (Image by ZHANG Jie)
Figure 2 The result of FAME-Net was compared against nine other methods on WorldView-III dataset. （Image by ZHANG Jie）