A research team led by Prof. XIE Chengjun and ZHANG Jie at the Hefei Institutes of Physical Science of the Chinese Academy of Sciences, has developed a new method for improving remote sensing images called Pan-Mamba. This method uses the Mamba model, which is known for efficiently capturing long-range dependencies in data. It combines low-resolution multispectral images with high-resolution panchromatic images to create clearer, high-resolution multispectral images.
This work has been published in Information Fusion.
Pan-sharpening is an important technique in remote sensing that combines low-resolution color images with high-resolution black-and-white images to create clearer, high-resolution color images.
In this study, the research team used the Mamba model, which is good at handling long-range relationships in data, similar to more complex models like Transformers, but with less computing power.
Pan-Mamba is the use of two key features: Channel Swapping Mamba and Cross-Modal Mamba. The Channel Swapping Mamba allows the model to start mixing the two types of images (panchromatic and multispectral) early in the process. This helps the model understand how the images relate to each other more efficiently. The Cross-Modal Mamba strengthens the mixing of both types of images by using multiple layers of interaction, ensuring that the model can make the most of the information from both image types to create high-quality results. This design keeps the process fast and efficient, even when dealing with large images.
To test Pan-Mamba, the team used images from WorldView-III, WorldView-II, and GaoFen-2. The results showed that Pan-Mamba works better than other methods and requires less computing power.
This research is an important step forward in improving the quality and resolution of remote sensing images.
The network structure of our proposed Pan-Mamba, which includes three Key components: Mamba block for long-range feature extraction, CW Mamba and cross modal mamba for shallow and deep feature fusion. (Image by ZHANG Jie)
