Local-global collaborative multi-scale feature augmentation for hyperspectral and multispectral image fusion

Images acquired by a single remote sensing sensor are inherently constrained by hardware and physical limitations, making it difficult to simultaneously achieve high spatial resolution and high spectral resolution. Hyperspectral images provide rich spectral information but typically suffer from low spatial resolution, whereas multispectral images contain finer spatial details at the cost of reduced spectral fidelity. To address this trade-off, this study proposes a local-global collaborative multi-scale feature augmentation method for hyperspectral and multispectral image fusion. The objective is to fully exploit the complementary spatial and spectral characteristics of heterogeneous data sources, thereby generating fused images that preserve spectral consistency while significantly enhancing spatial detail expression.

The proposed fusion framework consisted of four cooperative modules: Feature extraction, feature fusion, feature augmentation, and image reconstruction. First, the feature extraction module independently encoded the hyperspectral and multispectral inputs using dedicated convolutional layers to obtain hierarchical spectral and spatial feature representations. Second, the feature fusion module integrated the extracted features into a shared latent space, enabling cross-modal interaction and alignment. The core component was the feature augmentation module, which was divided into local and global sub-modules. The local feature augmentation sub-module employed multiple convolutional blocks with different receptive fields to strengthen fine-grained spatial details such as edges, textures, and local structures. The global feature augmentation sub-module introduced a spectral-spatial fusion Transformer architecture combined with multi-scale convolutions to model long-range dependencies and enhance global contextual information as well as spectral consistency. Finally, the image reconstruction module mapped the augmented fusion features back to the image domain to produce the final high-resolution hyperspectral image.

Extensive experiments were conducted on several benchmark hyperspectral and multispectral datasets, including both quantitative evaluations and qualitative visual comparisons. The proposed method consistently outperformed state-of-the-art fusion methods across multiple evaluation indicators. In terms of spatial detail preservation, the fused images exhibited sharper edges and clearer textures with significantly improved spatial resolution. Regarding spectral fidelity, the proposed method achieved low spectral distortion, with SAM (spectral angle mapper) values comparable to those of the best-performing competitors. Comprehensive evaluation indicators such as PSNR, SSIM, and ERGAS also demonstrated superior performance. For example, on the widely used CAVE and Harvard datasets, the proposed method achieved average PSNR improvements of 1.5-2.5 dB over the best baseline methods. Visual comparisons further confirmed that the proposed method effectively avoided common artifacts such as blurring and spectral aliasing. Moreover, the method showed robust performance across different scenes and varying degradation conditions.

The proposed local-global collaborative multi-scale feature augmentation method effectively mitigates the inherent spatial-spectral trade-off in single-sensor imaging systems. By jointly enhancing local fine-grained details and global contextual dependencies, the method generates fused images with both high spatial resolution and high spectral fidelity. Experimental results demonstrate its superiority over existing approaches in terms of accuracy, robustness, and visual quality. The proposed framework provides a powerful and versatile solution for hyperspectral and multispectral image fusion, with strong potential for practical applications in remote sensing, environmental monitoring, and beyond.