文献类型：期刊文献

英文题名：MDC-FusFormer: Multiscale Deep Cross-Fusion Transformer Network for Hyperspectral and Multispectral Image Fusion

作者：Sun, Le[1,2] Zhou, Jianxiao[1] Ye, Qiaolin[3] Wu, Zebin[4] Chen, Qiao[5,6] Xu, Zhongqi[7] Fu, Liyong[5]

第一作者：Sun, Le

通信作者：Fu, LY[1]

机构：[1]Nanjing Univ Informat Sci & Technol, Sch Comp Sci, Nanjing 210044, Peoples R China;[2]Nanjing Univ Informat Sci & Technol, Jiangsu Collaborat Innovat Ctr Atmospher Environm, Nanjing 210044, Peoples R China;[3]Nanjing Forestry Univ, Sch Informat Sci & Technol, Nanjing 210037, Peoples R China;[4]Nanjing Univ Sci & Technol, Sch Comp Engn, Nanjing 210094, Peoples R China;[5]Chinese Acad Forestry, Inst Forest Resource Informat Tech, Beijing 100091, Peoples R China;[6]NFGA, Key Lab Forest Management & Growth Modelling, Beijing 100091, Peoples R China;[7]Hebei Agr Univ, Coll Forestry, Baoding 071000, Peoples R China

年份：2024

卷号：62

外文期刊名：IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING

收录：;EI(收录号：20243616982872);Scopus(收录号：2-s2.0-85202757486);WOS:【SCI-EXPANDED(收录号:WOS:001311214200001)】；

基金：This work was supported in part by the National Natural Science Foundation of China under Grant 62471239 and Grant U23B2006 and in part by the Fundamental Research Funds for the Central Nonprofit Research Institution of CAF under Grant CAFYBB2022ZB002.

语种：英文

外文关键词：Convolutional neural network (CNN); cross-attention mechanism; hyperspectral and multispectral image fusion; transformer

摘要：The spatial resolution of hyperspectral images (HSIs) is usually limited due to internal imaging mechanisms. To obtain imagery with high spectral and high spatial resolutions, which is essential for subsequent HSI processing tasks, a costeffective approach is to fuse HSI with multispectral images (MSIs). One highly effective fusion method is the convolutional neural network (CNN). However, CNNs have limitations in capturing global information and complex features. Recently, visual transformers (ViTs) have garnered interest for their ability to process non-local information. Despite this, existing HSI-MSI fusion methods suffer from insufficient spatial-spectral feature interaction, resulting in suboptimal fusion quality. To address these challenges, we propose a multiscale deep cross-fusion transformer (MDC-FusFormer) network for HSI and MSI fusion. This network effectively performs the interactive fusion of spatial-spectral features, thereby enhancing the quality of the fused images. MDC-FusFormer employs a three-branch network architecture consisting of two independent progressive feature mining modules (PFMMs), a multiscale deep cross-fusion attention module, and a spatial-spectral feature fusion module. Initially, shallow features at different scales of MSI and HSI are recursively extracted through successive up- and down-sampling using CNNs. These features then interact with the deep cross-modal information at corresponding scales through the attention block. Finally, a multidimensional refinement convolution block (MRCB) is applied to refine the feature information, which is then combined with cascaded up-sampling to reconstruct the high-resolution fused image step by step. Experimental results on five datasets indicate that, compared to nine other methods, MDC-FusFormer delivers superior performance.