文献类型：期刊文献

英文题名：TSCMDL: Multimodal Deep Learning Framework for Classifying Tree Species Using Fusion of 2-D and 3-D Features

作者：Liu, Bingjie[1] Hao, Yuanshuo[2] Huang, Huaguo[1] Chen, Shuxin[3] Li, Zengyuan[3] Chen, Erxue[3] Tian, Xin[3] Ren, Min[4]

第一作者：Liu, Bingjie

通信作者：Tian, X[1]

机构：[1]Beijing Forestry Univ, Res Ctr Forest Management Engn State Forestry & Gr, Beijing 100083, Peoples R China;[2]Northeast Forestry Univ, Sch Forestry, Key Lab Sustainable Forest Ecosyst Management, Minist Educ, Harbin 150040, Heilongjiang, Peoples R China;[3]Chinese Acad Forestry, Inst Forest Resource Informat Tech, Beijing 100091, Peoples R China;[4]China Mobile Grp Shanxi Design Inst Co Ltd, Taiyuan 030032, Peoples R China

年份：2023

卷号：61

起止页码：1-1

外文期刊名：IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING

收录：;EI(收录号：20231714014837);Scopus(收录号：2-s2.0-85153334508);WOS:【SCI-EXPANDED(收录号:WOS:000979605400019)】；

基金：This work was supported in part by the National Key Research and Development Program of China under Grant 2021YFE0117700, in part by the National Science and Technology Major Project of China's High Resolution Earth Observation System under Grant 21-Y20B01-9001-19/22, and in part by the National Natural Science Foundation of China under Grant 42130111 and Grant 41871279. (Corresponding author: Xin Tian.)

语种：英文

外文关键词：Vegetation; Feature extraction; Point cloud compression; Laser radar; Deep learning; Forestry; Data models; light detection and ranging (LiDAR); multimodal; point cloud; tree species classification

摘要：Accurate tree species information is a prerequisite for forest resource management. Combining light detection and ranging (LiDAR) and image data is one main method of tree species classification. Traditional machine learning methods rely on expert knowledge to calculate a large number of feature parameters. Deep learning technology can directly use the original image and point cloud data to classify tree species. However, data with different patterns require the use of different types of deep learning methods. In this study, a tree species classification multimodal deep learning (TSCMDL) that fuses 2-D and 3-D features was constructed and then used to combine data from multiple sources for tree species classification. This framework uses an improved version of the PointMLP model as its backbone network and uses ResNet50 and PointMLP networks to extract the image features and point cloud features, respectively. The proposed framework was tested using unmanned aerial vehicle LiDAR (UAV LiDAR) data and red, green, blue (RGB) orthophotos. The results showed that the accuracy of the tree species classification using the TSCMDL framework was 98.52%, which was 4.02% higher than that based on point cloud features only. In addition, when the same hyperparameters were used for training the model, the efficiency of the model training was not significantly lower than for models based on point cloud features only. The proposed multimodal deep learning framework extracts features directly from the original data and integrates them effectively, thus avoiding manual feature screening and achieving more accurate classification. The feature extraction network used in the TSCMDL framework can be replaced by other suitable frameworks and has strong application potential.