基于深度特征交互與層次化多模態(tài)融合的情感識別模型
打開文本圖片集
關鍵詞：多模態(tài)情感識別；層次化融合；多尺度融合；特征融合
中圖分類號：TP391 文獻標志碼：A 文章編號：1001-3695（2025）07-008-1978-08
doi：10. 19734/j.issn.1001-3695.2024.11.0487
Abstract：Multimodalemotionrecognitionhasrecentlybecomeanimportantresearchdirectioninafectivecomputing，aiming to moreaccuratelyrecognizeandunderstand human emotional states by integrating various modalitiessuchasspeechandtext. However，existingmethodslacktheprocessngofinter-modalcorelationsduringfeatureextractionandoverlook multi-scale emotionalcuesduring feature fusion.Toaddresstheseisues，thisstudyproposedadeepfeature interactionand hierarchical multimodal fusionemotionrecognition model（DFIHMF）.Inthefeature extraction stage，themodel enhanced interactionsbetweendifferentmodalitiesandextractedmulti-scaleinformationbyintroducinglocalknowledgetokens（LKT）andcrosmodal interaction tokens（CIT）.Inthefeature fusionstage，the model integratedcomplexmultimodalfeaturesandmulti-scaleemotionalcesusingahierarchical fusionstrategy.ExperimentalresultsontheMOSIandMOSEIdatasetsshow thatthemodel achieves accuracy rates of 45.6% and 53.5% on the ACC7 evaluation metric，demonstrating that the proposed method outperforms existing technologies in multimodal emotion recognition tasks.
Key Words：multimodal emotion recognition；hierarchical fusion；multi-scale fusion；feature fusion
0 引言
情感識別是自然語言處理（naturallanguageprocessing，NLP）中的一項核心任務，其目標在于分析和處理輸入文本，以估計對象的情緒狀態(tài)。（剩余21459字）