基于特性分流的多模態(tài)對話情緒感知算法
打開文本圖片集
doi：10.19734/j. issn.1001-3695.2024.12.0466
Multimodal dialogue emotion perception algorithm based on feature divergence
Ren Qinze a，b ，Yuan Yea，b，F(xiàn)u Ketinga，?，F(xiàn)u Junxiua，?，Xu Kanga，b，Liu Na，bt （a.Institutefcneelie，ooflhe&，Ueitffone&oi ，China）
Abstract：Multimodalemotion perceptioniscrucialfor monitoring personal healthand providing medicalcareinthe fieldof proactive health.Currnt multimodal dialogue emotionperceptiontechnologiesfacechallenges ifusing informationacross differentmodalities，particularlyincapturinglocalrelationshipsbetweenmodalies.Theproposedmultimodalfusionalgorithm basedonfeaturediversion，MEPAD（multimodalemotionperceptionalgorithmwith featurediversion），addressedthesechallenges bycapturing global information indialogues using graphneural networks and integrating homogeneous and specific features across modalities through thehypercomplex number system and pairwise feature fusion mechanisms.Experiments on he IEMOCAP and MOSEI datasets demonstratethat MEPAD significantlyoutperforms existing methodsin multimodal dialogue emotionperceptiontasks，highlightingitsefectiveessandpotentialinhandlingomplexemotionaldata.Thisresearchoffs newinsights for theapplication of multimodal emotion perception technology in proactive health.
Key words：multimodal emotionrecognition；graph neural networks；hypercomplex number system；pairwisefeature fusion; dialogue emotion perception
0 引言
情緒感知是人類交流中的一個關鍵組成部分，在主動健康領域，對話者情緒變化也可以作為評估人類健康信息監(jiān)控的重要指標[1]。（剩余18982字）