基于信息互補與交叉注意力的跨模態(tài)檢索方法
打開文本圖片集
關(guān)鍵詞：信息互補；交叉注意力；圖卷積網(wǎng)絡(luò)；跨模態(tài)檢索
中圖分類號：TP391 文獻標(biāo)志碼：A 文章編號：1001-3695（2025）07-015-2032-07
doi：10.19734/j.issn.1001-3695.2025.01.0003
Abstract：WiththerapidgrowthofmultimodaldataontheInternet，cross-modalretrievaltechnologyhasatractedwidespread atention.However，some multimodaldataoftenlacksemanticinformation，whichleadstotheinabilityof modelstoaccurately extracttheinherentsemanticfeatures.Aditionally，somemultimodaldatacontainredundantinformationunrelatedtosemantics，whichinterfereswiththemodelextractionofkeyinformation.Toaddresstis，thispaperproposedacrossmodalretrieval methodbasedoninformationcomplementarityandcross-atention（ICCA）.The methodusedaGCN tomodeltherelationships betweenmulti-labelsanddata，supplementing the mising semantic informationinmultimodaldataandthe missing sampledetailinformationinmulti-bels.Moreover，acrossattntionsubmoduleusedulti-labelinformationtoflerouttedudant semantic-irelevantdata.Toachievebetter matchingofsemanticallysimilarimagesand textsinthecommonrepresentation space，this paperproposed asemantic matching lossThislossintegrated multi-labelembeddings intothe image-text matching process，further enhancingthesemanticqualityof thecommonrepresentation.Experimentalresultsonthree widelyuseddatasets NUS-WIDE，MIRFlickr-25K，and MS-COCO demonstrate that ICCA achieves mAPvaluesof0.808，0.859，and0.837， respectively， significantly outperforming existing methods.
KeyWords：informationcomplementarity；cross-attention；graph convolutional network（GCN）；cros-modalretrieval
0 引言
近年來，隨著互聯(lián)網(wǎng)技術(shù)的飛速發(fā)展，視頻、圖像、文本等多媒體數(shù)據(jù)呈現(xiàn)出急劇增長的趨勢。（剩余18541字）