English

应用科学学报 ›› 2010, Vol. 28 ›› Issue (3): 266-270.

• 信号与信息处理 • 上一篇    下一篇

基于概念选择和重要性度量的多模态语义融合

郭戈， 平西建， 张涛   

  1. 解放军信息工程大学信息工程学院，郑州450002
  • 收稿日期:2010-03-25 修回日期:2010-05-05 出版日期:2010-05-21 发布日期:2010-05-21
  • 作者简介:郭戈，博士生，研究方向：图像、视频处理与模式识别，E-mail：guoge800819@163.com；平西建，教授，博导，研究方向：图像处理与模式识别、计算机视觉等，E-mail：pingxj@126.com
  • 基金资助:

    国家自然科学基金(No.60903221)资助

Multimodal Fusion Based on Concept Selection and Importance Measure

GUO Ge, PING Xi-jian, ZHANG Tao   

  1. Institute of Information Engineering, PLA Information Engineering University, Zhengzhou 450002, China
  • Received:2010-03-25 Revised:2010-05-05 Online:2010-05-21 Published:2010-05-21

PDF

2001

被引次数

2

摘要：

根据人类认知过程的特性，提出语义选择和重要性度量的多模态融合算法. 分别在单个模态下获取语义概念，并利用相关性检测得到用于融合的语义概念，从而减少误检语义带来的扩散误差. 考虑到概率融合无法体现语义的时间特性，提出重要性度量的概念进行融合以获取高级语义. 实验结果表明，该方法能准确提取视频的高级语义信息，与其他融合算法相比时体现出良好的性能.

关键词: 语义信息, 多模态融合, 重要性度量, 相关性, 概念选择

Abstract:

According to the characteristics of human cognitive processes, a multimodal fusion method of semantic concept selection and importance measure is proposed. Semantic concepts are first obtained in single mode, and a correlation detection method is used to decide semantic concept of fusion. Correlation detection can reduce the influence of diffusion error due to false detection semantic. Since the method of probability fusion cannot effectively handle the temporal characteristics of the semantic, an importance measure is introduced for high level semantic fusion. Experimental results show that the proposed method using temporal characteristics and relativity between concepts can better extract high-level semantic contents as compared to other fusion methods.

Key words: semantic information, multimodal fusion, importance measure, relativity, concept selection

中图分类号: