面向嵌入式系统的音视频解码与同步播放实现

doi:10.3969/j.issn.0255-8297.2018.03.010

应用科学学报 ›› 2018, Vol. 36 ›› Issue (3): 504-514.doi: 10.3969/j.issn.0255-8297.2018.03.010

面向嵌入式系统的音视频解码与同步播放实现

张森永, 高树立, 陈为刚

天津大学电子信息工程学院, 天津 300072

收稿日期:2016-10-26 修回日期:2017-01-04 出版日期:2018-05-31 发布日期:2018-05-31
通信作者: 陈为刚,副教授,研究方向:无线通信理论与系统、信息理论与编码技术,E-mail:chenwg@tju.edu.cn E-mail:chenwg@tju.edu.cn
基金资助:
青岛海洋科学与技术国家实验室项目基金资助

Implementation of Audio and Video Decoding and Synchronous Playback Based on Embedded Systems

ZHANG Sen-yong, GAO Shu-li, CHEN Wei-gang

School of Electronic Information Engineering, Tianjin University, Tianjin 300072, China

Received:2016-10-26 Revised:2017-01-04 Online:2018-05-31 Published:2018-05-31

摘要/Abstract

摘要：

基于ARM嵌入式系统设计并实现了MPEG-2 TS流的音视频解码和同步播放系统.在该系统中，由于ARM处理器内嵌了硬件视频解码器，为充分利用该资源，采用硬件MFC（multi-format codec）解码器对视频数据进行解码，同时采用开源的Mad （libmad）库对音频数据进行解码.进一步利用V4L2（Video for Linux 2）驱动接口和ALSA（advancedLinux sound architecture）库实现视频和音频的播放.由于视频帧播放周期和硬件处理时延不一致，系统对音视频数据操作存在时间差，从而造成音视频播放不同步.于是采用将视频播放同步到音频播放流的方法实现音视频的播放同步.测试结果表明，设计的音视频解码和同步播放系统可以实现对音视频数据进行解码与同步播放.

关键词: 音视频解码, 音视频同步, 多线程, 嵌入式系统

Abstract:

An MPEG-2 TS stream decoding and synchronous playback system is designed and implemented based on ARM embedded system. In order to make full use of the resource of video codec embed in ARM processor, a hardware MFC codec is adopted to decode video data, and the open-source Mad- library (libmad) is used to decode audio data. Furthermore, V4L2 driver interface and ALSA library are adopted to play video and audio frame. Due to the delay of hardware decoding from video frame period, there exists a time difference between audio and video playbacks, causing audio and video unsynchronized. In the designed system, a method that synchronizes the video playback to the audio playback is used to achieve synchronization of audio and video playback. Test results show that the designed system can realize audio and video data decoding and synchronous playback.

Key words: audio and video decoding, audio and video synchronization, embedded system, multithreading

中图分类号:

TN911

张森永, 高树立, 陈为刚. 面向嵌入式系统的音视频解码与同步播放实现[J]. 应用科学学报, 2018, 36(3): 504-514.

ZHANG Sen-yong, GAO Shu-li, CHEN Wei-gang. Implementation of Audio and Video Decoding and Synchronous Playback Based on Embedded Systems[J]. Journal of Applied Sciences, 2018, 36(3): 504-514.

参考文献

[1] Park K, Lim Y, Suh D Y. Delivery of ATSC 3.0 services with MPEG media transport standard considering redistribution in MPEG-2 TS format[J]. IEEE Transactions on Broadcasting, 2016, 62(1):338-351.
[2] Lin Y Y, Hsu H H. General Architecture for MPEG-2/H.263/H.264/AVC to H.264/AVC intra frame transcoding[J]. Journal of Signal Processing Systems, 2015, 79(3):299-311.
[3] Yuste L B, Boronat F, Montagud M. Understanding timelines within MPEG standards[J]. IEEE Communications Surveys & Tutorials, 2016, 18(1):368-400.
[4] 黄晓伟,徐元欣,王匡. 数字电视系统中的音视频同步实现[J]. 电视技术,2005(7):45-46. Huang X W, Xu Y X, Wang K. Synchronization between audio and video in digital[J]. TV System, 2005(7):45-46. (in Chinese)
[5] Wu W, Chang Y. A novel algorithm of media synchronization for MPEG-2[J]. IEEE Transactions on Consumer Electronics, 2006, 52(2):646-650.
[6] Abdallah F B, Trabelsi C, Atitallah R B. Early power-aware design space exploration for embedded systems:MPEG-2 case study[C]//Proceedings of 2014 International Symposium on System-on-Chip (SoC). Tampere, Finland, IEEE, 2014:1-8.
[7] Fei W, Cong K. Implementation of linux video capture driver based on V4L2 architecture[J]. Electronic Science and Technology, 2012, 25(2):86-87.
[8] Belloch J A, Gonzalez A, Mayo R. Evaluating the potential of low power systems for headphone-based spatial audio applications[J]. Procedia Computer Science, 2015, 51:191-200.
[9] Binh N, Chung S T, Cho S. Design and implementation of an embedded multimedia live streaming decoder system[C]//2014 International Conference on Advanced Technologies for Communications (ATC 2014). Hanoi, Vietnam, IEEE, 2014:377-382.
[10] Lei X, Jiang X, Wang C. Design and implementation of a real-time video stream analysis system based on FFmpeg[C]//Proceedings of 4th IEEE World Congress on Software Engineering (WCSE). Beijing, China, 2013:212-216.
[11] International Organization for Standardization (ISO). Information technology-generic coding of moving pictures and associated audio information:systems[S]. Geneva, Switzerland:ISO, 2000.
[12] Zhang J, Li Y, Wei Y. Using timestamp to realize audio-video synchronization in real-time streaming media transmission[C]//Proceedings of International Conference on Audio, Language and Image Processing. Shanghai, China, 2008:1073-1076.
[13] Mosharafa S M, Ebrahim G A, Zekry A A. A novel algorithm for synchronizing audio and video streams in MPEG-2 system layer[C]//20149th International Conference on Computer Engineering & Systems (ICCES). Cairo, Egypt, IEEE, 2014:142-147.
[14] Yang M, Bourbakis N, Chen Z. An efcient audio-video synchronization methodology[C]//Proceedings of IEEE International Conference on Multimedia and Expo. Beijing, China, 2007:767-770.

面向嵌入式系统的音视频解码与同步播放实现

Implementation of Audio and Video Decoding and Synchronous Playback Based on Embedded Systems

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 6

编辑推荐

Metrics

本文评价

[1]	王德明, 隋修武, 张杨, 万凯新, 石峰. 基于DMA与改进GPC算法的超细缝合线直径控制技术[J]. 应用科学学报, 2019, 37(3): 349-358.
[2]	肖哲;陈兵. 嵌入式无线电台SNMP网管模型的设计与实现[J]. 应用科学学报, 2008, 26(2): 117-117 .
[3]	杨洪斌;吴悦;刘权胜. 同时多线程微处理器分布式保留站结构的数据流技术[J]. 应用科学学报, 2008, 26(2): 188-188 .
[4]	卜爱国, 刘昊, 胡晨, 王超. 基于AR模型的PCF动态电源管理预测策略[J]. 应用科学学报, 2005, 23(5): 483-488.
[5]	陈立全, 胡爱群, 胡骏, 李伟征. IEEE802.11 MAC层协议栈的均衡多线程实现方法[J]. 应用科学学报, 2005, 23(2): 155-159.
[6]	朱敏, 徐国治. 嵌入式系统中音频模块的实现[J]. 应用科学学报, 2004, 22(3): 415-418.