收稿日期: 2016-10-26
修回日期: 2017-01-04
网络出版日期: 2018-05-31
基金资助
青岛海洋科学与技术国家实验室项目基金资助
Implementation of Audio and Video Decoding and Synchronous Playback Based on Embedded Systems
Received date: 2016-10-26
Revised date: 2017-01-04
Online published: 2018-05-31
基于ARM嵌入式系统设计并实现了MPEG-2 TS流的音视频解码和同步播放系统.在该系统中，由于ARM处理器内嵌了硬件视频解码器，为充分利用该资源，采用硬件MFC（multi-format codec）解码器对视频数据进行解码，同时采用开源的Mad （libmad）库对音频数据进行解码.进一步利用V4L2（Video for Linux 2）驱动接口和ALSA（advancedLinux sound architecture）库实现视频和音频的播放.由于视频帧播放周期和硬件处理时延不一致，系统对音视频数据操作存在时间差，从而造成音视频播放不同步.于是采用将视频播放同步到音频播放流的方法实现音视频的播放同步.测试结果表明，设计的音视频解码和同步播放系统可以实现对音视频数据进行解码与同步播放.
张森永, 高树立, 陈为刚 . 面向嵌入式系统的音视频解码与同步播放实现[J]. 应用科学学报, 2018 , 36(3) : 504 -514 . DOI: 10.3969/j.issn.0255-8297.2018.03.010
An MPEG-2 TS stream decoding and synchronous playback system is designed and implemented based on ARM embedded system. In order to make full use of the resource of video codec embed in ARM processor, a hardware MFC codec is adopted to decode video data, and the open-source Mad- library (libmad) is used to decode audio data. Furthermore, V4L2 driver interface and ALSA library are adopted to play video and audio frame. Due to the delay of hardware decoding from video frame period, there exists a time difference between audio and video playbacks, causing audio and video unsynchronized. In the designed system, a method that synchronizes the video playback to the audio playback is used to achieve synchronization of audio and video playback. Test results show that the designed system can realize audio and video data decoding and synchronous playback.
[1] Park K, Lim Y, Suh D Y. Delivery of ATSC 3.0 services with MPEG media transport standard considering redistribution in MPEG-2 TS format[J]. IEEE Transactions on Broadcasting, 2016, 62(1):338-351.
[2] Lin Y Y, Hsu H H. General Architecture for MPEG-2/H.263/H.264/AVC to H.264/AVC intra frame transcoding[J]. Journal of Signal Processing Systems, 2015, 79(3):299-311.
[3] Yuste L B, Boronat F, Montagud M. Understanding timelines within MPEG standards[J]. IEEE Communications Surveys & Tutorials, 2016, 18(1):368-400.
[4] 黄晓伟,徐元欣,王匡. 数字电视系统中的音视频同步实现[J]. 电视技术,2005(7):45-46. Huang X W, Xu Y X, Wang K. Synchronization between audio and video in digital[J]. TV System, 2005(7):45-46. (in Chinese)
[5] Wu W, Chang Y. A novel algorithm of media synchronization for MPEG-2[J]. IEEE Transactions on Consumer Electronics, 2006, 52(2):646-650.
[6] Abdallah F B, Trabelsi C, Atitallah R B. Early power-aware design space exploration for embedded systems:MPEG-2 case study[C]//Proceedings of 2014 International Symposium on System-on-Chip (SoC). Tampere, Finland, IEEE, 2014:1-8.
[7] Fei W, Cong K. Implementation of linux video capture driver based on V4L2 architecture[J]. Electronic Science and Technology, 2012, 25(2):86-87.
[8] Belloch J A, Gonzalez A, Mayo R. Evaluating the potential of low power systems for headphone-based spatial audio applications[J]. Procedia Computer Science, 2015, 51:191-200.
[9] Binh N, Chung S T, Cho S. Design and implementation of an embedded multimedia live streaming decoder system[C]//2014 International Conference on Advanced Technologies for Communications (ATC 2014). Hanoi, Vietnam, IEEE, 2014:377-382.
[10] Lei X, Jiang X, Wang C. Design and implementation of a real-time video stream analysis system based on FFmpeg[C]//Proceedings of 4th IEEE World Congress on Software Engineering (WCSE). Beijing, China, 2013:212-216.
[11] International Organization for Standardization (ISO). Information technology-generic coding of moving pictures and associated audio information:systems[S]. Geneva, Switzerland:ISO, 2000.
[12] Zhang J, Li Y, Wei Y. Using timestamp to realize audio-video synchronization in real-time streaming media transmission[C]//Proceedings of International Conference on Audio, Language and Image Processing. Shanghai, China, 2008:1073-1076.
[13] Mosharafa S M, Ebrahim G A, Zekry A A. A novel algorithm for synchronizing audio and video streams in MPEG-2 system layer[C]//20149th International Conference on Computer Engineering & Systems (ICCES). Cairo, Egypt, IEEE, 2014:142-147.
[14] Yang M, Bourbakis N, Chen Z. An efcient audio-video synchronization methodology[C]//Proceedings of IEEE International Conference on Multimedia and Expo. Beijing, China, 2007:767-770.
/
| 〈 |
|
〉 |
