packet video error concealment with auto regressive model yongbing zhang, xinguang xiang, debin...
TRANSCRIPT
![Page 1: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/1.jpg)
Packet Video Error Concealment With Auto Regressive Model
Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow, IEEE
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 22, NO. 1, JANUARY 2012
![Page 2: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/2.jpg)
Outline
• Introduction• Auto-Regressive Model-Based Error Concealment• AR Coefficient Derivation Under Spatial Continuity
Constraint• AR Coefficient Derivation Under Temporal Continuity• Experimental Results and Analysis• Conclusion
![Page 3: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/3.jpg)
Introduction
![Page 4: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/4.jpg)
Introduction
• Error resilience and error concealment are two major techniques to combat the problem.
• Error resilience adds redundant information at the encoder and decreases the compression efficiency.
• Error concealment is a post-processing technique utilizing the correctly received information at the decoder side
![Page 5: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/5.jpg)
Introduction
• error concealment– Spatial approaches– Temporal approaches– hybrid approaches (mixed above)
![Page 6: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/6.jpg)
Error concealment
• Spatial approach– utilizing the correctly decoded surrounding pixels
under smoothness constraint– Spatial approaches may yield better performance
than temporal ones in scenes with high motion
![Page 7: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/7.jpg)
Error concealment
• Temporal approaches– Temporal approaches restore the corrupted blocks
by exploiting temporal correlation between successive frames.
![Page 8: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/8.jpg)
Error concealment
• hybrid approaches(proposed)– The interpolation filters are separable and the
coefficients are fixed– achieved good performance for isotropic
regions,poor for anisotropic regions.– auto-regressive(AR) model based error
concealment proposed
![Page 9: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/9.jpg)
Auto-Regressive Model-Based Error Concealment
![Page 10: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/10.jpg)
Auto-Regressive Model-Based Error Concealment
![Page 11: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/11.jpg)
Auto-Regressive Model-Based Error Concealment
• we have to estimate the AR coefficients by exploring the spatial and temporal correlations of the corrupted block with its available spatial and temporal neighboring pixels.
![Page 12: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/12.jpg)
AR Coefficient Derivation Under Spatial Continuity Constraint
• we assume all the pixels within the corrupted block possess the same AR coefficients
![Page 13: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/13.jpg)
AR Coefficient Derivation Under Spatial Continuity Constraint
• If any of the neighboring blocks are correctly received, the correctly received neighboring blocks are utilized to train AR coefficients of the corrupted block.
• If all the neighboring blocks are lost, the already concealed neighboring blocks are utilized to train AR coefficients of the corrupted block.
![Page 14: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/14.jpg)
AR Coefficient Derivation Under Temporal Continuity
• xt (k, l), the corresponding motion aligned pixel et−1 (k, l) in the extended block is first found, and then the corresponding pixel
xt−2 (k, l) in the second closest reconstructed frame is also found by the same MV.
![Page 15: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/15.jpg)
AR Coefficient Derivation Under Temporal Continuity
![Page 16: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/16.jpg)
AR Coefficient Derivation Under Temporal Continuity
• After having obtained the AR coefficients α and β, we merge the two regression results as
• If no solution to α and β, use the traditional methods (BMA or STBMA)
![Page 17: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/17.jpg)
Experimental Results and Analysis
• The encoding group of picture (GOP) is set to be IPPP. . . , where I frames are encoded every 16 frames.
• The transmission errors are assumed to only occur in P frames
![Page 18: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/18.jpg)
Experimental Results and Analysis
![Page 19: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/19.jpg)
Experimental Results and Analysis
![Page 20: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/20.jpg)
Experimental Results and Analysis
![Page 21: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/21.jpg)
![Page 22: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/22.jpg)
![Page 23: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/23.jpg)
![Page 24: Packet Video Error Concealment With Auto Regressive Model Yongbing Zhang, Xinguang Xiang, Debin Zhao, Siwe Ma, Student Member, IEEE, and Wen Gao, Fellow,](https://reader031.vdocuments.mx/reader031/viewer/2022012311/5516b345550346f6208b5331/html5/thumbnails/24.jpg)
Conclusion
• For each corrupted block, we first derived the motion vector and then replenished each corrupted pixel.
• We proposed two block-dependent AR coefficient derivation algorithms under spatial and temporal continuity constraints.
• The results outperforms the method without AR with acceptable computational complexity.