4106

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 21, NO. 9, SEPTEMBER 2012

Mode-Dependent Templates and Scan Order for H.264/AVC-Based Intra Lossless Coding
Zhouye Gu, Weisi Lin, Senior Member, IEEE , Bu-Sung Lee, Member, IEEE , Chiew Tong Lau, Member, IEEE , and Ming-Ting Sun, Fellow, IEEE

Abstract In H.264/advanced video coding (AVC), lossless coding and lossy coding share the same entropy coding module. However, the entropy coders in the H.264/AVC standard were original designed for lossy video coding and do not yield adequate performance for lossless video coding. In this paper, we analyze the problem with the current lossless coding scheme and propose a mode-dependent template (MD-template) based method for intra lossless coding. By exploring the statistical redundancy of the prediction residual in the H.264/AVC intra prediction modes, more zero coefcients are generated. By designing a new scan order for each MD-template, the scanned coefcients sequence ts the H.264/AVC entropy coders better. A fast implementation algorithm is also designed. With little computation increase, experimental results conrm that the proposed fast algorithm achieves about 7.2% bit saving compared with the current H.264/AVC delity range extensions high prole. Index Terms Adaptive scanning, intra video coding, lossless coding, mode-dependent templates.

I. I NTRODUCTION .264/ADVANCED VIDEO CODING (AVC), is a blockoriented motion-compensation-based codec standard developed by the ITU-T Video Coding Experts Group together with the International Organization for Standardization and International Electrotechnical Commission Moving Picture Experts Group. Currently, H.264/AVC has become one of the most commonly used formats for recording, compression, and distribution of various video content with a wide range of applications, including multimedia streaming and video conferencing [1][4]. The H.264/AVC standard has been developed by mainly focusing on lossy coding. However, lossless video coding is also important in areas such as source data archive and distribution [29], medical imaging [30] and machine-based video analysis. For source video archive and distribution, a video needs to be re-encoded repeatedly, with lossy coding, coding error will accumulate. For medical imaging, disease
Manuscript received August 27, 2011; revised April 10, 2012; accepted April 14, 2012. Date of publication May 3, 2012; date of current version August 22, 2012. This work was supported by MoEAcRF Tier 2, Singapore, under Grant T208B1218. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Hsueh-Ming Hang. Z. Gu, W. Lin, B.-S. Lee, and C. T. Lau are with the School of Computer Engineering, Nanyang Technological University, 639798 Singapore (e-mail: [email protected]). M.-T. Sun is with the Department of Electronic Engineering, University of Washington, Seattle, WA 98195 USA. Color versions of one or more of the gures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identier 10.1109/TIP.2012.2197630

diagnoses require medical images to have no data loss. Thus, rather than the use of lossy compression with relatively high compression ratio, mathematically lossless compression techniques are favored in this eld. For image/video analysis tasks performed by a machine, e.g., military object recognition and automatic product examination, being perceptual lossless is meaningless and lossless image/video coding is also preferred. Lured by the growing lossless coding market, H.264/AVC standard decided to provide improved functionality for lossless video coding. Firstly, pulse-code modulation (PCM) macroblock coding mode has been added in H.264/AVC standard. In this mode, the values of the original image pixels are sent directly without prediction, transform, and quantization. Although the PCM mode achieves lossless coding, its compression ratio is limited [5]. Later on, an extension to the original H.264/AVC, known as the delity range extensions (FRExt) [6], [7], has been developed by joint video team. In FRExt, a transform-bypass [8] lossless mode was included. Compared to the PCM mode, this mode employs prediction and entropy coding for lossless coding. To further enhance the coding performance for lossless coding, however, more efcient coding techniques for prediction are still required. In the meantime, a new intraprediction method called sample-wise differential PCM (DPCM) [5], [9], [10] was introduced for lossless intraprediction. Its main consideration is that the sample next to the one to be predicted is often a better predictor than a sample in a neighboring block several samples farther away. As a result, sample-wise DPCM has been shown to provide better compression performance without incurring a major increase in the computational complexity, and was subsequently adopted as a part of the new draft amendment for the H.264/AVC standard [11]. In H.264/AVC intra coding, given a 4 4 block, nine modes of intra prediction are used for spatial redundancy removal as shown in Fig. 1. After intra prediction, we get the residual block. For lossy intra coding, as shown in Fig. 2(a), integer discrete cosine transform (ICT) and quantization are rst applied to the input intra prediction residual block. Followed by the zigzag scan, the scanned coefcients are sent to the entropy coders for statistical redundancy removal. After ICT, the high magnitude coefcients tend to concentrate at the top-left part of the block and after quantization, the top-left part will have higher probability of having nonzero coefcients compared with the bottom-right part. Therefore, after

10577149/$31.00 2012 IEEE

GU et al.: MD-TEMPLATES AND SCAN ORDER FOR H.264/AVC-BASED INTRA LOSSLESS CODING

4107

Image Block

Residual Block Intra Prediction

Quantized Transform Coefficients ICT & Quantization

75 81 85 89 81 82 85 84 72 75 78 81 79 80 84 89

3 7 1 -2

6 1 3 1

4 2 3 4

4 -1 3 5

5 3 2 1

2 4 0 1

1 2 1 0

1 1 0 0

ZigZag Scan

Entropy Coder

Scanned Sequence

(a)
Image Block Residual Block Intra Prediction

Fig. 1.

Nine intra prediction modes [18].

75 81 85 89 81 82 85 84 72 75 78 81 79 80 84 89

3 7 1 -2

6 1 3 1

4 2 3 4

4 -1 3 5

the zigzag scan, the beginning part is more likely to have nonzero coefcients compared with the ending part. In the meanwhile, since the coefcients at the bottom-right corner tend to be small, after quantization, they are more likely to be quantized to consecutive zeros, the larger the quantization step is, the more zeros will be generated. The two entropy coders in H.264/AVC, context-based adaptive variable length coder (CAVLC) [12], and context-based adaptive binary arithmetic coder (CABAC) [13], are designed to explore these statistical characteristics. However, in lossless coding, the data sent to entropy coders are residual errors, which are the differential pixel values between the original and the predicted pixel values without transform and quantization, as shown in Fig. 2(b). Residual data differ from quantized transform coefcients in at least two aspects: 1) it has fewer zero entries and 2) all the positions in the prediction residual block have almost the same probability of having nonzero entries. Thus, the zigzag scan and the conventional entropy coders in H.264/AVC cannot provide the best coding performance for lossless video coding. Several studies [14][16], [19][22], [31], [32] have improved lossless coding based on H.264/AVC. Since H.264 lossless coding adopts a transform-bypass scheme, studies in [14][16] have explored possible improvements using transforms due to the energy compaction benet. Wei et al. [14] proposed an adaptive truncation algorithm for Hadamard transformed H.264/AVC lossless video coding. Ding et al. [15] proposed lossless coding algorithms that use the discrete wavelet transform to substitute the ICT in H.264/AVC. Ding et al. [16] also suggested two-layer coding algorithms to achieve both lossy and lossless coding using H.264/AVC. Moreover, in [15] and [16] the authors use CABAC and CAVLC adaptively based on the image content variation. Besides transforms, some other techniques [19][21] have been investigated for improving the lossless coding of H.264/AVC. Lee et al. [5] presented an efcient intra lossless coding method based on sample by sample DPCM for ve modes of H.264/AVC lossless coding and this technique was adopted in a new draft amendment for the H.264/AVC standard [11]. Nam et al. [19] proposed an additional pixel-wise prediction mode to co-operate with the original intra modes

ZigZag Scan

Entropy Coder

3 -2 1

3 -1 3

Scanned Sequence

(b)
Fig. 2. Comparison of H.264 intra lossy and lossless coding process. (a) Demonstration of H.264 lossy intra coding process. (b) Demonstration of H.264 lossless intra coding process.

in H.264/AVC. Heo et al. [20] modied three main coding processes of the conventional CAVLC in level and zero coding parts and achieved considerable performance improvement. Kim et al. proposed an improved entropy coding method [32] as well as an efcient two-layered residual coding scheme [31] that boost the compression efciency of H.264 intra lossless coding signicantly. In [21], Zhang et al. observed that the prediction residuals are not white. The redundancies can be reduced by further predictions since the residuals are in the form of granular noise. They thus proposed a compression scheme based on residual image prediction and coding. In [22], an adaptive scan order for CAVLC was proposed. For each intra mode, the authors rst analyze the average power of each position in a residual block and derive the new scan order in the descending order of average power of each position. However, its improvement is very limited, around 1% bit saving compared with [11]. For the mode-dependent transform (MDT) in the high efciency video coding (HEVC) testing prole [28], a pair of transform matrices is learned for each intra prediction mode and stored in both encoder and decoder. Inspired by MDT, we explore the statistical redundancy of prediction residual within intra prediction modes in this paper. By designing MD-templates to be subtracted by the intra prediction residual, more zero entries are generated; by designing the scan order corresponding to each template, the beginning part of the scanned coefcient sequence has a higher probability of having nonzero coefcients compared with the ending part. In this way, the resultant scanned sequence is more suitable for the entropy coding methods in H.264/AVC and experiment results conrm that considerable saving, in terms of bits, is achieved by the proposed MD-templates method

4108

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 21, NO. 9, SEPTEMBER 2012

Residual Block Xi

A MD-Template Wj

0.3

(a)
Fig. 3.

(b)

(c)

Probability

-5 7 -4 -2

6 -3 3 1

8 2 3 9

4 -1 3 5

ai 3

0 0 1 0

0 0 1 0

0 0 1 0

0 0 1 0

-5 7 -7 -2

6 -3 0 1

8 2 0 9

4 -1 0 5

0.25

0.2

0.15

Demonstration of X i classication process. TABLE I T RAINING S EQUENCES S ET A

0.1

0.05

Resolution QCIF (144 176)

No. of frames

200

CIF (288 352)

50

1080P (1080 1920)

Sequences Container Bus Silent Hall Waterfall City Paris Football Ducks_take_off Blue_sky Sunower Rush_hour

10

12

14

16

18

20

Absolute Value of Non-zero Residual Pixels

Fig. 4. Probability distribution of absolute value of rst 20 nonzero coefcients in residual blocks. TABLE III C ODEWORD TABLE FOR C OEFFICIENTS ai Value of ai 1 1 2 2 3 3 4 4 Sign bit 1 0 1 0 1 0 1 0 Code bits 1 1 01 01 001 001 000 000

TABLE II P ROBABILITY D ISTRIBUTION OF S ELECTED I NTRA P REDICTION M ODES Intra prediction mode Mode 0 Mode 1 Mode 2 Mode 3 Mode 4 Mode 5 Mode 6 Mode 7 Mode 8 Probability 0.302 0.262 0.182 0.071 0.074 0.024 0.035 0.021 0.029

compared with H.264/AVC delity range extensions high prole [11]. The rest of this paper is organized as follows. The methodology we use to derive MD-templates is described in Section II. In Section III, the two MD-template-based coding algorithms are given and the statistical comparison before and after using the MD-templates is also illustrated. Experimental results, presented in Section IV, show that the proposed scheme improves the compression efciency in terms of bit saving. Section V concludes this paper. II. P ROBLEM F ORMULATION A. MD-Template Learning Block-based templates are often used for both image/video processing [23] and coding [24] tasks. In this paper, we formulate the problem of MD-template learning for each intra prediction mode as follows. Firstly, we will classify residual blocks into different sets according to the given templates. Here we dene a set W j as
Wj

where X i is a given 4 4 matrix, which represents an intra predicted residual block belonging to one of the nine modes; W j is a 4 4 binary matrix with each entry being either zero or one. W j is what we are going to choose for MDtemplates; ai is a scale integer ranging from 255 to 255. The symbol is a multiplication operator. The operator vec() is a vectorization operator, which turns all the matrix elements to a vector form in the row by row order. Suppose Y is any m m matrix, after vectorization, we have the vector in (2), where each yr,c is an element of Y at the position (r, c) Vec(Y ) = [ y1,1, y1,2 , . . . , y1,m , y2,1 , y2,2 , . . . , y2,m , . . . , (2) ym ,1 , ym ,2 , . . . , ym ,m ] . The L0-norm, ||||0 , of a vector counts the total number of nonzero entries in the vector. In (1), minai ||vec(Xi ai W j )||0 means that given a W j , we try to nd an ai , such that for each X i , after being subtracted by ai W j the nonzero entries are minimized. With the condition of ||vec(Xi )||0 minai ||vec(Xi ai W j )||0 1, (1) means that given a W j , if more zero entries are generated in X i after being subtracted by ai W j , then X i belongs to the set W j . To classify X i to different W j , we resort to exhaustive search. Since W j is a 4 4 binary matrix, we have 244 = 65 536 sets to derive. Fig. 3 shows an example for the proposed classication. Suppose that we are given a W j as Fig. 3(b), and a residual block X i as Fig. 3(a). If we choose ai to be either 3 or 4, more zeros entries are generated. However, if ai = 4, only one more zero entry is generated, while if ai = 3, three more zero entries are generated, as shown in Fig. 3(c). Therefore, we will set ai = 3 since it generated minimal number of nonzero entries as stated in

= { X i | vec( X i )

min vec( X i ai W j )
ai

1} (1)

s.t. W j (r, c) {0, 1}, ai {255, 254, . . . , 1, 0, 1, . . . , 254, 255}

GU et al.: MD-TEMPLATES AND SCAN ORDER FOR H.264/AVC-BASED INTRA LOSSLESS CODING

4109

0 0 0 0
0 0 0 0

0 0 0 0
0 1 0 0
(i)

1 0 0 0
0 1 0 0

1 0 0 0
0 0 0 0

0 0 0 0
0 1 0 0

0 0 0 1
0 1 0 0
(j)

0 0 0 1
0 1 0 0

0 0 0 0
0 1 0 0

0 0 0 1
1 1 0 0

0 0 0 1
0 0 0 0

0 0 0 0
0 0 0 0

0 0 0 0
0 0 0 0

0 1 0 0
0 0 0 0

0 1 0 0
0 0 0 0
(l)

0 0 0 0
0 0 0 1

0 0 0 0
0 0 0 1

0 0 0 0
0 0 1 0

0 0 0 0
0 0 1 0

0 1 0 0
0 0 1 0

0 1 0 0
0 0 1 0

0 0 1 0
0 0 1 0

0 0 1 0
(f)

0 0 0 0
0 0 0 0

0 0 0 0
0 0 0 0

0 0 0 0
0 0 0 0

0 1 1 0
1 0 0 0

0 0 0 0
1 0 0 0

0 0 0 0
0 0 0 0

0 0 0 1
1 0 0 0

0 0 0 1
1 0 0 0

0 0 0 1
0 0 0 0

0 0 0 1
0 0 0 0

(a)

(b)

(c)

(d)

(e)

(g)

(h)

0 1 0 0

(k)

(m)

(n)

(o)

(p)

Fig. 5.

(a)-(p) Respectively, top 1st to 16th learned MD-templates from training set A for Mode 0.

minai ||vec(Xi ai W j )||0 . At the same time, we classify X i to W j according to the denition in (1). After we have W j for a certain intra mode, we dene S (W j ) = s.t. X i
i

TABLE IV T RAINING S EQUENCES S ET B Resolution QCIF (144 176) No. of frames Sequences Carphone Foreman Mobile Salesman Bus Hall Silent Tempete Ducks_take_off Blue_sky Station Old_town_cross

( B ( X i ) B ( X i ai W j ) Si deI n f o)
W j.

200

(3)
CIF (288 352) 50

Here B (X i ) is the number of bits required for coding residual block X i using H.264 Intra lossless coding, B (X i ai W j ) is the number of bits required for coding X i ai W j using H.264 Intra lossless coding, SideInfo represents bits required to code ai and W j . Therefore, function S (W j ) shows the total number of bits saved before and after using W j for residual blocks in a certain intra prediction mode. In [25][27], the learning and testing are conducted by learning on a set of sequences and testing on another set of sequences. In this paper, we follow this learning methodology. Specically, the sequences for MD-templates learning are summarized in Table I. All the training sequences are available at http://media.xiph.org/video/derf/. Among the nine intra prediction modes, the rst ve modes have higher occurrence probabilities than the rest four modes [17] and in [5] only rst ve modes are modied. We also record the overall selected intra prediction modes based on the learning sequences in Table I, and show the data in Table II. As indicated, Modes 0 to 4 account for more than 89% of prediction modes for intra prediction. Therefore, follow the methodology in [5], we only learn the MD-templates for these ve modes. Furthermore, we also generate the statistical distribution of the absolute values of nonzero entries for the residual blocks derived from the training sequences set A and plot the statistics of the rst 20 in terms of absolute value in Fig. 4. As the statistical data indicated, more than 75% of the nonzero prediction residual values range from 4 to 4. Therefore, we constrain ai at {4, 3, 2, 1, 1, 2, 3, 4}. The code word for ai is shown in Table III. The side information, SideInfo in (3), includes the code word for ai and 4 bits for each MDtemplate, since we are going to learn 16 MD-templates (in Section IV-A, we will discuss the number of MD-templates we used here). For a certain intra prediction mode, we try 65 536 W j and rank the derived S(W j ) in an descending order as S1 ,S2 ,,S65536. Then we name W 1 for the MD-template that

1080P (1080 1920)

corresponds to S1 , W 2 for the MD-template that corresponds to S2 and so on so forth. In other words, W 1 yields the most number of bit saving for a certain intra prediction mode, and W 2 yields the second most number of bit saving for a certain intra prediction mode. In Fig. 5, we show W 1 to W 16 from (a) to (p), which are the top 16 MD-templates learned from training set A for Mode 0. It is worth noting that these 16 MD-templates have strong directional patterns. For most of them, the positions of 1s are along the horizontal direction except Fig. 5(g), (k), and (n). On the other hand, if we use the training sequences set B shown in Table IV to learn the MD-templates for Mode 0, W 1 to W 16 are shown in Fig. 6. Most of MD-templates in Fig. 6 also have 1 entries distributed along horizontal direction, except Fig. 6(k), (l), and (o). Therefore, we take the directional exceptions of MD-templates, Figs. 5 (g), (k), (n), and 6(k), (l) and (o) in Mode 0, as a result of training set bias and we use the union set of top horizontally distributed MD-templates in Figs. 5 and 6 as the 16 MD-templates for Mode 0 as demonstrated in Fig. 7. Similarly, we derive the 16 MD-templates for Modes 1, 3, and 4 in Figs. 810, respectively. For Mode 2, W 1 to W 16 do not show strong directional patterns, therefore, we just choose the learned 16 MD-templates, W 1 to W 16, as shown in Fig. 11. B. Scan Order for Each MD-Template After we subtract the residual block, X i , by one of the 16 MD-templates multiplied by the corresponding scaling

4110

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 21, NO. 9, SEPTEMBER 2012

0 0 0 0

0 0 0 0

0 1 0 0

0 1 0 0

0 0 0 0

0 0 0 0

1 0 0 0

1 0 0 0

0 0 1 0

0 0 1 0

0 0 0 0

0 0 0 0

0 1 0 0

0 1 0 0

0 0 0 0

0 0 0 0

0 0 0 1

0 0 0 1

0 0 0 0

0 0 0 0

1 0 0 0

1 0 0 0

0 0 0 0
(f)

0 0 0 0

0 0 0 0

0 1 0 0

0 1 0 0

0 0 0 0

0 0 0 0

0 0 1 0

0 0 1 0

0 0 0 0

(a)

(b)

(c)

(d)

(e)

(g)

(h)

0 0 0 0

0 0 0 1
(i)

0 0 0 1

0 0 0 0

0 0 0 0

0 0 0 0
(j)

0 0 1 0

0 0 1 0

0 0 0 0

0 0 0 0

1 1 0 0

0 0 0 0

0 0 0 0

1 1 0 0

0 0 0 0
(l)

0 0 0 0

0 0 0 1

0 0 0 1

0 0 0 1

0 0 0 1

0 0 0 0

1 0 0 0

1 0 0 0

0 0 0 0

0 1 1 0

0 0 0 0

0 0 0 0

0 0 0 0

1 0 0 0

1 0 0 0

1 0 0 0

1 0 0 0

(k)

(m)

(n)

(o)

(p)

Fig. 6.

(a)-(p) Respectively, top 1st to 16th learned MD-templates from training set B for Mode 0.

1 0 0 0 0 0 0 0

1 0 0 0 0 0 0 0
(i)

1 0 0 0 1 0 0 0

1 0 0 0 1 0 0 0

0 1 0 0 0 0 0 0

0 1 0 0 0 0 0 0
(j)

0 1 0 0 0 1 0 0

0 1 0 0 0 1 0 0

0 0 1 0 0 0 0 0

0 0 1 0 0 0 0 0

0 0 1 0 0 0 1 0

0 0 1 0 0 0 1 0

0 0 0 1 0 0 0 0

0 0 0 1 0 0 0 0
(l)

0 0 0 1 0 0 0 1

0 0 0 1 0 0 0 1

1 0 0 0 0 0 0 0

1 0 0 0 1 0 0 0

0 0 0 0 1 0 0 0

0 0 0 0 0 0 0 0

0 1 0 0 0 0 0 0

0 1 0 0
(f)

0 0 0 0 0 1 0 0

0 0 0 0 0 0 0 0

0 0 1 0 0 0 0 0

0 0 1 0 0 0 1 0

0 0 0 0 0 0 1 0

0 0 0 0 0 0 0 0

0 0 0 1 0 0 0 0

0 0 0 1 0 0 0 1

0 0 0 0 0 0 0 1

0 0 0 0 0 0 0 0

(a)

(b)

(c)

(d)

(e)

(g)

(h)

0 1 0 0

(k)
Fig. 7.

(m)

(n)

(o)

(p)

16 MD-templates used for Mode 0.

1 1 1 1 0 0 1 1

0 0 0 0 0 0 0 0
(i)

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

1 1 1 1 0 0 1 1
(j)

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

1 1 1 1 0 0 1 1

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0
(l)

0 0 0 0 0 0 0 0

1 1 1 1 0 0 1 1

1 1 0 0 0 1 1 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

1 1 0 0
(f)

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

1 1 0 0 0 1 1 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

1 1 0 0 0 1 1 0

(a)

(b)

(c)

(d)

(e)

(g)

(h)

0 1 1 0

(k)
Fig. 8.

(m)

(n)

(o)

(p)

16 MD-templates used for Mode 1.

integer ai , we also dene a new scan order based on the selected MD-template. For the scan order, any design that reorders zero coefcients to the end of the scanned sequence is applicable. In this paper, we use the scan order demonstrated in Fig. 12. For Mode 0, the scan order is in a row by row, left to right, top to bottom manner. When the position labeled with 1 is scanned, it will be skipped rst and scanning in the same manner when all the 0 labeled positions are scanned, as shown in Fig. 12(a) and (b). Similarly, the scan order for Mode 1 is in a column by column, top to bottom, left to right manner with 1 positions reordered as shown in Fig. 12(c) and (d). For Mode 3, the scan order is the reection of the zigzag order with 1 positions reordered, as shown in Fig. 12(e) and (f). For Mode 4, the scan

order is the zigzag order with 1 positions reordered, as shown in Fig. 12(g) and (h). For Mode 2, the scan order of MD-templates (a)(b) is the same as in Mode 0, the scan order of MD-templates (c)(f) is the same as Mode 1, the scan order of MD-templates (g)(k) is the same as Mode 3, and the scan order of MD-templates (l)(p) is the same as Mode 4. In Fig. 13 we show the comparison of the H.264 intra lossless coding and proposed MD-template-based intra lossless coding. As observed in Fig. 13(b), after abstracting a MD-template multiplied by the corresponding scaling integer ai , three more zero coefcients are generated; after the MD scan, these three zero coefcients are arranged at the end of the scanned sequence. The resultant sequence ts the

GU et al.: MD-TEMPLATES AND SCAN ORDER FOR H.264/AVC-BASED INTRA LOSSLESS CODING

4111

1 0 0 0 0 1 0 0

0 1 0 0 0 0 0 0
(i)

0 0 1 0 0 0 0 1

0 0 0 1 0 0 0 0

1 0 0 0 0 0 0 0

0 1 0 0 1 0 0 0
(j)

0 0 0 0 0 0 0 0

0 0 0 0 0 0 1 0

0 0 0 0 0 1 0 0

0 0 0 0 0 0 1 0

0 0 1 0 0 0 0 0

0 0 0 1 0 0 0 0

0 0 0 0 0 0 0 0

0 1 0 0 0 0 1 0
(l)

0 0 1 0 0 0 0 1

0 0 0 0 0 0 0 0

1 0 0 0 0 0 0 0

0 0 0 0 1 0 0 0

0 0 1 0 0 1 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 1 0 0
(f)

0 0 0 0 0 1 0 0

0 0 0 1 0 0 1 0

0 0 0 0 0 0 1 0

1 0 0 0 0 0 0 1

0 1 0 0 0 0 0 0

0 0 1 0 0 0 0 0

0 1 0 0 0 0 0 0

0 0 1 0 0 0 0 0

0 0 0 1 1 0 0 0

0 0 0 0 0 1 0 0

(a)

(b)

(c)

(d)

(e)

(g)

(h)

0 0 0 0

(k)
Fig. 9.

(m)

(n)

(o)

(p)

16 MD-templates used for Mode 3.

0 0 0 1

0 0 1 0

0 1 0 0

1 0 0 0

0 0 0 0

0 0 0 0

0 1 0 0

1 0 0 0

0 0 0 0

0 0 1 0

0 1 0 0

0 0 0 0

0 0 0 1

0 0 1 0

0 0 0 0

0 0 0 0

0 0 0 0

0 0 1 0

0 0 0 0

1 0 0 0

0 0 0 1

0 0 0 0
(f)

0 1 0 0

0 0 0 0

0 0 0 0

0 0 0 1

0 0 1 0

0 1 0 0

0 0 1 0

0 1 0 0

1 0 0 0

0 0 0 0

(a)

(b)

(c)

(d)

(e)

(g)

(h)

0 0 1 0

0 0 0 0
(i)

1 0 0 0

0 0 0 0

0 0 0 0

0 0 0 1
(j)

0 0 0 0

0 1 0 0

0 0 0 0

0 1 0 0

1 0 0 0

0 0 0 0

0 0 1 0

0 1 0 0
(l)

0 0 0 0

0 0 0 0

0 0 0 0

0 0 0 0

0 0 1 0

0 1 0 0

0 0 0 0

0 0 0 1

0 0 1 0

0 0 0 0

0 1 0 0

1 0 0 0

0 0 0 0

0 0 0 0

0 0 0 0

0 0 0 0

0 0 0 1

0 0 1 0

(k)
Fig. 10.

(m)

(n)

(o)

(p)

16 MD-templates used for Mode 4.

1 0 0 0 0 0 0 0

1 0 0 0 0 1 0 0
(i)

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 1

0 0 0 0 0 1 0 0

0 0 0 0 0 0 0 0
(j)

0 1 0 0 0 0 1 0

0 1 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 0 0 0 1 0 0 0

0 1 1 0 0 0 1 0

0 0 0 0 0 0 0 0

1 1 0 0 0 1 0 0

0 0 0 0 1 0 0 0
(l)

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 1 1 0 0 0 0 0

0 0 0 0 0 0 1 0

0 0 0 0 0 1 0 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0

0 1 1 0
(f)

0 0 0 0 0 0 1 0

0 0 0 0 0 0 0 0

0 0 0 0 0 0 1 0

0 1 0 0 0 0 0 0

0 0 1 0 1 0 0 0

0 0 0 0 0 0 0 0

0 1 0 0 0 0 0 0

0 0 1 0 0 0 0 0

0 0 0 0 0 0 1 0

0 0 0 0 0 1 0 0

(a)

(b)

(c)

(d)

(e)

(g)

(h)

0 0 0 1

(k)
Fig. 11.

(m)

(n)

(o)

(p)

16 MD-templates used for Mode 2.

entropy coder in H.264 better compared with the sequence in Fig. 13(a). III. P ROPOSED A LGORITHMS A. Proposed Algorithms In this subsection, we give the pseudo-code of the MD-template-based intra lossless coding in Figs. 14 and 15. In Fig. 14, X i is the residual block belonging to a certain mode, l H .264 is the number of bits required to code X i by H.264 intra lossless (H.264-LS) with CAVLC coder. In Algorithm 1, all the possible combination of ai and MDtemplates for each block are tested. Then the coding result with minimal bits, L, is compared with the coded H.264 intra lossless bitstream l H .264 and the one with least bits is chosen.

However, Algorithm 1 is very time consuming since each combination of ai and MD-templates for each block is coded. Therefore, in Fig. 15, we give a fast algorithm (Algorithm 2) for practical implementation. In Algorithm 2, not all the combinations of ai and MD-templates for each block are coded and compared with the H.264-LS result. For an input residual block X i , after substracting the scaled MD-templates, if more than 1 zero coefcients are generated, it will be coded to compare the bits length with the H.264-LS result, and decide which method to use. Although Algorithm 2 is much faster than Algorithm 1, it reduces coding efciency since not all the combination of ai and MD-templates are coded to compare with the H.264-LS result.

4112

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 21, NO. 9, SEPTEMBER 2012

0 1 0 0
1 0 0 0

0 1 0 0
0 1 0 0

0 1 0 0
0 0 1 0

0 1 0 0
0 0 0 0

1 2 3 4 13 14 15 16 5 6 7 8 9 10 11 12
(b)

1 1 1 1
0 0 0 1

0 0 0 0
(c)

0 0 0 0
0 1 0 0

0 0 0 0
0 0 0 0

13 14 15 16
1 3 4 16

1 2 3 4
2 5 8 9

5 6 7 8

9 10 11 12

Inputs: Xi,lH.264

Outputs: The coded bitstream 1) forai= -4:-1,1:4 forW = MD-Template(a): MD-Template(p) calculate Ri = Xi aiW; scan Ri with the conrresponding scan order of W. code the scanned Ri with entropy coders in H.264/AVC and record the length of the derived bitstream l(ai ,W) end end 2) add the corresponding overhead bits H(ai ,W) to l(ai ,W) and choose the one with least bits, L=min (H(ai ,W)+l(ai ,W)). 3) ifL<lH.264 code the residual block with MD-Templates based method else code the residual block with convential H.264/AVC lossless method end

(a)

(d)

14 6 2 10 15 5 11 9 16 13 12 8
(f)

1 3 4 7

0 0 0 0

6 7 15 11 10 12 13 14

(e)

(g)

(h)

Fig. 12. (a) MD-template for Mode 1. (b) Corresponding scan order of MDtemplate (a). (c) MD-template for mode 0. (d) Corresponding scan order of MD-template (c). (e) MD-template for mode 3. (f) Corresponding scan order of MD-template (e). (g) MD-template for mode 4. (h) Corresponding scan order of MD-template (g).
Image Block Residual Block Intra Prediction

75 81 85 89 81 82 85 84 72 75 78 81 79 80 84 89 3 7 1 -2 6 1 3 1 4 2 3 4 4 -1 3 5

Fig. 14.

Pseudo-code of Algorithm 1.

Inputs: Xi,lH.264

ZigZag Scan

Outputs: The coded bitstream

3 -1 3 4 5

Entropy Coder

3 -2 1

Scanned Sequence

(a)
Image Block Residual Block Intra Prediction A MD-Template For Mode0

75 81 85 89 81 82 85 84 72 75 78 81 79 80 84 89

3 7 1 -2

6 1 3 1

4 2 3 4

4 -1 3 5

0 0 1 0

0 0 1 0

0 0 1 0

0 0 1 0

3 7 -2 -2

6 1 0 1

4 2 0 4

4 -1 0 5

Mode Dependent Scan

Entropy Coder

1 -2 1

5 -2 0

Scanned Sequence

(b)
Fig. 13. Demonstration of H.264 intra lossless coding with and without MD-templates. (a) Demonstration of H.264 lossless intra coding process. (b) Demonstration of H.264 lossless intra coding process using MD-templates.

1) forai= -4:-1,1:4 forW = MD-Template(a): MD-Template(p) calculate Ri = Xi aiW; if ||vec(Ri)||0 - ||vec(Xi)||0 2 scan Ri with the conrresponding scan order of W. code the scanned Ri with entropy coders in H.264/AVC and record the length of the derived bitstream l add the corresponding overhead bits Hto l,L= H+l ifL<lH.264 code the residual block with MD-Templatesbased method break end end ifW == MD-Template(p)&ai== 4 code the residual block with convential H.264/AVC lossless method end end end

Fig. 15. Pseudo-code of Algorithm 2. TABLE V SR C OMPARISON Resolution QCIF CIF Sequences Foreman Mobile Tempete Flowergarden Crowdrun Parkrun SR (Algorithm 1) 81.3 78.2 76.6 78.5 68.5 65.7 74.8 SR (Algorithm 2) 48.4 49.4 47.7 51.3 39.3 38.8 45.8

B. Statistics Before and After Coding With the Proposed Algorithms To give a more detailed analysis of our method, we dene the selection rate (SR) as Selection Rate(%) = No. of BlocksMDtemplates 100 No. of BlocksAll (4)

1080P

where No. of BlocksMDtemplates is the number of blocks choose to use the MD-templates to code and No. of BlocksAll is the total number of blocks in a video. In Table V, we record the SR for six sequences, each with 100 frames of the YUV 4:2:0 format, coded with Algorithms 1 and 2. As we can observe, with Algorithm 1, about 75% of blocks could be further compressed with the proposed MDtemplate method. On the other hand, with Algorithm 2, the proposed MD-template method improves the coding efciency

Average SR

for 46% of blocks. Anyway, Algorithm 2 still achieves considerable bit saving as recorded in Tables VIII and IX. We further calculate the number of zero coefcients before and after using the MD-templates in Table VI for the six sequences with different resolutions. As observed, after

GU et al.: MD-TEMPLATES AND SCAN ORDER FOR H.264/AVC-BASED INTRA LOSSLESS CODING

4113

TABLE VI N UMBER OF Z ERO C OEFFICIENTS C OMPARISON Proposed (Algorithm 2) 4.87 3.78 2.45 3.02 2.31 2.29 3.12 1.47 Fig. 17.
0.08 0.07 0.06 0.05 0.04 0.03 0.02 0 Proposed H.264-LS

Sequences Foreman Mobile Flowergarden CIF Tempete Crowdrun 1080P Parkrun Average zero coefcients QCIF

H.264-LS(CAVLC) [11] 2.25 1.90 1.37 1.73 1.37 1.26 1.65

Average zero coefcients over H.264 lossless

Coding time increase comparison in percentage.

6 8 10 Scanning Position

12

14

16

Fig. 16. Probability distribution of existence of nonzero coefcients versus scanning position.

we need to add some syntax elements in the bitstream for decoding. For each coded block in the bitstream, following the syntax of intra-prediction mode information, 1 bit is used to indicate whether this block is coded with proposed method. If this block is coded with the standard intra H.264-LS, CAVLC code is followed. If this block is coded with proposed method, the code of scaling integer ai (please refer to Table III) and code of MD-template used for this block (4 bits for each template) are inserted before the CAVLC code. If better entropy coding is used, we can expect even higher compression ratio. CAVLC is used in all the experiments. Here the compression ratio and the bit saving are dened in (5) and (6). A. Number of MD-Templates to Use

applying the MD-templates, 1.47 more zero coefcients are generated on average. In Fig. 16, we plot the probability distribution of the exsisting nonzero coefcients versus the scanning position before and after using the MD-templates for the CIF Tempete sequence. As observed, before using the MD-templates, each position almost has the same probability of having nonzero coefcients (the solid curve), after using the MD-templates, the position at the beginning part tends to be more likely to have nonzero coefcients compared with the ending part of the scanning position (the dash curve). IV. OVERALL E XPERIMENTS AND R ESULTS In this section, ve QCIF sequences, two CIF sequences, ve 512 512 images and four HD sequences are selected to show the effectiveness of the proposed method as well as to provide a comprehensive comparison with the state-of-art intra lossless coding method based on H.264-LS [14], [20] with CAVLC entropy coding Original Video Size (5) Compression Ratio = Bit rateMethod Bit Saving(%) = Bit rateH.264 Bit rateMethod 100. BitrateH.264 (6)

Obviously, with more MD-templates to choose, e.g., 32 MD-templates or 64 MD-templates, we have higher probability of getting a block with more zeros and this will give more potential for further compression. However, more MDtemplates will lead to more computation. More importantly, we also need more bits to represent these MD-templates, e.g., 5 bits for 32 MD-templates and 6 bits for 64 MD-templates, which will negatively impact the coding efciency. Therefore, we need to choose a reasonable MD-templates number. In Table VII, we show the coding result with eight MD-templates, 16 MD-templates, and 32 MD-templates for the six sequences. As indicated in Table VII, when we increase the number of MD-templates from 8 to 16, the averaged bit saving increases by about 2%. However, when we further increase the number of MD-templates from 16 to 32, the bit saving starts to drop. Therefore, we use 16 MD-templates in our method. B. Coding Performance Comparison In Tables VIII and IX, we compare the compression performance among H.264-LS (CAVLC), Heos method [20], Weis method [14] and the proposed method with Algorithm 1 and Algorithm 2. All the data for Heos and Weis methods are directly extracted from [14] and [20]. If a sequence is used for training, it will not be included in the testing results. As indicated in the bold numbers, the proposed method with fast coding algorithm (Algorithm 2) achieves about 7.2% bit saving compared with H.264-LS on average. The proposed

The CAVLC decoding part does not need to change since the MD-templates are added after CAVLC decoding. However,

4114

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 21, NO. 9, SEPTEMBER 2012

TABLE VII C OMPRESSION E FFICIENCY C OMPARISON W ITH D IFFERENT MD-T EMPLATES N UMBERS ( THE N UMBER IN B OLD F ONT S HOWS THE B EST R ESULT ) H.264-LS (CAVLC) [11] Compression ratio 2.16 1.53 1.86 1.76 2.11 2.05 1.91

8 MD-templates (Algorithm 2) Compression ratio 2.25 1.64 1.99 1.89 2.19 2.14 2.02 Saving (%)* 4.02 7.14 6.35 6.78 3.65 4.18 5.35

16 MD-templates (Algorithm 2) Compression ratio 2.31 1.69 2.02 1.93 2.22 2.18 2.06 Saving (%)* 6.59 9.08 8.00 8.76 5.10 5.91 7.24

32 MD-templates (Algorithm 2) Compression ratio 2.31 1.68 2.02 1.92 2.21 2.17 2.05 Saving (%)* 6.54 8.84 8.08 8.46 4.93 5.81 7.16

Sequences Foreman# Mobile# Tempete$ Flowergarden$ Crowdrun& Parkrun& Average

Bit saving calculated relative to H.264-LS (CAVLC). # 150 frames of YUV 4:2:0 format with QCIF (144 176) resolution. $ 150 frames of YUV 4:2:0 format with CIF (288 352) resolution. & 150 frames of YUV 4:2:0 format with 1080P (1080 1920) resolution.

TABLE VIII C OMPRESSION P ERFORMANCE C OMPARISON OF H EO S [20] M ETHOD AND THE P ROPOSED M ETHOD ( THE N UMBER IN B OLD F ONT S HOWS THE B EST R ESULT B ETWEEN H EO S [20] M ETHOD AND THE P ROPOSED A LGORITHM 2) H.264-LS (CAVLC) [11] Compression ratio 2.16 1.53 1.86 1.76 2.11 2.05 1.91

Heo [20] Compression ratio 2.27 1.71 2.01 1.96 2.19 2.16 2.05 Saving (%)* 4.93 10.67 7.30 9.88 3.58 4.85 6.87

Proposed (Algorithm 2) Compression ratio 2.31 1.69 2.02 1.93 2.22 2.18 2.06 Saving (%)* 6.59 9.08 8.00 8.76 5.10 5.91 7.24

Proposed (Algorithm 1) Compression ratio 2.36 1.71 2.07 1.97 2.27 2.23 2.10 Saving (%)* 8.60 10.88 10.24 10.89 7.12 8.14 9.31

Sequences Foreman# Mobile# Tempete$ Flowergarden$ Crowdrun& Parkrun& Average

Bit saving calculated relative to H.264-LS (CAVLC). # 150 frames of YUV 4:2:0 format with QCIF (144 176) resolution. $ 150 frames of YUV 4:2:0 format with CIF (288 352) resolution. & 150 framesof YUV 4:2:0 format with 1080P (1080 1920) resolution.

MD-templates-based method with Algorithm 2 gives better performance compared with Heos and Weis methods, for about 0.4% and 0.8% bit saving improvement, respectively. If a longer encoding process is allowed, Algorithm 1 could give even higher performance, about 2.4% and 3% bit saving improvements on average compared with Heos and Weis methods, respectively. C. Computation Time Comparison For intra lossless coding, the most time consuming part is the CAVLC module. Before subtracting MD-templates, the encoder needs to code each block nine times to decide which mode this block belongs to, thus it requires to conduct CAVLC for nine times. In Algorithm 1, all the possible combinations of ai and MD-templates for each block are tested. Then the combination with minimal bits, L, is compared with the coded intra H.264-LS bitstream l H .264 and the one with least bits is chosen. Since we have eight values for ai and 16 MD-templates, we need to code (with CAVLC) each block 8 16 = 128 times, which takes almost 13 times more

computation than that of the standard intra H.264-LS coding (to conduct CAVLC for nine times). For Algorithm 2, for most of the cases, CAVLC is conducted once to decide whether the residual block X i is coded with the MD-template-based method or not. In Table X, we show the distribution of times of CAVLC used in Algorithm 2. As summarized in Table X, on average, for 62% of residual blocks, Algorithm 2 requires to conduct CAVLC only once to decide whether to code with the MD-template-based method or not. For 15% of residual blocks, Algorithm 2 requires to conduct CAVLC more than twice to decide whether to code with the MD-template-based method or not. Therefore, the increased computation by Algorithm 2 is very limited. In Fig. 17, we show the computation time increase comparison between intra H.264-LS and Algorithm 2. Sequences Akiyo, News and Coastguard are QCIF sequences with 30 frames. Sequences Tempete and Flowergarden are CIF sequences with 30 frames. House and Lena are 512 512 images. With Intel X5560 CPU at 2.80 GHz, 16 GB RAM and 64-bit Operation System, we record the coding time increase for the seven test sequences and images. As indicated

GU et al.: MD-TEMPLATES AND SCAN ORDER FOR H.264/AVC-BASED INTRA LOSSLESS CODING

4115

TABLE IX C OMPRESSION P ERFORMANCE C OMPARISON OF W EI S [14] M ETHOD AND THE P ROPOSED M ETHOD ( THE N UMBER IN B OLD F ONT S HOWS
THE

B EST R ESULT B ETWEEN W EI S [14] M ETHOD AND THE P ROPOSED A LGORITHM 2)

Sequences Akiyo# Coastguard# Foreman# Mobile# News# Avion@ Baboon@ House@ Lena@ Peppers@ InToTree& TownCross& Average

H.264-LS (CAVLC) [11] Compression ratio 2.53 2.13 2.23 1.45 2.25 2.09 1.24 1.88 1.78 1.63 1.73 1.62 1.88

Wei [14] Compression ratio 2.74 2.25 2.43 1.57 2.43 2.26 1.35 2.03 1.86 1.66 1.82 1.67 2.00 Saving (%)* 7.81 5.74 8.23 8.16 7.50 7.77 8.77 7.75 4.47 2.17 5.05 3.33 6.40

Proposed (Algorithm 2) Compression ratio 2.74 2.29 2.41 1.58 2.46 2.25 1.35 2.05 1.92 1.72 1.83 1.71 2.03 Saving (%)* 7.63 6.83 7.62 8.49 8.62 7.13 8.53 8.25 7.12 5.02 5.29 5.42 7.16

Proposed (Algorithm 1) Compression ratio 2.80 2.34 2.47 1.62 2.52 2.31 1.39 2.10 1.96 1.75 1.87 1.75 2.07 Saving (%)* 9.79 8.80 9.79 10.75 10.76 9.33 10.70 10.31 9.24 7.04 7.27 7.41 9.27

Bit saving calculated relative to H.264-LS (CAVLC). # 30 frames of YUV 4:4:4 format with QCIF (144 176) resolution. @ Imagewith 512 512 resolution. & 10 framesof YUV 4:4:4 format with 1080P (1080 1920) resolution.

TABLE X P ROBABILITIES OF T IMES OF CAVLC U SED IN A LGORITHM 2 Times of CAVLC used Twice More than twice 0.250 0.170 0.257 0.219 0.254 0.213 0.238 0.229 0.242 0.175 0.095 0.169 0.164 0.083 0.134 0.151

Sequences/Images Akiyo News Coastguard FlowerGarden Tempete House Lena Average

Once 0.508 0.655 0.648 0.612 0.582 0.704 0.628 0.620

QCIF

in H.264/AVC. With insignicant computational increase, the proposed method with fast coding algorithm (Algorithm 2) achieves about 7.2% bit saving compared with intra H.264-LS. If the coding condition allows a longer encoding process, the Algorithm 1 that codes all the possible combinations of ai and MD-templates will give even better performance, about 9.3% of bit saving compared with intra H.264-LS. R EFERENCES
[1] Advanced Video Coding for Generic Audiovisual Services, ITU-T Standard H.264 and ISO/IEC 14496-10, May 2003. [2] Advanced Video Coding[S] Final Committee Draft, ITU-T Standard H.264/ISO/IEC 11496-10, 2002. [3] T. Wiegand, G. J. Sullivan, G. Bjntegaard, and A. Luthra, Overview of the H.264/AVC video coding standard, IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 560576, Jul. 2003. [4] A. Luthra, G. J. Sullivan, and T. Wiegand, Introduction to the special issue on the H.264/AVC video coding standard, IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 557559, Jul. 2003. [5] Y. L. Lee, K. H. Han, and G. J. Sullivan, Improved lossless intra-coding for H.264/MPEG-4 AVC, IEEE Trans. Image Process., vol. 15, no. 9, pp. 26102615, Sep. 2006. [6] Draft Text of H.264/AVC Fidelity Range Extensions Amendment to ITUTRec.H.264, Standard ISO/IEC14496-10 AVC, ISO/IEC JTC1/SC29/WG11, and ITU-T Q.6/SG16, Jul. 2004. [7] G. J. Sullivan, P. Topiwala, and A. Luthra, The H.264/AVC standard: Overview and introduction to the delity range extensions, Proc. SPIE, vol. 5558, pp. 454474, Aug. 2004. [8] Intra-Lossless Coding and QP Range Selection, Standard ISO/IEC JTC1/SC29/WG11 and ITU-TQ.6/SG16, May 2002. [9] Lossless Intra-Coding for Improved 4:4:4 Coding in H.264/MPEG-4 AVC, Standard ISO/IEC JTC1/SC29/WG11 and ITU-T Q.6/SG16, Jul. 2005. [10] Complexity of the Proposed Lossless Intra, Standard ISO/IEC JTC1/SC29/WG11 and ITU-T Q.6/SG16, Oct. 2005. [11] Draft Text of H.264/AVC Advanced 4:4:4 Prole Amendment to ITU-TRec. H.264|ISO/IEC 14496-10 AVC, Standard ISO/IEC JTC1/SC29/WG11 and ITU-T Q.6/SG16, Oct. 2005. [12] Context-Adaptive VLC (CVLC) Coding of Coefcients, Standard ISO/IEC JTC1/SC29/WG11 and ITU-TQ6/SG16m, May 2002.

CIF Image

in Fig. 17, the coding time is increased modestly by 26.8% on average with Algorithm 2. V. C ONCLUSION The entropy coding module in the H.264/AVC standard was originally designed for lossy video coding and did not yield adequate performance for lossless video coding. For the MD Transform (MDT) in the HEVC testing prole, a pair of transform matrices is learned for each intra prediction mode and stored in both encoder and decoder. Inspired by MDT, we have explored the statistical correlation of the intra prediction residual for the ve intra prediction modes. By designing templates to be subtracted for each intra prediction modes, more zero entries are generated. By further designing the scan order for each template, the beginning part of the scanned coefcient sequence has higher probability of having nonzero coefcients compared with the ending part. In this way, the resultant sequence is more suitable for entropy coding

4116

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 21, NO. 9, SEPTEMBER 2012

[13] D. Marpe, H. Schwarz, and T. Wiegand, Context-based adaptive binary arithmetic codingin-the H.264/AVC video compression, IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 620636, Jul. 2003. [14] S. T. Wei, C. W. Tien, B. D. Liu, and J. F. Yang, Adaptive truncation algorithm for hadamard-transformed H.264/AVC lossless video coding, IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 5, pp. 538549, May 2011. [15] J. R. Ding and J. F. Yang, Adaptive entropy coding with (5, 3) DWT for H.264 lossless image coding, in Proc. IEEE Region 10 Conf. TENCON, Oct. 2007, pp. 14. [16] J. R. Ding, J. Y. Chen, F. C. Yang, and J. F. Yang, Two-layer and adaptive entropy coding algorithms for H.264-based lossless image coding, in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., Mar. 2008, pp. 13691372. [17] H. Q. Zeng, K. K. Ma, and C. H. Cai, Hierarchical intra mode decision for H.264/AVC, IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 6, pp. 907912, Jun. 2010. [18] Y. W. Huang, B.-Y. Hsieh, T.-C. Chen, and L.-G. Chen, Analysis, fast algorithm, and VLSI architecture design for H.264/AVC intra frame coder, IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 3, pp. 378401, Mar. 2005. [19] J. H. Nam and D. Sim, Lossless video coding based on pixelwise prediction, Multimedia Syst., vol. 14, no. 5, pp. 291298, Nov. 2008. [20] J. Heo and Y. S. Ho, Efcient level and zero coding methods for H.264/AVC lossless intra coding, IEEE Signal Process. Lett., vol. 17, no. 1, pp. 8790, Jan. 2010. [21] Q. Zhang, Y. Dai, and C. C. J. Kuo, Lossless video compression with residual image prediction and coding (RIPC), in Proc. IEEE Int. Symp. Circuits Syst., May 2009, pp. 617620. [22] Y. C. Wei, Performance improvements for intra-prediction in advanced video coding, Ph.D. dissertation, Dept. Electr. Eng., Nat. Cheng Kung Univ., Tainan, Taiwan, 2008. [23] Y. H. Huang and K. L. Chung, Texture-and multiple-templatebased algorithm for lossless compression of error-diffused images, IEEE Trans. Image Process., vol. 16, no. 5, pp. 12581268, May 2007. [24] M. Paul, W. Lin, C. T. Lau, and B. S. Lee, Direct inter-mode selection for H.264 video coding using phase correlation, IEEE Trans. Image Process., vol. 20, no. 2, pp. 461473, Feb. 2011. [25] O. G. Sezer, R. A. Cohen, and A. Vetro, Robust learning of 2-D separable transforms for next-generation video coding, in Proc. Data Compress. Conf., 2011, pp. 6372. [26] Y. Ye and M. Karczewicz, Improved intra coding, ITU-T Q.6/SG11VCEG, Shenzhen, China, Tech. Rep. ITU-T Q.6/SG11 VCEG, Oct, 2007. [27] M. Budagavi and M. Zhou, Orthogonal MDDT and mode dependent DCT, ITU-T Q.6/SG16VCEG, Kyoto, Japan, Tech. Rep. ITU-T Q.6/SG16 VCEG, Jan. 2010. [28] Test Model Under Consideration, Standard JCTVC-A205, Apr. 2010. [29] R. Dvila and I. Gilmour. (2006, Jan.). Lossless Video Compression for Archives: Motion JPEG2k and Other Options. Media Matters, LLC, New York [Online]. Available: http://www.mediamatters.net/docs/WhitePapers/WPMJ2k.pdf [30] D. Clunie, Lossless compression of grayscale medical imageseffectiveness of traditional and state of the art approaches, Proc. SPIE Med. Imag., vol. 3980, pp. 7484, Feb. 2000. [31] S. H. Kim, J. W. Kang, and C.-C. J. Kuo, Improved H.264/AVC lossless intra coding with two-layered residual coding (TRC), IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 7, pp. 10051010, Jul. 2011. [32] S.-H. Kim, J. Heo, and Y.-S. Ho, Efcient entropy coding scheme for H.264 lossless video coding, Signal Process., Image Commun., vol. 25, no. 9, pp. 687696, Oct. 2010.

Weisi Lin (M92SM98) received the B.Sc. degree in electronics and the M.Sc. degree in digital signal processing from Zhongshan University, Guangzhou, China, and the Ph.D. degree in computer vision from Kings College, London University, London, U.K. He was the Project Leader of over ten successfully delivered major projects in digital multimedia technology development. Currently, he is an Associate Professor with the School of Computer Engineering, Nanyang Technological University, Singapore. He has published over 190 refereed papers in international journals and conferences. His areas of expertise include image processing, perceptual modeling, video compression, multimedia communication, and computer vision. Dr. Lin served as the Lab Head, Visual Processing, and the Acting Department Manager, Media Processing, for the Institute for Infocomm Research. He is a Chartered Engineer (U.K.), a fellow of the Institution of Engineering Technology, and an Honorary Fellow of the Singapore Institute of Engineering Technologists. He is currently on the editorial boards of the IEEE T RANSACTIONS ON M ULTIMEDIA, the IEEE S IGNAL P ROCESSING L ETTERS and the Journal of Visual Communication and Image Representation, and was elected as a Distinguished Lecturer of APSIPA in 2012. Bu-Sung Lee (M08) received the B.Sc. (Hons.) and Ph.D. degrees from the Electrical and Electronics Department, Loughborough University of Technology, U.K., in 1982 and 1987, respectively. He is an Associate Professor with Nanyang Technological University, Singapore. He is active in the development of the Research and Education Network, Singapore and the Asia Pacic Region, e.g., Trans-EuroAsia Network. His current research interests include grid and cloud computing, wireless communication, and multimedia communication. Chiew-Tong Lau (M84) received the B.Eng. degree from Lakehead University, Thunder Bay, ON, Canada, in 1983, and the M.A.Sc. and Ph.D. degrees in electrical engineering from the University of British Columbia, Vancouver, BC, Canada, in 1985 and 1990, respectively. He is currently an Associate Professor with the School of Computer Engineering, Nanyang Technological University, Singapore. His current research interests include wireless communications systems and signal processing. Ming-Ting Sun (M81SM89F96) received the B.S. degree from National Taiwan University in 1976, the M.S. degree from the University of Texas at Arlington, Arlington, in 1981, and the Ph.D. degree from the University of California, Los Angeles, in 1985, all in electrical engineering. He joined the Faculty of the University of Washington in September 1996. Before that, he was the Director of the Video Signal Processing Group, Bellcore (now Telcordia). He has been a Chair Professor with Tsing Hwa University, and a Visiting Professor with Tokyo University, National Taiwan University, and National ChungCheng University. His current research interests include machine learning and video processing. He holds 13 patents and has published more than 200 technical publications, including 15 book chapters in the area of video technology. He was actively involved in the development of H.261, MPEG-1, and MPEG-2 video coding standards. He has also co-edited a book entitled Compressed Video Over Networks (CRC Press, 2000). Dr. Sun is currently the Editor-in-Chief of the Journal of Visual Communications and Image Representation. He was the Editor-in-Chief of the IEEE T RANSACTIONS ON M ULTIMEDIA from 2000 to 2001. He was a recipient of an IEEE CASS Golden Jubilee Medal in 2000 and he was a co-chair of the Visual Communication and Image Processing (SPIE VCIP) Conference in 2000. He was the Editor-in-Chief of the IEEE T RANSACTIONS ON C IRCUITS AND S YSTEMS FOR V IDEO T ECHNOLOGY (T-CSVT) from 1995 to 1997 and the Express Letter Editor of T-CSVT from 1993 to 1994. He was a co-recipient of the T-CSVT Best Paper Award in 1993 and the Award of Excellence from Bellcore in 1987 for his work on digital subscriber line technology. From 1988 to 1991, he served as the Chairman of the IEEE CAS Standards Committee and established an IEEE Inverse Discrete Cosine Transform Standard.

Zhouye Gu received the B.Sc. degree in electronic information science and technology from the Nanjing University of China, Nanjing, China, in 2007. He is currently pursuing the Ph.D. degree with the School of Computer Engineering, Nanyang Technological University, Singapore. His current research interests include image and video coding and processing and machine learning.