3

While debugging HM 16.2's decoding, I see CUs that look like they have been subdivided into PUs. Each PU has the same MV. Allowing different MVs within a CU is the only reason I (currently) know to split a CU into PUs.

I wonder if I am misinterpreting the CTU data structure (TComDataCU)*. Can anyone help me with this? Do you know if there are other reasons to split a CU into PUs?

Related questions:

  1. Why does TComDataCU split a 64x64 CTU split into 256 parts? (I initially expected to see 64 parts, one part for each of the smallest possible 8x8 CUs. Now I assume that the additional parts allow for smaller PUs/TUs.)
  2. Is it correct to interpret TComMv::getHor() and TCovMv::getVer() directly as an MV, or must some additional information (e.g. merge/skip info, deltas, etc..) be incorporated to get a "true" MV?

* For a TComDataCU* called ctu, I see

ctu->getTotalNumPart() == 256
ctu->getDepth(48) == 3
ctu->getPredictionMode(48) == INTER_MODE
ctu->getPartitionSize(48) == Nx2N

and where mvf = ctu->getCUMvField(REF_PIC_LIST_0) (Only uni-prediction is used, with a simple GOP: I <- P <- P <- P ...),

mvf->getMv(48).getHor() == mvf->getMv(50).getHor() &&
mvf->getMv(49).getHor() == mvf->getMv(51).getHor() &&
mvf->getMv(48).getVer() == mvf->getMv(50).getVer() &&
mvf->getMv(49).getVer() == mvf->getMv(51).getVer() &&
mvf->getMv(48).getHor() == mvf->getMv(49).getHor() &&
mvf->getMv(48).getVer() == mvf->getMv(49).getVer()

Linking these observations to my question, for this CU I understand that the CU has two PUs encompassing indices 48,50 and 49,51 like

+--+--+
|48|49|
+--+--+
|50|51|
+--+--+

and so I expect

mvf->getMv(48).getHor() == mvf->getMv(50).getHor() &&
mvf->getMv(49).getHor() == mvf->getMv(51).getHor() &&
mvf->getMv(48).getVer() == mvf->getMv(50).getVer() &&
mvf->getMv(49).getVer() == mvf->getMv(51).getVer() && (
mvf->getMv(48).getHor() != mvf->getMv(49).getHor() ||
mvf->getMv(48).getVer() != mvf->getMv(49).getVer()
)

Why do the two PUs (appear to) have identical MVs?

1 Answer 1

1

First of all, as you noticed, the smallest block size in HEVC is 4x4. CUs can only take sizes between 64x64 and 8x8 but PUs or TUs can go down to 4x4. Apart from the reason you cited, a CU can also be split into 4 PUs in the case of intra-encoding, and the 4 PUs could have different intra-prediction directions.

As the smallest block size is 4x4, a 64x64 CTU is composed of 256 parts.

In the HM reference software, the data structure of a CTU TComDataCU will always comprise all possible smallest blocks, irrespectively of the actual block structure. That's why it's possible (and is often the case), that the CU data is stored redundantly.

For example, a 64x64 CTU called ctu which contains a single CU (so CU of size 64x64) will have 256 depths 0 stored. If you check the depth at (z-scan) index 0, ctu->getDepth(0), you will get 0. This would be sufficient to describe the size of the CU, but if you check the depth at another index ctu->getDepth(48), you will also get 0, which is redundant.

This is what you observe in your case. Your CU is split into 2 vertical PUs (Nx2N), so (48,50) is one single PU and is described with one motion vector. However, the data structure in HM stores the MV at both places 48 and 50.

To address your second related question, yes the horizontal and vertical parts of an MV "truly" describe it. However, you might additionally need the index of the referenced frame. This can be found in TComMvField which additionally to the TComMv also has an Int for the index.

3
  • Thanks for your help so far. You explain that 48 and 50 have the same MV due to TComDataCU's redundant storage; and that makes sense to me. It does not (yet) make sense to me that 48 and 49 have the same MV when ctu->getPartitionSize(48) == Nx2N. Apr 5, 2015 at 4:01
  • @chris.redekop Right, this doesn't really seem to make sense. One possible explanation that I see, is that the 2 PUs don't have the same reference frame. I see you are using IPP structure in your case. If you are using the encoder_lowdelay_P_main config file from HM, it looks like each P frame can reference 4 prior frames. In that case, the 2 PUs could have "same" motion vector but different reference index.
    – damjeux
    Apr 7, 2015 at 11:45
  • Yes, if I was using a GOP structure other than IPP I could see multiple references coming into play here. With the IPP structure, however, I don't think multiple references come into play... Apr 11, 2015 at 20:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.