32

I'm trying to figure out what "continuation bytes" are (for curiousity sake) in the UTF-8 encoding.

Wikipedia introduces this term in the UTF-8 article without defining it at all

Google search returns no useful information either. I'm about to jump into the official specification, but would preferably read a high-level summary first.

1
  • Looks like somebody just edited the Wikipedia article. (:
    – tripleee
    Feb 20, 2012 at 13:08

3 Answers 3

57

A continuation byte in UTF-8 is any byte where the top two bits are 10.

They are the subsequent bytes in multi-byte sequences. The following table may help:

Unicode code points  Encoding  Binary value
-------------------  --------  ------------
 U+000000-U+00007f   0xxxxxxx  0xxxxxxx

 U+000080-U+0007ff   110yyyxx  00000yyy xxxxxxxx
                     10xxxxxx

 U+000800-U+00ffff   1110yyyy  yyyyyyyy xxxxxxxx
                     10yyyyxx
                     10xxxxxx

 U+010000-U+10ffff   11110zzz  000zzzzz yyyyyyyy xxxxxxxx
                     10zzyyyy
                     10yyyyxx
                     10xxxxxx

Here you can see how the Unicode code points map to UTF-8 multi-byte byte sequences, and their equivalent binary values.

The basic rules are this:

  1. If a byte starts with a 0 bit, it's a single byte value less than 128.
  2. If it starts with 11, it's the first byte of a multi-byte sequence and the number of 1 bits at the start indicates how many bytes there are in total (110xxxxx has two bytes, 1110xxxx has three and 11110xxx has four).
  3. If it starts with 10, it's a continuation byte.

This distinction allows quite handy processing such as being able to back up from any byte in a sequence to find the first byte of that code point. Just search backwards until you find one not beginning with the 10 bits.

Similarly, it can also be used for a UTF-8 strlen by only counting non-10xxxxxx bytes.

11
  • I think this is a bit of a stretch/overbroading... anyways, it contains useful info! What I can't understand though, is why 11 is ever needed. One can say that the leading byte starts with 0, and the continuation ones (they can be arbitarily many) start with 1.
    – EKons
    Aug 27, 2016 at 18:37
  • @ΈρικΚωνσταντόπουλος "the leading byte starts with 0" -- this is not correct. A byte that starts with 0 is a single-byte code point, so it is neither a leading byte nor a continuation. It stands alone. That's what makes it distinct from bytes starting with 11, which indicate it is the first byte of a sequence and more bytes are expected to follow in order to represent a single code point. Mar 21, 2017 at 20:45
  • @WilliamPrice Dunno why I posted that off-topic comment, but I think it was me trying to invent my own encoding.
    – EKons
    Mar 22, 2017 at 11:40
  • You say that 110xxxxx has two bytes, 1110xxxx has three and 11110xxx has four. Shouldn't it be the other way? 010 for 2, 011 for 3, 100 for 4? And if not, why not in binary code?
    – Cornelius
    Aug 19, 2020 at 2:44
  • @Cornelius: no, the bit patterns you've given are the binary values for 2, 3, and 4, but that has nothing do do with UTF-8 encoding. If the first byte starts with 110, there is one continuation byte. If it starts 1110, there are two continuation bytes. Ditto for 11110 having three continuation bytes. It may help to think of the first bit of the first byte deciding whether it's a multi-byte (if 1) or not (if 0). Then the number of consecutive 1 bits after that is the number of continuation bytes that follow (1-3).
    – paxdiablo
    Aug 19, 2020 at 2:52
1

In short words, continuation bytes are the bytes except first byte or single byte. In UTF-8, continuation bytes are started with 0x10.

1
  • I think you mean to say 0b10000000, which is 0x80 (the relevant part being the first two bits.) They use x differently in the specifcation. See tools.ietf.org/html/rfc3629#section-3
    – tay10r
    Nov 3, 2020 at 19:21
-5

“Continuation byte” isn’t a term but a normal English word and the term “byte.” If used as a pseudo-term, it may confuse the reader.

The Unicode Standard uses this expression in one place only, Ch. 5, clause 5.22: “For example, consider the first three bytes of a four-byte UTF-8 sequence, followed by a byte which cannot be a valid continuation byte: .” In this context, the meaning is clear: it’s just a byte that continues something, namely a sequence of bytes.

The Wikipedia page apparently uses “continuation byte” to mean any byte in the UTF-8 encoding except the first byte of the encoded form of a character.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.