I'm writing a simple website parser on PHP 5.2.10.
When using default internal encoding (which is ISO-8859-1), I get an error always at the same function call:
$start = mb_strpos($index, '<a name=gr1>');
Fatal error: Allowed memory size of 50331648 bytes exhausted (tried to allocate 11924760 bytes)
The length of the string $index in this case was 2981190 bytes - exactly 4 times less than PHP tried to allocate.
Now, if I use
mb_internal_encoding('UTF-8')
the error disappears. Does that mean that PHP uses more memory for single-byte strings that for multibyte ones? How's that possible? Any ideas?
UPD: Memory usage doesn't seem to depend on encoding: average memory_get_usage() is almost the same using UTF-8 and ISO-8859-1. I think that the problem might be in mb_strpos. In fact, the string $index has Windows-1251 encoding (cyrillic), so it contains symbols that are not valid for UTF-8. This may cause mb_strpos to somehow try to convert or just use the additional memory for some needs. Will try to find the answer in the sources of mb_strpos.
mb_strpos()
, but it's worth the upgrade in any case.iconv
can be a good way to make sure your strings are "sane" and match detected/set encoding. Would like to profile and see what it's doing with those 1252 control codes. Oh evil m-dash.