Fatal error with very big strings conversion

Hello, I got a problem today with a very big text mail (~

Mon, 28 Apr 2008 14:07:58 -0400

Hello, I got a problem today with a very big text mail (~4Mo) due to an infinite loop in a log deamon. This causes my whole mailbox page to crash with an PHP Fatal error: PHP Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 16187725 bytes) in /usr/share/horde3/lib/Horde/String.php on line 128 I managed to prevent the error with the following hack: /* HACK by Elektordi 2008-04-28: Prevent crashs caused by very big text mails */ if(strlen($input)>1048576 && String::extensionExists('mbstring')) { return @mb_convert_encoding($input, $to, $from); } in file String.php and function _convertCharset, just before the line /* Use utf8_[en|de]code() if possible. */ I did this because Mbstring seems to allocate much less memory than utf8_xxcode functions. But a better test in the "if" is to check the free memory instead of using the constant 1048576 (=1024*1024) Hope this will help you! Regards, Elektordi

Do you have any speed numbers for mbstring vs. utf8-en/decod

Tue, 29 Apr 2008 00:51:03 -0400

Do you have any speed numbers for mbstring vs. utf8-en/decode? If it's more efficient with memory and the same or faster for speed, we should just use it first, period.

mb_convert_encoding is way slower than utf8_encode on my sys

Tue, 29 Apr 2008 02:14:54 -0400

mb_convert_encoding is way slower than utf8_encode on my system. To convert a 117 bytes (with 1,000,000 iterations) from ASCII to UTF-8 takes: utf8_encode time: 15 seconds mb_convert_encoding time: 46 seconds Howerver, mb_convert_encoding() is able to deal with larger strings. Eg: 50% of max memory instead of %25 max memory.

Can you both test this patch, please? http://lists.horde.or

Thu, 01 May 2008 17:49:48 -0400

Can you both test this patch, please? http://lists.horde.org/archives/cvs/Week-of-Mon-20080428/077940.html

Shouldn't this be an && operator?

Thu, 01 May 2008 18:37:09 -0400

Shouldn't this be an && operator?

No, the logic is: if the string is small, OR neither mbstrin

Thu, 01 May 2008 18:38:44 -0400

No, the logic is: if the string is small, OR neither mbstring or iconv are available, then try utf8-en/de-code

> Can you both test this patch, please? Ping?

Sun, 11 May 2008 17:56:37 -0400

> Can you both test this patch, please? Ping?

Since the recommended memory limit is 64MB should we use utf

Sun, 11 May 2008 18:51:12 -0400

Since the recommended memory limit is 64MB should we use utf8_encode for <= 16MB instead of just 1MB?

Raising the limit to 16MB seems about 50% faster.

Sun, 11 May 2008 19:05:25 -0400

Raising the limit to 16MB seems about 50% faster.

Done.

Mon, 12 May 2008 16:40:41 -0400

Done.