[#6660] Fatal error with very big strings conversion
Summary Fatal error with very big strings conversion
Queue Horde Framework Packages
Queue Version FRAMEWORK_3
Type Bug
State Resolved
Priority 2. Medium
Owners Chuck Hagenbuch <chuck (at) horde (dot) org>
Requester admin (at) elektordi (dot) net
Created 04/28/2008 (71 days ago)
Due
Updated 05/12/2008 (57 days ago)
Assigned 04/29/2008 (70 days ago)
Resolved 05/12/2008 (57 days ago)
Attachments
Milestone
Patch No

History
05/12/2008 Chuck Hagenbuch Comment #10
State ⇒ Resolved
Assigned to Chuck Hagenbuch
Reply to this comment
Done.
05/11/2008 Matt Selsky Comment #9 Reply to this comment
Raising the limit to 16MB seems about 50% faster.
05/11/2008 Matt Selsky Comment #8 Reply to this comment
Since the recommended memory limit is 64MB should we use utf8_encode 
for <= 16MB instead of just 1MB?
05/11/2008 Jan Schneider Comment #7 Reply to this comment
Can you both test this patch, please?
Ping?
05/01/2008 Chuck Hagenbuch Comment #6 Reply to this comment
No, the logic is: if the string is small, OR neither mbstring or iconv 
are available, then try utf8-en/de-code
05/01/2008 Jan Schneider Comment #5 Reply to this comment
Shouldn't this be an && operator?
05/01/2008 Chuck Hagenbuch Comment #4
Priority ⇒ 2. Medium
Reply to this comment
04/29/2008 Matt Selsky Comment #3 Reply to this comment
mb_convert_encoding is way slower than utf8_encode on my system.  To 
convert a 117 bytes (with 1,000,000 iterations) from ASCII to UTF-8 
takes:

utf8_encode time: 15 seconds
mb_convert_encoding time: 46 seconds

Howerver, mb_convert_encoding() is able to deal with larger strings.   
Eg: 50% of max memory instead of %25 max memory.
04/29/2008 Chuck Hagenbuch Patch ⇒
 
04/29/2008 Chuck Hagenbuch Comment #2
State ⇒ Feedback
Reply to this comment
Do you have any speed numbers for mbstring vs. utf8-en/decode? If it's 
more efficient with memory and the same or faster for speed, we should 
just use it first, period.
04/28/2008 admin (at) elektordi (dot) net Comment #1
Patch ⇒ [See the hack in the description.]
Milestone ⇒
Queue ⇒ Horde Framework Packages
Summary ⇒ Fatal error with very big strings conversion
Type ⇒ Bug
Priority ⇒ 3. High
State ⇒ Unconfirmed
Reply to this comment
Hello,

I got a problem today with a very big text mail (~4Mo) due to an 
infinite loop in a log deamon. This causes my whole mailbox page to 
crash with an PHP Fatal error:

PHP Fatal error:  Allowed memory size of 33554432 bytes exhausted 
(tried to allocate 16187725 bytes) in 
/usr/share/horde3/lib/Horde/String.php on line 128

I managed to prevent the error with the following hack:

         /* HACK by Elektordi 2008-04-28: Prevent crashs caused by 
very big text mails */
         if(strlen($input)>1048576 && String::extensionExists('mbstring')) {
                 return @mb_convert_encoding($input, $to, $from);
         }

in file String.php and function _convertCharset, just before the line 
/* Use utf8_[en|de]code() if possible. */

I did this because Mbstring seems to allocate much less memory than 
utf8_xxcode functions. But a better test in the "if" is to check the 
free memory instead of using the constant 1048576 (=1024*1024)

Hope this will help you!

Regards,

Elektordi