The changes were made to Horde 3.2/IMP 4.2.

I would recommend that you reopen this bug, we experience problems

downloading an 36 mb attachment with a 128 mb memory limit.

With a 256 mb memory limit it is ok.



Running latest stable horde / imp and php.



It seems that allot of the memory is consumed by the str_replace calls

in lib/MIME/Contents.php -- One could wonder if a unix/windows line 
shift conversion is really nessecery in a binary attachment, eg. zip 
file. - One might also think it would be smarter streaming the 
attachment directly to the client at download and not drag it through 
php memory?



I haven't looked much into this as other things take priority at the 
moment, does anyone experience similar problems?



Best regards, Martin

Seems to me it's safe to resolve this now.

Moving to HEAD and downgrading a bit since changes have been made, and 
we're waiting to see what happens.

For ease of reference, changes made here:

http://lists.horde.org/archives/cvs/Week-of-Mon-20061204/063600.html

Quite honestly, the best way to tackle this may just be to make this 
change in CVS and play around with it and see what it breaks (if 
anything).  Therefore, all tracking of eol related bugs should occur 
in this ticket.  Also, will get rid of some foreach() calls in favor 
of array_shift() to also help with memory usage.

We were running into pretty consistent runaway httpd children when 
users would download large (> 30 MB) attachments.  Memory usage would 
balloon to at least 4 times the size of the attachment and the process 
would monopolize the CPU for literally several minutes.  (This was 
after removing or sufficiently increasing various Horde, PHP, and 
system-level limits as the httpd childs would segfault previously.)   
We also found that even moderately sized attachments (>14MB) would 
monopolize the CPU for several minutes and if more than one medium to 
large sized attachments were downloaded at the same time from the same 
server the browser would time out before the download to the client 
even began.



After tracing through way too much PHP, two lines looked suspicious:





--- horde/lib/Horde/MIME/Part.php.bak Wed Jul  5 10:30:47 2006

+++ horde/lib/Horde/MIME/Part.php     Tue Nov 21 03:26:12 2006

@@ -1152 +1152 @@

-            $message = preg_replace("/=\r?\n/", '', $this->_contents);

+//            $message = preg_replace("/=\r?\n/", '', $this->_contents);



--- horde/imp/lib/MIME/Contents.php.bak       Thu May  4 21:30:21 2006

+++ horde/imp/lib/MIME/Contents.php   Tue Nov 21 03:26:26 2006

@@ -173 +173 @@

-            $this->_bodypart[$id] = str_replace("\r\n", "\n", 
$this->_bodypart[$id]);

+//            $this->_bodypart[$id] = str_replace("\r\n", "\n", 
$this->_bodypart[$id]);







These calls were being made for any and all attachments regardless of 
encoding or MIME type.  Completely commenting out both lines 
immediately solved the runaway processes.  A 100 MB JPEG attachment 
took under 10 seconds of processing before download began compared to 
over 10 minutes before (but it still used just over 400 MB of memory).



Where N is the size of the attachment, I'm gonna take a stab in the 
dark and assume the 4N memory requirement is most likely attributed to 
each byte in the attachment being represented as a 32-bit value.  As 
for the CR+LF regexps, I admit I haven't gone through the RFC, but I'm 
guessing this isn't explicitly necessary for the majority of non-text 
attachments especially when they're a base64 blob.  So far, plain 
text, JPEG, TIFF, and random binary blobs have downloaded without 
incident.