Tickets :: Comment on [#4753] Attachment rewrites causing HUGE memory bloat

* Your Email Address
* Spam protection	Enter the letters below: .__ . . __ . .._. [__)\|__\|/ `\| \| \| \| \| \|\__.\|__\|_\|_
Comment	> > > We were running into pretty consistent runaway httpd children when > users would download large (> 30 MB) attachments. Memory usage would > balloon to at least 4 times the size of the attachment and the > process would monopolize the CPU for literally several minutes. > (This was after removing or sufficiently increasing various Horde, > PHP, and system-level limits as the httpd childs would segfault > previously.) We also found that even moderately sized attachments > (>14MB) would monopolize the CPU for several minutes and if more than > one medium to large sized attachments were downloaded at the same > time from the same server the browser would time out before the > download to the client even began. > > > > After tracing through way too much PHP, two lines looked suspicious: > > > > > > --- horde/lib/Horde/MIME/Part.php.bak Wed Jul 5 10:30:47 2006 > > +++ horde/lib/Horde/MIME/Part.php Tue Nov 21 03:26:12 2006 > > @@ -1152 +1152 @@ > > - $message = preg_replace("/=\r?\n/", '', $this->_contents); > > +// $message = preg_replace("/=\r?\n/", '', $this->_contents); > > > > --- horde/imp/lib/MIME/Contents.php.bak Thu May 4 21:30:21 2006 > > +++ horde/imp/lib/MIME/Contents.php Tue Nov 21 03:26:26 2006 > > @@ -173 +173 @@ > > - $this->_bodypart[$id] = str_replace("\r\n", "\n", > $this->_bodypart[$id]); > > +// $this->_bodypart[$id] = str_replace("\r\n", "\n", > $this->_bodypart[$id]); > > > > > > > > These calls were being made for any and all attachments regardless of > encoding or MIME type. Completely commenting out both lines > immediately solved the runaway processes. A 100 MB JPEG attachment > took under 10 seconds of processing before download began compared to > over 10 minutes before (but it still used just over 400 MB of memory). > > > > Where N is the size of the attachment, I'm gonna take a stab in the > dark and assume the 4N memory requirement is most likely attributed > to each byte in the attachment being represented as a 32-bit value. > As for the CR+LF regexps, I admit I haven't gone through the RFC, but > I'm guessing this isn't explicitly necessary for the majority of > non-text attachments especially when they're a base64 blob. So far, > plain text, JPEG, TIFF, and random binary blobs have downloaded > without incident. > >
Attachment
Watch this ticket

Saved Queries