6.0.0-beta1
7/5/25

[#1163] IMP Memory usage
Summary IMP Memory usage
Queue IMP
Queue Version HEAD
Type Enhancement
State Resolved
Priority 2. Medium
Owners
Requester jigermano (at) uolsinectis (dot) com (dot) ar
Created 01/14/2005 (7477 days ago)
Due
Updated 04/18/2006 (7018 days ago)
Assigned 01/14/2005 (7477 days ago)
Resolved 02/07/2005 (7453 days ago)
Milestone
Patch No

History
04/18/2006 05:06:58 PM Chuck Hagenbuch Comment #7 Reply to this comment
I asked the original reporter to look at HEAD, and didn't get further 
feedback. It's likely that the improvements are not in Horde 3.1, as 
the MIME code in HEAD is a bit diverged at this point.
04/18/2006 04:35:42 PM nick_horde (at) navaho (dot) co (dot) uk Comment #6 Reply to this comment
Sorry if I've missed why this ticket is closed...

I'm still having this problem with  horde 3.1.1 and imp 4.1.1   - we 
need to send at least 10mb attachments.  Even with php's memory_limit 
at 75mb I can't upload a 9.5mb file.  If I go upto 100mb I can and it 
works fine.  I get the same errors as mentioned in this ticket.

Is there nothing more than can be done about this - doesn't quite site 
right allowing all those system resources just for a 10mb attachment.


02/07/2005 12:42:47 AM Chuck Hagenbuch Comment #5
State ⇒ Resolved
Reply to this comment
Some of these changes don't make sense (references - PHP is reference 
counted); some I've applied (avoiding changing strings when possible - 
that's what defeats reference counting). See what current CVS looks 
like for you.
01/28/2005 10:32:05 AM Jan Schneider Comment #4 Reply to this comment
As a completely unrelated side note: if you have such a big user base, 
you might want to add some info to http://horde.org/survey/
01/28/2005 03:36:53 AM jigermano (at) uolsinectis (dot) com (dot) ar Comment #3 Reply to this comment
Yes. My patch was incomplete. The error message is because a constant 
is passed as argument to a method which defines that argument must be 
passed by reference. I did see one of the issues was corrected, 
though. this return line:



return ''.$large_string;



is gone now.

The fatal error you're mentioning can be easily fixed like this:

$i->setContents("string constant');

should be changed to something like:

$i->setContents($dummy = 'string constant');



Moving on.

After reinstaling the framework with the patch I had posted applied 
I've got the error about memory again. I will try to look into it.

Anyway: the patch I posted is not correct from a formal point of view: 
::setContent() is a setter, so the argument shoud not be passed by 
reference. But we know we are calling this with a 10Mb string. The 
solution may be not be my patch (I don't like either, I was making 
changes trying to find problems like the one corrected, end up finding 
part of the problem comes from CORRECT programing). :(

Don't know how should it be fixed. But I think it is a problem that 
should be addressed. Until it is fixed somehow I will continue to 
patch the sources (altough I will try to find what I've lost with my 
overwriting of the changes when reinstaling the framework, I will post 
once I have).

I have 300k clients with a mean of 3 email accounts each. Dimensioning 
for that kind of numbers makes this kind of problems important :)



Thanks for the time you took checking this out, sorry for not posting 
a complete patch.



Juan
01/28/2005 03:03:13 AM mail (at) kai-becker (dot) com Comment #2 Reply to this comment
I did those changes on my  system.

Before that 32MB PHP memory limit wasn't enough to download a 6,5 MB 
mail. 64 MB worked. With those changes I got at least an error message 
at 32 MB. I didn't do any further memory testing.

But I do have one  problem. Mail sending doesn't work anymore:

Fatal error: Cannot pass parameter 1 by reference in 
/home/webmail/html/horde/lib/Horde/MIME/Part.php on line 324
01/14/2005 10:02:50 PM Chuck Hagenbuch Summary ⇒ IMP Memory usage
State ⇒ Assigned
 
01/14/2005 06:46:28 PM jigermano (at) uolsinectis (dot) com (dot) ar Comment #1
Priority ⇒ 2. Medium
Type ⇒ Enhancement
Summary ⇒ IMP Memory usageok,
Queue ⇒ IMP
State ⇒ New
Reply to this comment
Hi.

I have been doing some simple tests with IMP4. It's a great piece of

  software and excellent functionality.

  I'm setting it up to be used by several thousands of clients and although

  it'll run in pretty big machines (a couple of Dual Xeon 2.8Ghz, 2Gb Ram,

  SCSI RAID 6) I am evaluating some more of the cpu/memory intensive

  caracteristics (compression, mime decode, text processing -Filter class-).

  We allow our clients mails up to 13Mb each. Now, I tested IMP with a 8Mb

  attachment and got:



  (I started using a 20Mb limit)



    PHP Fatal error:  Allowed memory size of 20971520 bytes exhausted (tried

  to allocate 10646241 bytes) in /usr/local/lib/php/Horde/MIME/Part.php on

  line 306



  and after moving to 50Mb limit:



    PHP Fatal error:  Allowed memory size of 50331648 bytes exhausted (tried

  to allocate 10646241 bytes) in /usr/local/lib/php/Horde/MIME/Part.php on

  line 1209



  I think 10646241 is the sizes of the base 64 encoded attachment.

  After a short following of the code, I found lot of memory copies. Don't

  know if they´re unnecesary, but I took a chance and made some changes:



  After this:

  Index: lib/MIME/Contents.php

  ===================================================================

  RCS file: /repository/imp/lib/MIME/Contents.php,v

  retrieving revision 1.155

  diff -u -b -r1.155 Contents.php

  --- lib/MIME/Contents.php       7 Jan 2005 19:09:07 -0000       1.155

  +++ lib/MIME/Contents.php       14 Jan 2005 15:28:03 -0000

  @@ -155,15 +155,14 @@

         *

         * @return string  The text of the part.

         */

  -    function getBodyPart($id)

  +    function &getBodyPart($id)

        {

  -        $contents = '';



            if (!isset($this->_bodypart[$id])) {

                $this->_bodypart[$id] =

  @imap_fetchbody($GLOBALS['imp']['stream'], $this->_index, $id, FT_UID);

            }



  -        return ($contents . $this->_bodypart[$id]);

  +        return !empty($this->_bodypart[$id])?$this->_bodypart[$id]:'';

        }



  And



  Index: Part.php

  ===================================================================

  RCS file: /repository/framework/MIME/MIME/Part.php,v

  retrieving revision 1.178

  diff -u -b -r1.178 Part.php

  --- Part.php    3 Jan 2005 13:09:09 -0000       1.178

  +++ Part.php    14 Jan 2005 15:40:09 -0000

  @@ -301,9 +301,9 @@

         * @param string $contents           The part body.

         * @param optional string $encoding  The current encoding of the

  contents.

         */

  -    function setContents($contents, $encoding = null)

  +    function setContents(&$contents, $encoding = null)

        {

  -        $this->_contents = $contents;

  +        $this->_contents = &$contents;

            $this->_flags['contentsSet'] = true;

            $this->_currentEncoding = (is_null($encoding)) ?

  $this->getCurrentEncoding() : MIME::encoding($encoding, MIME_STRING);

        }

  @@ -382,7 +382,7 @@

         */

        function transferDecodeContents()

        {

  -        $contents = $this->transferDecode();

  +        $contents = &$this->transferDecode();

            $this->_currentEncoding = $this->_flags['lastTransferDecode'];

            $this->setTransferEncoding($this->_currentEncoding);



  @@ -1190,7 +1190,8 @@

         */

        function transferDecode()

        {

  -        $contents = $this->getContents();

  +

  +        $contents = &$this->getContents();

            $encoding = $this->getCurrentEncoding();



            /* If the contents are empty, return now. */



  I was able to download the attachment with the same php.ini settings. Can

  anyone see a problem with this?



Juan

Saved Queries