Summary | IMP Memory usage |
Queue | IMP |
Queue Version | HEAD |
Type | Enhancement |
State | Resolved |
Priority | 2. Medium |
Owners | |
Requester | jigermano (at) uolsinectis (dot) com (dot) ar |
Created | 01/14/2005 (7477 days ago) |
Due | |
Updated | 04/18/2006 (7018 days ago) |
Assigned | 01/14/2005 (7477 days ago) |
Resolved | 02/07/2005 (7453 days ago) |
Milestone | |
Patch | No |
feedback. It's likely that the improvements are not in Horde 3.1, as
the MIME code in HEAD is a bit diverged at this point.
I'm still having this problem with horde 3.1.1 and imp 4.1.1 - we
need to send at least 10mb attachments. Even with php's memory_limit
at 75mb I can't upload a 9.5mb file. If I go upto 100mb I can and it
works fine. I get the same errors as mentioned in this ticket.
Is there nothing more than can be done about this - doesn't quite site
right allowing all those system resources just for a 10mb attachment.
State ⇒ Resolved
counted); some I've applied (avoiding changing strings when possible -
that's what defeats reference counting). See what current CVS looks
like for you.
you might want to add some info to http://horde.org/survey/
is passed as argument to a method which defines that argument must be
passed by reference. I did see one of the issues was corrected,
though. this return line:
return ''.$large_string;
is gone now.
The fatal error you're mentioning can be easily fixed like this:
$i->setContents("string constant');
should be changed to something like:
$i->setContents($dummy = 'string constant');
Moving on.
After reinstaling the framework with the patch I had posted applied
I've got the error about memory again. I will try to look into it.
Anyway: the patch I posted is not correct from a formal point of view:
::setContent() is a setter, so the argument shoud not be passed by
reference. But we know we are calling this with a 10Mb string. The
solution may be not be my patch (I don't like either, I was making
changes trying to find problems like the one corrected, end up finding
part of the problem comes from CORRECT programing). :(
Don't know how should it be fixed. But I think it is a problem that
should be addressed. Until it is fixed somehow I will continue to
patch the sources (altough I will try to find what I've lost with my
overwriting of the changes when reinstaling the framework, I will post
once I have).
I have 300k clients with a mean of 3 email accounts each. Dimensioning
for that kind of numbers makes this kind of problems important :)
Thanks for the time you took checking this out, sorry for not posting
a complete patch.
Juan
Before that 32MB PHP memory limit wasn't enough to download a 6,5 MB
mail. 64 MB worked. With those changes I got at least an error message
at 32 MB. I didn't do any further memory testing.
But I do have one problem. Mail sending doesn't work anymore:
Fatal error: Cannot pass parameter 1 by reference in
/home/webmail/html/horde/lib/Horde/MIME/Part.php on line 324
State ⇒ Assigned
Priority ⇒ 2. Medium
Type ⇒ Enhancement
Summary ⇒ IMP Memory usageok,
Queue ⇒ IMP
State ⇒ New
I have been doing some simple tests with IMP4. It's a great piece of
software and excellent functionality.
I'm setting it up to be used by several thousands of clients and although
it'll run in pretty big machines (a couple of Dual Xeon 2.8Ghz, 2Gb Ram,
SCSI RAID 6) I am evaluating some more of the cpu/memory intensive
caracteristics (compression, mime decode, text processing -Filter class-).
We allow our clients mails up to 13Mb each. Now, I tested IMP with a 8Mb
attachment and got:
(I started using a 20Mb limit)
PHP Fatal error: Allowed memory size of 20971520 bytes exhausted (tried
to allocate 10646241 bytes) in /usr/local/lib/php/Horde/MIME/Part.php on
line 306
and after moving to 50Mb limit:
PHP Fatal error: Allowed memory size of 50331648 bytes exhausted (tried
to allocate 10646241 bytes) in /usr/local/lib/php/Horde/MIME/Part.php on
line 1209
I think 10646241 is the sizes of the base 64 encoded attachment.
After a short following of the code, I found lot of memory copies. Don't
know if they´re unnecesary, but I took a chance and made some changes:
After this:
Index: lib/MIME/Contents.php
===================================================================
RCS file: /repository/imp/lib/MIME/Contents.php,v
retrieving revision 1.155
diff -u -b -r1.155 Contents.php
--- lib/MIME/Contents.php 7 Jan 2005 19:09:07 -0000 1.155
+++ lib/MIME/Contents.php 14 Jan 2005 15:28:03 -0000
@@ -155,15 +155,14 @@
*
* @return string The text of the part.
*/
- function getBodyPart($id)
+ function &getBodyPart($id)
{
- $contents = '';
if (!isset($this->_bodypart[$id])) {
$this->_bodypart[$id] =
@imap_fetchbody($GLOBALS['imp']['stream'], $this->_index, $id, FT_UID);
}
- return ($contents . $this->_bodypart[$id]);
+ return !empty($this->_bodypart[$id])?$this->_bodypart[$id]:'';
}
And
Index: Part.php
===================================================================
RCS file: /repository/framework/MIME/MIME/Part.php,v
retrieving revision 1.178
diff -u -b -r1.178 Part.php
--- Part.php 3 Jan 2005 13:09:09 -0000 1.178
+++ Part.php 14 Jan 2005 15:40:09 -0000
@@ -301,9 +301,9 @@
* @param string $contents The part body.
* @param optional string $encoding The current encoding of the
contents.
*/
- function setContents($contents, $encoding = null)
+ function setContents(&$contents, $encoding = null)
{
- $this->_contents = $contents;
+ $this->_contents = &$contents;
$this->_flags['contentsSet'] = true;
$this->_currentEncoding = (is_null($encoding)) ?
$this->getCurrentEncoding() : MIME::encoding($encoding, MIME_STRING);
}
@@ -382,7 +382,7 @@
*/
function transferDecodeContents()
{
- $contents = $this->transferDecode();
+ $contents = &$this->transferDecode();
$this->_currentEncoding = $this->_flags['lastTransferDecode'];
$this->setTransferEncoding($this->_currentEncoding);
@@ -1190,7 +1190,8 @@
*/
function transferDecode()
{
- $contents = $this->getContents();
+
+ $contents = &$this->getContents();
$encoding = $this->getCurrentEncoding();
/* If the contents are empty, return now. */
I was able to download the attachment with the same php.ini settings. Can
anyone see a problem with this?
Juan