Every message displays as empty

In IMP 5 alpha 1 I get a perfect mailbox view, but when I cl

Wed, 09 Mar 2011 16:59:32 +0000

In IMP 5 alpha 1 I get a perfect mailbox view, but when I click any message it opens with [No Subject] and in the body there is only an "Unnamed message part", and then "This part is empty". Clicking the Unnamed part gives me an empty window. Enabling debug and debug_raw in backends.php while trying to see a a very simple test message (just one text part) yelds the output that I include at the end of this ticket (it is long and I XXXed out my email address). If you want me to attach logs, packet captures (I am using SSL on the web side but not on the IMAP side), screenshot, configs, etc. just ask. ------------------------------ (1299688102.5800) S: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE STARTTLS AUTH=PLAIN] Dovecot ready. (1299688102.6300) C: [SASL-IR AUTHENTICATE Command - username: bergonz] (1299688102.6400) S: 1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UN SELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in (1299688102.6400) C: 2 SELECT INBOX (1299688102.6800) S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft unknown-2 unknown-3 $MDNSent $Forwarded Junk NonJunk $label1 $has_cal) (1299688102.6800) S: * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft unknown-2 unknown-3 $MDNSent $Forwarded Junk NonJunk $label1 $has_cal \*)] Flags permitted. (1299688102.6800) S: * 702 EXISTS (1299688102.6800) S: * 0 RECENT (1299688102.6800) S: * OK [UNSEEN 2] First unseen. (1299688102.6800) S: * OK [UIDVALIDITY 1221554981] UIDs valid (1299688102.6800) S: * OK [UIDNEXT 368331] Predicted next UID (1299688102.6800) S: * OK [HIGHESTMODSEQ 49797] Highest (1299688102.6800) S: 2 OK [READ-WRITE] Select completed. (1299688102.6900) C: 3 UID SORT RETURN (ALL COUNT) (ARRIVAL) US-ASCII ALL (1299688102.7400) S: * ESEARCH (TAG "3") UID ALL 367272,367274,367294,367301,367307:367313,367315:367317,367319:367321,367323,367325:367330,367332:367335,3673 37:367338,367340:367346,367348:367353,367355:367356,367358:367362,367365:367370,367372,367374:367384,367386:367388,367390:367392,367394,367396:367399,367402,3 67423:367424,367426:367428,367430:367433,367436:367454,367457:367459,367461:367466,367468:367472,367474:367483,367485:367494,367496,367498:367509,367511,36751 3,367515:367519,367522:367526,367529,367531,367533:367534,367536:367537,367539:367544,367547:367548,367550,367572:367573,367575:367582,367586:367612,367614:36 7617,367626:367627,367629:367635,367637:367639,367641:367649,367654:367657,367659,367661:367663,367665,367667:367681,367685:367688,367690:367692,367700,367708 ,367710:367716,367718:367738,367740:367741,367743:367744,367746,367748,367750:367755,367757:367772,367775:367777,367779,367781:367787,367790:367793,367796:367 804,367806:367809,367811,367814:367821,367823:367826,367829:367835,367857:367859,367865,367868:367869,367918,367920:367925,367928,367930,367933:367934,367952: 367954,367956:367958,367960:367968,367970:367976,367979,367982:367987,367991,367993:368003,368005:368008,368010:368016,368018:368032,368034,368037:368045,3680 47,368049,368051:368056,368058:368065,368085:368089,368091:368094,368096:368103,368105,368107:368110,368113,368115:368118,368120,368122:368123,368126,368129:3 68130,368133:368138,368140:368143,368145:368152,368154:368156,368158,368160,368162:368165,368167,368169,368172:368179,368181:368183,368185:368186,368188,36819 2,368194:368196,368200:368330 COUNT 702 (1299688102.7500) S: 3 OK Sort completed (0.007 secs). (1299688102.8300) C: 4 UID FETCH 368326 (BODYSTRUCTURE) (1299688102.8600) S: * 698 FETCH (UID 368326 BODYSTRUCTURE ("text" "plain" ("charset" "ISO-8859-1" "format" "flowed") NIL NIL "7bit" 200 5 NIL NIL NIL NIL)) (1299688102.8700) S: 4 OK Fetch completed. (1299688102.8700) C: 5 UID FETCH 368326 (FLAGS) (1299688102.9000) S: * 698 FETCH (UID 368326 FLAGS (\Seen)) (1299688102.9000) S: 5 OK Fetch completed. (1299688102.9100) C: 6 UID FETCH 368326 (ENVELOPE BODY[HEADER]) (1299688102.9500) S: * 698 FETCH (UID 368326 ENVELOPE ("Wed, 09 Mar 2011 17:15:33 +0100" "Test message" (("Michele Bergonzoni" NIL "bergonz" "labs.it")) (("Michele Bergonzoni" NIL "bergonz" "labs.it")) (("Michele Bergonzoni" NIL "bergonz" "labs.it")) ((NIL NIL "bergonz" "labs.it")) NIL NIL NIL "<4D77A7A5.5080701@labs.it>") BODY[HEADER] {951} (1299688102.9500) S: Received: (qmail 24961 invoked by alias); 9 Mar 2011 16:15:34 -0000 Delivered-To: labs-XXX Received: (qmail 24957 invoked by uid 1058); 9 Mar 2011 16:15:34 -0000 Received: from 1.0.14.107 by mason (envelope-from , uid 1050) with qmail-scanner-2.01-rej (clamdscan: 0.91.2/4456. uvscan: v5.1.00/v5132. spamassassin: 3.1.8. Clear:RC:1(1.0.14.107):. Processed in 0.019371 secs); 09 Mar 2011 16:15:34 -0000 Received: from unknown (HELO ?10.0.14.107?) (10.0.14.107) by mason.internal.labs.it with SMTP; 9 Mar 2011 16:15:34 -0000 Message-ID: <4D77A7A5.5080701@labs.it> Date: Wed, 09 Mar 2011 17:15:33 +0100 From: Michele Bergonzoni User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.14) Gecko/20110223 Thunderbird/3.1.8 MIME-Version: 1.0 To: XXX Subject: Test message Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit (1299688102.9500) S: ) (1299688102.9500) S: 6 OK Fetch completed. (1299688103.3000) C: 7 UID FETCH 368326 (BODY.PEEK[1]) (1299688103.3300) S: * 698 FETCH (UID 368326 BODY[1] {200} (1299688103.3300) S: This is a test. -- Ing. Michele Bergonzoni - Laboratori Guglielmo Marconi S.p.a. Phone:+39-051-4392826 Fax:+39-051-6153683 e-mail: XXX alt.advanced.networks.design.configure.operate (1299688103.3400) S: ) (1299688103.3400) S: 7 OK Fetch completed. (1299688103.6500) C: 8 SEARCH RETURN (COUNT) UNSEEN (1299688103.6800) S: * ESEARCH (TAG "8") COUNT 335 (1299688103.6900) S: 8 OK Search completed (0.000 secs). (1299688104.0900) C: 9 LOGOUT (1299688104.1200) S: * BYE Logging out (1299688104.1200) S: 9 OK Logout completed. ------------------------------

More details: if I click "Message Source", the source is OK

Wed, 09 Mar 2011 17:14:45 +0000

More details: if I click "Message Source", the source is OK (has all the body, parts, base64 if there are attachments), so it doesn't seem to be an imap issue. If I open a message with attachments, it shows me an empty message with [No Subject] and the attachments. If I open an attachment I get an empty file with the correct name and of the correct mime type.

> More details: if I click "Message Source", the source is O

Thu, 10 Mar 2011 09:22:54 +0000

> More details: if I click "Message Source", the source is OK (has all > the body, parts, base64 if there are attachments), so it doesn't seem > to be an imap issue. If I open a message with attachments, it shows > me an empty message with [No Subject] and the attachments. If I open > an attachment I get an empty file with the correct name and of the > correct mime type. > I had the same issue. It was related to the PHP version. After upgrade everything worked fine. Which version are you using?

I have: PHP 5.2.6-1+lenny9 with Suhosin-Patch 0.9.6.2 (cl

Thu, 10 Mar 2011 10:30:53 +0000

I have: PHP 5.2.6-1+lenny9 with Suhosin-Patch 0.9.6.2 (cli) (built: Aug 4 2010 03:25:57) Copyright (c) 1997-2008 The PHP Group Zend Engine v2.2.0, Copyright (c) 1998-2008 Zend Technologies I will try to change machine (or dist-upgrade) if I can. The requirement in the relnotes is PHP 5.2.0 or above, it may be easier to change the relnotes...

> PHP 5.2.6-1+lenny9 with Suhosin-Patch 0.9.6.2 (cli) (bui

Thu, 10 Mar 2011 19:11:20 +0000

> PHP 5.2.6-1+lenny9 with Suhosin-Patch 0.9.6.2 (cli) (built: Aug 4 > 2010 03:25:57) > I will try to change machine (or dist-upgrade) if I can. The > requirement in the relnotes is PHP 5.2.0 or above, it may be easier > to change the relnotes... IMP works fine with PHP 5.2. It may just be your particular installation of PHP that is broken.

This PHP installation is currently running horde 3.3.8 + imp

Fri, 11 Mar 2011 08:22:34 +0000

This PHP installation is currently running horde 3.3.8 + imp 4.3.7 + kronolith 2.3.4 with no issues. Of course it is possible that it is botched in some subtle way: in this case I would really like to understand where the problem is. I set the loglevel to debug and when I open a message (in DIMP, where the issue is the same) and I append the output below, but it seems pretty much normal noise to me. Maybe someone can suggest good spots where to add print/debug statements: following the flow in the software is not easy for me. Mar 11 09:06:33 cecere HORDE: HORDE Horde_Registry: retrieved app with cache ID horde_registry|app|1299667947|a3e9df9e0233273bfceda6ec6abc9689 [pid 633 on line 1611 of "/usr/share/php/Horde/Registry.php"] Mar 11 09:06:33 cecere HORDE: HORDE [imp] Load config file (conf.php; app: imp) [pid 633 on line 785 of "/usr/share/php/Horde.php"] Mar 11 09:06:33 cecere HORDE: HORDE [imp] SQL (0.0057s) #012#011SELECT * FROM horde_histories WHERE object_uid =#012#011 'imp.bergonz.<20110310191122.Horde.bBf6Rhha3zhNeSJa7ioDUVA@bugs.horde.#012#011 org>' [pid 633 on line 773 of "/usr/share/php/Horde/Db/Adapter/Base.php"] Mar 11 09:06:33 cecere HORDE: HORDE [imp] Load config file (mime_drivers.php; app: horde) [pid 633 on line 785 of "/usr/share/php/Horde.php"] Mar 11 09:06:33 cecere HORDE: HORDE [imp] Load config file (mime_drivers.php; app: imp) [pid 633 on line 785 of "/usr/share/php/Horde.php"] Mar 11 09:06:33 cecere HORDE: HORDE [imp] SQL (0.0055s) #012#011SELECT alarm_id, alarm_uid, alarm_start, alarm_end, alarm_methods,#012#011 alarm_params, alarm_title, alarm_text, alarm_snooze, alarm_internal#012#011 FROM horde_alarms WHERE alarm_dismissed = 0 AND ((alarm_snooze IS NULL#012#011 AND alarm_start <= '2011-03-11T08:06:33') OR alarm_snooze <=#012#011 '2011-03-11T08:06:33') AND (alarm_end IS NULL OR alarm_end >=#012#011 '2011-03-11T08:06:33') AND (alarm_uid IS NULL OR alarm_uid = '' OR#012#011 alarm_uid = 'bergonz') ORDER BY alarm_start, alarm_end [pid 633 on line 773 of "/usr/share/php/Horde/Db/Adapter/Base.php"] Mar 11 09:06:33 cecere HORDE: HORDE [imp] Max memory usage: 11010048 bytes [pid 633 on line 462 of "/usr/share/php/Horde/Registry.php"]

I modified /usr/share/php/Horde/Imap/Client/Base.php at line

Fri, 11 Mar 2011 10:01:35 +0000

> Array > ( > [368578] => Horde_Imap_Client_Data_Fetch

Fri, 11 Mar 2011 21:10:08 +0000
> Array > ( > [368578] => Horde_Imap_Client_Data_Fetch Object > ( > [_data:protected] => Array > ( > [14] => 866 > [13] => 368578 > [6] => Array > ( > [1.2] => Array > ( > [d] => > [t] => Resource id #157 > ) > > ) > > ) > > ) > > ) This looks entirely correct. A BODYPART fetch from UID 368578 is returning bodytext (Resource id #157). You can do further debugging in imp/lib/Contents.php around line 492 (this is where the "This part is empty" status message is being set). The data is written to the resource stream in Horde_Imap_Client_Socket line 3705. So you can also poke around there.

Looking back through this ticket, I see: PHP 5.2.6-1+lenn

Fri, 11 Mar 2011 21:14:58 +0000
Looking back through this ticket, I see: PHP 5.2.6-1+lenny9 with Suhosin-Patch 0.9.6.2 Maybe a suhosin issue? We are writing to a temporary stream, which uses the system temporary directory. Don't know if this is the kind of thing suhosin (incorrectly) is trying to block.

suhosin.simulation=On makes no difference, but I followed yo

Mon, 14 Mar 2011 10:08:37 +0000
suhosin.simulation=On makes no difference, but I followed your guidance and found this workaround. In line 3743 of /usr/share/php/Horde/Imap/Client/Socket.php (just before the return) I inserted: "rewind($data);". This resolves the issue, now I see messages, download attachments, etc. I also tried if it was a problem with the imap debug, but with: - debug disabled in backends.cfg - rewind omitted the issue is still there. What really makes a difference is the rewind. Hope this ticket and workaround helps you find something interesting. Thank you very much for your help.

Confirmed, problem and solution. This is on Ubuntu 8.04 LTS

Mon, 14 Mar 2011 14:31:31 +0000
Confirmed, problem and solution. This is on Ubuntu 8.04 LTS eg. PHP 5.2.4-2ubuntu5.14

> Confirmed, problem and solution. This is on Ubuntu 8.04 LT

Tue, 15 Mar 2011 02:50:14 +0000
> Confirmed, problem and solution. This is on Ubuntu 8.04 LTS eg. PHP > 5.2.4-2ubuntu5.14 Now we're getting somewhere... namely that your PHP installations are broken. There is NO need to rewind the data stream at the code location mentioned. I have verified that the stream is not rewound on my installation, and I see everything just fine. The stream is only rewound when needed in Horde_Imap_Client_Data_Fetch. Can you try running this test script on your machine?

And here you go: http://bugs.php.net/bug.php?id=46426

Tue, 15 Mar 2011 02:52:56 +0000
And here you go: http://bugs.php.net/bug.php?id=46426

Changes have been made in Git for this ticket: Bug #9644: W

Tue, 15 Mar 2011 03:00:45 +0000
Changes have been made in Git for this ticket: Bug #9644: Workaround PHP < 5.2.6 (broken stream handling) 5 files changed, 16 insertions(+), 8 deletions(-) http://git.horde.org/horde-git/-/commit/1c3da149c4d76a2c5984f15e6c9e235c8fd9c40a

Yes, both your script and the one found in the PHP bug repor

Tue, 15 Mar 2011 09:10:36 +0000
Yes, both your script and the one found in the PHP bug report show that my PHP is broken as you described. I removed my workaround and applied your patch and everything is OK. Thanx!

Can someone explain the last comment at http://bugs.php.net/

Tue, 15 Mar 2011 11:02:12 +0000
Can someone explain the last comment at http://bugs.php.net/bug.php?id=46426? Is this Bug fixed or flagged as Bogus for 5.2.x? According to the first comment it is fixed since 30.10.2008 according to the last comment it is no PHP Bug?

> Is this Bug fixed or flagged as Bogus for 5.2.x? AFAIU

Tue, 15 Mar 2011 11:28:44 +0000
> Is this Bug fixed or flagged as Bogus for 5.2.x? AFAIU the bogus state refers to the (not) required documentation.