6.0.0-git
2019-09-20

[#13711] ActiveSync via Outlook 2013: Empty Mails
Summary ActiveSync via Outlook 2013: Empty Mails
Queue Synchronization
Queue Version FRAMEWORK_5_2
Type Bug
State Resolved
Priority 2. Medium
Owners mrubinsk (at) horde (dot) org
Requester christian (at) hailer (dot) eu
Created 2014-11-22 (1763 days ago)
Due
Updated 2014-12-06 (1749 days ago)
Assigned 2014-12-04 (1751 days ago)
Resolved 2014-12-04 (1751 days ago)
Milestone
Patch No

History
2014-12-06 08:10:47 patrick (at) spamreducer (dot) eu Comment #40 Reply to this comment

[Show Quoted Text - 12 lines]
Good morning to all!
The problem seems to be solved, I have written a simple PHP script for 
all out there to "reset" the Outlook 2013 wrong MIME truncation 
settings.

Download it from here:
https://github.com/zorpat/Horde-Tools-Collection

It's very simple to use!
Usage: php horde_reset_wrong_mimetruncation.php mysql_host 
horde_DB_name horde_DB_username horde_DB_password activesync_device_id

Example: php horde_reset_wrong_mimetruncation.php localhost horde_db 
horde_user horde_password 3278A0C997A24217BB1A105CC9569D12
Feel free to use it!
2014-12-04 19:50:24 Michael Rubinsky Comment #39
State ⇒ Resolved
Reply to this comment
All new received messages in OL show some of the MIME headers in 
message body. Special chars like the german "ä" appear as ecoded 
strings.. Messages are very hard to read since a lot of escaping 
chars in body tend to be unreadable..
could this be an issue with this mime truncation problem?
Yes.

Closing the ticket since all outstanding issues are fixed on my end. 
Please use the mailing list for further support regarding resetting 
the account.
2014-12-04 19:43:00 patrick (at) spamreducer (dot) eu Comment #38 Reply to this comment

[Show Quoted Text - 15 lines]
Thank you very much! I will try to do that tomorrow.

Another thought:
All new received messages in OL show some of the MIME headers in 
message body. Special chars like the german "ä" appear as ecoded 
strings.. Messages are very hard to read since a lot of escaping chars 
in body tend to be unreadable..
could this be an issue with this mime truncation problem?
2014-12-04 19:37:27 Michael Rubinsky Comment #37 Reply to this comment
...and of course, you will want to do this while NO clients are able 
to connect or you run the risk of the client overwriting your changes.
2014-12-04 19:35:39 Michael Rubinsky Comment #36 Reply to this comment
You would need to find all the horde_activesync_cache table entries 
that correspond to your OL clients, then look at the "collections" 
array in the serialized data, and change the value of mimetruncation 
to false (which in serialized format would be b:0. Better yet, you 
should write a small script to read the cache data, unserialize it, 
change the mimetruncation value and rewrite it out to the database.

Additionally, there are a few different ways that a client can 
indicate it wants truncated data (which caused this problem in the 
first place). I'm going on the assumption that your OL is requesting 
this data in the same way as my test install.

Before doing anything this destructive though, I would recommend 
resetting at least one of your accounts to be SURE that this is the 
issue.
2014-12-04 19:22:41 patrick (at) spamreducer (dot) eu Comment #35 Reply to this comment
Unfortunately, if you are still seeing this issue, this is the 
likely reason. The incorrect truncation constant is probably cached 
in the state. Alternatively, you could *try* to manually change this 
value in the database for every cache/state entry. This would be 
error prone, time consuming and would not change any of the emails 
already on the device.
OL2013 is making me crazy..! On the Android clients there are no 
problems ever.

Ok, this workarround sounds much better..
Could you provide me with a short description on how exactly I could 
accomplish this?
2014-12-04 19:10:52 Michael Rubinsky Comment #34 Reply to this comment
Unfortunately, if you are still seeing this issue, this is the likely 
reason. The incorrect truncation constant is probably cached in the 
state. Alternatively, you could *try* to manually change this value in 
the database for every cache/state entry. This would be error prone, 
time consuming and would not change any of the emails already on the 
device.

2014-12-04 18:56:34 patrick (at) spamreducer (dot) eu Comment #33 Reply to this comment
Delete the account from OL and add it again. You can/should also 
remove the old account's data from Horde in the activesync admin 
page just to keep your database from becoming cluttered with unused 
account data.
Oh my god.
Is there any other possibility?
There are a lot of heavy accounts.. they would take days to resynch..
I have done this a lot of times before because other bugs..and it's 
just not possible for me..

2014-12-04 18:55:50 patrick (at) spamreducer (dot) eu Comment #32 Reply to this comment
Delete the account from OL and add it again. You can/should also 
remove the old account's data from Horde in the activesync admin 
page just to keep your database from becoming cluttered with unused 
account data.
Oh my god.
Is there any other possibility?
There are a lot of heavy accounts.. they would take days to resynch..
I have done this a lot of times before because other bugs..and it's 
just not possible for me..

2014-12-04 18:38:47 Michael Rubinsky Comment #31 Reply to this comment
Delete the account from OL and add it again. You can/should also 
remove the old account's data from Horde in the activesync admin page 
just to keep your database from becoming cluttered with unused account 
data.
2014-12-04 18:35:59 patrick (at) spamreducer (dot) eu Comment #30 Reply to this comment

[Show Quoted Text - 10 lines]
What do you exactly meen with recreating the account?


2014-12-04 18:35:29 Michael Rubinsky State ⇒ Feedback
 
2014-12-04 18:31:27 Michael Rubinsky Comment #29 Reply to this comment
Horde_ActiveSync 2.20.5
Just updated to GIT master (2194a19), same issue.
OL2013 is receiving empty messages , just "Return-P" is in there..
This sounds like the issue that the first commit on this ticket 
solved. Did you recreate the account in OL?

If not, I would need a complete synclog. I cannot reproduce this at all.
2014-12-04 15:46:53 patrick (at) spamreducer (dot) eu Comment #28 Reply to this comment
Horde_ActiveSync 2.20.5
Just updated to GIT master (2194a19), same issue.
OL2013 is receiving empty messages , just "Return-P" is in there..

2014-12-04 09:25:32 Michael Rubinsky Comment #27
State ⇒ Resolved
Reply to this comment
Horde_ActiveSync 2.20.5
2014-12-04 09:24:10 Git Commit Comment #26 Reply to this comment
Changes have been made in Git (master):

commit 2194a1964929e055eafed1d66d5a06bd5bdbb0fa
Author: Michael J Rubinsky <mrubinsk@horde.org>
Date:   Thu Dec 4 04:22:52 2014 -0500

     Add basic unit test for Bug: 13711

  framework/ActiveSync/package.xml                   |   10 ++-
  .../test/Horde/ActiveSync/ImapAdapterTest.php      |   74 +++++++++++++++
  .../test/Horde/ActiveSync/Stub/ImapFactory.php     |   95 
++++++++++++++++++++
  3 files changed, 177 insertions(+), 2 deletions(-)

http://github.com/horde/horde/commit/2194a1964929e055eafed1d66d5a06bd5bdbb0fa
2014-12-04 09:24:04 Git Commit Comment #25 Reply to this comment
Changes have been made in Git (master):

commit 16d46bc1f260ddb9a72d3faee1bb6141b05e0e8e
Author: Michael J Rubinsky <mrubinsk@horde.org>
Date:   Thu Dec 4 04:21:31 2014 -0500

     Bug: 13711 Don't assume we have a text/plain or text/html.

     It's possible the email may contain only "attachment" data.

  .../lib/Horde/ActiveSync/Imap/Adapter.php          |    6 +++---
  1 files changed, 3 insertions(+), 3 deletions(-)

http://github.com/horde/horde/commit/16d46bc1f260ddb9a72d3faee1bb6141b05e0e8e
2014-12-03 08:55:37 marth (at) tsvschlieben (dot) de Comment #24 Reply to this comment
Our dirty quickfix for this issue (no warranty for correctness):

Line  1278 in /usr/share/php/Horde/Mime/Part.php:

   $boundary = trim($this->getContentTypeParameter('boundary'), '"');
+$this->_parts = array_filter($this->_parts);
   reset($this->_parts);

This just removed the empty elements in the array that causes the 
error messages. Probably a real fix will need to make sure that the 
array contains correct values only...


2014-12-02 18:54:31 patrick (at) spamreducer (dot) eu Comment #23 Reply to this comment
And then you have no "emptyness" error anymore??
No empty messages anymore. The problem with empty messages occured 
also on iOS devices, but after the updates of more than a week ago, 
at least the iOS issues were gone.
oh god.. then hopefully this BUG is resolved soon and pushed to GIT..

It's not an option to keep this problem in code till it will appear on 
another message..
2014-12-02 18:40:58 horde40 (at) volkerthen (dot) com Comment #22 Reply to this comment
And then you have no "emptyness" error anymore??
No empty messages anymore. The problem with empty messages occured 
also on iOS devices, but after the updates of more than a week ago, at 
least the iOS issues were gone.
2014-12-02 18:24:06 patrick (at) spamreducer (dot) eu Comment #21 Reply to this comment

[Show Quoted Text - 31 lines]
And then you have no "emptyness" error anymore??
2014-12-02 18:15:38 horde40 (at) volkerthen (dot) com Comment #20 Reply to this comment
Michael, the developer in charge of this ticket, isn't available til 
the end of the week, as far as I know. In the meantime I did a bit of 
debugging myself. Please follow the procedure, as I did, and let's 
find out what kind of messages (or contact or calendar data) is 
breaking the sync process.

- Delete the account in your Outlook
- Delete the corresponding ActiveSync device in Horde
- Enable per device logging in Horde
- Restart Outlook (as I found out, it has to be restarted. If not, 
then it's not getting a new Sync ID from Horde)
- Set up a new ActiveSync account in Outlook
- Take a look at your email folders in Outlook. Some messages will 
most likely get synced, but at some point it stops. Check the content 
of the first message, that is not getting synced. You might see these 
messages in the log file. In my test case it was the sent folder that 
was responsible for at least this one specific problem on the client 
side. Outlook didn't finish the sync process, and started again and 
again syncing the sent folder to that specific point (in my case 
2014-11-07, 13:10), duplicating every message at every sync on the 
client side.

- The message that interfered with the sync was a vcalendar invitation 
of my client (date 2014-11-07, 13:18), generated by Outlook, sent over 
ActiveSync.

- We saved that message, deleted it from the sent and trash folder. 
Thereafter we had a complete sync.

- I sent the content of said message to Mike.

2014-12-02 07:29:22 marth (at) tsvschlieben (dot) de Comment #19 Reply to this comment
Is there any development on this serious bug? ActiveSync is nearly 
useless at the moment. Do you need some additional test data?

2014-11-25 17:20:57 skaman (dot) tx (at) gmail (dot) com Comment #18 Reply to this comment
Yes,

I should have been cleared in my earlier post as I get the continuous 
message in my ssh terminal as well of:


HORDE: Call to a member function toString() on a non-object [pid 4282 
on line 1282 of "/usr/share/php/Horde/Mime/Part.php"]

When I activate "per device" debug in EAS settings, the OL2013device   
log files are flooded with information, even when set to "Protocol".

Updated to latest versions a few seconds ago but as soon as OL2013 
queries EAS the error beaps/message begins on ssh terminal still.   
Also no emails appear in the OL2013 inbox or other folders..

Scott
2014-11-25 12:59:39 richard (dot) beltrame (at) tritem (dot) net Comment #17
New Attachment: horde_log2.txt Download
Reply to this comment
Michael,

I ran extra tests to dig the problem.
I have updated to the latest versions before running.

I emptied totally my mailbox (inbox, sent, ...), deleted in horde 
activesync administration my device, deleted the OL2013 profile, 
recreated EAS profile it and sync again.

After the sync of my calendar and contacts (a few mails during the 
interval have a normal display, the blank mails have disappeared), i 
heard regular bips from the SSH console displaying continusously the 
following message :

Message from syslogd@master1 at Nov 25 13:40:27 ...
  HORDE: Call to a member function toString() on a non-object [pid 
29535 on line 1282 of "/usr/share/pear/Horde/Mime/Part.php"]

I looked at the log and the contact sync seems to be in a kind of loop.
It seems to detect endless changes for my entire adress book.
Maybe not the real problem, but perhaps a clue ...

I can't join the entire log but you'll find one of the iteration.

Closing OL2013 and the bip/message is not occuring anymore.

Hope this helps.

Thanks again for your reactive support when we face problems.

Kind regards

Richard

[Show Quoted Text - 18 lines]
2014-11-25 10:59:01 patrick (at) spamreducer (dot) eu Comment #16
New Attachment: screenshot OL2013 header.PNG Download
Reply to this comment
Truncated Header in OL2013
2014-11-25 10:57:15 patrick (at) spamreducer (dot) eu New Attachment: sample message.txt Download
 
2014-11-25 10:55:52 patrick (at) spamreducer (dot) eu Comment #15
New Attachment: screenshot OL2013.PNG Download
Reply to this comment
A backtrace, or at the very least a copy of an email that is 
triggering this error, would be very helpful.
Every new email is causing the "emptyness" error..

Have updated to GIT master (090e816) seconds ago, new mails are empty 
like before updating Horde.

I'll upload a sample message and a screenshot from OL2013.

Patrick.
2014-11-25 07:50:38 Michael Rubinsky Comment #14 Reply to this comment
I think there are two different issues going on in this ticket. This 
latest commit fixes the issue shown in the log from comment #10.  The 
tell-tale sign of this issue would be a log entry similar to the 
following:

2014-11-25T06:35:41+00:00 INFO: [28787] Checking MIMETRUNCATION: 9, 
ServerData: 665

Note the "9" - this is the broken part.

I'm not sure what is causing the issue that results in:

   HORDE: Call to a member function toString() on a non-object [pid
3565 on line 1282 of "/usr/share/php/Horde/Mime/Part.php"]

A backtrace, or at the very least a copy of an email that is 
triggering this error, would be very helpful.

2014-11-25 07:38:13 Git Commit Comment #13 Reply to this comment
Changes have been made in Git (master):

commit c2e121841d12e943762b801940b79d58d3be1999
Author: Michael J Rubinsky <mrubinsk@horde.org>
Date:   Tue Nov 25 02:36:38 2014 -0500

     Fix and implement hotfix for incorrect truncationsize defaults.

     Bug: 13711

  .../lib/Horde/ActiveSync/Imap/Adapter.php          |    8 +++++++-
  1 files changed, 7 insertions(+), 1 deletions(-)

http://github.com/horde/horde/commit/c2e121841d12e943762b801940b79d58d3be1999
2014-11-25 07:01:42 richard (dot) beltrame (at) tritem (dot) net Comment #12
New Attachment: horde_log.txt Download
Reply to this comment
Hi Michael,

Same problem for me.
EAS is now broken on my 2 computers running OL2013.
EAS is working fine in Windows 8 Mail UI application and in Windows 
Phone 8.1 App mail.

Tried to delete my OST files and resync but sync is blocking at a 
moment so i can't see the whole content of my mailbox anymore. IMAP is 
working fine.

Upgraded to Horde_Mime 2.5.2 this morning but no change.

See the AS log attached.

Thanks for help

Kind regards

Richard


2014-11-25 06:58:08 christian (at) hailer (dot) eu Comment #11
New Attachment: empty_mail_header.png Download
Reply to this comment
One more thing: it looks like the mail gets truncated while 
downloaded, the header of the mail object just shows "Return-Pa", 
nothing more.
2014-11-25 06:51:13 christian (at) hailer (dot) eu Comment #10
New Attachment: empty_mail.zip Download
Reply to this comment
Hi Michael,

here is the sync-log and a screenshot from OL2013. The subject of the 
test email was "testmail", this is logged correctly in the sync-log... 
however, OL shows an empty mail.

Regards, Christian
2014-11-24 23:36:22 skaman (dot) tx (at) gmail (dot) com Comment #9 Reply to this comment
Same thing happening for me in OL2013.  IMAP works just fine but all 
AS accounts show blank email FROM, SUBJECT, and CONTENT.  I am also 
see the following error when an AS client attempts to connect.

  HORDE: Call to a member function toString() on a non-object [pid 
3565 on line 1282 of "/usr/share/php/Horde/Mime/Part.php"]
I cannot reproduce with latest code.

I see proper email content on all of my test devices, including 
OL2013, various versions of Android and various versions of iOS.

I would need a complete sync log to investigate further.
2014-11-24 15:27:18 Michael Rubinsky Comment #8
State ⇒ Feedback
Reply to this comment
I cannot reproduce with latest code.

I see proper email content on all of my test devices, including 
OL2013, various versions of Android and various versions of iOS.

I would need a complete sync log to investigate further.
2014-11-24 13:03:59 christian (at) hailer (dot) eu Comment #7 Reply to this comment
Still happening to me, too.

The following packages were updated this morning:

upgrade ok: channel://pear.horde.org/Horde_Mail-2.5.0
upgrade ok: channel://pear.horde.org/Horde_Smtp-1.7.0
upgrade ok: channel://pear.horde.org/Horde_ListHeaders-1.2.0
upgrade ok: channel://pear.horde.org/Horde_Mime-2.5.1
upgrade ok: channel://pear.horde.org/Horde_Core-2.17.0
upgrade ok: channel://pear.horde.org/Horde_Imap_Client-2.25.4
upgrade ok: channel://pear.horde.org/Horde_ActiveSync-2.20.3

I reverted to my nightly backup in the meantime.

Regards, Christian
2014-11-24 10:07:46 patrick (at) spamreducer (dot) eu Comment #6 Reply to this comment
Definitely NOT SOLVED!
Empty Messages are still generated..
..in Outlook 2013, Android is OK.

2014-11-24 08:42:14 patrick (at) spamreducer (dot) eu Comment #5 Reply to this comment
Definitely NOT SOLVED!
Empty Messages are still generated..
2014-11-24 05:41:04 Michael Rubinsky Comment #4
State ⇒ Resolved
Reply to this comment
Similar, but not quite the same as Bug: 13714.

Either way, both are now fixed.
2014-11-23 07:33:04 gavin (at) northeys (dot) net Comment #3 Reply to this comment
I am experiencing the same problem as well.  In addition, in Android, 
my emails are displaying the correct Subject and To headers, but the 
email body is empty, has a less than symbol (<), or in one instance 
contains the first 9 characters of the email body.  This is on Android 
version 4.4.4 and 4.4.2.

Hopefully this has been addressed in the update to Horde_Core but if 
not, I wanted to make you aware of this additional information.

Thanks
2014-11-22 15:29:51 Michael Rubinsky Assigned to Michael Rubinsky
 
2014-11-22 15:29:40 Michael Rubinsky Comment #2
State ⇒ Assigned
Reply to this comment
I'm fairly certain this is fixed in Horde_Core already, but it is not 
upgradeable at the moment since some of it's dependencies need to be 
upgraded first.

Keeping the ticket open until Core is pushed and can verify.
2014-11-22 04:00:02 christian (at) hailer (dot) eu Comment #1
Type ⇒ Bug
State ⇒ Unconfirmed
Priority ⇒ 2. Medium
Summary ⇒ ActiveSync via Outlook 2013: Empty Mails
Queue ⇒ Synchronization
Milestone ⇒
Patch ⇒ No
New Attachment: 2.JPG Download
Reply to this comment
Hello Michael,

I updated ActiveSync from 2.20.1 to 2.20.2 today. After the update 
Outlook 2013 showed a completely blank email, no email address or 
body. Android displeyed the mail correctly.
After reverting to 2.20.1 it worked again.

Saved Queries