[#12007] Messages marked for deletion do not catch changes to the seen flag.
Summary Messages marked for deletion do not catch changes to the seen flag.
Queue Synchronization
Queue Version Git master
Type Bug
State Resolved
Priority 1. Low
Owners mrubinsk@horde.org
Requester horde@agotnes.com
Created 2013-01-30 (2752 days ago)
Due
Updated 2013-02-11 (2740 days ago)
Assigned
Resolved 2013-02-01 (2750 days ago)
Milestone
Patch No

Comments
horde@agotnes.com 2013-01-30 22:52:07
1. Receive a new email
   AS synchs it to the device as expected
2. Delete the email from an IMAP client which only marks the message 
as deleted (e.g. Mac Mail)
   i.e. it shows in IMP with a strikethrough font in the Inbox
   AS doesn't delete the message after a manual refresh of the Inbox
3. Delete the message with a strikethrough font using IMP
   AS deletes the message on the local device after a manual refresh 
of the inbox

If I skip 2. above and delete the message in IMP the message is 
subsequently deleted immediately via AS.

I.e. it looks like AS doesn't honour the 'deleted' flag for messages 
in the inbox whilst once the message is removed from the inbox to the 
Trash folder AS removes it.

This is using AS 2.1.5, IMAP 2.5.0 and iOS 6.1 ...

Happy to provide traces if required - let me know.

horde@agotnes.com 2013-01-31 00:43:21
Bit more detail;

Exact same issue on an Android 4.1.1 device (Samsung Galaxy S3)

Also - the email isn't just not deleted - it also remains marked as 
unread on both devices even though it is marked as deleted.

Michael Rubinsky <mrubinsk@horde.org> 2013-01-31 14:13:24
ActiveSync knows nothing about the /deleted flag. The *only* way to 
delete a message in ActiveSync is to expunge it from the mailbox. 
Period.

horde@agotnes.com 2013-01-31 21:36:24
Can I ask to clarify for others finding this behaviour in future and 
wondering;

This is a choice in the Horde AS code? I.e. when seeing a message 
marked as /deleted on IMAP AS leaves it there as the AS protocol it 
has no facility to tell a client the message is marked for deletion 
(but not yet deleted/expunged)?

And in theory (and probably with some strange side effects) - the AS 
code *could* choose to tell the client the message was deleted at this 
point?

And lastly - shouldn't the message be marked as 'read' via AS? Is that 
a separate bug? (happy to raise one).

thx

Michael Rubinsky <mrubinsk@horde.org> 2013-02-01 17:15:47
> Can I ask to clarify for others finding this behaviour in future and 
> wondering;
>
> This is a choice in the Horde AS code? I.e. when seeing a message 
> marked as /deleted on IMAP AS leaves it there as the AS protocol it 
> has no facility to tell a client the message is marked for deletion 
> (but not yet deleted/expunged)?

It's not really a choice, because there *is* no such thing as a 
message marked for deletion in AS.  A message is either in the mailbox 
or it is not. The /deleted flag is not a supported flag in the 
protocol, there is no mechanism to tell the client it is "marked for 
deletion" and there is no way of displaying such messages in a 
different manner (such as with a line though them).

Technically, what the code does is ask for any changes since the last 
known modseq value. It then discards any messages returned from this 
query that have the /deleted flag now set.

> And in theory (and probably with some strange side effects) - the AS 
> code *could* choose to tell the client the message was deleted at 
> this point?

No, because that would put the client and the server in an 
inconsistent state. There is a command called SOFTDELETE that could be 
abused for this purpose, but it still needs to be implemented (and is 
a current TODO listed in the library docs). It's normally used for 
removing messages from the client (email or appointments) that fall 
outside of the client's configured time window.

> And lastly - shouldn't the message be marked as 'read' via AS? Is 
> that a separate bug? (happy to raise one).

Actually, yes, I suppose it should. The way the code works currently 
is the entire message is ignored, so the /seen flag is never caught. 
No need for a new ticket, I'll take care of it.

Git Commit <commits@lists.horde.org> 2013-02-01 17:29:19
Changes have been made in Git (master):

commit 1a2da08abce3ff7c2ede0c55a77803e44238ab84
Author: Michael J Rubinsky <mrubinsk@horde.org>
Date:   Fri Feb 1 12:26:28 2013 -0500

     Don't filter out /deleted messages, other flags may have changed.

     This will cause in increase in traffic, but is necessary to properly
     catch changes to the /seen flag.

     See Bug: 12007

  .../lib/Horde/ActiveSync/Imap/Adapter.php          |    6 ++----
  1 files changed, 2 insertions(+), 4 deletions(-)

http://git.horde.org/horde-git/-/commit/1a2da08abce3ff7c2ede0c55a77803e44238ab84

horde@agotnes.com 2013-02-11 08:26:01
Thanks for the detailed response - one last note for posterity below 
for others finding this annoying (like I did!)

>
> No, because that would put the client and the server in an 
> inconsistent state. There is a command called SOFTDELETE that could 
> be abused for this purpose, but it still needs to be implemented 
> (and is a current TODO listed in the library docs). It's normally 
> used for removing messages from the client (email or appointments) 
> that fall outside of the client's configured time window.

Looking forward to SoftDelete being implemented then :)

I did some checking on the behaviour of the Mac client I use (Mountain 
Lion, Mac Mail) and found it effectively does the following;

1. Moves the message to Trash folder
2. Marks message as 'deleted'

It then leaves the message in this state until, depending on settings, 
it decides to Expunge. I couldn't find a way to make a step 3. above 
being 'expunge' which is what I'd like so I sorted it out server side 
with Dovecot like so;

#!/bin/sh
doveadm expunge -A mailbox Inbox DELETED

Set up as a cron job to run hourly. Works a treat in my scenario 
although other mail clients may well work differently making this not 
work so well...

Jan Schneider <jan@horde.org> 2013-02-11 17:32:02
> Thanks for the detailed response - one last note for posterity below 
> for others finding this annoying (like I did!)
>
>>
>> No, because that would put the client and the server in an
>> inconsistent state. There is a command called SOFTDELETE that could
>> be abused for this purpose, but it still needs to be implemented (and
>> is a current TODO listed in the library docs). It's normally used for
>> removing messages from the client (email or appointments) that fall
>> outside of the client's configured time window.
>
> Looking forward to SoftDelete being implemented then :)
>
> I did some checking on the behaviour of the Mac client I use 
> (Mountain Lion, Mac Mail) and found it effectively does the following;
>
> 1. Moves the message to Trash folder
> 2. Marks message as 'deleted'
>
> It then leaves the message in this state until, depending on 
> settings, it decides to Expunge. I couldn't find a way to make a 
> step 3. above being 'expunge' which is what I'd like so I sorted it 
> out server side with Dovecot like so;
>
> #!/bin/sh
> doveadm expunge -A mailbox Inbox DELETED
>
> Set up as a cron job to run hourly. Works a treat in my scenario 
> although other mail clients may well work differently making this 
> not work so well...