Great!

First test by manually triggering resync from the device: WORKS.
Completely deleting the account and the device, re-pair etc.: WORKS

Great, seems fixed! Thank you a lot!

Can you try now?

Changes have been made in Git (master):

commit 1633b590a0a024f9a329e4aad5cff7b5f0d43f93
Author: Michael J Rubinsky <mrubinsk@horde.org>
Date:   Thu Dec 13 13:45:17 2012 -0500

     Bug: 11754 Server may advertise CONDSTORE but not support per 
mailbox MODSEQ.

  .../lib/Horde/ActiveSync/Imap/Adapter.php          |    4 ++--
  1 files changed, 2 insertions(+), 2 deletions(-)

http://git.horde.org/horde-git/-/commit/1633b590a0a024f9a329e4aad5cff7b5f0d43f93

The IMAP server is a Cyrus imapd - I just noticed that the package 
name doesn't make that clear and not everybody might know that Kolab 
uses that.

I forgot to mention: the test email was sent and INBOXed at 18:39 CET. 
I've yet to see it on the mobile device.

The test user has no specific contacts - that'll be the entries coming 
from the LDAP directory. Someone doing the Kolab and/or LDAP 
integration stuff (Jan?) might have an idea whether to synthesize the 
UIDs or who should add them when into which attribute.
sorry, no new account, as I'm currently running out of time - but 
here's the email-only log. Same procedure, same effect. (I've let my 
client run on "poll emails immediately" or this short test, only 
switch to "5 min interval" at the end.)

The IMAP server is imapd-2.3.13-20081020_kolab2, tracing that will 
have to wait until tomorrow, too - please let me know if this 
shortened log is sufficient or you'd need a clean account and/or IMAP 
trace.

Also, you never answered the question about your IMAP server.  What 
server/version are you using? An IMAP debug log would probably be 
helpful as well.

To start with, you have a bunch of errors from other collections, such 
as contacts due to invalid data on your server. E.g., you have a bunch 
of contacts with no UID value. You MUST have some unique UID field 
identified for your contacts.

Can you create a new account, but ONLY select the INBOX for synching? 
The log is extremely difficult to read since your device is sending 
multiple SYNC requests for each collection simultaneously.

I noticed my test server was still on Horde_ActiveSync 2.0.8, so I 
upgraded this to 2.0.10 and reran the test.

First observation: The existing test message was transferred via the 
initial sync. The older messages were skipped as I have set a time 
limit for syncing messages.

Second observation: Nothing else has changed: A test email sent to the 
account (after initial sync) doesn't get synced to the device, neither 
via periodic sync nor via explicit sync.

Installed packages, channel pear.horde.org:
===========================================
Package                   Version State
Horde_ActiveSync          2.0.10  stable
Horde_Alarm               2.0.2   stable
Horde_Argv                2.0.2   stable
Horde_Auth                2.0.1   stable
Horde_Autoloader          2.0.1   stable
Horde_Browser             2.0.2   stable
Horde_Cache               2.0.1   stable
Horde_Cli                 2.0.1   stable
Horde_Compress            2.0.1   stable
Horde_Constraint          2.0.1   stable
Horde_Controller          2.0.1   stable
Horde_Core                2.1.0   stable
Horde_Crypt               2.1.0   stable
Horde_Crypt_Blowfish      1.0.1   stable
Horde_Data                2.0.1   stable
Horde_Date                2.0.1   stable
Horde_Date_Parser         2.0.1   stable
Horde_Db                  2.0.1   stable
Horde_Editor              2.0.1   stable
Horde_ElasticSearch       1.0.1   stable
Horde_Exception           2.0.1   stable
Horde_Feed                2.0.1   stable
Horde_Form                2.0.1   stable
Horde_Group               2.0.1   stable
Horde_History             2.0.1   stable
Horde_Http                2.0.1   stable
Horde_Icalendar           2.0.1   stable
Horde_Image               2.0.1   stable
Horde_Imap_Client         2.3.2   stable
Horde_Imsp                2.0.1   stable
Horde_Injector            2.0.1   stable
Horde_Itip                2.0.1   stable
Horde_Kolab_Format        2.0.1   stable
Horde_Kolab_Server        2.0.1   stable
Horde_Kolab_Session       2.0.1   stable
Horde_Kolab_Storage       2.0.2   stable
Horde_Ldap                2.0.1   stable
Horde_ListHeaders         1.0.1   stable
Horde_Lock                2.0.1   stable
Horde_Log                 2.0.1   stable
Horde_LoginTasks          2.0.1   stable
Horde_Mail                2.0.2   stable
Horde_Memcache            2.0.1   stable
Horde_Mime                2.0.1   stable
Horde_Mime_Viewer         2.0.1   stable
Horde_Nls                 2.0.1   stable
Horde_Notification        2.0.1   stable
Horde_Oauth               2.0.1   stable
Horde_Pdf                 2.0.1   stable
Horde_Perms               2.0.1   stable
Horde_Prefs               2.0.1   stable
Horde_Rdo                 2.0.1   stable
Horde_Role                1.0.1   stable
Horde_Routes              2.0.1   stable
Horde_Rpc                 2.0.1   stable
Horde_Scribe              2.0.1   stable
Horde_Secret              2.0.2   stable
Horde_Serialize           2.0.1   stable
Horde_Service_Facebook    2.0.1   stable
Horde_Service_Twitter     2.0.1   stable
Horde_Service_Weather     2.0.1   stable
Horde_SessionHandler      2.0.1   stable
Horde_Share               2.0.1   stable
Horde_SpellChecker        2.0.1   stable
Horde_Stream              1.2.0   stable
Horde_Stream_Filter       2.0.1   stable
Horde_Stream_Wrapper      2.0.1   stable
Horde_Support             2.0.2   stable
Horde_SyncMl              2.0.1   stable
Horde_Template            2.0.1   stable
Horde_Text_Diff           2.0.1   stable
Horde_Text_Filter         2.0.3   stable
Horde_Text_Filter_Csstidy 2.0.1   stable
Horde_Text_Flowed         2.0.1   stable
Horde_Thrift              2.0.1   stable
Horde_Timezone            1.0.1   stable
Horde_Token               2.0.1   stable
Horde_Translation         2.0.1   stable
Horde_Tree                2.0.1   stable
Horde_Url                 2.0.1   stable
Horde_Util                2.0.1   stable
Horde_Vfs                 2.0.3   stable
Horde_View                2.0.1   stable
Horde_Xml_Element         2.0.1   stable
Horde_Xml_Wbxml           2.0.1   stable
content                   2.0.1   stable
horde                     5.0.2   stable
imp                       6.0.2   stable
ingo                      3.0.1   stable
kronolith                 4.0.2   stable
mnemo                     4.0.1   stable
nag                       4.0.1   stable
timeobjects               2.0.1   stable
turba                     4.0.1   stable
webmail                   5.0.2   stable

Hi Michael,
sorry for taking my time - I wanted to already use your updated code 
(last weekend), to have a clean & simple test case, and data I could 
send you without too much disclosure.

The attached trace covers the following:

- I removed all ActiveSync devices via H5 admin web interface
- I removed all ActiveSync configurations from my test device (HP Pre3, WebOS)
- I have a test account (INBOX contained 5 read messages)

- The ActiveSync account was configured on the mobile device 
2012-12-12, 16:59 CET
- The system was given some hours to create a stable sync basis
- at 2012-12-12, 22:30 CET, a test email was sent to the test account 
(and was visible via that test user's H5 IMP)

The message never made it to the mobile device.

We have had Apache restarts during that test window because of "Base 
lambda function for closure not found" errors, just for reference:

Wed Dec 12 17:00:40 CET 2012: restarting Apache2 web server
Wed Dec 12 18:46:38 CET 2012: restarting Apache2 web server
Wed Dec 12 20:02:45 CET 2012: restarting Apache2 web server
Wed Dec 12 21:39:26 CET 2012: restarting Apache2 web server
Wed Dec 12 22:47:23 CET 2012: restarting Apache2 web server
Wed Dec 12 22:56:25 CET 2012: restarting Apache2 web server
Wed Dec 12 23:11:45 CET 2012: restarting Apache2 web server
Wed Dec 12 23:11:48 CET 2012: restarting Apache2 web server
Wed Dec 12 23:11:51 CET 2012: restarting Apache2 web server
Wed Dec 12 23:11:54 CET 2012: restarting Apache2 web server
Wed Dec 12 23:11:56 CET 2012: restarting Apache2 web server
Wed Dec 12 23:27:00 CET 2012: restarting Apache2 web server

With earlier tests on a different account, sometimes and after *days* 
some email updates made it to the device. I'll keep the trace running 
and will check from time to time if the test email got synced to the 
device. If I happen to identify that case in the trace, I can append 
the according trace section here if you'd like.

Oh, and a test appointment I created a few minuted ago got synced to 
the device just nicely. My INBOX, as displayed on the mobile device, 
is still empty though.

I'll provide a full log (plus description of the test case it is based 
on) in a few days.

1). Are you marking the message as deleted, or are you expunging the 
message from the INBOX?

2). What IMAP server/version are you using? Even better would be the 
list of reported capabilities.

Any further debugging of ActiveSync traffic would require the *full* 
synclog - from device pairing to problem development, as the actual 
problem often occurs before the symptoms present.

Normal, and harmless.
This means that more than one process is syncing the same collection 
at the same time. So the current process dies, letting the first 
process complete. This can sometimes occur when you explicitly trigger 
a sync on the device while it is already PINGing.
This means that, for whatever reason, the device did not receive 
and/or understand the response to it's last request, so it is 
repeating the request. We count the number of times this happens to 
prevent infinite sync loops.
This means that the current syncState has been removed, as would 
happen when you remove the device from Horde (or manually from the DB).
This is similar error to the above error that occurs when the state is 
removed during a running PING.
Without more information, it's impossible to tell what is wrong. The 
error messages above can all be attributed to normal/expected behavior.

Ok, this is quite an interesting problem. I most of the older mails 
from Inbox to another sub-folder and it didn't solve the problem. 
However, after I started bisecting the commits I figured out that also 
the phone was somehow misbehaving ---> I restarted the phone once 
again. ---> I continued bisecting and none of the tested commits were 
bad anymore ---> Now the activesync works again even with the latest 
commit and therefore I'm unable to re-produce the problem. I even 
moved all the inbox mails back and the activesync still works just fine.

The obvious conclusion would be therefore that the problem was on 
android client end, but I don't believe that is the real reason, as 
rebooting the phone didn't help originally at all, until I emptied my 
inbox and only after that the android reboot got the activesync 
working again. In case the activesync gets stuck again, I'll try to 
debug it again hopefully with better success.

I seem to have lost email syncing with Samsung Galaxy S3 <-> Horde git 
(e813ee5a85abc0a9a0aeada5ca417fea30ee8197):

The combination has been working quite fine until git commit 
33c28c5b3e4f09e6c7d750f276186336e7f3a8e0, after which I had some bug 
login problems and/or mail sending was completely broken for a while 
--> I was forced back to this commit again.

Currently the problem is only related to email. For example calendar 
syncing works just fine.
I can even send emails from my android phone and they get delivered 
properly (although a sent copy gets copied to sent base folder, not to 
the configured 2012 sub-folder) ---> However, the just sent email 
newer shows up on the phone. I checked the cyrus imapd communication, 
but I don't see any failures there, so I suspect that the error 
message about sync keys might be causing this problem itself.

This is just to confirm that somebody else is also seend a similar 
problem with current activesync.

2012-11-30T17:25:12+00:00 INFO: [SEC155335ACE18C3] Folder server id: INBOX
2012-11-30T17:25:12+00:00 DEBUG: I     </FolderId>
2012-11-30T17:25:12+00:00 DEBUG: I     <DeletesAsMoves>
2012-11-30T17:25:12+00:00 DEBUG: I      1
2012-11-30T17:25:12+00:00 DEBUG: I     </DeletesAsMoves>
2012-11-30T17:25:12+00:00 DEBUG: I     <GetChanges/>
2012-11-30T17:25:12+00:00 DEBUG: I     <WindowSize>
2012-11-30T17:25:12+00:00 ERR: Unmatched content:
2012-11-30T17:25:12+00:00 ERR: Array
(
     [1] => 1
     [2] => WindowSize
     [4] => 1
)

2012-11-30T17:25:12+00:00 DEBUG: I      50
2012-11-30T17:25:12+00:00 DEBUG: I     </WindowSize>
2012-11-30T17:25:12+00:00 DEBUG: I     <Options>
2012-11-30T17:25:12+00:00 DEBUG: I      <FilterType>
2012-11-30T17:25:12+00:00 DEBUG: I       0
2012-11-30T17:25:12+00:00 DEBUG: I      </FilterType>
2012-11-30T17:25:12+00:00 DEBUG: I      <AirSyncBase:BodyPreference>
2012-11-30T17:25:12+00:00 DEBUG: I       <AirSyncBase:Type>
2012-11-30T17:25:12+00:00 DEBUG: I        2
2012-11-30T17:25:12+00:00 DEBUG: I       </AirSyncBase:Type>
2012-11-30T17:25:12+00:00 DEBUG: I       <AirSyncBase:TruncationSize>
2012-11-30T17:25:12+00:00 DEBUG: I        0
2012-11-30T17:25:12+00:00 DEBUG: I       </AirSyncBase:TruncationSize>
2012-11-30T17:25:12+00:00 DEBUG: I      </AirSyncBase:BodyPreference>
2012-11-30T17:25:12+00:00 DEBUG: I      <AirSyncBase:BodyPreference>
2012-11-30T17:25:12+00:00 DEBUG: I       <AirSyncBase:Type>
2012-11-30T17:25:12+00:00 DEBUG: I        4
2012-11-30T17:25:12+00:00 DEBUG: I       </AirSyncBase:Type>
2012-11-30T17:25:12+00:00 DEBUG: I      </AirSyncBase:BodyPreference>
2012-11-30T17:25:12+00:00 DEBUG: I     </Options>
2012-11-30T17:25:12+00:00 DEBUG: I    </Folder>
2012-11-30T17:25:12+00:00 ERR: Incrementing counter, we saw this 
synckey before.
2012-11-30T17:25:12+00:00 DEBUG: I   </Folders>
2012-11-30T17:25:12+00:00 DEBUG: I  </Synchronize>
2012-11-30T17:25:12+00:00 DEBUG: Have syncable collections
2012-11-30T17:25:12+00:00 DEBUG: All synckeys confirmed. Continuing with SYNC

I had no change even after a few hours.

As my IMAP store is rather large, I added another (test) account to my 
mobile device, in parallel to my regular account. That allowed me to 
test sending messages between these two accounts without having a 
second device at hand - and those test emails were immediately shown, 
on both respective inboxes!

But not all is well, I still have more than slight sync trouble, i.e. 
when deleting a message on the mobile device, it'll re-appear on the 
next sync.

This sync is something I will have to spend some time on, it seems.

I definitely hate Murphy :D

I've had a new (and unread) message lingering in my INBOX for minutes 
now (sent and received around 18:52:xx), which wasn't reported to the 
mobile device.

Once I sent my bug report, the resulting confirmation message showed 
up in my INBOX (and wasn't transfered either).

Using teh web interface, I moved that (and another) message into a 
sub-folder and expunged old deleted emails from my INBOX (there were 
several of those) - then suddenly, I received the initial test 
message, which still was (is) in my INBOX.

For completeness, I've included messages for some of the preceding polls:

2012-11-25T18:16:21+00:00 DEBUG: [6110] Initializing state for 
collection: INBOX, synckey: {50b25f1c-0e34-478a-847d-1450c0a86742}2
2012-11-25T18:16:21+00:00 DEBUG: [6110] Loading state for synckey 
{50b25f1c-0e34-478a-847d-1450c0a86742}2
2012-11-25T18:16:21+00:00 DEBUG: [6110] Initializing message diff 
engine for INBOX
2012-11-25T18:16:21+00:00 DEBUG: 
Horde_Core_ActiveSync_Driver::getServerChanges(INBOX, 1353867038, 
1353867381, 0, 1)
2012-11-25T18:16:21+00:00 DEBUG: [6110] Found 0 message changes in INBOX.
2012-11-25T18:16:26+00:00 DEBUG: [6110] Initializing state for 
collection: INBOX, synckey: {50b25f1c-0e34-478a-847d-1450c0a86742}2
2012-11-25T18:16:26+00:00 DEBUG: [6110] Loading state for synckey 
{50b25f1c-0e34-478a-847d-1450c0a86742}2
2012-11-25T18:16:26+00:00 DEBUG: [6110] Initializing message diff 
engine for INBOX
2012-11-25T18:16:26+00:00 DEBUG: 
Horde_Core_ActiveSync_Driver::getServerChanges(INBOX, 1353867038, 
1353867386, 0, 1)
2012-11-25T18:16:26+00:00 DEBUG: [6110] Found 0 message changes in INBOX.
2012-11-25T18:16:31+00:00 DEBUG: [6110] Initializing state for 
collection: INBOX, synckey: {50b25f1c-0e34-478a-847d-1450c0a86742}2
2012-11-25T18:16:31+00:00 DEBUG: [6110] Loading state for synckey 
{50b25f1c-0e34-478a-847d-1450c0a86742}2
2012-11-25T18:16:31+00:00 DEBUG: [6110] Initializing message diff 
engine for INBOX
2012-11-25T18:16:31+00:00 DEBUG: 
Horde_Core_ActiveSync_Driver::getServerChanges(INBOX, 1353867038, 
1353867391, 0, 1)
2012-11-25T18:16:31+00:00 DEBUG: [6110] Found 0 message changes in INBOX.
2012-11-25T18:16:36+00:00 DEBUG: [6110] Initializing state for 
collection: INBOX, synckey: {50b25f1c-0e34-478a-847d-1450c0a86742}2
2012-11-25T18:16:36+00:00 DEBUG: [6110] Loading state for synckey 
{50b25f1c-0e34-478a-847d-1450c0a86742}2
2012-11-25T18:16:36+00:00 DEBUG: [6110] Initializing message diff 
engine for INBOX
2012-11-25T18:16:36+00:00 DEBUG: 
Horde_Core_ActiveSync_Driver::getServerChanges(INBOX, 1353867038, 
1353867396, 0, 1)
2012-11-25T18:16:36+00:00 DEBUG: [6110] Found 1 message changes in INBOX.
2012-11-25T18:16:36+00:00 DEBUG: [6110] No PIM changes present, 
returning all messages.
2012-11-25T18:16:36+00:00 INFO: [6110] Changes available!

Then I sent another message (test2) to my account - it wouldn't up on 
the mobile device, even after an explicit sync initiated on the mobile 
device.
I initiated another purge (which did nothing, as no deleted messages 
were in the INBOX), deleted the old message (test1), manually resynced 
- no change.

As I know had a deleted (but not purged) message "test1" in the INBOX, 
I again purged and manually resynced. Again, no change on the mobile 
device: "test1" (gone according to web UI) was still displayed and 
"test2" was not to be seen.

I sent "test3" and "test4", moved "test3" to some folder, resync - no change.
Deleted "test2", resync - no change
Purged INBOX (only "test4" left in INBOX), resync - no change.

What's funny... when I manually resync, there's tons of traffic (I 
have some 150 folders) - but the initial messages above were truly 
terse, so there must have been a difference between the manual syncs 
and what had happened initial to this report. The client is set to 
"poll messages when received" rather than some time interval. Changing 
that to i.e. 5 minutes results in short bursts of sync traffic, but no 
change on the device... at least no yet, I'll recheck in a couple of 
hours...

Summary: Messages may make it to the device - bu manually polling for 
them is insufficient :(

I'm running a WebOS Pre3 against Horde5, ActiveSync. I've tried 
various settings and scenarios now, but always get to a point where 
the Pre3 will not receive new emails.

My test workflow:
- delete account from mobile device
- delete ActiveSync device in Horde
- reboot mobile device
- add account to mobile device (and select "email" only, to avoid 
mixed polls in the logs)
   * I see all current email on the device, no errors detected
- let it run for some time
- send new email to my email address
   * new emails show up in Horde5 web interface, but not on mobile device

I've tried with protocol 2.5 and 12.1, to the same effect. As I'm 
running ActiveSync with "custom logging", I have a device-specific log 
file to analyze.

Something that caught my eye in that log file - I have various "ERR:" 
entries, usually a pointer to trouble:

2012-11-25T14:22:39+00:00 ERR: Unmatched content:
2012-11-25T14:22:39+00:00 ERR: Array

2012-11-25T14:22:44+00:00 ERR: Changes detected in sync_cache during 
wait interval, exiting without updating cache.

2012-11-25T14:43:26+00:00 ERR: Incrementing counter, we saw this 
synckey before.

2012-11-25T14:53:39+00:00 ERR: [5199] State gone, PING terminating and 
forcing a SYNC:

2012-11-25T15:08:28+00:00 ERR: [5323] PING terminating: Empty synckey 
for INBOX

Most of these appear several times, but not necessarily in a specific pattern.

Since I started my latest test run, I have not been struck by "Base 
lambda function for closure not found in" bail-outs, which I earlier 
suspected as the root cause of these problems.

Can anyone shed some light on this and please give me a helping hand 
with debugging this issue?