6.0.0-alpha14
6/24/25

[#13273] Possible PING / FOLDERCREATE race condition can result in duplicate folders being sent to clients.
Summary Possible PING / FOLDERCREATE race condition can result in duplicate folders being sent to clients.
Queue Synchronization
Queue Version Git master
Type Bug
State Resolved
Priority 1. Low
Owners mrubinsk (at) horde (dot) org
Requester patrick (at) spamreducer (dot) eu
Created 06/18/2014 (4024 days ago)
Due
Updated 08/20/2014 (3961 days ago)
Assigned 08/19/2014 (3962 days ago)
Resolved 08/20/2014 (3961 days ago)
Github Issue Link
Github Pull Request
Milestone
Patch No

History
08/20/2014 05:46:04 PM patrick (at) spamreducer (dot) eu Comment #30 Reply to this comment
I am not being ignorant towards a client - you are not my client. I 
have, however, given you hours of my time in trying to troubleshoot 
your issue(s).
Sorry to repeat..but I think you have misunderstood something.
*I* was like ignorant to *my* clients. Just to justify why I was 
getting nervous.

It's *not* in my interest, and sure not in yours to turn this into an 
argument.

I will cooperate with my questions and bugtracing as I did before this 
big issue.

I really like and appreciate all of your work!
08/20/2014 04:57:14 PM Michael Rubinsky Comment #29 Reply to this comment

[Show Quoted Text - 11 lines]
The reason this was not fixed earlier is because the logs you provided 
me did not show the full picture - not until your most recent log.

Look. I don't want to turn this into an argument, but i can't let this 
go unanswered. You have to realize that this "free" support is done on 
our free time. If we don't respond, it's because we have more 
important things to do, or have already answered your question to the 
best of our abilities given the information that has been provided. I 
am not being ignorant towards a client - you are not my client. I 
have, however, given you hours of my time in trying to troubleshoot 
your issue(s). As stated on multiple occasions, if you want quicker, 
more "client-like" support - pay for it and become a client.

08/20/2014 04:22:27 PM patrick (at) spamreducer (dot) eu Comment #28 Reply to this comment
If you are unhappy with the support/response time that you receive 
for free then it's time to pay for support.
..just to conclude: the free services are *pretty* *good* , but for 
this ticket I was really disappointed since I wrote the informations I 
had.
I asked several times for help, also just a hint for debugging or 
trying to debug. nobody gave me an answer till yesterday.
For me it was like beeing the ignorant guy towards my clients; this 
leaves me exasperated.
08/20/2014 03:34:28 PM Michael Rubinsky Comment #27 Reply to this comment
Suppose this folder structure: "INBOX/sub/subsub"

When I rename the "sub" folder to "subfolder" then all the 
"subsub"-folders are renamed to "subsub [1]" and OL resyncs 
*everything* in this subsub folders..
Again, cannot reproduce. Will need, at a minimum, the synchlog showing 
the supposed broken behavior.

Also, please create a separate ticket, as the original issue in this 
ticket has been resolved.

08/20/2014 03:33:13 PM Michael Rubinsky Comment #26
State ⇒ Resolved
Reply to this comment
This support is very important to us as users, since we are not able 
to use you *great* product in productive environments if we have to 
wait for months to get rid of bugs/enhancements.
If you are unhappy with the support/response time that you receive for 
free then it's time to pay for support.

08/20/2014 02:16:24 PM patrick (at) spamreducer (dot) eu Comment #25 Reply to this comment
Well, first of all, *THANK YOU Michael*!
For now it seems as it is resolved, but I have to try out all 
possible conditions next days to give you a definite OK.

This support is very important to us as users, since we are not able 
to use you *great* product in productive environments if we have to 
wait for months to get rid of bugs/enhancements.
Found a BUG..

Suppose this folder structure: "INBOX/sub/subsub"

When I rename the "sub" folder to "subfolder" then all the 
"subsub"-folders are renamed to "subsub [1]" and OL resyncs 
*everything* in this subsub folders..

08/20/2014 07:27:56 AM patrick (at) spamreducer (dot) eu Comment #24 Reply to this comment
Well, first of all, *THANK YOU Michael*!
For now it seems as it is resolved, but I have to try out all possible 
conditions next days to give you a definite OK.

This support is very important to us as users, since we are not able 
to use you *great* product in productive environments if we have to 
wait for months to get rid of bugs/enhancements.
08/19/2014 11:27:24 PM Git Commit Comment #23 Reply to this comment
Changes have been made in Git (master):

commit de72dce8ca03ee0b8389b4ae744563f92cce7122
Author: Michael J Rubinsky <mrubinsk@horde.org>
Date:   Tue Aug 19 19:27:03 2014 -0400

     *sigh* Actually enforce the parameter.

     Bug: 13273

  .../lib/Horde/ActiveSync/Collections.php           |    2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)

http://github.com/horde/horde/commit/de72dce8ca03ee0b8389b4ae744563f92cce7122
08/19/2014 11:25:28 PM Git Commit Comment #22 Reply to this comment
Changes have been made in Git (master):

commit 2a925e59c82ee029fff097f55acc984d6faa432c
Author: Michael J Rubinsky <mrubinsk@horde.org>
Date:   Tue Aug 19 19:24:22 2014 -0400

     Correctly fix Bug: 13273

  .../lib/Horde/ActiveSync/Collections.php           |    8 +++++++-
  .../lib/Horde/ActiveSync/Request/Ping.php          |    4 ++--
  .../lib/Horde/ActiveSync/Request/Sync.php          |   10 +++++-----
  3 files changed, 14 insertions(+), 8 deletions(-)

http://github.com/horde/horde/commit/2a925e59c82ee029fff097f55acc984d6faa432c
08/19/2014 11:10:39 PM Git Commit Comment #21 Reply to this comment
Changes have been made in Git (master):

commit 5f3c55b8d1ff39e25d85f21eab805e79db0dbc87
Author: Michael J Rubinsky <mrubinsk@horde.org>
Date:   Tue Aug 19 19:09:50 2014 -0400

     Revert "Prevent the folder cache from being overwritten."

     This reverts commit e596382d79bc4c2ea2938c8a74974666ca0edace.

     Bug: 13273  This won't work as-is. It will prevent any folder
     changes from sticking. Correct fix shortly.

  .../lib/Horde/ActiveSync/Collections.php           |    8 +-------
  .../ActiveSync/lib/Horde/ActiveSync/SyncCache.php  |   12 ------------
  2 files changed, 1 insertions(+), 19 deletions(-)

http://github.com/horde/horde/commit/5f3c55b8d1ff39e25d85f21eab805e79db0dbc87
08/19/2014 10:59:50 PM Michael Rubinsky Comment #20
State ⇒ Feedback
Reply to this comment
Does this fix it for you?
08/19/2014 10:59:02 PM Git Commit Comment #19 Reply to this comment
Changes have been made in Git (master):

commit e596382d79bc4c2ea2938c8a74974666ca0edace
Author: Michael J Rubinsky <mrubinsk@horde.org>
Date:   Tue Aug 19 18:58:32 2014 -0400

     Prevent the folder cache from being overwritten.

     Possible fix for Bug: 13273

  .../lib/Horde/ActiveSync/Collections.php           |    8 +++++++-
  .../ActiveSync/lib/Horde/ActiveSync/SyncCache.php  |   12 ++++++++++++
  2 files changed, 19 insertions(+), 1 deletions(-)

http://github.com/horde/horde/commit/e596382d79bc4c2ea2938c8a74974666ca0edace
08/19/2014 10:10:17 PM Michael Rubinsky Comment #18
State ⇒ Assigned
Summary ⇒ Possible PING / FOLDERCREATE race condition can result in duplicate folders being sent to clients.
Reply to this comment
Updating bug title to better represent problem.
08/19/2014 10:08:45 PM Michael Rubinsky Comment #17 Reply to this comment
While I still cannot reproduce this, I *think* that this is the result 
of a race condition. In this latest log, it looks like you have 
created a new folder in OL - INBOX/Einkauf/Amazon. This was correctly 
dealt with on the server. The problem is that a PING request was 
already running on another thread. Since it found some changes, it 
updated the cache - overwriting the information about the newly 
created Amazon folder that was added to the cache during the separate 
FOLDERCREATE request.

I already have plans to refactor the various caches currently in use, 
but this will not happen until Horde 6. I'll have to come up with some 
workaround to implement in the meantime.
08/19/2014 09:21:55 AM patrick (at) spamreducer (dot) eu Comment #16 Reply to this comment
It's next to impossible for me to help any further since I cannot 
reproduce the issue locally.  The only thing I can suggest is that 
you can pay for support and have us ssh into your dev box to try to 
diagnose the problem.
I really don't know WHY your Horde installation is not having this issues!
I have installed a "normal" dovecot server - really nothing special or 
tuned - just "/" in my namespace as the separator (since "." is used 
in foldernames), the "listescape" feature and some "special_use" 
folders.
All the folders are created on the IMAP-side; there MUST be a problem 
in Horde; I am going to be crazy since I don't know from where the 
problem arise and I am not able to reverse engineer this problem or 
your code.

Maybe the new log is showing you some hints? Again, could you look at 
this logfile: https://db.tt/XH0GWxJV

Would it be possible to have some configuration files from a 
testserver you are using?
Or even better, SSH to a testserver; like we did times ago on my 
windows machine to find out the strange behavior of OL clients?
08/18/2014 02:47:03 PM Michael Rubinsky Comment #15 Reply to this comment
It's next to impossible for me to help any further since I cannot 
reproduce the issue locally.  The only thing I can suggest is that you 
can pay for support and have us ssh into your dev box to try to 
diagnose the problem.
08/18/2014 10:30:23 AM patrick (at) spamreducer (dot) eu Comment #14 Reply to this comment
..could, or maybe should I delete the erraneous profiles in Horde?
If YES, how can I do that?
The "horde-remove-user-data"-command is not removing ALL the data as 
I've seen some times ago?

Again, please provide me with a hint, or any help!

Thank you very much!
So is there really no one who could just *try* to help me out of this 
situation?
I have reinstalled the complete server
.. it *WILL NOT RUN WITH OL2013*..
07/14/2014 05:34:36 PM patrick (at) spamreducer (dot) eu Comment #13 Reply to this comment
..could, or maybe should I delete the erraneous profiles in Horde?
If YES, how can I do that?
The "horde-remove-user-data"-command is not removing ALL the data as 
I've seen some times ago?

Again, please provide me with a hint, or any help!

Thank you very much!
07/14/2014 05:20:47 PM patrick (at) spamreducer (dot) eu Comment #12 Reply to this comment
Hi guys,
what's happening so far?

Have you any new idea, hints, or anything else?

I'm in big trouble. My clients are expering this problem and there is 
no one who likes to help in any way?
07/09/2014 03:08:27 PM patrick (at) spamreducer (dot) eu Comment #11 Reply to this comment
I have numerous test folders, deep structures and weird characters 
in the folder names (including similar names to what you have 
suggested). I've had them for ages, just for testing purposes. Have 
not seen this.

The log shows nothing other than sending the correct folder 
structure to the client. If, at some point after this, the client 
decides to mess with them, that's the client's problem.
Hi!
Some new logs..

At least now there are error in logs..!!
---8<---------------------------------------------------------------------------------------------
2014-07-09T14:48:50+00:00 INFO: [40798] Obtained synckey for 
collection Fe0797ea1 from cache: {53bd556b-0f08-48af-9488-a1995f6ed0ca}3
2014-07-09T14:48:50+00:00 ERR: [40798] 
Horde_ActiveSync_Collections::getBackendIdForFolderUid failed because 
folder was not found in cache.
2014-07-09T14:48:49+00:00 INFO: UNSETTING collection INBOX/!Garni 
Helfer (Fe0797ea1) PINGABLE flag.
---8<---------------------------------------------------------------------------------------------

There are 2 new folders with the problem..
1. is the "INBOX/!Garni Helfer" folder => after the error it is 
renamed in "INBOX/!Garni Helfer [1]"
2. is the "INBOX/Privat/Übersetzungen/APP-MobileCounter" folder => 
renames to  "INBOX/Privat/Übersetzungen/APP-MobileCounter [1]"

The logs are on dropbox:
1. logfile: 
https://dl.dropboxusercontent.com/u/12623450/1.%20Creation%20of%20new%20folders%20Garni%20Helfer%2BMobileCounter.tgz
2. logfile: 
https://dl.dropboxusercontent.com/u/12623450/2.%20Error%20is%20appearing.tgz
3. logfile: 
https://dl.dropboxusercontent.com/u/12623450/3.%20After%20closing%20and%20reopening%20OL2013.tgz

I hope this provides some usable feedback..

Please provide me with a solution; I can't find a solution..

07/02/2014 02:31:19 PM Michael Rubinsky Comment #10 Reply to this comment
I have numerous test folders, deep structures and weird characters in 
the folder names (including similar names to what you have suggested). 
I've had them for ages, just for testing purposes. Have not seen this.

The log shows nothing other than sending the correct folder structure 
to the client. If, at some point after this, the client decides to 
mess with them, that's the client's problem.
07/02/2014 08:17:55 AM patrick (at) spamreducer (dot) eu Comment #9 Reply to this comment
Still cannot reproduce, even with email boxes containing a '.' as 
suggested in an off-list email.
Please create a testfolder and wait for some days, it will not show up 
everytime you create it..

This is strange..and perhaps, very difficult to reproduce..
07/01/2014 05:14:28 PM Michael Rubinsky Comment #8 Reply to this comment
Additionally, the log provided off-list shows a number of absolutely 
normal FOLDERSYNC commands and responses that pick up the new folders. 
Nothing abnormal.
07/01/2014 05:06:08 PM Michael Rubinsky Comment #7 Reply to this comment
Still cannot reproduce, even with email boxes containing a '.' as 
suggested in an off-list email.
07/01/2014 01:24:28 PM patrick (at) spamreducer (dot) eu Comment #6 Reply to this comment

[Show Quoted Text - 10 lines]
Problem Folder = "collection: INBOX/Maillisten/SQLgrey/2014.07"
In OL2013 it's displayed as "INBOX/Maillisten/SQLgrey/2014.07 [1]"


07/01/2014 01:21:47 PM patrick (at) spamreducer (dot) eu Comment #5 Reply to this comment
Cannot reproduce.
Hi guys,
happened again some minutes ago!

Clients log from today uploaded on dropbox: https://db.tt/XH0GWxJV

Please try to reproduce.. ;-)

Thanks!
06/23/2014 08:11:58 AM patrick (at) spamreducer (dot) eu Comment #4 Reply to this comment
Hi guys,
I don't know why, this morning after opening OL2013 again the same problem!

A folder has appeared as "foldername [1]"..

Could you retry my steps and wait for 1-2days?

I haven't opened OL2013 since saturday..

I'll switch on debugging on a daily base, when I catch the problem 
again I will upload new logs..
06/21/2014 04:31:12 PM Michael Rubinsky State ⇒ Not A Bug
 
06/21/2014 08:03:00 AM patrick (at) spamreducer (dot) eu Comment #3 Reply to this comment
Cannot reproduce.
You are TRUE mike!

Now, the problem has gone..
The problem arised with the "INBOX" subscriptions BUG Ticket 13261 
(Folder Inbox not shown in IMP)..

After updating to GIT the problem persists, I've to delete and 
recreate a new Outlook profile..
All the OL profiles have to be recreated if someone has worked with 
the buggy version prior fixing the "INBOX" subscription problem..
06/20/2014 08:26:58 PM Michael Rubinsky Comment #2
State ⇒ Feedback
Reply to this comment
Cannot reproduce.
06/20/2014 11:01:13 AM Jan Schneider Assigned to Michael Rubinsky
State ⇒ Assigned
Priority ⇒ 1. Low
 
06/18/2014 03:13:13 PM patrick (at) spamreducer (dot) eu Comment #1
Priority ⇒ 3. High
State ⇒ Unconfirmed
Patch ⇒ No
Milestone ⇒
Queue ⇒ Synchronization
Type ⇒ Bug
Summary ⇒ Outlook 2013 subfolder creation results in unusability of OL profile
Reply to this comment
Hey guys,
again me..

And again OL2013.. ;-)

Done this so far:

Updated Horde from 7b459d4 to d49790c.

1. Create a new folder "test" as a subfolder of "Mailinglisten"
2. Create a new folder "2014.06" as a subfolder of "test"
---------- Grafical explanation ----------
--INBOX
   --Mailinglisten
     --test
          --2014.06
------------------------------------------
3. Move some messages to this folder
4. OL2013 is now logging-in and off on the IMAP-Server (over horde 
ActiveSync) - this would never end, something like an infinite loop..
5. Close OL2013
6. Reopen OL2013
7. OL2013 now shows the folder "test [1]" instead of "test"
8. Folder "test [1]" with his subfolder "2014.06" is empty
9. Moved messages are not more available on filesystem, they where gone..
10. Try to rename folder "test [1]" to "test" - this is obviously not 
possible since the folder "test [1]" does not exists on IMAP, OL2013 
disconnects from ActiveSync..
11. Close OL2013

In Dovecot IMAP folder both folders are created as expected:
Jun 18 16:57 .INBOX.Maillisten.test/
Jun 18 16:57 .INBOX.Maillisten.test.2014\2e06/
..but no messages in it.

Logs are uploaded on dropbox (400K in size): https://db.tt/XH0GWxJV

Thanks!

Saved Queries