6.0.0-git
2018-12-15

[#14039] Kolab: Initial folder sync is done twice
Summary Kolab: Initial folder sync is done twice
Queue Horde Framework Packages
Type Bug
State Stalled
Priority 1. Low
Owners mrubinsk (at) horde (dot) org
Requester thomas.jarosch (at) intra2net (dot) com
Created 2015-07-01 (1263 days ago)
Due
Updated 2015-07-23 (1241 days ago)
Assigned 2015-07-23 (1241 days ago)
Resolved 2015-07-23 (1241 days ago)
Milestone
Patch No

History
2015-07-23 16:57:22 Michael Rubinsky State ⇒ Stalled
 
2015-07-23 16:46:08 Thomas Jarosch Comment #6 Reply to this comment
Glad you found it.

I'd say "won'tfix" for now, the surgery needed to fix this
is probably quite a bit of labor. The modseq based sync is
a huge speed up, so let's try the new code in production first.

2015-07-23 15:15:59 Michael Rubinsky Comment #5
State ⇒ Feedback
Reply to this comment
Found it. It really has nothing to do with the sync strategy in use. 
The issue is that Kolab/Turba treat distribution lists and contacts as 
essentially two different data types. In Turba_Driver_Kolab::connect, 
each is fetched separately, and following the code down the stack for 
each $type we eventually get to Kolab_Storage_Base::getData(), which 
caches each $type separately. Therefore, even if we already 
synchronized the contact objects, when we query for the 
distribution-lists, this will create (or fetch an already created) a 
separate cache.

This is deep in the existing design of the storage library, not sure 
how to work around that if the two types need to be treated separately.
2015-07-23 04:07:33 Michael Rubinsky Comment #4
State ⇒ Assigned
Reply to this comment
Ok. Nevermind. I was testing by synchronizing with Outlook. This only 
apparently happens (at least with Turba) when the sync is triggered by 
using the Turba UI to browse an address book.
2015-07-23 03:39:05 Michael Rubinsky Comment #3 Reply to this comment
Are you sure you are using the code from Git master?
....or even the kolab topic branch we were using, as it is the same 
code that was merged into master.
2015-07-23 03:36:40 Michael Rubinsky Comment #2
State ⇒ Feedback
Reply to this comment
Spent some time trying to reproduce this to no avail.

Are you sure you are using the code from Git master?  I only see the 
synchronization taking place once - while watching both the activesync 
log and the general horde log where the KOLAB_STORAGE entries are 
logged. Also, the line numbers and files that the logging is done from 
in your log file don't match up at all and the text of the messages is 
slightly different than from our code.
2015-07-01 14:00:24 Thomas Jarosch Assigned to Michael Rubinsky
State ⇒ Assigned
 
2015-07-01 14:00:08 Thomas Jarosch Comment #1
Type ⇒ Bug
State ⇒ Unconfirmed
Priority ⇒ 1. Low
Summary ⇒ Kolab: Initial folder sync is done twice
Queue ⇒ Horde Framework Packages
Milestone ⇒
Patch ⇒ No
New Attachment: horde.log Download
Reply to this comment
Hi Michael,

using the new Token sync based strategy, I noticed the initial sync is 
done twice.
Looks like we don't store the initial token / stamp before doing 
another full, initial sync.

---------------------------------
[root@horde-git tmp]# egrep "Full|token" horde.log
2015-07-02T15:50:42+02:00 DEBUG: HORDE [turba] [KOLAB_STORAGE] 
Fetching stamp without token. [pid 25205 on line 248 of 
"/datastore/DEVEL/framework/Kolab_Storage/lib/Horde/Kolab/Storage/Data/Cached.php"]
2015-07-02T15:50:42+02:00 DEBUG: HORDE [turba] [KOLAB_STORAGE] Full 
folder sync details: user: mueller, folder: INBOX/Contacts, uids:   
[pid 25205 on line 248 of 
"/datastore/DEVEL/framework/Kolab_Storage/lib/Horde/Kolab/Storage/Data/Cached.php"]
2015-07-02T15:50:42+02:00 DEBUG: HORDE [turba] [KOLAB_STORAGE] Full 
Horde_History sync for user: mueller, folder: INBOX/Contacts, 
is_reset: 0, prefix: turba:sZ_eVtpyGKcNvfK4lzOlMs6: [pid 25205 on line 
73 of 
"/datastore/DEVEL/framework/Kolab_Storage/lib/Horde/Kolab/Storage/Data/Query/History/Cache.php"]
2015-07-02T15:50:42+02:00 DEBUG: HORDE [turba] [KOLAB_STORAGE] 
Fetching stamp without token. [pid 25205 on line 248 of 
"/datastore/DEVEL/framework/Kolab_Storage/lib/Horde/Kolab/Storage/Data/Cached.php"]
2015-07-02T15:50:42+02:00 DEBUG: HORDE [turba] [KOLAB_STORAGE] Full 
folder sync details: user: mueller, folder: INBOX/Contacts, uids:   
[pid 25205 on line 248 of 
"/datastore/DEVEL/framework/Kolab_Storage/lib/Horde/Kolab/Storage/Data/Cached.php"]
---------------------------------


The full sync request is probably in the code for ages but was never 
noticed before:
The time based sync triggers only every XX seconds and the session 
based sync triggers one per login only.

Personally I don't mind triggering multiple sync requests from the 
specific Kolab drivers right now, we should just make use of the token 
/ stamp if we already have it. Otherwise huge folders will be twice as 
slow to sync as they need to be :)

Attached is a debug log from a newly created user "mueller" + 
completely cleared Horde_Cache.

Cheers,
Thomas

Saved Queries