UID search is non-strict

Hi, I've been struggling for a few weeks now, trying to m

Sat, 17 Mar 2012 15:14:14 +0000

Hi, I've been struggling for a few weeks now, trying to make contacts synchronization work via activesync on my Android phone (xPeria X10), without success. It seems to be able to synchronize 6 (random) contacts but no more. To be more precise, I'm using an LDAP as container for a personal adressbook. I've setup turba to map : '__key' => 'dn', '__uid' => 'cn', Entries look like this in my LDAP : dn: cn=blah blah,ou=addressbook,cn=kyoshiro,ou=users,dc=kyoshiro,dc =org objectClass: top objectClass: person objectClass: organizationalPerson objectClass: inetOrgPerson givenName: blah sn: blah cn: blah blah mail: ... I had an issue in the dedicated log file for my device saying : 2012-03-17T13:29:38+01:00 ERR: Internal Horde Error: multiple turba objects with same objectId. 2012-03-17T13:29:38+01:00 DEBUG: [androidc1376193414] Saving state: Array ( [0] => {4f625e28-fca4-44da-9d51-40e958be147c}788 [1] => a:494:{i:0;a:3:{s:2:"id";s:12:"Some Friend";s:4:"type";s:6:"change";s:5:"flags";s:10:"NewMessage";}i:1;... [2] => androidc1376193414 [3] => 1331987377 [4] => Contacts [5] => kyoshiro ) I upgraded to the latest PEAR version and the serialize and ERR line seem not to appear in the logfiles anymore, but there's still only 6 contacts on my phone. I've attached the current log of what happens when I ask my device to resync contacts. Please let me know if you need more data on this issue. Thanks ! Regards

The UID value MUST be a globally unique identifier. The erro

Sat, 17 Mar 2012 16:02:36 +0000

The UID value MUST be a globally unique identifier. The error is telling you that more than one entry in your datastore has the same UID value.

> The UID value MUST be a globally unique identifier. The er

Sat, 17 Mar 2012 16:30:25 +0000

> The UID value MUST be a globally unique identifier. The error is > telling you that more than one entry in your datastore has the same > UID value. Yes, but it is a unique identifier : ~$ grep 'cn:' slapcat.ldif > cn.list ~$ grep 'cn:' slapcat.ldif |sort -u > cn2.list ~$ wc -l cn* 524 cn2.list 524 cn.list Plus, at the moment I don't have this error in my log anymore, so I don't understand why it's not syncing :/ LDAP is great for handling accounts and providing detailed information about each user. And in this case using "uid" as a globally unique identifier is easy. But in my case, I'm using LDAP as a storage for a personal LDAP, so these entries don't have any uid since they're not real users. I decided to use "cn" as identifier, and even if someday something could lead to a non unique issue, it's not the case at the moment. I'm a bit lost, I don't really understand why - even now this issue is gone - sync is only working for a few contacts. You can see the log I attached in my previous post for more information. Thanks

>> The UID value MUST be a globally unique identifier. The e

Sat, 17 Mar 2012 16:58:42 +0000

>> The UID value MUST be a globally unique identifier. The error is >> telling you that more than one entry in your datastore has the same >> UID value. > > Yes, but it is a unique identifier : > ~$ grep 'cn:' slapcat.ldif > cn.list > ~$ grep 'cn:' slapcat.ldif |sort -u > cn2.list > ~$ wc -l cn* > 524 cn2.list > 524 cn.list Then you have to find out why Turba's search functions return more than one result for for the UID field see Turba_Api#export() in turba/lib/Api.php. > Plus, at the moment I don't have this error in my log anymore, so I > don't understand why it's not syncing :/ Because you already have done the initial SYNC. After the first SYNC, only changes reported by the history system are taken into account. You can clear the ActiveSync binding to your device from either the user ActiveSync prefs, or the device list in the administrative section. This will re-do the initial sync.

> Then you have to find out why Turba's search functions r

Mon, 19 Mar 2012 08:59:58 +0000

> Then you have to find out why Turba's search functions return more > than one result for for the UID field see Turba_Api#export() in > turba/lib/Api.php. Ok I'll try to use these functions and I'll come back to you afterwards > Because you already have done the initial SYNC. After the first SYNC, > only changes reported by the history system are taken into account. > You can clear the ActiveSync binding to your device from either the > user ActiveSync prefs, or the device list in the administrative > section. This will re-do the initial sync. > I tried reprovisionning devices and removing the device from the administrative section, but no luck. I suppose there still are issues with the results returned by my LDAP... or in the configuration.

> >> Then you have to find out why Turba's search functions

Mon, 19 Mar 2012 15:32:09 +0000

> >> Then you have to find out why Turba's search functions return more >> than one result for for the UID field see Turba_Api#export() in >> turba/lib/Api.php. > > Ok I'll try to use these functions and I'll come back to you afterwards > >> Because you already have done the initial SYNC. After the first SYNC, >> only changes reported by the history system are taken into account. >> You can clear the ActiveSync binding to your device from either the >> user ActiveSync prefs, or the device list in the administrative >> section. This will re-do the initial sync. >> > > I tried reprovisionning devices and removing the device from the > administrative section, but no luck. I suppose there still are issues > with the results returned by my LDAP... or in the configuration. I've got my answer, I think. I'm using cn as identifier (__uid) When I look for Joris, the Turba_Api::export('Joris', 'text/x-vcard') returns these 2 similar entries : cn: Joris cn: zzz Old Joris I think the uid search is not an exact seach but a "contains" search, which I suppose is not valid behavior. But then again, I'm not an expert so maybe I'm wrong ? Is there a quick way to fix this (other than searching for contacts having their "cn" contained in another contact's "cn" and changing their __uid) ? Thanks for your support.

Ah, ok. That makes sense then. The UID should be a strict s

Mon, 19 Mar 2012 15:44:52 +0000

Ah, ok. That makes sense then. The UID should be a strict search. Probably haven't run into this issue since it would only be an issue if the search term contains spaces (if the search is "strict" then the string is split on spaces and an OR search is performed on the parts). Typically UIDs would not have spaces which is why we haven't run into this earlier I guess. The quick fix - and the correct fix too - is to add the field to the "strict" array in your backends.local.php file where your ldap source is defined.

Changes have been made in Git (develop): commit 8f9a27c43f0

Mon, 19 Mar 2012 15:49:17 +0000

Changes have been made in Git (develop): commit 8f9a27c43f01d717155249f7fe24b4e4aacff510 Author: Michael J Rubinsky Date: Mon Mar 19 11:47:25 2012 -0400 Bug: 11083 UID should always be a strict search. turba/config/backends.php | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) http://git.horde.org/horde-git/-/commit/8f9a27c43f01d717155249f7fe24b4e4aacff510

Updated example/default configurations to reflect this.

Mon, 19 Mar 2012 15:50:35 +0000

Updated example/default configurations to reflect this.

> Ah, ok. That makes sense then. The UID should be a strict

Mon, 19 Mar 2012 16:05:12 +0000

> Ah, ok. That makes sense then. The UID should be a strict search. > > Probably haven't run into this issue since it would only be an issue > if the search term contains spaces (if the search is "strict" then > the string is split on spaces and an OR search is performed on the > parts). Typically UIDs would not have spaces which is why we haven't > run into this earlier I guess. > > The quick fix - and the correct fix too - is to add the field to the > "strict" array in your backends.local.php file where your ldap source > is defined. Oh yes of course ! It's my fault too, I didn't really understood what was the meaning of the approximate and strict arrays / parameters in the configuration. It's currently synchronizing, thanks a lot for your help :)

Hmm I don't know if it's related but some contacts are still

Mon, 19 Mar 2012 17:15:48 +0000

Hmm I don't know if it's related but some contacts are still missing. I can't find them in the serialized array logged in the dedicated logfile in DEBUG mode. I suspect they're not in the contacts list sent to my phone, and thus not synced. I tried using the API to export() the corresponding uid, and it works. I also get the contacts whenever I do a Api::search('part of the name/uid'). How can I use the API to get the same contact list as the one sent by the rpc.php / Activesync module ? Regards

The serialized array in the horde_activesync_state table onl

Mon, 19 Mar 2012 17:29:35 +0000

The serialized array in the horde_activesync_state table only holds the entries for items that have NOT yet been sync'd - not all entries are sent in a single request - we hold the pending items in that array, and they are removed once sent to the device. Once the initial sync is done, only items that have changed are sent to the device. This initial list is obtained by calling $registry->contacts->listUids() - which returns the UIDs of all the contacts we want to send. We then call $registry->contacts->export() to fetch each entry.

> The serialized array in the horde_activesync_state table o

Tue, 20 Mar 2012 10:06:37 +0000

> The serialized array in the horde_activesync_state table only holds > the entries for items that have NOT yet been sync'd - not all entries > are sent in a single request - we hold the pending items in that > array, and they are removed once sent to the device. Once the initial > sync is done, only items that have changed are sent to the device. > > This initial list is obtained by calling > $registry->contacts->listUids() - which returns the UIDs of all the > contacts we want to send. We then call $registry->contacts->export() > to fetch each entry. Thanks. I checked the uid list and some were really missing. In fact, I tried a ldapsearch and saw a "size limit exceeded". My server was by default configured to limit any query to 500 results max. I raised the value to 1000 but maybe there should be a notification somewhere (logs, user/admin feedback ?) so that people may fix the issue ? I might not be the only one having more than 500 contacts and a configured limit since it's debian's default ? ^^' Anyways, now I've got my full contact list on my phone, thanks a lot for your help !

Changes have been made in Git (master): commit 8f9a27c43f01

Wed, 29 Aug 2012 12:33:19 +0000

Changes have been made in Git (master): commit 8f9a27c43f01d717155249f7fe24b4e4aacff510 Author: Michael J Rubinsky Date: Mon Mar 19 11:47:25 2012 -0400 Bug: 11083 UID should always be a strict search. turba/config/backends.php | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) http://git.horde.org/horde-git/-/commit/8f9a27c43f01d717155249f7fe24b4e4aacff510