6.0.0-git
2019-03-23

[#11083] UID search is non-strict
Summary UID search is non-strict
Queue Turba
Queue Version 3.0.12
Type Bug
State Resolved
Priority 2. Medium
Owners mrubinsk (at) horde (dot) org
Requester horde (at) kyoshiro (dot) org
Created 2012-03-17 (2562 days ago)
Due
Updated 2012-08-29 (2397 days ago)
Assigned
Resolved 2012-03-19 (2560 days ago)
Milestone
Patch No

History
2012-08-29 12:33:19 Git Commit Comment #14 Reply to this comment
Changes have been made in Git (master):

commit 8f9a27c43f01d717155249f7fe24b4e4aacff510
Author: Michael J Rubinsky <mrubinsk@horde.org>
Date:   Mon Mar 19 11:47:25 2012 -0400

     Bug: 11083 UID should always be a strict search.

  turba/config/backends.php |    8 +++++---
  1 files changed, 5 insertions(+), 3 deletions(-)

http://git.horde.org/horde-git/-/commit/8f9a27c43f01d717155249f7fe24b4e4aacff510
2012-03-20 10:06:37 horde (at) kyoshiro (dot) org Comment #13 Reply to this comment

[Show Quoted Text - 11 lines]
Thanks.

I checked the uid list and some were really missing. In fact, I tried 
a ldapsearch and saw a "size limit exceeded". My server was by default 
configured to limit any query to 500 results max.

I raised the value to 1000 but maybe there should be a notification 
somewhere (logs, user/admin feedback ?) so that people may fix the 
issue ? I might not be the only one having more than 500 contacts and 
a configured limit since it's debian's default ? ^^'

Anyways, now I've got my full contact list on my phone, thanks a lot 
for your help !
2012-03-19 17:29:35 Michael Rubinsky Comment #12 Reply to this comment
The serialized array in the horde_activesync_state table only holds 
the entries for items that have NOT yet been sync'd - not all entries 
are sent in a single request - we hold the pending items in that 
array, and they are removed once sent to the device. Once the initial 
sync is done, only items that have changed are sent to the device.

This initial list is obtained by calling 
$registry->contacts->listUids() - which returns the UIDs of all the 
contacts we want to send. We then call $registry->contacts->export() 
to fetch each entry.
2012-03-19 17:15:48 horde (at) kyoshiro (dot) org Comment #11 Reply to this comment
Hmm I don't know if it's related but some contacts are still missing. 
I can't find them in the serialized array logged in the dedicated 
logfile in DEBUG mode. I suspect they're not in the contacts list sent 
to my phone, and thus not synced.

I tried using the API to export() the corresponding uid, and it works. 
I also get the contacts whenever I do a Api::search('part of the 
name/uid'). How can I use the API to get the same contact list as the 
one sent by the rpc.php / Activesync module ?

Regards
2012-03-19 16:05:12 horde (at) kyoshiro (dot) org Comment #10 Reply to this comment

[Show Quoted Text - 11 lines]
Oh yes of course ! It's my fault too, I didn't really understood what 
was the meaning of the approximate and strict arrays / parameters in 
the configuration.

It's currently synchronizing, thanks a lot for your help :)
2012-03-19 15:50:35 Michael Rubinsky Comment #9
State ⇒ Resolved
Reply to this comment
Updated example/default configurations to reflect this.
2012-03-19 15:50:02 Michael Rubinsky Summary ⇒ UID search is non-strict
 
2012-03-19 15:49:17 Git Commit Comment #8 Reply to this comment
Changes have been made in Git (develop):

commit 8f9a27c43f01d717155249f7fe24b4e4aacff510
Author: Michael J Rubinsky <mrubinsk@horde.org>
Date:   Mon Mar 19 11:47:25 2012 -0400

     Bug: 11083 UID should always be a strict search.

  turba/config/backends.php |    8 +++++---
  1 files changed, 5 insertions(+), 3 deletions(-)

http://git.horde.org/horde-git/-/commit/8f9a27c43f01d717155249f7fe24b4e4aacff510
2012-03-19 15:44:52 Michael Rubinsky Comment #7 Reply to this comment
Ah, ok. That makes sense then.  The UID should be a strict search.

Probably haven't run into this issue since it would only be an issue 
if the search term contains spaces (if the search is "strict" then the 
string is split on spaces and an OR search is performed on the parts). 
  Typically UIDs would not have spaces which is why we haven't run 
into this earlier I guess.

The quick fix - and the correct fix too - is to add the field to the 
"strict" array in your backends.local.php file where your ldap source 
is defined.
2012-03-19 15:32:09 horde (at) kyoshiro (dot) org Comment #6 Reply to this comment

[Show Quoted Text - 18 lines]
I've got my answer, I think. I'm using cn as identifier (__uid)

When I look for Joris, the Turba_Api::export('Joris', 'text/x-vcard') 
returns these 2 similar entries :
cn: Joris
cn: zzz Old Joris

I think the uid search is not an exact seach but a "contains" search, 
which I suppose is not valid behavior.
But then again, I'm not an expert so maybe I'm wrong ?
Is there a quick way to fix this (other than searching for contacts 
having their "cn" contained in another contact's "cn" and changing 
their __uid) ?

Thanks for your support.
2012-03-19 08:59:58 horde (at) kyoshiro (dot) org Comment #5 Reply to this comment
Then you have to find out why Turba's search functions return more 
than one result for for the UID field see Turba_Api#export() in 
turba/lib/Api.php.
Ok I'll try to use these functions and I'll come back to you afterwards
Because you already have done the initial SYNC. After the first 
SYNC, only changes reported by the history system are taken into 
account.  You can clear the ActiveSync binding to your device from 
either the user ActiveSync prefs, or the device list in the 
administrative section. This will re-do the initial sync.
I tried reprovisionning devices and removing the device from the 
administrative section, but no luck. I suppose there still are issues 
with the results returned by my LDAP... or in the configuration.
2012-03-17 16:58:42 Michael Rubinsky Comment #4 Reply to this comment

[Show Quoted Text - 10 lines]
Then you have to find out why Turba's search functions return more 
than one result for for the UID field see Turba_Api#export() in 
turba/lib/Api.php.
Plus, at the moment I don't have this error in my log anymore, so I 
don't understand why it's not syncing :/
Because you already have done the initial SYNC. After the first SYNC, 
only changes reported by the history system are taken into account.   
You can clear the ActiveSync binding to your device from either the 
user ActiveSync prefs, or the device list in the administrative 
section. This will re-do the initial sync.

2012-03-17 16:30:25 horde (at) kyoshiro (dot) org Comment #3 Reply to this comment
The UID value MUST be a globally unique identifier. The error is 
telling you that more than one entry in your datastore has the same 
UID value.
Yes, but it is a unique identifier :
~$ grep 'cn:' slapcat.ldif  > cn.list
~$ grep 'cn:' slapcat.ldif |sort -u  > cn2.list
~$ wc -l cn*
   524 cn2.list
   524 cn.list

Plus, at the moment I don't have this error in my log anymore, so I 
don't understand why it's not syncing :/

LDAP is great for handling accounts and providing detailed information 
about each user. And in this case using "uid" as a globally unique 
identifier is easy. But in my case, I'm using LDAP as a storage for a 
personal LDAP, so these entries don't have any uid since they're not 
real users. I decided to use "cn" as identifier, and even if someday 
something could lead to a non unique issue, it's not the case at the 
moment.

I'm a bit lost, I don't really understand why - even now this issue is 
gone - sync is only working for a few contacts. You can see the log I 
attached in my previous post for more information.

Thanks
2012-03-17 16:02:36 Michael Rubinsky Comment #2
Assigned to Michael Rubinsky
State ⇒ Not A Bug
Reply to this comment
The UID value MUST be a globally unique identifier. The error is 
telling you that more than one entry in your datastore has the same 
UID value.
2012-03-17 15:14:14 horde (at) kyoshiro (dot) org Comment #1
Type ⇒ Bug
State ⇒ Unconfirmed
Priority ⇒ 2. Medium
Summary ⇒ Activesync showing only 6 contacts out of 400+
Queue ⇒ Turba
Milestone ⇒
Patch ⇒ No
New Attachment: androidc1376193414.txt Download
Reply to this comment
Hi,

I've been struggling for a few weeks now, trying to make contacts 
synchronization work via activesync on my Android phone (xPeria X10), 
without success. It seems to be able to synchronize 6 (random) 
contacts but no more.

To be more precise, I'm using an LDAP as container for a personal 
adressbook. I've setup turba to map :
          '__key' => 'dn',
         '__uid' => 'cn',
Entries look like this in my LDAP :
dn: cn=blah blah,ou=addressbook,cn=kyoshiro,ou=users,dc=kyoshiro,dc
  =org
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
givenName: blah
sn: blah
cn: blah blah
mail: ...

I had an issue in the dedicated log file for my device saying :
2012-03-17T13:29:38+01:00 ERR: Internal Horde Error: multiple turba 
objects with same objectId.
2012-03-17T13:29:38+01:00 DEBUG: [androidc1376193414] Saving state: Array
(
     [0] => {4f625e28-fca4-44da-9d51-40e958be147c}788
     [1] => a:494:{i:0;a:3:{s:2:"id";s:12:"Some 
Friend";s:4:"type";s:6:"change";s:5:"flags";s:10:"NewMessage";}i:1;...

     [2] => androidc1376193414
     [3] => 1331987377
     [4] => Contacts
     [5] => kyoshiro
)

I upgraded to the latest PEAR version and the serialize and ERR line 
seem not to appear in the logfiles anymore, but there's still only 6 
contacts on my phone.

I've attached the current log of what happens when I ask my device to 
resync contacts. Please let me know if you need more data on this issue.

Thanks !
Regards

Saved Queries