6.0.0-git
2019-03-23

[#12480] Autocomplete includes all addresses for a user where any of them match
Summary Autocomplete includes all addresses for a user where any of them match
Queue Horde Framework Packages
Queue Version Git master
Type Enhancement
State Resolved
Priority 1. Low
Owners slusarz (at) horde (dot) org
Requester simon (at) simonandkate (dot) net
Created 2013-07-19 (2073 days ago)
Due
Updated 2016-07-04 (992 days ago)
Assigned
Resolved 2014-07-28 (1699 days ago)
Milestone
Patch No

History
2016-07-04 14:48:22 pascal (dot) rigaux (at) univ-paris1 (dot) fr Comment #28 Reply to this comment
I suggest a commit limiting the filtering on e-mail addresses :

https://github.com/horde/horde/pull/197

restrict post search "filtering on e-mail addresses matching search 
string" when there are multiple e-mail adresses per result

Rationale: there is no reason to completely discard some search results.
Example: result for "Rigaux Pascal" (that matches our LDAP "cn") are 
discarded because my e-mail address is "Pascal.Rigaux@univ-paris1.fr".

Limitation: what should be done if 2 e-mail addresses are returned, 
none matching the search string?
Better fix would be to include all e-mails, but sort them?

2015-06-14 21:18:53 l (dot) kiraly (at) madalbal (dot) hu Comment #27 Reply to this comment
This commit breaks the behaviour to display all contacts when
composing an email and then click on the "To" link.
Verified and fixed.
This fix prevents searching by alias name, which was worked before.
2014-08-07 18:15:31 Michael Slusarz Comment #26 Reply to this comment
This commit breaks the behaviour to display all contacts when 
composing an email and then click on the "To" link.
Verified and fixed.
2014-08-07 18:15:13 Git Commit Comment #25 Reply to this comment
Changes have been made in Git (master):

commit 8c16e12c176e17c394452258ff8d6a5c8a71cd05
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Thu Aug 7 12:04:50 2014 -0600

     Bug #12480: Don't do email filter on name if name is not part of 
the search result

  turba/lib/Api.php |   30 ++++++++++++++++++------------
  1 files changed, 18 insertions(+), 12 deletions(-)

http://github.com/horde/horde/commit/8c16e12c176e17c394452258ff8d6a5c8a71cd05
2014-08-07 18:11:23 Git Commit Comment #24 Reply to this comment
Changes have been made in Git (FRAMEWORK_5_2):

commit 3928f11a32c030edc92f020eac769edec033af71
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Thu Aug 7 12:04:50 2014 -0600

     Bug #12480: Don't do email filter on name if name is not part of 
the search result

  turba/lib/Api.php |   30 ++++++++++++++++++------------
  1 files changed, 18 insertions(+), 12 deletions(-)

http://github.com/horde/horde/commit/3928f11a32c030edc92f020eac769edec033af71
2014-08-07 10:41:16 marth (at) tsvschlieben (dot) de Comment #23 Reply to this comment
This commit breaks the behaviour to display all contacts when 
composing an email and then click on the "To" link.

Old: You see all contacts (until you filter them with your own search rules)
New: You see no contacts (until you filter them with your own search rules)

Proposed fix (Line 1275ff)

OLD:
if ((stripos($e_val, $name) !== false) ||
  (stripos($display_name, $name) !== false)) {

NEW:
if ((strlen($name) == 0) ||
  (stripos($e_val, $name) !== false) ||
  (stripos($display_name, $name) !== false)) {

However I'm not sure if that breaks something else - it just restores 
the old (and wanted) behaviour in compose message view.

[Show Quoted Text - 15 lines]
2014-08-06 00:08:35 simon (at) simonandkate (dot) net Comment #22 Reply to this comment
Temp fix in Turba 4.2.1.

Better way of fixing in future is to handle e-mail searches in a 
separate API search. See #10913.
I'll check the log for what is actually being sent in the next couple 
of days, but with 4.2.1 it is now only returning a match if it matches 
the primary email address, not the others at all.

So if I type deb.tony which is the start of my sister's personal email 
address, as it goes through 'deb' it shows her work address 
(deborah...) which is in the primary email address field, then when I 
hit the '.' for deb.tony all hints disappear.
2014-08-05 23:02:57 Git Commit Comment #21 Reply to this comment
Changes have been made in Git (master):

commit e6615d84fc8c14f2565b057721ffbf7f2ab446d7
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Tue Aug 5 16:38:57 2014 -0600

     For e-mail search, use transliteration to broaden result pool

     This filtering doesn't have to be exact, and I've been convinced by
     those that use languages with non-ascii characters that this is the
     proper (or, at least, better) behavior.

     See request #12480

  turba/lib/Api.php |   16 +++++++++++++---
  1 files changed, 13 insertions(+), 3 deletions(-)

http://github.com/horde/horde/commit/e6615d84fc8c14f2565b057721ffbf7f2ab446d7
2014-08-05 22:40:45 Git Commit Comment #20 Reply to this comment
Changes have been made in Git (FRAMEWORK_5_2):

commit c1dd0d496b5de85659e4316d9b4f76bc98d84c3e
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Tue Aug 5 16:38:57 2014 -0600

     For e-mail search, use transliteration to broaden result pool

     This filtering doesn't have to be exact, and I've been convinced by
     those that use languages with non-ascii characters that this is the
     proper (or, at least, better) behavior.

     See request #12480

  turba/lib/Api.php |   16 +++++++++++++---
  1 files changed, 13 insertions(+), 3 deletions(-)

http://github.com/horde/horde/commit/c1dd0d496b5de85659e4316d9b4f76bc98d84c3e
2014-08-05 19:32:56 Git Commit Comment #19 Reply to this comment
Changes have been made in Git (master):

commit 76f51c2f57110e2d694e369629e7b6553603856b
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Tue Aug 5 13:31:19 2014 -0600

     [mms] Make case-insensitive API e-mail search work on non-ASCII 
data (Request #12480).

     Conflicts:
             turba/docs/CHANGES
             turba/package.xml

  turba/docs/CHANGES |    9 +++++++++
  turba/lib/Api.php  |    6 +++---
  turba/package.xml  |    7 ++++---
  3 files changed, 16 insertions(+), 6 deletions(-)

http://github.com/horde/horde/commit/76f51c2f57110e2d694e369629e7b6553603856b
2014-08-05 19:31:33 Git Commit Comment #18 Reply to this comment
Changes have been made in Git (FRAMEWORK_5_2):

commit 4c64ffc02e94acbe7cca9338f62d03f2d4208fbe
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Tue Aug 5 13:31:19 2014 -0600

     [mms] Make case-insensitive API e-mail search work on non-ASCII 
data (Request #12480).

  turba/docs/CHANGES |    2 ++
  turba/lib/Api.php  |    6 +++---
  turba/package.xml  |    4 +++-
  3 files changed, 8 insertions(+), 4 deletions(-)

http://github.com/horde/horde/commit/4c64ffc02e94acbe7cca9338f62d03f2d4208fbe
2014-07-28 06:28:29 Michael Slusarz Comment #17
Assigned to Michael Slusarz
State ⇒ Resolved
Reply to this comment
Temp fix in Turba 4.2.1.

Better way of fixing in future is to handle e-mail searches in a 
separate API search. See #10913.
2014-07-28 06:24:50 Git Commit Comment #16 Reply to this comment
Changes have been made in Git (master):

commit a58ffee8f7430324467a11b018446019888acead
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Mon Jul 28 00:21:56 2014 -0600

     [mms] Only include e-mail addresses that match the search string 
in the return from the search() API call (Request #12480).

     Conflicts:
             turba/docs/CHANGES
             turba/package.xml

  turba/docs/CHANGES |    4 +++-
  turba/lib/Api.php  |   26 ++++++++++++++++++--------
  turba/package.xml  |    1 +
  3 files changed, 22 insertions(+), 9 deletions(-)

http://github.com/horde/horde/commit/a58ffee8f7430324467a11b018446019888acead
2014-07-28 06:24:31 Git Commit Comment #15 Reply to this comment
Changes have been made in Git (FRAMEWORK_5_2):

commit 854721c2298a1b4de149b0b12523119f3dec203d
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Mon Jul 28 00:21:56 2014 -0600

     [mms] Only include e-mail addresses that match the search string 
in the return from the search() API call (Request #12480).

  turba/docs/CHANGES |    2 ++
  turba/lib/Api.php  |   26 ++++++++++++++++++--------
  turba/package.xml  |    2 ++
  3 files changed, 22 insertions(+), 8 deletions(-)

http://github.com/horde/horde/commit/854721c2298a1b4de149b0b12523119f3dec203d
2014-07-11 14:33:31 Michael Slusarz Version ⇒ Git master
Queue ⇒ Horde Framework Packages
State ⇒ Accepted
 
2014-07-10 00:34:55 simon (at) simonandkate (dot) net Comment #14 Reply to this comment

[Show Quoted Text - 15 lines]
Just noticed it's already an enhancement request... :) Any chance of 
it getting accepted?

Thanks Michael.
2014-07-10 00:31:43 simon (at) simonandkate (dot) net Comment #13 Reply to this comment

[Show Quoted Text - 9 lines]
Which is I imagine way beyond my limited skills... :(

Can this ticket be changed to an enhancement request? It seems like a 
fairly useful capability to me for auto complete to work with whatever 
email addresses are configured to search...
2014-07-09 21:43:08 Michael Slusarz Comment #12 Reply to this comment
How do I get it to when it matches, DISPLAY object_homeemail and 
object_workemail if it is matching one of them instead of only 
object_email?
Pretty sure you are going to have to edit the autocomplete code 
(believe it is in Horde_Core_Ajax_Imple_ContactAutoCompleter) to have 
it return these extra fields (right now it only returns 'name' and 
'email').  Then you will have to write code to parse the search 
results to pull email addresses out of these fields.
2014-07-09 13:33:51 simon (at) simonandkate (dot) net Comment #11 Reply to this comment
OK, finally. :)

After upgrading everything to x.2 I have come back to this, and my 
Turba install is now aligned with the object_email, object_homeemail, 
object_workemail standard that is now the default.

Typing in "deb.ton" to a new email should match object_homeemail for 
my sister. Auto-complete in Imp does this query:

2014-07-09T22:47:26+10:00 DEBUG: HORDE [turba] SQL  (0.0041s)
        SELECT object_id, object_type, owner_id, object_members,
          object_lastname, object_firstname, object_middlenames,
          object_namesuffix, object_email FROM turba_objects WHERE (owner_id =
          'simon' AND ((LOWER(object_lastname) LIKE LOWER('%deb.ton%') OR
          LOWER(object_firstname) LIKE LOWER('%deb.ton%') OR
          LOWER(object_middlenames) LIKE LOWER('%deb.ton%') OR
          LOWER(object_namesuffix) LIKE LOWER('%deb.ton%')) OR
          LOWER(object_email) LIKE LOWER('%deb.ton%') OR LOWER(object_homeemail)
          LIKE LOWER('%deb.ton%') OR LOWER(object_workemail) LIKE
          LOWER('%deb.ton%'))) [pid 9860 on line 208 of 
"/usr/share/pear/Horde/Db/Adapter/Pdo/Base.php"]
2014-07-09T22:47:26+10:00 DEBUG: HORDE [turba] LDAP query by 
Turba_Driver_ldap::_search(): user = simon, root = 
ou=shared_addressbook,dc=simonandkate,dc=lan (ldap.simonandkate.lan); 
filter = "(|(cn=*deb.ton*)(mail=*deb.ton*))"; attributes = "dn, 
turbaType, turbaMembers, cn, mail, sn"; deref = "0"  ; sizelimit = 200 
[pid 9860 on line 203 of "/var/www/horde/turba/lib/Driver/Ldap.php"]

This is matching correctly, finding "deb.ton" in object_homeemail - 
which I know because an auto-complete entry is shown. But - the 
auto-complete prompt in Imp presents me with ONLY the contents of 
object_email to select. That single email suggestion remains as long 
as I type a valid email for her, even if it is matching one that is 
not shown.

I have the following in my Imp pref for search:

simon        imp        search_fields        {"mdmVRibMvw2mw-W5vSljVQ1":["name","email","homeEmail","workEmail"],"localldap":["name","email"],"favourites":["name","email"]}

It is searching the right fields, matching them (because the AC entry 
remains visible as long as one of the underlying emails matches what 
is being typed) but it only ever shows the email address that is in 
object_email.

How do I get it to when it matches, DISPLAY object_homeemail and 
object_workemail if it is matching one of them instead of only 
object_email?
2014-02-17 01:58:46 Michael Rubinsky State ⇒ No Feedback
 
2013-07-31 11:17:11 simon (at) simonandkate (dot) net Comment #10 Reply to this comment
Thanks, I will sort out on the weekend and report back on success. 
Concentrating on Bug 12486 at the moment.
2013-07-30 06:11:43 Michael Slusarz Comment #9 Reply to this comment
So if I have email addresses in object_email, object_homeemail, 
object_workemail (is that what the new ones are called?), will 
autocomplete in Imp search all (if those fields specified in Mail 
Prefs), and return all matching objects?
Yes, if that's how you have configured Turba's backend.  But turba 
isn't going to return these values unless you've configured it to do so.

So I guess I'm still not seeing the issue(?)
2013-07-26 05:57:41 simon (at) simonandkate (dot) net Comment #8 Reply to this comment
So where/why are you defining object_email2 and object_email3?  That 
doesn't appear anywhere in Turba (or Horde for that matter).  We 
only use object_email.

I would guess you have a badly outdated backends.local.php config.
http://wiki.horde.org/TurbaExtraFields

I wrote that 2 years ago when the extra email addresses were not 
supported through ActiveSync, to give full support for three addresses 
through Horde / ActiveSync / iOS.

Upgrades along the way have not broken anything with that 
configuration that I setup, so I have never needed to change it.

I will look through the newer setup and see about migrating the 
addresses into the newer objects.

So if I have email addresses in object_email, object_homeemail, 
object_workemail (is that what the new ones are called?), will 
autocomplete in Imp search all (if those fields specified in Mail 
Prefs), and return all matching objects?

Thanks.
2013-07-25 19:15:47 Michael Slusarz Comment #7
Version ⇒ Git master
Queue ⇒ Turba
Reply to this comment
So where/why are you defining object_email2 and object_email3?  That 
doesn't appear anywhere in Turba (or Horde for that matter).  We only 
use object_email.

I would guess you have a badly outdated backends.local.php config.
2013-07-25 11:33:06 simon (at) simonandkate (dot) net Comment #6 Reply to this comment
I got it...

I see this when typing "simona" in for AC:

2013-07-25T10:56:12+00:00 DEBUG: HORDE [turba] SQL  (0.0023s)
        SELECT object_id, object_type, owner_id, object_members,
          object_lastname, object_firstname, object_middlenames,
          object_namesuffix, object_email FROM turba_objects WHERE (owner_id =
          'simon' AND ((LOWER(object_lastname) LIKE LOWER('%simona%') OR
          LOWER(object_firstname) LIKE LOWER('%simona%') OR
          LOWER(object_middlenames) LIKE LOWER('%simona%') OR
          LOWER(object_namesuffix) LIKE LOWER('%simona%')) OR
          LOWER(object_email) LIKE LOWER('%simona%') OR LOWER(object_email2)
          LIKE LOWER('%simona%') OR LOWER(object_email3) LIKE
          LOWER('%simona%'))) [pid 12252 on line 558 of 
"/usr/share/pear/Horde/Db/Adapter/Base.php"]
2013-07-25T10:56:12+00:00 DEBUG: HORDE [turba] LDAP query by 
Turba_Driver_ldap::_search(): user = simon, root = 
ou=shared_addressbook,dc=simonandkate,dc=lan (ldap.simonandkate.lan); 
filter = "(|(cn=*simona*)(mail=*simona*))"; attributes = "dn, 
turbaType, turbaMembers, cn, mail, sn"; deref = "0"  ; sizelimit = 200 
[pid 12252 on line 195 of "/var/www/horde/turba/lib/Driver/Ldap.php"]

The "aquenta" email address that is being returned in the graphic 
attached to this ticket is located in SQL object_email only, not in 
LDAP at all.

Manually executing that SQL statement shows me that the script is 
matching the "simona" as LIKE against object_email2 for Katie (where 
that address resides), but then the SELECT statement is only returning 
object_email, which contains the aquenta email address.

If I exclude the LDAP directory from the search, I see object_email 
returned for the two SQL address book entries that match:

Mine, where object_email is simon AT simonandkate.net, and katie's 
where it is AT aquenta..., thus returning an email address that does 
not match the search query. So I have no way of auto-completing 
Katie's simonandkate email address unless I move it to object_email 
instead of object_email2, despite that AC actually searches them all 
for matches.



2013-07-23 19:22:09 Michael Slusarz Comment #5 Reply to this comment
I can verify that this works fine for me.  Here's the SQL statement 
that appears in the DEBUG logs (searching for the string "test"):

2013-07-23T13:21:01-06:00 DEBUG: HORDE [turba] SQL  (0.0144s)
         SELECT object_id, object_type, owner_id, object_members,
           object_nameprefix, object_firstname, object_middlenames,
           object_lastname, object_namesuffix, object_email FROM turba_objects
           WHERE (owner_id = 'slusarz' AND (object_firstname ILIKE '%test%' OR
           object_lastname ILIKE '%test%' OR object_middlenames ILIKE 
'%test%' OR
           object_nameprefix ILIKE '%test%' OR object_namesuffix ILIKE '%test%'
           OR (object_nameprefix ILIKE '%test%' OR object_firstname ILIKE
           '%test%' OR object_middlenames ILIKE '%test%' OR 
object_lastname ILIKE
           '%test%' OR object_namesuffix ILIKE '%test%') OR object_alias ILIKE
           '%test%' OR object_yomifirstname ILIKE '%test%' OR 
object_yomilastname
           ILIKE '%test%' OR object_bday ILIKE '%test%' OR object_anniversary
           ILIKE '%test%' OR object_spouse ILIKE '%test%' OR object_phototype
           ILIKE '%test%' OR object_homestreet ILIKE '%test%' OR object_homepob
           ILIKE '%test%' OR object_homecity ILIKE '%test%' OR
           object_homeprovince ILIKE '%test%' OR object_homepostalcode ILIKE
           '%test%' OR object_homecountry ILIKE '%test%' OR (object_nameprefix
           ILIKE '%test%' OR object_firstname ILIKE '%test%' OR
           object_middlenames ILIKE '%test%' OR object_lastname ILIKE 
'%test%' OR
           object_namesuffix ILIKE '%test%' OR object_homestreet ILIKE '%test%'
           OR object_homecity ILIKE '%test%' OR object_homeprovince ILIKE
           '%test%' OR object_homepostalcode ILIKE '%test%') OR 
object_workstreet
           ILIKE '%test%' OR object_workpob ILIKE '%test%' OR object_workcity
           ILIKE '%test%' OR object_workprovince ILIKE '%test%' OR
           object_workpostalcode ILIKE '%test%' OR object_workcountry ILIKE
           '%test%' OR (object_nameprefix ILIKE '%test%' OR object_firstname
           ILIKE '%test%' OR object_middlenames ILIKE '%test%' OR 
object_lastname
           ILIKE '%test%' OR object_namesuffix ILIKE '%test%' OR
           object_homestreet ILIKE '%test%' OR object_homecity ILIKE 
'%test%' OR
           object_homeprovince ILIKE '%test%' OR object_homepostalcode ILIKE
           '%test%' OR object_workstreet ILIKE '%test%' OR 
object_workcity ILIKE
           '%test%' OR object_workprovince ILIKE '%test%' OR
           object_workpostalcode ILIKE '%test%') OR object_otherstreet ILIKE
           '%test%' OR object_otherpob ILIKE '%test%' OR object_othercity ILIKE
           '%test%' OR object_otherprovince ILIKE '%test%' OR
           object_otherpostalcode ILIKE '%test%' OR object_othercountry ILIKE
           '%test%' OR (object_nameprefix ILIKE '%test%' OR object_firstname
           ILIKE '%test%' OR object_middlenames ILIKE '%test%' OR 
object_lastname
           ILIKE '%test%' OR object_namesuffix ILIKE '%test%' OR
           object_homestreet ILIKE '%test%' OR object_homecity ILIKE 
'%test%' OR
           object_homeprovince ILIKE '%test%' OR object_homepostalcode ILIKE
           '%test%' OR object_workstreet ILIKE '%test%' OR 
object_workcity ILIKE
           '%test%' OR object_workprovince ILIKE '%test%' OR
           object_workpostalcode ILIKE '%test%' OR object_otherstreet ILIKE
           '%test%' OR object_othercity ILIKE '%test%' OR object_otherprovince
           ILIKE '%test%' OR object_otherpostalcode ILIKE '%test%') OR
           object_department ILIKE '%test%' OR object_manager ILIKE '%test%' OR
           object_assistant ILIKE '%test%' OR object_tz ILIKE '%test%' OR
           object_email ILIKE '%test%' OR object_homephone ILIKE '%test%' OR
           object_homephone2 ILIKE '%test%' OR object_homefax ILIKE '%test%' OR
           object_workphone ILIKE '%test%' OR object_workphone2 ILIKE 
'%test%' OR
           object_cellphone ILIKE '%test%' OR object_carphone ILIKE '%test%' OR
           object_radiophone ILIKE '%test%' OR object_companyphone 
ILIKE '%test%'
           OR object_assistantphone ILIKE '%test%' OR object_fax ILIKE '%test%'
           OR object_pager ILIKE '%test%' OR object_title ILIKE '%test%' OR
           object_role ILIKE '%test%' OR object_company ILIKE '%test%' OR
           object_logotype ILIKE '%test%' OR object_category ILIKE '%test%' OR
           object_notes ILIKE '%test%' OR object_url ILIKE '%test%' OR
           object_freebusyurl ILIKE '%test%' OR object_pgppublickey ILIKE
           '%test%' OR object_smimepublickey ILIKE '%test%' OR object_imaddress
           ILIKE '%test%' OR object_imaddress2 ILIKE '%test%' OR
           object_imaddress3 ILIKE '%test%')) [pid 7223 on line 558 of 
"/disk2/src/horde/framework/Db/lib/Horde/Db/Adapter/Base.php"]

2013-07-22 00:04:23 simon (at) simonandkate (dot) net Comment #4 Reply to this comment
I have an SQL backend for Turba, with the primary address book having 
three email address fields selected for name matching in Imp 
preferences. I wonder if it's something to do with that multiple match?
2013-07-19 15:28:34 Jan Schneider Comment #3 Reply to this comment
Cannot reproduce. I actually don't even get matches for email 
addresses at the moment.
Nevermind, I expected the AC to default to searching all available 
search fields, it defaults to name only though. Still cannot reproduce 
what you see though.
2013-07-19 15:23:32 Jan Schneider Comment #2
State ⇒ Feedback
Reply to this comment
Cannot reproduce. I actually don't even get matches for email 
addresses at the moment.
2013-07-19 12:12:26 simon (at) simonandkate (dot) net Comment #1
Type ⇒ Enhancement
State ⇒ New
Priority ⇒ 1. Low
Summary ⇒ Autocomplete includes all addresses for a user where any of them match
Queue ⇒ IMP
Milestone ⇒
Patch ⇒ No
New Attachment: ac.png Download
Reply to this comment
See attached example - typing simonandkate returns both Katie's email 
addresses, even the one that does not include that string. Same with 
names, not just domains.

Not too much of an issue in a small address book, but in a larger one 
this could be troublesome.

Ideally the a/c should only return addresses that match the entered string.

Saved Queries