6.0.0-git
2019-05-19

[#13554] String contains non-ASCII characters.
Summary String contains non-ASCII characters.
Queue Horde Framework Packages
Queue Version Git master
Type Bug
State Resolved
Priority 2. Medium
Owners slusarz (at) horde (dot) org
Requester azurit (at) pobox (dot) sk
Created 2014-09-08 (1714 days ago)
Due
Updated 2014-09-17 (1705 days ago)
Assigned 2014-09-09 (1713 days ago)
Resolved 2014-09-16 (1706 days ago)
Milestone
Patch No

History
2014-09-17 07:38:25 Git Commit Comment #26 Reply to this comment
Changes have been made in Git (master):

commit 77acd38f939d3b1642ea72d210790466320f9a48
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Tue Sep 16 14:36:50 2014 -0600

     [mms] IMAP driver needs to explicitly search for UTF-8 text, 
since we don't restrict charset input at the UI level (Bug #13554).

     Conflicts:
             ingo/docs/CHANGES
             ingo/package.xml

  ingo/docs/CHANGES        |    3 +++
  ingo/lib/Script/Imap.php |    5 +++++
  ingo/package.xml         |    3 ++-
  3 files changed, 10 insertions(+), 1 deletions(-)

http://github.com/horde/horde/commit/77acd38f939d3b1642ea72d210790466320f9a48
2014-09-17 07:38:17 Git Commit Comment #25 Reply to this comment
Changes have been made in Git (master):

commit ecea895075319793c034a30a625be1bc5db60a6c
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Tue Sep 16 13:59:50 2014 -0600

     [mms] Better error handling when attempting to authenticate via 
IMAP LOGIN command with non-ASCII credentials (Bug #13554).

  .../Imap_Client/lib/Horde/Imap/Client/Socket.php   |   16 ++++++++++++++--
  framework/Imap_Client/package.xml                  |    2 ++
  2 files changed, 16 insertions(+), 2 deletions(-)

http://github.com/horde/horde/commit/ecea895075319793c034a30a625be1bc5db60a6c
2014-09-16 20:44:06 Michael Slusarz Comment #24
State ⇒ Resolved
Reply to this comment
Ingo 3.2.2

This is the best thing that could have happened.  IMAP driver was 
completely broken for non-ASCII searches in ingo since forever, so the 
recent change in Horde_Imap_Client easily/quickly identified that issue.

This still won't work as expected though.  e.g. blacklisting IDN 
addresses probably won't work due to the way IDN messages are 
sent/stored.  There's a bunch of recent RFCs involved with trying to 
handle this better and, in fact, I'm working with at least one Gmail 
guy right now in terms of trying to get more widespread support for 
non-ASCII data at both the client and server level.
2014-09-16 20:38:29 Git Commit Comment #23 Reply to this comment
Changes have been made in Git (FRAMEWORK_5_2):

commit bbdbdc665cb631104eaab305212384880d58b829
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Tue Sep 16 14:36:50 2014 -0600

     [mms] IMAP driver needs to explicitly search for UTF-8 text, 
since we don't restrict charset input at the UI level (Bug #13554).

  ingo/docs/CHANGES        |    2 ++
  ingo/lib/Script/Imap.php |    5 +++++
  ingo/package.xml         |    2 ++
  3 files changed, 9 insertions(+), 0 deletions(-)

http://github.com/horde/horde/commit/bbdbdc665cb631104eaab305212384880d58b829
2014-09-16 15:43:07 steven (at) tkfast (dot) com Comment #22 Reply to this comment
The user I have that has this problem has a blacklist filter that has 
the email
noreply@körung.com
in the blacklist list that causes the error.
2014-09-16 08:15:10 Michael Slusarz Comment #21 Reply to this comment
One of our users experienced this problem with apostrophe (´) in one 
of the filtering rules - don't know if this is ascii or non-ascii 
char.
That's not an apostrophe.  That's an acute diacritic and it is non-ASCII.
2014-09-16 07:58:02 azurit (at) pobox (dot) sk Comment #20 Reply to this comment
Happing when executing IMAP filter on login was the crucial
information. This needs to be investigated.
Between this report, and a report in #13555, can I safely assume 
that everybody that has this problem is both:
- using imap filtering in ingo
- has a non-ascii character in one of their filter strings?
One of our users experienced this problem with apostrophe (´) in one 
of the filtering rules - don't know if this is ascii or non-ascii char.

2014-09-16 07:54:01 Michael Slusarz Comment #19 Reply to this comment
Happing when executing IMAP filter on login was the crucial 
information. This needs to be investigated.
Between this report, and a report in #13555, can I safely assume that 
everybody that has this problem is both:
- using imap filtering in ingo
- has a non-ascii character in one of their filter strings?
2014-09-16 07:51:20 Michael Slusarz Comment #18 Reply to this comment
Then you posted an incorrect, unrelated backtrace, because it 
explicitly points to what I said. Plus, I can reproduce this as soon 
as I provide a non-ascii password.
This should be fixed to simply report a generic authentication 
failure.  Which should be done internally in Horde_Imap_Client.
2014-09-09 13:31:18 azurit (at) pobox (dot) sk Comment #17 Reply to this comment
I don't see how this "leaves the application in an useless state" though.
for a common user, when application shows big error message and all 
control elements disappears, this is 'useless state' which he/she 
doesn't know how to correct and is usually contacting technical support

2014-09-09 12:47:52 Jan Schneider Assigned to Michael Slusarz
State ⇒ Assigned
 
2014-09-09 12:46:53 Jan Schneider Comment #16 Reply to this comment
Hi Jan,

can you reproduce this cases too?

- folder name contain special chars
No.
- filter contain special chars
Yes.
Both cases weren't problematic before this commit (At least here). 
Now they brake the whole application.

We're using client side filtering (no sieve or similar). I'm not 
sure if the new sanitizing process is necessary in that case. From 
my view it should degrade graceful instead of just throwing an 
exception and leave the application in an useless state.
Happing when executing IMAP filter on login was the crucial 
information. This needs to be investigated.

I don't see how this "leaves the application in an useless state" though.
2014-09-09 12:41:57 azurit (at) pobox (dot) sk Comment #15 Reply to this comment
if i get back to the password you mentioned - just login with random 
login and random password which will contain unicode chars and the 
whole horde crashes. i don't think this is correct behavior of any 
application.
2014-09-09 12:38:21 Jan Schneider Comment #14 Reply to this comment
the mailing list is starting to be full of reports about this error 
and you are just saying there's no problem, how typical to Horde 
project..
No, but we're trying to be precise. And your report doesn't match what 
you said, and is indeed not a problem with Horde.
That doesn't mean that others may not weigh in with further reports, 
like it just happened.
2014-09-09 12:28:40 azurit (at) pobox (dot) sk Comment #13 Reply to this comment
the mailing list is starting to be full of reports about this error 
and you are just saying there's no problem, how typical to Horde 
project..
2014-09-09 12:24:13 marth (at) tsvschlieben (dot) de Comment #12 Reply to this comment
Hi Jan,

can you reproduce this cases too?

- folder name contain special chars
- filter contain special chars

Both cases weren't problematic before this commit (At least here). Now 
they brake the whole application.

We're using client side filtering (no sieve or similar). I'm not sure 
if the new sanitizing process is necessary in that case. From my view 
it should degrade graceful instead of just throwing an exception and 
leave the application in an useless state.
Then you posted an incorrect, unrelated backtrace, because it 
explicitly points to what I said. Plus, I can reproduce this as soon 
as I provide a non-ascii password.
2014-09-09 12:11:28 Jan Schneider Comment #11 Reply to this comment
Then you posted an incorrect, unrelated backtrace, because it 
explicitly points to what I said. Plus, I can reproduce this as soon 
as I provide a non-ascii password.
2014-09-09 11:25:17 azurit (at) pobox (dot) sk Comment #10 Reply to this comment
and i'm able to reproduce the problem exactly how i describe it in bug 
report. after i delete that folder (via shell), login start to work 
again.
2014-09-09 11:23:54 azurit (at) pobox (dot) sk Comment #9 Reply to this comment
my password contains only ascii letters and digits
2014-09-09 11:20:46 Jan Schneider Comment #8
Version ⇒ Git master
Queue ⇒ Horde Framework Packages
Reply to this comment
This is in Horde_Imap_Client.
2014-09-09 11:20:03 Jan Schneider Comment #7 Reply to this comment
This doesn't have anything to do with non-ascii folder names, but you 
are using a password that contains non-ascii characters which is not 
supported.
2014-09-09 07:14:19 marth (at) tsvschlieben (dot) de Comment #6 Reply to this comment
It seems, this commit breaks several other things too:

https://github.com/horde/horde/commit/10a822329f9fcaa8026de4c1e56de637554cf261

For exapmle if the Blacklist of an user contain non-ascii chars he's 
not able to access webmail anymore.
2014-09-08 19:04:26 azurit (at) pobox (dot) sk Comment #5 Reply to this comment
this is it?



2014-09-08T21:01:48+02:00 EMERG: HORDE [horde] String contains 
non-ASCII characters. [pid 25718 on line 73 of 
"/usr/share/php/Horde/Imap/Client/Data/Format/String.php"]
2014-09-08T21:01:48+02:00 DEBUG: HORDE  1. 
Horde_Core_Auth_Application->authenticate() 
/usr/share/horde/login.php:155
  2. Horde_Core_Auth_Application->authenticate() 
/usr/share/php/Horde/Core/Auth/Application.php:126
  3. Horde_Auth_Base->authenticate() 
/usr/share/php/Horde/Core/Auth/Application.php:129
  4. Horde_Core_Auth_Application->_authenticate() 
/usr/share/php/Horde/Auth/Base.php:160
  5. Horde_Registry->callAppMethod() 
/usr/share/php/Horde/Core/Auth/Application.php:158
  6. call_user_func_array() /usr/share/php/Horde/Registry.php:1201
  7. IMP_Application->authAuthenticate()
  8. IMP_Auth::authenticate() /usr/share/horde/imp/lib/Application.php:371
  9. IMP_Imap->login() /usr/share/horde/imp/lib/Auth.php:86
10. IMP_Imap->__call() /usr/share/horde/imp/lib/Auth.php:86
11. call_user_func_array() /usr/share/horde/imp/lib/Imap.php:718
12. Horde_Imap_Client_Base->login()
13. Horde_Imap_Client_Socket->_login() 
/usr/share/php/Horde/Imap/Client/Base.php:794
14. Horde_Imap_Client_Socket->_tryLogin() 
/usr/share/php/Horde/Imap/Client/Socket.php:497
15. Horde_Imap_Client_Data_Format_String->__construct() 
/usr/share/php/Horde/Imap/Client/Socket.php:730

2014-09-08 18:51:21 Jan Schneider Comment #4 Reply to this comment
Yes, the backtrace. Enable debug logging.
2014-09-08 17:06:44 azurit (at) pobox (dot) sk Comment #3 Reply to this comment
i can see only this in logs:
2014-09-04T18:13:37+02:00 EMERG: HORDE [imp] String contains non-ASCII 
characters. [pid 25850 on line 73 of 
"/usr/share/php/Horde/Imap/Client/Data/Format/String.php"]

do you need something more?

2014-09-08 16:56:35 Jan Schneider Comment #2
State ⇒ Feedback
Priority ⇒ 2. Medium
Reply to this comment
Backtrace?
2014-09-08 12:50:52 azurit (at) pobox (dot) sk Comment #1
Type ⇒ Bug
State ⇒ Unconfirmed
Priority ⇒ 3. High
Summary ⇒ String contains non-ASCII characters.
Queue ⇒ IMP
Milestone ⇒
Patch ⇒ No
Reply to this comment
While logging into IMP, this error is displayed and login is not possible:
String contains non-ASCII characters.

How to reproduce:
1.) Login into account.
2.) Create folder 'Odoslaná po?ta' (without quotes)
3.) Logout and try to login again.

Saved Queries