[#10462] search looping in IE 9
Summary search looping in IE 9
Queue IMP
Queue Version 5.0.11
Type Bug
State Resolved
Priority 3. High
Owners Michael Slusarz <slusarz (at) horde (dot) org>
Requester dom (dot) lalot (at) gmail (dot) com
Created 08/26/2011 (635 days ago)
Due
Updated 09/30/2011 (600 days ago)
Assigned 09/26/2011 (604 days ago)
Resolved 09/30/2011 (600 days ago)
Attachments debug-horde2.htm Download
debug-horde3.htm Download
Milestone 5.0.14
Patch No

History
09/30/2011 07:43:22 AM Michael Slusarz State ⇒ Resolved
 
09/26/2011 03:41:03 AM Git Commit Comment #25 Reply to this comment
Changes have been made in Git for this ticket:

Bug #10462: Many more dynamic view/base64url mailbox encoded fixes
Thought I had gone through all the non-message viewing code and done
this... guess not.

  1 files changed, 134 insertions(+), 119 deletions(-)
http://git.horde.org/horde-git/-/commit/8ff98d8fa0e4f256516cf77ec67e9504b5dbff1e
09/26/2011 02:50:28 AM Michael Slusarz Comment #24
Milestone ⇒ 5.0.14
State ⇒ Feedback		 Reply to this comment
Should be fixed for 5.0.14.  Will leave open for feedback/reports 
since it touches a lot of code.
09/26/2011 02:49:15 AM Git Commit Comment #23 Reply to this comment
Changes have been made in Git for this ticket:

Bug #10462: changelog entries

  2 files changed, 9 insertions(+), 3 deletions(-)
http://git.horde.org/horde-git/-/commit/ba0d765a1fd2833ec3bbcf2ba2b0f421c733eebb
09/26/2011 02:49:11 AM Git Commit Comment #22 Reply to this comment
Changes have been made in Git for this ticket:

Bug #10462: Don't use null characters in browser javascript

  22 files changed, 334 insertions(+), 365 deletions(-)
http://git.horde.org/horde-git/-/commit/26eba7b031f8a8a63b72a2f76cf2cd4726ae1de6
09/21/2011 03:50:35 PM Michael Slusarz Comment #21 Reply to this comment
I'd rather have this blocking on 5.0.12.  Since it essentially will 
break an installation (server-side) if using IE 9.
Nevermind - I see that .12 was already released.
09/21/2011 03:49:01 PM Michael Slusarz Comment #20 Reply to this comment
I'd rather have this blocking on 5.0.12.  Since it essentially will 
break an installation (server-side) if using IE 9.
09/21/2011 08:20:52 AM Jan Schneider Comment #19
Milestone ⇒ 5.0.13		 Reply to this comment
This has to wait for the next release, needs some testing anyway.
09/21/2011 07:15:42 AM dom (dot) lalot (at) gmail (dot) com Comment #18 Reply to this comment

[Hide Quoted Text]
It seems to me that you use the \0 as a delimiter between the mailbox
name and the message ID. If this is true, without refactoring all
your code, why not use another character for that ?
Because all other ASCII characters are valid IMAP mailbox characters.
Hi Mike,

We are waiing for a solution. We can't go in prod whith a loop like
this. It's eating resources! Is it difficult and have you an idea
about when it can be solved?
Yes, it is difficult.  I have to completely rewrite how we handle 
mailbox names in javascript.  Considering we use these name's 
everywhere, and there are thousands of line of javascript code, this 
is not trivial.

You can instead hack the code and force people using IE9 to use the 
traditional view instead.
Hi Mike,

Could you give me some advice. I suppose we have to dynamically lock   
a pref depending on the browser recognition and we should do that in a 
  place outside of horde internal code.

Thanks

Dom

09/21/2011 06:49:28 AM Michael Slusarz Comment #17
Milestone ⇒ 5.0.12		 Reply to this comment
Preliminary fix has been pushed to the imp-nulljsmbox branch.  As part 
of the refactor, the way messages are reported as deleted to the 
viewport has changed.  Good news: we will only need to report the list 
of deleted messages once (as opposed to twice currently).  Bad news: 
this doesn't work yet, since the browser cache is being purged every 
time we access the PHP server.  So more work to do before this is 
completed.
09/19/2011 04:07:17 PM Michael Slusarz Comment #16 Reply to this comment

[Hide Quoted Text]
It seems to me that you use the \0 as a delimiter between the mailbox
name and the message ID. If this is true, without refactoring all
your code, why not use another character for that ?
Because all other ASCII characters are valid IMAP mailbox characters.
Hi Mike,

We are waiing for a solution. We can't go in prod whith a loop like 
this. It's eating resources! Is it difficult and have you an idea 
about when it can be solved?
Yes, it is difficult.  I have to completely rewrite how we handle 
mailbox names in javascript.  Considering we use these name's 
everywhere, and there are thousands of line of javascript code, this 
is not trivial.

You can instead hack the code and force people using IE9 to use the 
traditional view instead.

09/19/2011 04:04:31 PM dom (dot) lalot (at) gmail (dot) com Comment #15 Reply to this comment
It seems to me that you use the \0 as a delimiter between the mailbox
name and the message ID. If this is true, without refactoring all
your code, why not use another character for that ?
Because all other ASCII characters are valid IMAP mailbox characters.
Hi Mike,

We are waiing for a solution. We can't go in prod whith a loop like 
this. It's eating resources! Is it difficult and have you an idea 
about when it can be solved?

Thanks

Dom
09/01/2011 04:21:25 PM Michael Slusarz Comment #14 Reply to this comment
It seems to me that you use the \0 as a delimiter between the 
mailbox name and the message ID. If this is true, without 
refactoring all your code, why not use another character for that ?
Because all other ASCII characters are valid IMAP mailbox characters.
09/01/2011 08:26:02 AM xavier (dot) montagutelli (at) unilim (dot) fr Comment #13 Reply to this comment
Excuse me if my thoughts miss the point.

It seems to me that you use the \0 as a delimiter between the mailbox 
name and the message ID. If this is true, without refactoring all your 
code, why not use another character for that ? The ASCII table has 
provisioned special characters for that purpose, more suitable than 
the NULL char

29 : group separator
30 : record separator
31 : unit separator


09/01/2011 05:54:14 AM Michael Slusarz Comment #12
Version ⇒ 5.0.11
Priority ⇒ 3. High		 Reply to this comment
Ugh... IE9 does not allow null characters in javascript string values 
/at all/.  Am going to have to refactor code to use the 
base64urlencoded mailbox value as the view name.
08/31/2011 10:36:56 PM xavier (dot) montagutelli (at) unilim (dot) fr Comment #11
New Attachment: debug-horde3.htm Download		 Reply to this comment
I have simplified the test.

The problem is that IMP sends NULL (\0) characters in the JSON flow. 
Without the \0, IE9 manages to decode the  JSON string. Bug in IE9 ?

08/31/2011 10:11:55 PM xavier (dot) montagutelli (at) unilim (dot) fr Comment #10
New Attachment: debug-horde2.htm Download		 Reply to this comment
Changes have been made in Git for this ticket:

Bug #10462: More workarounds for broken suhosin extension
Thank you, I will test this tomorrow.

But I don't think this bug is related to Suhoshin, as it is only in 
simulation mode on my server.

I have investigated a bit more. I have isolated a JSON response coming 
from my server after a search. It contains all relevant data. But IE9 
in strict mode doesn't seem to "decipher" correctly the JSON response. 
You will find attached a HTML file with the javascript code to 
demonstrate that : under chrome, it shows all the content of the 
"rowlist" inner object. Under IE9 strict mode, only the last item is 
shown !

The JSON string seems to be correct : http://jsonlint.com/
Can it be a bug in IE9 JSON.parse function ??
  6 files changed, 73 insertions(+), 79 deletions(-)
http://git.horde.org/horde-git/-/commit/90a41bc5b0f93701f543fa0ee45561ea3396b6f9
08/31/2011 08:55:32 PM Git Commit Comment #9 Reply to this comment
Changes have been made in Git for this ticket:

Bug #10462: More workarounds for broken suhosin extension

  6 files changed, 73 insertions(+), 79 deletions(-)
http://git.horde.org/horde-git/-/commit/90a41bc5b0f93701f543fa0ee45561ea3396b6f9
08/31/2011 05:16:31 PM Michael Slusarz Comment #8
Priority ⇒ 2. Medium
State ⇒ Assigned
Assigned to Michael Slusarz		 Reply to this comment
It loops when IE9 is in "native" mode. The search works if the 
browser is in "compatibility" mode.
Now we are getting somewhere.
Aug 31 11:57:32 webmail4 HORDE: [imp] Array#012(#012    [INBOX] => 
Array#012        (#012            [0] => NaN#012         
)#012#012)#012 [pid 29629 on line 247 of 
"/var/www/horde/imp/lib/Views/ListMessages.php"]
'NaN' indiciates that this is somehow related to IE9's javascript 
implementation.
08/31/2011 05:14:28 PM Michael Slusarz Comment #7 Reply to this comment
Aug 31 10:52:16 webmail4 suhosin[4957]: ALERT-SIMULATION - ASCII-NUL 
chars not allowed within request variables - dropped variable 'view' 
(attacker '164.81.3.151', file '/var/www/horde/services/ajax.php')
Don't use suhosin.  It breaks things.
08/31/2011 10:42:05 AM xavier (dot) montagutelli (at) unilim (dot) fr Comment #6 Reply to this comment
A few more information :
I have the same problem with IE9 under my installation.
It loops when IE9 is in "native" mode. The search works if the browser 
is in "compatibility" mode.
I also have the following logs :


Aug 31 10:52:15 webmail4 HORDE: [imp] PHP ERROR: array_flip(): Can 
only flip STRING and INTEGER values! [pid 12219 on line 248 of 
"/var/www/horde/imp/lib/Views/ListMessages.php"]
I added a "print_r" in this script. I get :

Aug 31 11:57:32 webmail4 HORDE: [imp] Array#012(#012    [INBOX] => 
Array#012        (#012            [0] => NaN#012        )#012#012)#012 
[pid 29629 on line 247 of 
"/var/www/horde/imp/lib/Views/ListMessages.php"]

This error does not appear using chrome.
Aug 31 10:52:16 webmail4 suhosin[4957]: ALERT-SIMULATION - ASCII-NUL 
chars not allowed within request variables - dropped variable 'view' 
(attacker '164.81.3.151', file '/var/www/horde/services/ajax.php')
I also get this message using chrome. It is related to MBOX_PREFIX 
ending with the NULL char in ./imp/lib/Search.php. Doesn't seem to be 
related with the problem.

The search doesn't loop when I search a non-existent pattern in my 
mailbox, or when there's only one result ...

The search returns something, I see it with the dev tools in IE9. But 
it loops. Example :

/*-secure-{"response":{"ViewPort":{"cacheid":"1","data":{"INBOX\u0000356703":{"flag":["list","unseen"],"imapuid":356703,"view":"INBOX","size":"6\u00a0Ko","date":"10:57:30","from":"Jan Schneider","subject":"Re: [dev] Horde License File Shipping and Horde License Header Cleanup","listmsg":1},"INBOX\u0000355303":{"flag":["\\seen","list"],"imapuid":355303,"view":"INBOX","size":"9\u00a0Ko","date":"23\/08\/2011","from":"Jan Schneider","subject":"Re: [dev] Horde_Ldap performance problem + patch","listmsg":1}},"label":"R\u00e9sultats de la recherche","metadata":{"nothread":1,"sortby":100,"sortdir":1,"search":1,"flags":["\\flagged","\\deleted","$junk","$notjunk","\\answered","\\draft","$forwarded","\\seen"],"slabel":"Recherche From (Header) for 'schneider' dans [Bo\u00eete de 
r\u00e9ception]","readonly":0},"rowlist":{"INBOX\u0000356703":1,"INBOX\u0000355303":2},"totalrows":2,"view":"impsearch\u0000dimpqsearch","requestid":315,"rownum":1,"search":1}}}*/
08/31/2011 09:02:37 AM xavier (dot) montagutelli (at) unilim (dot) fr Comment #5 Reply to this comment
I can not reproduce in either dynamic or standard view, doing
quicksearch (dynamic), advanced search (both), or basic search
(standard).  I did a simple search of my Inbox searching for a From
name,
I have the same problem with IE9 under my installation. I also have 
the following logs :


Aug 31 10:52:15 webmail4 HORDE: [imp] PHP ERROR: array_flip(): Can 
only flip STRING and INTEGER values! [pid 12219 on line 248 of 
"/var/www/horde/imp/lib/Views/ListMessages.php"]

Aug 31 10:52:16 webmail4 suhosin[4957]: ALERT-SIMULATION - ASCII-NUL 
chars not allowed within request variables - dropped variable 'view' 
(attacker '164.81.3.151', file '/var/www/horde/services/ajax.php')

Aug 31 10:52:16 webmail4 suhosin[4957]: ALERT-SIMULATION - ASCII-NUL 
chars not allowed within POST variables - dropped variable 'view' 
(attacker '164.81.3.151', file '/var/www/horde/services/ajax.php')

Suhoshin is in simulation mode only :
suhosin.simulation = On

and the search works under IE8.
08/31/2011 07:43:22 AM dom (dot) lalot (at) gmail (dot) com Comment #4 Reply to this comment
I can not reproduce in either dynamic or standard view, doing 
quicksearch (dynamic), advanced search (both), or basic search 
(standard).  I did a simple search of my Inbox searching for a From 
name,
As I was using my own auth driver, I went back to ldap auth, but 
symptom is the same

You can see the apache log. I am asking a colleague to do the same 
test. simple search in from. Chrome and FF are OK

Dom

61795054e5de1322bc16#bWJveDpJTkJPWA" "Mozilla/5.0 (compatible; MSIE 
9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:27:46 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2480 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:27:47 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2480 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:27:49 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2481 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:27:51 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2480 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:27:52 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2480 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:27:54 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2480 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:27:55 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2480 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:27:57 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2480 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:27:59 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2480 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:28:00 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2480 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:28:02 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2480 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:28:04 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2480 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:28:05 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2481 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:28:07 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2480 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:28:09 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2480 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:28:10 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2480 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:28:12 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2480 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:28:14 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2480 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"
139.124.208.6 - - [31/Aug/2011:09:28:16 +0200] "POST 
/horde/services/ajax.php/imp/viewPort HTTP/1.1" 200 2481 
"https://horde4.univmed.fr/horde/imp?u=18061795054e5de1322bc16#bWJveDpJTkJPWA" 
"Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"

08/31/2011 06:23:18 AM Michael Slusarz Comment #3 Reply to this comment
I can not reproduce in either dynamic or standard view, doing 
quicksearch (dynamic), advanced search (both), or basic search 
(standard).  I did a simple search of my Inbox searching for a From 
name,
08/30/2011 07:23:53 PM Michael Slusarz Comment #2 Reply to this comment
Possibly a javascript issue.

For further debugging information, please read 
http://www.horde.org/apps/horde/docs/INSTALL#dynamic-view-troubleshooting
08/26/2011 02:15:44 PM dom (dot) lalot (at) gmail (dot) com Comment #1
State ⇒ Unconfirmed
Patch ⇒ No
Milestone ⇒
Queue ⇒ IMP
Summary ⇒ search looping in IE 9
Type ⇒ Bug
Priority ⇒ 1. Low		 Reply to this comment
Hello,

Doing some tests, and trying under IE, the search is looping:

Consuming 30% CPU
Generating 731 messages found of the same message after 2 minutes

I just look the header From and try my name

Everything is fine under chrome and FF.
I don't normaly use IE, so it's standard configuration

Dom