Changes have been made in Git for this ticket:

Bug #9842: Base64url encode mailbox names in form data

  22 files changed, 152 insertions(+), 122 deletions(-)
http://git.horde.org/horde-git/-/commit/32c3fac7b3a089cd7914cfdd7578d5e8775f0d15

No.  I committed a prior patch to the imp_5_1 branch, but then deleted 
the branch.  I don't really understand the concept of the develop 
branch at this time, so I did not repush there.

What you are probably seeing is base64url encoding of dimp's hash 
(breadcrumb) information.  But this has nothing to do with encoding 
mailbox names in form data/url links.

Unless I'm missing something, you *did* already commit this. At least 
my URL hashes in DIMP are all encoded since a few days now.

Changing milestone.  Will commit to master once 5.0.4 is released.

Unfortunately, it appears that debian may ship PHP by default with 
suhosin.  Which is stupid but whatever.  (The benefits of suhosin (are 
there any?) are entirely debatable.  Completely breaking 
functionality, in the name of quote security unquote, is not a valid 
solution.  But I guess you throw the security term around and people 
will pick up on it as a buzz word so they will use your product, no 
matter what it actually does.  Off soapbox).

So to avoid IMP being broken on all debian servers by default, and to 
work around this issue, we will instead always pass around the mailbox 
in form data (both POST and GET) as base64url encoded.  I have locally 
fixed this and will commit to the develop branch (or a imp 5.1 topic 
branch) as soon as I am finished testing/squashing all the bugs.

Changes have been made in Git for this ticket:

Bug #9842: IMP 5 will not work with broken suhosin extension

  1 files changed, 5 insertions(+), 0 deletions(-)
http://git.horde.org/horde-git/-/commit/63ba95c6b2027fc7995ec122ad200c33bbbeaa80

My PHP 5.3.3 (debian) is suhosin patched AND had the suhosin 
extension. Without the extension, IMAP search does work now.

Thanks for the hint!

Did you use the Suhosin extension or the patch? The extension already 
caused other problems in the past, so I don't have any problem with 
discouraging its usage altogether.

Ugh.  Suhosin sucks then.  Apparently it feels it is a security threat 
to have a null character in URL parameters, even though it is 
perfectly permissible.  I'm not sure there is anything we can do about 
this other than to say IMP does not work with suhosin.

That is true.. I disabled suhosin and everything is working:-)
Is there something to do to able working with suhosin?
Where is the problem?
Thank you for your time

And there's your problem.  The mailbox entry is nowhere to be found.   
Either your PHP or web server is incorrectly munging the URL parameter 
data.

What version of PHP are you using?  You wouldn't happen to be using 
something like suhosin either?

<516> root@duff/tmp/h4# less horde_debug.txt

2011-04-14T00:41:21+02:00 DEBUG: Variable information:
object(Horde_Variables)#231 (3) {
   ["_vars":protected]=>
   array(7) {
     ["horde_sidebar_expanded"]=>
     string(1) "0"
     ["horde_menu_expanded"]=>
     string(21) "expimp,administration"
     ["imp_key"]=>
     string(32) "c6ad08b2e81994328a077e2962a2741a"
     ["default_horde_view"]=>
     string(11) "traditional"
     ["Horde4"]=>
     string(32) "c6ad08b2e81994328a077e2962a2741a"
     ["auth_key"]=>
     string(23) "18847229984da5c8a601048"
     ["SESSc0d2b4ced4b5e9b5a4f261e22dcfe7ba"]=>
     string(32) "08db564168fd858ea6bcd95cd5bbb32a"
   }
   ["_expectedVariables":protected]=>
   array(0) {
   }
   ["_sanitized":protected]=>
   bool(false)
}

Backtrace:
1. Horde::debug() /var/www/h4/imp/mailbox.php:55

2011-04-14T00:41:21+02:00 DEBUG: Variable information:
array(7) {
   ["horde_sidebar_expanded"]=>
   string(1) "0"
   ["horde_menu_expanded"]=>
   string(21) "expimp,administration"
   ["imp_key"]=>
   string(32) "c6ad08b2e81994328a077e2962a2741a"
   ["default_horde_view"]=>
   string(11) "traditional"
   ["Horde4"]=>
   string(32) "c6ad08b2e81994328a077e2962a2741a"
   ["auth_key"]=>
   string(23) "18847229984da5c8a601048"
   ["SESSc0d2b4ced4b5e9b5a4f261e22dcfe7ba"]=>
   string(32) "08db564168fd858ea6bcd95cd5bbb32a"
}

Backtrace:
1. Horde::debug() /var/www/h4/imp/mailbox.php:56

Now we're getting somewhere.  Looks like your server is munging the 
URL parameters somehow.

Now we need a Horde::debug($vars) on line 55 of mailbox.php.  And for 
good measure, put a Horde::debug($_REQUEST) on the line after that.

Mail :: Inbox, it's the normal INBOX without any sign of the search.
2011-04-13T23:17:17+02:00 DEBUG: Variable information:
object(IMP_Mailbox)#112 (2) {
   ["_cache":protected]=>
   array(0) {
   }
   ["_mbox":protected]=>
   string(5) "INBOX"
}

Backtrace:
1. Horde::debug() /var/www/h4/imp/mailbox.php:55

Unfortunately, this tells me nothing other than the search is 
successfully being stored in the session.  Which is what is expected.

What does the mailbox look like after you perform the search.  Does it 
say Search Results at the top?  Or does it say INBOX (or some other 
mailbox)?

A Horde::debug(IMP::$mailbox) on line 55 of imp/mailbox.php would be useful.

The IMAP search is still not working for me. The result page is the 
same mailbox again, with search icon and all the mails.

Although I use a cyrus 2.2.13 without the ESEARCH, I patched Socket.php
(CAPABILITY IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS ID 
NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY SORT 
THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE XIMAPPROXY)

I attached the log for $vars, $q_ob and $imp_search for a simple 
search for Subject 'XXX' on INBOX.

I am attaching log

I have no idea.  You will need to debug yourself to determine what is 
wrong because no developers can reproduce.

I would focus on search-basic.php since it is much simpler than the 
advanced search.  Perform a simple search and track some variables 
using Horde::debug() (see instructions below).

I would like to see the debug results for $vars (put 
Horde::debug($vars) on line 77 of search-basic.php), $q_ob (put 
Horde::debug($q_ob) on line 84 of search-basic.php), and $imp_search 
(put Horde::debug($imp_search) on line 55 of mailbox.php).

Horde::debug() documentation: http://wiki.horde.org/Doc/Dev/DebugH4

It is not working for me. I am using Dovecot Imap server. May you 
please help me?

Changes have been made in Git for this ticket:

Bug #9842: Work around broken ESEARCH on Cyrus

  2 files changed, 5 insertions(+), 4 deletions(-)
http://git.horde.org/horde-git/-/commit/a8c5884b2ec30eed062f614c360901666a356345

It sure does.  Thanks a million!  Will keep an eye out for cyrus 2.4.8 
as well :)

cheers!

See if this fixes things.

Changes have been made in Git for this ticket:

Bug #9842: Possible workaround for broken Cyrus IMAP behavior

  1 files changed, 14 insertions(+), 1 deletions(-)
http://git.horde.org/horde-git/-/commit/d2b5777a18dd8b740de9cb2a43b34f3e9ad2dbf2

This is flat-out wrong.  Pursuant to RFC 4466 [2.6.1], the order of 
the arguments goes as follows:

    Arguments:  OPTIONAL result specifier
                OPTIONAL [CHARSET] specification
                searching criteria (one or more)
That's because RFC 5182 is not using the optional result specifier argument.

Sure enough, this is a Cyrus bug:
http://bugzilla.cyrusimap.org/bugzilla3/show_bug.cgi?id=3435

Ok, making some progress by spying on another client that its working 
with, then duplicating results in telnet.  here is a paste of my 
telnet session:

Working: (no charset)
-------------------------------
. SEARCH RETURN (ALL COUNT) FROM horde
* ESEARCH (TAG ".") ALL 
380,766,1051,1300:1301,1303,1927:1928,3102:3103,4880,4900,4921,4923,5722,5749,5757,6051:6052,22753,22778,22782,22788,22796:22799 COUNT 
27
. OK Completed (27 msgs in 0.060 secs)


Working w/ charset:
----------------------------------
. SEARCH CHARSET UTF-8 RETURN (ALL COUNT) FROM horde
* ESEARCH (TAG ".") ALL 
380,766,1051,1300:1301,1303,1927:1928,3102:3103,4880,4900,4921,4923,5722,5749,5757,6051:6052,22753,22778,22782,22788,22796:22799 COUNT 
27
. OK Completed (27 msgs in 0.060 secs)

Broken in imp:
--------------------------------------------------
. SEARCH RETURN (ALL COUNT) CHARSET UTF-8 FROM horde
. BAD Invalid Search criteria

So it appears the order of the CHARSET option.  Not sure if a cyrus 
specific workaround is needed or if this how its supposed to be.   
Every mention of charset in this paper shows it at the beginning: 
http://www.faqs.org/rfcs/rfc5182.html

Thanks!

Older versions of IMP (and other clients) are probably not sending 
extended search commands.

Thanks for looking at the log.  Strangely, other clients, including 
the previous version of horde (I think H3? over a year+ old)  appear 
to work fine with imap search.  I am not sure if they are searching in 
the same manner or not, but definitely performing successful imap 
search against cyrus 2.4.7, but will see if I can get a log.

Your IMAP server is broken.  Your server (Cyrus 2.4.7) advertises that 
it supports ESEARCH (RFC 4731).  Thus we send an extended search 
command, but the IMAP server is telling us it is bad:

(1302667094.7783) C: 5 UID SEARCH RETURN (ALL COUNT) CHARSET UTF-8 FROM horde
(1302667094.7785) S: 5 BAD Invalid Search criteria

However, there's nothing wrong with that search command.  Here's what 
it looks like when I run it on a dovecot installation:

5 UID SEARCH RETURN (ALL COUNT) CHARSET UTF-8 FROM horde
* ESEARCH (TAG "5") UID COUNT 0
5 OK Search completed (0.031 secs).

It's not a charset issue since the server must return a NO response 
rather than a BAD response (and it should return a BADCHARSET response 
also).

Jan, I think you run Cyrus.  Are you running 2.4.7?  Can you verify 
that this command works (or is broken) on your installation?

The imap log is attached.  I have snipped (marked with <snip> ) a 
lengthly list of mailboxes out of the log, as well as changed user 
ids's but nothing else has been changed.  I do see the Invalid Search 
Criteria in here so hopefully it helps!

Thanks!

Can you provide imap logs? (see previous entries for instructions on 
how to do this)  We might be getting somewhere here ... 'Invalid 
Search criteria' appears nowhere in the Horde codebase so it MUST be 
something that is returned from the IMAP server.

Same here as well.  Cyrus 2.4.7 with firefox 3, 4, chrome, and safari 
on mac 10.6.7.

Upon searching in traditional interface, I land at url:

https://webmail.xyz.com/imp/mailbox.php?mailbox=impsearch%00OBeSBUt1iUdNo-sg5QESIHA

and 3 times at the top I see errors:
Mailbox listing failed: Bad IMAP request: Invalid Search criteria

cheers,
-scott

/beta/imp/mailbox.php?mailbox=impsearch%00lJNSYRiV5G1No_BwtrJCIcA

What is the URL of your search page?  It should look something like:

[...]/horde/imp/mailbox.php?mailbox=impsearch%00impbsearch

Iam using chrome and firefox under ubuntu and fedora.

Safari 5 and FF4 on MacOS

What browser are you using?

forgot to tell: that's true for all search forms I found as well as 
for dynview and traditional view (dont know about mobile ones)

IMAP search is broken for me, too.
Using current Horde 4.0, IMP 5.0, cyrus imapd 2.2.13

Neither the imp debug log nor the protocol log of cyrus imapd show any 
sign of the search commands. Seems they are not issued at all.

my ./imp/config/backends.local.php
$servers['imap'] = array(
     'disabled' => false,
     'name' => 'duff IMAP',
     'hostspec' => 'localhost',
     'hordeauth' => true,
     'protocol' => 'imap',
     'port' => 143,
     'secure' => false,
     'maildomain' => '',
     'acl' => true,
     'cache' => true,
     //'debug' => '/tmp/mail4.log',
     //'debug_raw' => true,
);

I logged in Horde -> IMP -> then I went into advanced search (simple 
search not working too) and insert simple search in From field in 
Inbox folder. Result is all messages.:-(

Nothing is being searched for there.

How exactly can you reproduce - e.g. what view 
(dynamic/traditional/mobile), what page, etc.

I am attaching imap log. I made mailbox with two messages. I made 
query which should return just one message. I returns both.
Thank you for your reply.

It works for everyone else, so at a minimum you will need to provide 
an IMAP log.

-----

To further debug this issue, we need details of the IMP -> IMAP/POP 
communication.

To enable debugging, see instructions contained in 
imp/config/backends.php (the 'debug' config parameter).

Debugging should not be enabled on a production server,   Attach/post 
only the portion of the log that directly deals with the problem 
reported (it may be simplest to clear the log file and then perform 
the event that causes the error).

I am using latest Horde 4 with latest IMP 5. Search doesn't work at 
all no matter what
option I choose. I try it in IMP and in DIMP. Search returns all 
messages everytime.
latest Horde 3 with Imp 4 works great with the same settings.