Please don't compare us with roundcube. :)  Especially in terms of 
performance.

Here's a fun trick (haven't looked at their IMAP code in awhile,  Did 
so to answer your question below, and analysis of their IMAP logs 
alarmed me).  In IMP 5 and Roundcube, open a mailbox with 20,000 
messages.

IMP should open in less than a second. (well.. depends a bit on 
whether the IMAP server-side cache is primed, but it is approximate).
Roundcube... well, let's just say you could go a brew a cup of coffee 
before the first results came up (in my local test, it took 2 minutes 
and 24 seconds).

Roundcube can only display X number of messages in a page (in other 
words, it does not have seamless scrolling like IMP).  However, for 
some reason, it downloads ALL of this information for EVERY message in 
the mailbox:

(UID RFC822.SIZE FLAGS INTERNALDATE BODY.PEEK[HEADER.FIELDS (DATE FROM 
TO SUBJECT CONTENT-TYPE LIST-POST DISPOSITION-NOTIFICATION-TO REPLY-TO 
IN-REPLY-TO CC BCC MESSAGE-ID CONTENT-TRANSFER-ENCODING REFERENCES 
X-PRIORITY X-DRAFT-INFO MAIL-FOLLOWUP-TO MAIL-REPLY-TO RETURN-PATH)])

ACK!  Why the hell is it doing this?  Why do you need 
"Disposition-Notification-To" data when displaying mailbox information 
(A: you don't).  Why do you need to download all of this data for 
every message in the mailbox, when you are only displaying the first X 
messages (A: you don't).  Etc, etc.

There's a specific command in IMAP to provide all (or most) of the 
information needed to display a mailbox listing: ENVELOPE.  The fact 
that they are not using this at all?  That is extremely disturbing.

For the record, here is what we request from the IMAP server on a 
mailbox listing:

(ENVELOPE FLAGS RFC822.SIZE BODY.PEEK[HEADER.FIELDS (IMPORTANCE 
LIST-POST X-PRIORITY CONTENT-TYPE)])
No.  Quite honestly, they just happen to be lucky that they are not 
triggering the bug.  They are using horribly inefficient queries that, 
for whatever reason, do not tickle the bug on the Exchange server.   
But that is neither intentional, nor desired behavior.
I would love to workaround.  Bur right now, I don't even know what the 
problem is.  You would have to investigate for me to figure out what 
precisely is triggering the bug.  Does Exchange not like IMAP commands 
that only return INTERNALDATE?  Or maybe it is only triggered if we do 
INTERNALDATE searches on a message list over 2,000 (e.g. it is not 
triggered if the queries are done are smaller subsets of the message 
list).

I wonder about the second statement.  I had the user try roundcube and using
re-sorting in that environment, and there was never a problem displaying the
message list within roundcube.  Maybe roundcube have a workaround or somehow
dodge the bug?

We'll probably direct users to Exchange's OWA website, or remove exchange
from our list of supported sites if there are too many problems.  But 
it would be
great if horde/imp can handle what other email clients can somehow manage.

This is a bug/fault in Exchange.  Unfortunately, there is no way to 
work around.

Highlights of log file:

Exchange 2010 does not support SORT:
(1313836978.3636) S: * CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM 
AUTH=GSSAPI AUTH=PLAIN STARTTLS UIDPLUS CHILDREN IDLE NAMESPACE LITERAL+

Also humorous: Exchange does not support UTF-8 searches either:
(1313836982.3986) C: 7 SEARCH CHARSET UTF-8 BODY a 1
(1313836982.5981) S: 7 NO [BADCHARSET (US-ASCII)] The specified 
charset is not supported.

This is 2010.  It can't be that difficult to have implemented UTF-8 by now...

Because Exchange does not support server side sort, for anything other 
than sequence sort IMP needs to grab the information from the IMAP 
server for all messages in the mailbox to do the requested sort.  In 
your case, the user wants to do a date sort.

So, we first grab the list of UIDs from the mailbox:
(1313836999.1513) C: 4 UID SEARCH ALL
(1313836999.3543) S: * SEARCH 122571 122591 122608 12261 [...and 2092 
other UIDs]

Then we try to fetch the internal date information for all of these messages:

(1313836999.4008) C: 5 UID FETCH 122571,122591,122608,122617,[... and 
2092 other UIDS] (INTERNALDATE)
(1313836999.4020) S: 5 BAD Command Error. 10

[Sidenote: this should be optimized in the IMAP library.  Instead of 
grabbing the list of UIDs and then fetching by this list of UIDs, we 
should instead Fetch all messages in the mailbox (by sequence number), 
and grab the UID during this fetch: XX FETCH 1:* (UID INTERNALDATE).   
This will save a roundtrip to the server.]

Our FETCH command is a perfectly valid command (and not a very 
complicated one at that).  Exchange is throwing a BAD, which according 
to the RFCs means an invalid command or invalid arguments.  But 
instead it looks like it should be throwing a NO instead (fetch error: 
can't fetch that data).

In either case, Exchange is at fault.  Almost certainly it is some 
sort of error in parsing a date string that is specific to one message 
in that user's mailbox that is causing that problem (date parsing is 
notorious for being buggy, since the possible inputs/formats are so 
numerous).

I have a log file from IMP.  It contains many email addresses, names, 
subject lines and other data the user probably doesn't want to share 
with the world, so I'm sending the log privately to Michael Slusarz.

This is the Horde log, not the IMAP log.  Although your Horde log 
indicates you have a bad value for 'namespace' in your backends.php 
configuration (Although you really, really shouldn't be setting that 
config value.

We need the IMAP log of the Horde/IMP -> IMAP conversation.

-----

To further debug this issue, we need details of the IMP -> IMAP/POP 
communication.

To enable debugging, see instructions contained in 
imp/config/backends.php (the 'debug' config parameter).

Debugging should not be enabled on a production server,   Attach/post 
only the portion of the log that directly deals with the problem 
reported (it may be simplest to clear the log file and then perform 
the event that causes the error).

Using IMP 5.0.9 within webmail 4.0.8.

Exchange server user testing horde webmail cannot see mailbox.  Initially
it did appear, but when Date field was clicked to sort by date, there
is an error "Mailbox listing failed".

Attachment is debug level log of user connecting to exchange server from IMP.
In DIMP the full error message doesn't display on the web app, but in 
IMP it does.
IMP displays error like that in log: "Mailbox listing failed: Bad IMAP 
request: Command Error. 10"

User has about 8000 messages in Exchange 2010.

Cannot reproduce error with similar size of my own mailbox moved onto 
exchange using imap client.