5.2.0-git
2014-07-23

[#12001] Courier - max atom size too small
Summary Courier - max atom size too small
Queue IMP
Queue Version 6.0.3
Type Bug
State Resolved
Priority 1. Low
Owners slusarz (at) horde (dot) org
Requester azurit (at) pobox (dot) sk
Created 2013-01-28 (541 days ago)
Due
Updated 2013-04-08 (471 days ago)
Assigned
Resolved 2013-02-17 (521 days ago)
Milestone
Patch No

History
2013-04-08 10:06:26 Git Commit Comment #10 Reply to this comment
Changes have been made in Git (master):

commit 885d1e6e2afb2071ac7847b33e417c0c898e43cd
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Wed Feb 13 20:01:58 2013 -0700

     [mms] Disable mailbox sorting by default if the remote server 
does not natively support it (Bug #12001).

  imp/config/backends.php                |    9 ++++
  imp/docs/CHANGES                       |    2 +
  imp/docs/UPGRADING                     |    8 +++
  imp/lib/Imap.php                       |   17 +++++++
  imp/lib/Mailbox.php                    |   22 ++++++---
  imp/lib/Prefs/Sort/FixedDate.php       |   47 +------------------
  imp/lib/Prefs/Sort/None.php            |   78 
++++++++++++++++++++++++++++++++
  imp/lib/Prefs/Sort/Sortpref/Locked.php |   37 +++++++++++++++
  imp/package.xml                        |    1 +
  9 files changed, 169 insertions(+), 52 deletions(-)

http://git.horde.org/horde-git/-/commit/885d1e6e2afb2071ac7847b33e417c0c898e43cd
2013-02-17 23:44:26 azurit (at) pobox (dot) sk Comment #9 Reply to this comment
thanks!!
2013-02-17 23:41:47 Michael Slusarz Comment #8
State ⇒ Resolved
Assigned to Michael Slusarz
Reply to this comment
Horde_Imap_Client 2.7.0.
2013-02-17 23:40:56 Git Commit Comment #7 Reply to this comment
Changes have been made in Git (master):

commit 044bbf012148ecb8f47ceef812433ea57e5f2267
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Sun Feb 17 17:12:38 2013 -0700

     [mms] Ensure that a FETCH call will not exceed maximum allowed 
command length on the IMAP server (Bug #12001).

     [mms] Add Horde_Imap_Client_ids#split().

  .../Imap_Client/doc/Horde/Imap/Client/UPGRADING    |    8 ++
  .../Imap_Client/lib/Horde/Imap/Client/Ids.php      |   23 +++++++
  .../Imap_Client/lib/Horde/Imap/Client/Socket.php   |   68 
++++++++++++--------
  framework/Imap_Client/package.xml                  |   12 ++-
  .../Imap_Client/test/Horde/Imap/Client/IdsTest.php |   33 ++++++++++
  5 files changed, 112 insertions(+), 32 deletions(-)

http://git.horde.org/horde-git/-/commit/044bbf012148ecb8f47ceef812433ea57e5f2267
2013-02-14 02:52:32 Michael Slusarz Comment #6 Reply to this comment
Because Courier is braindead and doesn't support SORT.
OK - I was actually wrong about this.  It does support SORT.  But for 
some reason it is doing a client-side sort anyway.

Maybe the commit I just added helps (this was already fixed several 
months ago in IMP 6.1).
2013-02-14 02:51:18 Git Commit Comment #5 Reply to this comment
Changes have been made in Git (master):

commit 818c8f709b5b6be512626737fbcaa78f84d9eae5
Author: Michael M Slusarz <slusarz@horde.org>
Date:   Wed Feb 13 20:25:25 2013 -0700

     Possible client-side sorting fix for Bug #12001

  imp/lib/Imap.php |    2 +-
  1 files changed, 1 insertions(+), 1 deletions(-)

http://git.horde.org/horde-git/-/commit/818c8f709b5b6be512626737fbcaa78f84d9eae5
2013-02-14 00:19:04 Michael Slusarz Comment #4 Reply to this comment
Why?
Let me count the ways:

1. RFC 2683 was written in 1999.  Back then, servers (both IMAP and 
the underlying OS) weren't as robust/were 32-bit and had limited 
memory/etc, etc.  But now it is 2013 and there is zero reason, 
performance or otherwise, to NOT allow something like this.  Or, at 
the very least, provide a much more reasonable command limit for 
authenticated users (e.g. 100 KB).
2. Regardless of #1, 2683 is informational and not part of the standard.
3. What's more - there are now IMAP extensions that will be completely 
broken if you limit the size of incoming data.  For example QRESYNC, 
since you MUST pass the entire known UID list at SELECT/EXAMINE time.   
If you have thousands of UIDs in the local cache, this is no longer 
possible (you can't break this up into multiple commands).
4. It is a significant performance penalty to have to break apart and 
send these commands separately, especially since IMP does not 
currently support pipelining.
i was trying to google it and it looks like i'm not able to set this 
in Courier. I don't know why exactly IMP requested so many messages 
but it happened automatically right after the login so i assumed 
it's not my fault.
Because Courier is braindead and doesn't support SORT.  So if you are 
sorting by something other than arrival sort, we have to grab the 
message data from every message in the mailbox to do the sorting on 
the PHP side.

I removed the code to prevent sorting on servers that don't support 
this in IMP, but I think I probably need to add it back (Gmail doesn't 
provide sort either).
Command line length limit in several IMAP servers (default values in bytes):
Courier: 16384 (looks like cannot be changed)
Dovecot: 65536 (can be set in config file)
Cyrus: probably 131072 (can be set in config file)
The latter 2 are a much more reasonable default on current machines, 
while preventing DoS attacks.
2013-01-28 19:04:42 azurit (at) pobox (dot) sk Comment #3 Reply to this comment
Why? i was trying to google it and it looks like i'm not able to set 
this in Courier. I don't know why exactly IMP requested so many 
messages but it happened automatically right after the login so i 
assumed it's not my fault.

Command line length limit in several IMAP servers (default values in bytes):
Courier: 16384 (looks like cannot be changed)
Dovecot: 65536 (can be set in config file)
Cyrus: probably 131072 (can be set in config file)

From RFC 2683:
    A client should limit the length of the command lines it generates to
    approximately 1000 octets (including all quoted strings but not
    including literals).  If the client is unable to group things into
    ranges so that the command line is within that length, it should
    split the request into multiple commands.  The client should use
    literals instead of long quoted strings, in order to keep the command
    length down.

    For its part, a server should allow for a command line of at least
    8000 octets.  This provides plenty of leeway for accepting reasonable
    length commands from clients.  The server should send a BAD response
    to a command that does not end within the server's maximum accepted
    command length.

2013-01-28 17:52:00 Michael Slusarz Comment #2 Reply to this comment
You've got to be kidding me.
2013-01-28 15:26:37 azurit (at) pobox (dot) sk Comment #1
State ⇒ Unconfirmed
New Attachment: imp-debug.txt Download
Patch ⇒ No
Milestone ⇒
Queue ⇒ IMP
Priority ⇒ 1. Low
Type ⇒ Bug
Summary ⇒ Courier - max atom size too small
Reply to this comment
One e-mail account cannot login, Courier-IMAP is printing this error:
max atom size too small

Attaching the IMAP communication log. Looks like the IMP sends too 
long command.