6.0.0-beta1
7/10/25

[#2341] Performance issue during recursive folder traversal
Summary Performance issue during recursive folder traversal
Queue IMP
Queue Version 4.0.3
Type Bug
State Resolved
Priority 1. Low
Owners slusarz (at) horde (dot) org
Requester maurer (at) amasol (dot) de
Created 07/26/2005 (7289 days ago)
Due
Updated 08/24/2005 (7260 days ago)
Assigned 07/26/2005 (7289 days ago)
Resolved 08/24/2005 (7260 days ago)
Github Issue Link
Github Pull Request
Milestone
Patch No

History
08/24/2005 11:03:55 PM Michael Slusarz Comment #3
State ⇒ Resolved
Reply to this comment
IMAP_Tree in HEAD now exclusively uses the imap_getmailboxes() 
function so we are a bit smarter about children determination.  We 
still have to pay the price of extra searches if only viewing 
subscribed, however, since there is no way to determine this unless we 
actually look for subscribed folders under the current folder.
07/29/2005 04:52:34 AM Michael Slusarz Comment #2
Priority ⇒ 1. Low
Reply to this comment
The issue you are seeing appears to be because the behavior of 
imap_list()/imap_lsub() and imap_getmailboxes()/imap_getsubscribed() 
is not the same.  Namely, that the subscribed versions of these 
functions do NOT set the NoChildren flag (it is not that IMAP_Tree 
ignores the flag - it doesn't - it is that this flag is not set at 
all).  So I will have to go back and look to see how I can work around 
this.
07/26/2005 04:11:28 PM Jan Schneider Assigned to Michael Slusarz
State ⇒ Assigned
 
07/26/2005 03:55:25 PM maurer (at) amasol (dot) de Comment #1
State ⇒ Unconfirmed
Priority ⇒ 2. Medium
Type ⇒ Bug
Summary ⇒ Performance issue during recursive folder traversal
Queue ⇒ IMP
Reply to this comment
After upgrading to IMP 4.0.3 (from IMP 3.2.1), I noticed a severe 
performance degradation.  After some analysis, I noticed that IMP send 
large amounts of LSUB queries to our IMAP server.  Most of these are 
unnecessary.



This is a particular big problem with our support account.  This 
account has about 2000 folders that are placed in a 2-3 level folder 
hierarchy with about 80 non-leaf nodes.



IMP starts with listing all subscribed folders using

   00000003 LSUB "" *

Then, however, it starts to query every single folder, including the 
leaf folders:

   00000007 LSUB "" some/folder/%

I.e. that results in about 2000 LSUB requests that put a high load on 
both the IMAP server and the Apache/PHP installation.



Since the imap-uw server that I was using didn't support the 
HasNoChildren flag in the LSUB response, I decided try courier-imap.   
However, the number of LSUB requests doesn't change:

   00000007 LSUB "" INBOX.some.folder.%

This server correctly returns the HasNoChildren flag on leaf nodes

   * LSUB (\HasNoChildren) "." "INBOX.some.folder"

but IMP apparently doesn't utilize this information (in this 
particular piece of code).



I'd suppose that the problem is somewhere in the recursive folder tree 
building code, probably somewhere around _getList - but that's just my 
guess.



IMP 3.2.1, OTOH, only used

   00000003 LSUB "" *

to get the complete list of folder and deducted everything else from 
that information.



I'd like to suggest that IMP either properly supports the 
HasNoChildren flag or - preferably - caches the results of the *-query 
and dispenses with subsequent calls of LSUB.


Saved Queries