On further reflection, agreed that merging to 4.2.1 makes the most sense.

Changes have been made in CVS for this ticket:

http://cvs.horde.org/diff.php/imp/config/servers.php.dist?r1=1.50.2.21&r2=1.50.2.22&ty=u
http://cvs.horde.org/diff.php/imp/docs/CHANGES?r1=1.699.2.319&r2=1.699.2.320&ty=u
http://cvs.horde.org/diff.php/imp/lib/IMAP/Tree.php?r1=1.25.2.51&r2=1.25.2.52&ty=u
http://cvs.horde.org/diff.php/imp/lib/Session.php?r1=1.74.2.41&r2=1.74.2.42&ty=u

Really, only lib/IMAP/Tree.php is needed.  The other 2 files remove a 
no-longer needed configuration variable.

I agree on merging it - it seems to have cleared up 4-5 bugs so far.

I vote for merging it into 4.2.1.  We'll be backporting it if it isn't.



So.. at this point, these changes effect lib/IMAP/Tree.php, 
lib/Session.php and config/servers.php?

This code seems to be working well for everyone.  I think we should 
keep it in HEAD a bit longer and if we see no further major issues, 
merge to FW_3 for 4.2.2.  Or maybe this is such an upgrade over the 
current code that it would be worth it to merge to 4.2.1, reasoning 
that any bugs in it are less critical than the bugs in the current code?

Great! It fixed both, the linking, and the empty node problem.

Works for me still.  I tried playing around with a few more things so 
maybe my last commit will help.

Changes have been made in CVS for this ticket:

http://cvs.horde.org/diff.php/imp/lib/IMAP/Tree.php?r1=1.193&r2=1.194&ty=u

There is something really wrong with your namespace detection.  Put in:

   print_r($this->_namespaces);

above line 469 ("foreach ($this->_namespaces as $key => $val) {") and 
report the output.

changing the pref value of tree_view gives me this in http's logs :

[Fri Jun 27 10:28:52 2008] [error] [client 192.168.100.188] PHP 
Notice:  Undefined variable: name in 
/var/www/html/horde/imp/lib/IMAP/Tree.php on line 483, referer: 
http://192.168.1.22/horde/services/prefs.php?app=imp&group=display

[Fri Jun 27 10:28:52 2008] [error] [client 192.168.100.188] PHP 
Notice:  Undefined index:   in 
/var/www/html/horde/imp/lib/IMAP/Tree.php on line 483, referer: 
http://192.168.1.22/horde/services/prefs.php?app=imp&group=display

[Fri Jun 27 10:28:52 2008] [error] [client 192.168.100.188] PHP 
Notice:  Undefined variable: name in 
/var/www/html/horde/imp/lib/IMAP/Tree.php on line 483, referer: 
http://192.168.1.22/horde/services/prefs.php?app=imp&group=display

Before the patch,we were able to see/use shared folders with the 
'imap_config' option.

now, we don't see shared folders any more with or without the 
imap_config option. We use courier-imap-4.3.0.



$servers['imap'] = array(

     'name' => 'Courier IMAP Server',

     'server' => '127.0.0.1',

     'hordeauth' => true,

     'protocol' => 'imap/notls',

     'port' => 143,

     'maildomain' => 'mbpgroup.com',

     'smtphost' => '127.0.0.1',

     'smtpport' => 25,

     'realm' => '',

     'preferred' => '',

     'quota' => array(

         'driver' => 'imap',

         'params' => array(),

     ),

     'acl' => array(

         'driver' => 'rfc2086',

/*    ),

    'imap_config' => array(

         'children' => true,

         'namespace' => array(

             '' => array(

                 'name' => '',

                 'delimiter' => '.',

                 'type' => 'personal',

                 'hidden' => false

             ),

             '#shared.' => array(

                 'name' => '#shared.',

                 'delimiter' => '.',

                 'type' => 'personal',

                 'hidden' => false

             ),

       ),

         'search_charset' => array(

             'UTF-8' => true

         )

*/    )

);

Not for me, the issue with the empty parent node still exists. I 
noticed though, that this doesn't happen from the start, but only 
after first showing all folders, and then going back to only showing 
subscribed folders (in the folders navigator).

With my latest changes, everything seems to be working (at least for me).

Changes have been made in CVS for this ticket:

http://cvs.horde.org/diff.php/imp/lib/IMAP/Tree.php?r1=1.192&r2=1.193&ty=u

At least for the shared folders namespace, you can't create any folder 
directly under it anyway. And we don't show the Public Folders node 
either, if there aren't any folders below it.

It only contains "namespace.username.folder".

I thought this was a feature?  Or did we decide in the past that if a 
public/other namespace is empty, we would only show if subscriptions 
were off?  We do need to show this namespace somewhere, even if it is 
empty, if simply to allow the user to create subfolders in this 
namespace.

Can you check the output from _getList() and see if 
"namespace.username.folder" is being returned from the IMAP server, or 
if both "namespace.username.folder" and "namespace.username" is being 
returned.  Obviously, if it is the former than IMP_Tree is buggy, and 
if it is the latter then the IMAP server is buggy.

I have prefs set to display namespaces separately and use 
subscriptions. Now, if there is a shared folder from another user, the 
parent node saying "Other Users' Folders" is displayed (empty), even 
if I didn't subscribe to any of those other users' folders.



Additionally, the user itself is clickable, though I'm not sure if 
this is an issue with the IMAP server or IMP, i.e. I have the hierarchy:



Other User's Folders

   username

     foldername



if the user "username" shared his folder "foldername". And both 
"username" and "foldername" appear as clickable, valid mailboxes.

I don't see the subscriptions issue with the latest version - maybe it 
was fixed in the last few days.



As for the other issue, dovecot is exhibiting correct behavior.  From 
RFC 3501 [6.3.9]:



       The returned untagged LSUB response MAY contain different mailbox

       flags from a LIST untagged response.  If this should happen, the

       flags in the untagged LIST are considered more authoritative.



       ...



       The server MUST NOT unilaterally remove an existing mailbox name

       from the subscription list even if a mailbox by that name no

       longer exists.



So if viewing only subscribed folders, we do need to cross-reference 
with imap_list() results to get an accurate representation of the 
current subscribed folders.

I'm not actually too worried about that, though that would be nice - 
what's more curious is that I don't use subscriptions (they're turned 
off in my prefs) and IMP started to use them.

I use dovecot and I saw this too - imap_lsub on dovecot appears to 
return the list of mailboxes in the subscribed folder regardless of 
whether they are present on the server or not.  Seems like this is 
broken IMAP behavior, but maybe I'll ask Timo (dovecot dev) if this is 
correct or not.



To workaround, we should probably get the list of folders from 
imap_list() and cross-reference with the list of folders from 
imap_lsub().

It started using folder subscriptions even though I have them turned 
off. but once I cleared my dovecot .subscriptions file (had some old 
folders in it) the current code is working well for me. I'm a pretty 
small test case though.

Changes have been made in CVS for this ticket:

http://cvs.horde.org/diff.php/imp/lib/IMAP/Tree.php?r1=1.188&r2=1.189&ty=u

Changes have been made in CVS for this ticket:

http://cvs.horde.org/diff.php/imp/config/servers.php.dist?r1=1.82&r2=1.83&ty=u
http://cvs.horde.org/diff.php/imp/lib/Session.php?r1=1.168&r2=1.169&ty=u

First attempt has been committed to HEAD.  Now testing is needed.   
Those using a non-HEAD version can download the file from:

http://cvs.horde.org/co.php/imp/lib/IMAP/Tree.php?r=1.188&p=1



and directly replace the existing imp/lib/IMAP/Tree.php

Changes have been made in CVS for this ticket:

http://cvs.horde.org/diff.php/imp/lib/IMAP/Tree.php?r1=1.187&r2=1.188&ty=u

I've been working on this for about a week now.  The patch has the 
right idea, but there is a boatload of code that needs to be taken out 
(10 KB so far) and there are several bugs not covered by this patch.   
Additionally, I am looking at not storing polled, subscribed, and 
unsubscribed information in separate lists to reduce session size even 
more (the data will be rebuilt from the tree).

Works okay for me so far.

After our recent horde/imp upgrade at umich, we've seen login times rise

from ~5-10 seconds to ~10-30 seconds.   The bulk of this time is spent

building the mailbox list in IMP_Tree (imp/lib/IMAP/Tree.php).  We've

made some changes which dramatically reduce the IMP_Tree startup time

and would love some feedback and basic sanity checking before we move

forward with them.



On an unloaded test server in our environment, imap_getmailboxes and

imap_list commands can take up to 1.7 seconds each against our Cyrus system.



First, IMP_Tree calls imap_getmailboxes for each namespace with a %

appended to get one level deep of subfolders.



Second, IMP_Tree::_initSubscribed executes imap_list for each namespace

with an * appended to get all sublevels of folders.



Last, it recursively crawls each folder found during the first set of

imap_getmailboxes and calls imap_getmailboxes on each folder with %

again until all branches have been plumbed.



Our improvements are premised on the fact that one level searches (%)

and searches which return full trees of results (*) actually take the

same time to return from Cyrus.  So we are bypassing all other searches

and starting off with one single full (*) search from the root of the

user's mailbox.



One quirk we had to side step then was that % searches will return

objects for folders which have children that can't be directly accessed

themselves.  The * search will not.



So we just iterate through the php data structure and detect parentless

children and fake up an object for them.  Then, if tree_view is turned

on, we separate non-personal mailboxes into  IMPTREE_OTHER_KEY and

IMPTREE_SHARED_KEY.



Since the list of folders at this point has all the children, we bypass

the recursive calls to expand.