6.0.0-git
2019-03-24

[#13438] Doesn't show unsubscribed folders
Summary Doesn't show unsubscribed folders
Queue IMP
Queue Version 6.2.1
Type Bug
State Not A Bug
Priority 2. Medium
Owners
Requester frank.richter (at) hrz (dot) tu-chemnitz (dot) de
Created 2014-08-13 (1684 days ago)
Due
Updated 2014-08-24 (1673 days ago)
Assigned
Resolved 2014-08-24 (1673 days ago)
Milestone
Patch No

History
2014-08-24 23:44:42 Michael Slusarz Comment #12
State ⇒ Not A Bug
Reply to this comment
With  suppress_capabilities: LIST-EXTENDED IMP works with LSUB and 
simple LIST commands,
and shows all folders. So this workaround works.
This can also be done from the IMP side by adding:

     'capability_ignore' => array('LIST-EXTENDED')

to the cyrus config in backends.local.php.  Obviously, it makes more 
sense to do this from the Cyrus side since this will fix this issue on 
ALL clients connecting to the server, rather than just IMP.
Question for you: Will you provide a workround for this cyrus-imapd 
bug in IMP,
See above for backend configuration workaround.

This can't be done automagically in the driver because there is no way 
of determining that the server is actually broken based on the 
response.  IMAP version sniffing is not an option either.  (Other 
broken IMAP servers/commands can be detected because they do things 
like send obviously incorrect information, or by sending the wrong 
response.  But absent not using LIST-EXTENDED for ANY server - which 
is not going to happen - there's not much we can do about server 
replies that are inaccurate.)
2014-08-22 10:22:16 frank (dot) richter (at) hrz (dot) tu-chemnitz (dot) de Comment #11 Reply to this comment
C: 4 LIST () "" (INBOX INBOX.* user.* *) RETURN (SUBSCRIBED)
If I understand RFC 5258 right, this means:
     List all folders returning the Subscribed status.
Incorrect.  This command means "return all mailboxes that match the 
given query (ignoring subscribed status) and provide information as 
to which mailboxes are subscribed"
Ok, I was trying to express exactly that, sorry.

[Show Quoted Text - 9 lines]
It's buggy in a Cyrus-murder environment (frontend/backend servers),
I got this answer from <brong@fastmail.fm> on info-cyrus mailing list:

    Yes, it's a bug :(  List code is hairy, and a couple of the edge 
cases are wrong.
     I recommend just turning off LIST-EXTENDED with:
     suppress_capabilities: LIST-EXTENDED

See http://comments.gmane.org/gmane.mail.imap.cyrus/38001

With  suppress_capabilities: LIST-EXTENDED IMP works with LSUB and 
simple LIST commands,
and shows all folders. So this workaround works.
Except a problem I still have is that the former LIST command is 
what IMP will issue when it is displaying ALL mailboxes.  (Whether 
cyrus is buggy is a separate issue.)  If IMP is instead configured 
to display only subscribed mailboxes, you should be seeing these 
kind of queries instead:

C: 4 LIST "" INBOX RETURN (SUBSCRIBED)
C: 3 LIST (SUBSCRIBED) "" (INBOX.* user.* *) RETURN (SUBSCRIBED)
Exactly this is the case when preference subscribe = 1
and cyrus-imapd replies correctly here.

My question was with subscribe = 0 -> Doesn't show unsubscribed folders

So, IMP is fine, cyrus-imapd in a murder environment is buggy.
Workaround for cyrus-imapd as described above

Question for you: Will you provide a workround for this cyrus-imapd 
bug in IMP,
e.g. not asking
   LIST () "" (INBOX INBOX.* user.* *) RETURN (SUBSCRIBED)
but intead just
   LIST () "" (INBOX INBOX.* user.* *)


Thanks,
Frank

2014-08-21 00:58:19 Michael Slusarz Comment #10 Reply to this comment
C: 4 LIST () "" (INBOX INBOX.* user.* *) RETURN (SUBSCRIBED)
If I understand RFC 5258 right, this means:
     List all folders returning the Subscribed status.
Incorrect.  This command means "return all mailboxes that match the 
given query (ignoring subscribed status) and provide information as to 
which mailboxes are subscribed"

"List all folders returning the Subscribed status" would instead look 
something like this:

C: 4 LIST (SUBSCRIBED) "" (INBOX INBOX.* user.* *)
But instead of this, the server returns ONLY subscribed folders.
If that is truly what you are seeing with your installation, then yes 
- it is buggy.
It seems that Cyrus-imapd .2.4.17 - at least in my installation - is 
buggy here!
Except a problem I still have is that the former LIST command is what 
IMP will issue when it is displaying ALL mailboxes.  (Whether cyrus is 
buggy is a separate issue.)  If IMP is instead configured to display 
only subscribed mailboxes, you should be seeing these kind of queries 
instead:

C: 4 LIST "" INBOX RETURN (SUBSCRIBED)
C: 3 LIST (SUBSCRIBED) "" (INBOX.* user.* *) RETURN (SUBSCRIBED)

This tells the IMAP server to only return INBOX, subscribed mailboxes, 
and provide flag information with each returned mailbox as to whether 
the mailbox is subscribed or not.

So I am still concerned that there is something in your IMP 
installation where internally it still thinks it is always displaying 
all mailboxes, regardless of subscribed status.  (Given your symptoms, 
this is actually the expected behavior if that is the case).  Which 
begs the question: are you absolutely sure you don't have the 
'subscribe' preference set to 0 for that user?  IMP is really acting 
like that is the case.
2014-08-20 08:41:23 frank (dot) richter (at) hrz (dot) tu-chemnitz (dot) de Comment #9 Reply to this comment
C: 4 LIST () "" (INBOX INBOX.* user.* *) RETURN (SUBSCRIBED)
If I understand RFC 5258 right, this means:
     List all folders returning the Subscribed status.

But instead of this, the server returns ONLY subscribed folders.

It seems that Cyrus-imapd .2.4.17 - at least in my installation - is 
buggy here!
This command can only be issued if the argument to 
Horde_Imap_Client_Base#listMailboxes() is either 
'Horde_Imap_Client::MBOX_ALL_SUBSCRIBED' or 
'Horde_Imap_Client::MBOX_UNSUBSCRIBED'.
If I comment out line 1317
/usr/share/pear/Horde/Imap/Client/Socket.php
             switch ($mode) {
             case Horde_Imap_Client::MBOX_ALL_SUBSCRIBED:
             case Horde_Imap_Client::MBOX_UNSUBSCRIBED:
                 # $return_opts->add('SUBSCRIBED');

it shows all the folders ->

C: 3 LIST () "" (INBOX INBOX.* user.* *)

So maybe this would be a workaround for me.
This can only happen if the argument to IMP_Ftree_Account_Imap 
includes the 'UNSUB' mask argument.  This argument can only occur in 
one of three situations that I can see:

1. Subscriptions are disabled (prefs 'subscribe' value)
Yes, $this->subscriptions is false -> mask gets 'UNSUB'

Ok, I'll dig into Cyrus-imapd then ...

Thanks,
Frank

2014-08-19 22:30:18 Michael Slusarz Comment #8 Reply to this comment
C: 4 LIST () "" (INBOX INBOX.* user.* *) RETURN (SUBSCRIBED)
This command can only be issued if the argument to 
Horde_Imap_Client_Base#listMailboxes() is either 
'Horde_Imap_Client::MBOX_ALL_SUBSCRIBED' or 
'Horde_Imap_Client::MBOX_UNSUBSCRIBED'.

This can only happen if the argument to IMP_Ftree_Account_Imap 
includes the 'UNSUB' mask argument.  This argument can only occur in 
one of three situations that I can see:

1. Subscriptions are disabled (prefs 'subscribe' value)
2. Session 'showunsub' variable is set (irrelevant here, since this 
can only be set in basic or minimal views and we are talking about 
dynamic view).
3. user does not have 'allow_folders' permission (not the issue here, 
since this would only query INBOX).

See imp/lib/Ftree.php line 169.


2014-08-19 14:26:29 frank (dot) richter (at) hrz (dot) tu-chemnitz (dot) de Comment #7 Reply to this comment
BUT I have    $_prefs['subscribe']['value'] = 0;
In previous Horde/IMP versions that meant
         Use IMAP mailbox subscriptions?   - no
Assuming you mean that you have entered that as a default in your 
prefs.local.php file, make sure the user you are testing with does 
not have that pref set in the prefs backend.
Right, I have a test user with no set prefs at all (besides 3 
automatically set last_login, etc..).
2014-08-19 13:51:18 Michael Rubinsky Comment #6 Reply to this comment
BUT I have    $_prefs['subscribe']['value'] = 0;
In previous Horde/IMP versions that meant
         Use IMAP mailbox subscriptions?   - no
Assuming you mean that you have entered that as a default in your 
prefs.local.php file, make sure the user you are testing with does not 
have that pref set in the prefs backend.
2014-08-19 12:57:20 frank (dot) richter (at) hrz (dot) tu-chemnitz (dot) de Comment #5 Reply to this comment
You'll have to trace the code itself and/or provide a reproducible 
test case.  I can't give you much more guidance than that since 
everything works fine here.
Thanks. We have Cyrus-imapd 2.4. I see the same here:

C: 4 LIST () "" (INBOX INBOX.* user.* *) RETURN (SUBSCRIBED)

Horde asks only for *subscribed* folders, the server returns the list,
and Horde shows these folders.

BUT I have    $_prefs['subscribe']['value'] = 0;
In previous Horde/IMP versions that meant
         Use IMAP mailbox subscriptions?   - no

How can I achieve that Horde5.2.X/IMP 6.2.X shows the full list of 
folders, not just the subscribed?

Thanks,
Frank



2014-08-15 06:48:05 Michael Slusarz Comment #4 Reply to this comment
You'll have to trace the code itself and/or provide a reproducible 
test case.  I can't give you much more guidance than that since 
everything works fine here.

I can verify that everything works fine on separate Cyrus 2.2, 2.3, 
and 2.4 installations.

2.2:

S: * OK bernard Cyrus IMAP4 v2.2.13-Debian-2.2.13-19+squeeze3 server ready
[snip]
C: 2 UNSUBSCRIBE INBOX.B
S: 2 OK Completed
[snip]
C: 2 LSUB "" INBOX
C: 3 LSUB "" INBOX.*
C: 4 LSUB "" user.*
C: 5 LSUB "" *
S: * LSUB (\Noselect \HasChildren) "." "INBOX"
S: 2 OK Completed (0.000 secs 2 calls)
Command 2 took 0.001 seconds.
S: * LSUB () "." "INBOX.A"
S: 3 OK Completed (0.000 secs 2 calls)
Command 3 took 0.0011 seconds.
S: 4 OK Completed (0.000 secs 1 calls)
Command 4 took 0.0011 seconds.
S: * LSUB () "." "INBOX.A"
S: 5 OK Completed (0.000 secs 2 calls)
Command 5 took 0.0012 seconds.
C: 6 LIST "" INBOX
C: 7 LIST "" INBOX.*
C: 8 LIST "" user.*
C: 9 LIST "" *
S: * LIST (\HasChildren) "." "INBOX"
S: 6 OK Completed (0.000 secs 4 calls)
Command 6 took 0.001 seconds.
S: * LIST (\HasNoChildren) "." "INBOX.A"
S: * LIST (\HasNoChildren) "." "INBOX.B"
S: 7 OK Completed (0.000 secs 3 calls)
Command 7 took 0.0012 seconds.
S: 8 OK Completed (0.000 secs 1 calls)
Command 8 took 0.0009 seconds.
S: * LIST (\HasChildren) "." "INBOX"
S: * LIST (\HasNoChildren) "." "INBOX.A"
S: * LIST (\HasNoChildren) "." "INBOX.B"
S: 9 OK Completed (0.000 secs 4 calls)

2.3:

S: * OK [CAPABILITY IMAP4 IMAP4rev1 LITERAL+ ID STARTTLS 
COMPRESS=DEFLATE] centos.curecanti.org Cyrus IMAP 
v2.3.16-Fedora-RPM-2.3.16-6.el6_2.5 server ready
[snip]
C: 2 UNSUBSCRIBE INBOX.A
S: 2 OK Completed
[snip]
C: 2 LSUB "" INBOX
C: 3 LSUB "" INBOX.*
C: 4 LSUB "" user.*
C: 5 LSUB "" *
S: * LSUB (\Noselect \HasChildren) "." "INBOX"
S: 2 OK Completed (0.000 secs 3 calls)
Command 2 took 0.0007 seconds.
S: * LSUB () "." "INBOX.123"
S: * LSUB () "." "INBOX.ABC"
S: 3 OK Completed (0.000 secs 3 calls)
Command 3 took 0.0009 seconds.
S: 4 OK Completed (0.000 secs 1 calls)
Command 4 took 0.0381 seconds.
S: * LSUB () "." "INBOX.123"
S: * LSUB () "." "INBOX.ABC"
S: 5 OK Completed (0.000 secs 3 calls)
Command 5 took 0.0383 seconds.
C: 6 LIST "" INBOX
C: 7 LIST "" INBOX.*
C: 8 LIST "" user.*
C: 9 LIST "" *
S: * LIST (\HasChildren) "." "INBOX"
S: 6 OK Completed (0.000 secs 5 calls)
Command 6 took 0.0011 seconds.
S: * LIST (\HasNoChildren) "." "INBOX.123"
S: * LIST (\HasNoChildren) "." "INBOX.A"
S: * LIST (\HasNoChildren) "." "INBOX.ABC"
S: 7 OK Completed (0.000 secs 4 calls)
Command 7 took 0.0013 seconds.
S: 8 OK Completed (0.000 secs 1 calls)
Command 8 took 0.0011 seconds.
S: * LIST (\HasChildren) "." "INBOX"
S: * LIST (\HasNoChildren) "." "INBOX.123"
S: * LIST (\HasNoChildren) "." "INBOX.A"
S: * LIST (\HasNoChildren) "." "INBOX.ABC"
S: 9 OK Completed (0.000 secs 5 calls)
Command 9 took 0.0015 seconds.
2.4:

S: * OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE AUTH=DIGEST-MD5 
AUTH=CRAM-MD5 AUTH=NTLM AUTH=LOGIN AUTH=PLAIN SASL-IR] ruraljuror 
Cyrus IMAP v2.4.16-Debian-2.4.16-4+deb7u2 server ready
[snip]
C: 3 UNSUBSCRIBE INBOX.BBB
S: 3 OK Completed
Command 3 took 0.4326 seconds.
[snip]
C: 4 LIST "" INBOX RETURN (SUBSCRIBED)
C: 3 LIST (SUBSCRIBED) "" (INBOX.* user.* *) RETURN (SUBSCRIBED)
S: * LIST (\Subscribed \HasChildren) "." INBOX (CHILDINFO (SUBSCRIBED))
S: 4 OK Completed (0.010 secs 23 calls)
Command 4 took 0.0016 seconds.
S: * LIST (\Subscribed \HasChildren) "." INBOX.A
S: * LIST (\Subscribed \HasChildren) "." INBOX.A.B
S: * LIST (\Subscribed) "." INBOX.A.B.C
S: * LIST (\Subscribed) "." INBOX.B.c
S: * LIST (\Subscribed) "." INBOX.C
S: * LIST (\Subscribed) "." INBOX.Trash
S: * LIST (\Subscribed \HasChildren) "." INBOX.A
S: * LIST (\Subscribed \HasChildren) "." INBOX.A.B
S: * LIST (\Subscribed) "." INBOX.A.B.C
S: * LIST (\Subscribed) "." INBOX.B.c
S: * LIST (\Subscribed) "." INBOX.C
S: * LIST (\Subscribed) "." INBOX.Trash
S: 3 OK Completed (0.000 secs 12 calls)
Command 3 took 0.0036 seconds.
C: 5 LIST () "" (INBOX.* user.* *) RETURN (SUBSCRIBED)
S: * LIST (\Subscribed \HasChildren) "." INBOX.A (CHILDINFO (SUBSCRIBED))
S: * LIST (\Subscribed \HasChildren) "." INBOX.A.B (CHILDINFO (SUBSCRIBED))
S: * LIST (\Subscribed \HasNoChildren) "." INBOX.A.B.C
S: * LIST (\Subscribed \HasNoChildren) "." INBOX.B.c
S: * LIST (\HasNoChildren) "." INBOX.BBB
S: * LIST (\Subscribed \HasNoChildren) "." INBOX.C
S: * LIST (\Subscribed) "." INBOX.Trash
S: * LIST (\HasChildren) "." INBOX (CHILDINFO (SUBSCRIBED))
S: * LIST (\Subscribed \HasChildren) "." INBOX.A (CHILDINFO (SUBSCRIBED))
S: * LIST (\Subscribed \HasChildren) "." INBOX.A.B (CHILDINFO (SUBSCRIBED))
S: * LIST (\Subscribed \HasNoChildren) "." INBOX.A.B.C
S: * LIST (\Subscribed \HasNoChildren) "." INBOX.B.c
S: * LIST (\HasNoChildren) "." INBOX.BBB
S: * LIST (\Subscribed \HasNoChildren) "." INBOX.C
S: * LIST (\Subscribed) "." INBOX.Trash
S: 5 OK Completed (0.000 secs 39 calls)
Command 5 took 0.0043 seconds.
2014-08-15 05:58:35 frank (dot) richter (at) hrz (dot) tu-chemnitz (dot) de Comment #3 Reply to this comment
Duplicate of #13261
I'm sorry, it doesn't work with
Horde_Imap_Client            2.24.2  stable

What can I do to debug this?

2014-08-15 00:03:24 Michael Slusarz Comment #2
State ⇒ Duplicate
Reply to this comment
Duplicate of #13261
2014-08-13 13:48:21 frank (dot) richter (at) hrz (dot) tu-chemnitz (dot) de Comment #1
Type ⇒ Bug
State ⇒ Unconfirmed
Priority ⇒ 2. Medium
Summary ⇒ Doesn't show unsubscribed folders
Queue ⇒ IMP
Milestone ⇒
Patch ⇒ No
Reply to this comment
With IMP 6.2.1 the folder list doesn't show IMAP folders I'm not 
subscribed to, regardless the imp setting for 'subscribe'.
E.g. if I unsubscribe to INBOX, no INBOX is shown.

Server (Cyrus-imapd) logs:
<1407937338<3 LIST () "" (INBOX INBOX.* user.* *) RETURN (SUBSCRIBED)^M

It's a fresh new up-to-date installation:
Channel pear.horde.org: No upgrades available

Thanks,
Frank

Saved Queries