6.0.0-git
2020-02-20

[#3877] Mailbox list caching tracking bug
Summary Mailbox list caching tracking bug
Queue IMP
Queue Version HEAD
Type Bug
State Resolved
Priority 2. Medium
Owners Horde Developers (at) , slusarz (at) horde (dot) org
Requester vilius (at) lnk (dot) lt
Created 2006-05-03 (5041 days ago)
Due
Updated 2006-05-30 (5014 days ago)
Assigned 2006-05-24 (5020 days ago)
Resolved 2006-05-30 (5014 days ago)
Milestone
Patch No

History
2006-05-30 15:43:01 Michael Slusarz Comment #54
State ⇒ Resolved
Reply to this comment
Almost positive this was the culprit:

http://lists.horde.org/archives/cvs/Week-of-Mon-20060522/058202.html



Closing ticket.
2006-05-29 19:59:05 adrieder (at) sbox (dot) tugraz (dot) at Comment #53 Reply to this comment
Seems also to work now here
2006-05-29 08:19:50 Jan Schneider Comment #52 Reply to this comment
I haven's seen this happening anymore during the last few days.
2006-05-24 16:09:01 Michael Slusarz Comment #51
State ⇒ Feedback
Reply to this comment
OK - think I finally figured this out.  Try latest HEAD.
2006-05-24 14:18:01 Michael Slusarz Comment #50 Reply to this comment
Then I am going to need you to trace the code in IMP_Message::delete() 
and/or IMP_Message::expunge() to figure out why the messages aren't 
being removed.  Because evidently I am never going to be able to 
reproduce this locally.
2006-05-24 10:13:34 Jan Schneider Comment #49 Reply to this comment
Did this, by chance, help:
Didn't fix my problem either.
2006-05-24 07:50:35 adrieder (at) sbox (dot) tugraz (dot) at Comment #48 Reply to this comment
I'm afraid no...

Deleted messages still do not always get moved to the trash folder and 
suddenly reappear in the mailboxlist marked as deleted.
2006-05-24 07:23:56 adrieder (at) sbox (dot) tugraz (dot) at Comment #47 Reply to this comment
Are you using PHP 4?  Seeing as how I believe Jan uses PHP 4 and has
seen this, and I use PHP 5 and have never seen this (I can't
reproduce this given either of your scenarios) - this may be an issue
with references somewhere.  In fact I'm hoping this is the issue or
else I'm stumped.
Yes I'm running PHP4
2006-05-23 17:36:29 Michael Slusarz Comment #45 Reply to this comment
2006-05-23 15:45:33 Michael Slusarz Comment #44 Reply to this comment

[Show Quoted Text - 9 lines]
Are you using PHP 4?  Seeing as how I believe Jan uses PHP 4 and has 
seen this, and I use PHP 5 and have never seen this (I can't reproduce 
this given either of your scenarios) - this may be an issue with 
references somewhere.  In fact I'm hoping this is the issue or else 
I'm stumped.
2006-05-22 10:52:22 Jan Schneider Comment #43 Reply to this comment
And more important: it shouldn't matter if Idisable the cache.
100% agreed.  But my issue is I have not been able to reproduce an
incorrect cache update once in the last 2 weeks, with mailbox caching
both on and off.  So I haven't ignored this ticket - I just don't
even know where to begin.
I have just seen this again. I saw new messages in the virtual inbox 
from the real inbox, deleted some of them, went to the inbox and 
didn't see any new messages. The status was correct though.

When I tried to track this down, another message arrived to inbox and 
all unread messages appeared.
2006-05-22 09:00:42 adrieder (at) sbox (dot) tugraz (dot) at Comment #42 Reply to this comment
Here is a reproduceable scenario:



This is with "use Trash folder" on.

Open a mailbox -> delete some messages -> open a message -> report it as SPAM

Now the message gets reported as SPAM and deleted, the mailbox list 
now shows the previously deleted messages again, but marked as 
deleted. They should no show up, because the should have been moved to 
the trash folder before.

Now they are in the trash folder and in the mailbox.
2006-05-18 21:35:39 Michael Slusarz Comment #41 Reply to this comment
When deleting a folder:
Notice: Undefined property: _arrival in
/home/jan/headhorde/imp/lib/IMAP/MailboxCache.php on line 571
Fixed.  Seems that unset(), as of PHP 5, now throws undefined errors
if you try to unset an array key that doesn't exist.
Never mind.  $_arrival doesn't exist in that file anymore - that's why 
the error was being thrown.  Duh, me.
2006-05-18 21:15:15 Jan Schneider Comment #40 Reply to this comment
That sounds very unlikely, especially since I use PHP 4.
It doesn't complain about unsetting the array key, the whole array, 
the property doesn't even exist at that point.
2006-05-18 21:12:11 Jan Schneider Comment #39 Reply to this comment
both on and off.  So I haven't ignored this ticket - I just don't
even know where to begin.
Does it make sense at all that the search (virtual inbox) returns 
something different than the regular mailbox view? If so, I can try to 
track down where IMP is fetching the (incomplete) mailbox list from if 
I see it the next time.
2006-05-18 21:09:41 Jan Schneider Comment #38 Reply to this comment
That sounds very unlikely, especially since I use PHP 4.
2006-05-18 21:08:12 Michael Slusarz Comment #37 Reply to this comment
And more important: it shouldn't matter if Idisable the cache.
100% agreed.  But my issue is I have not been able to reproduce an 
incorrect cache update once in the last 2 weeks, with mailbox caching 
both on and off.  So I haven't ignored this ticket - I just don't even 
know where to begin.
2006-05-18 21:06:43 Michael Slusarz Comment #36 Reply to this comment
When deleting a folder:
Notice: Undefined property: _arrival in
/home/jan/headhorde/imp/lib/IMAP/MailboxCache.php on line 571
Fixed.  Seems that unset(), as of PHP 5, now throws undefined errors 
if you try to unset an array key that doesn't exist.
2006-05-18 11:07:06 vilius (at) lnk (dot) lt Comment #35 Reply to this comment
Also, when you are in INBOX folder and new mail arrives, new message
count near folder name in header increases, but you can't see them in
the list. After relogin messages shows up.
I'm still seeing this sometimes on some accounts.
2006-05-18 11:00:08 Jan Schneider Comment #34 Reply to this comment
When deleting a folder:

Notice: Undefined property: _arrival in 
/home/jan/headhorde/imp/lib/IMAP/MailboxCache.php on line 571
2006-05-18 10:59:24 Jan Schneider Comment #33 Reply to this comment
And more important: it shouldn't matter if Idisable the cache.
2006-05-16 17:52:06 Jan Schneider Comment #32 Reply to this comment
No.
2006-05-16 17:46:48 Michael Slusarz Comment #31 Reply to this comment
Jan - do you have concurrent clients accessing your IMAP system?
2006-05-11 11:35:59 Jan Schneider Comment #30 Reply to this comment
I have the mailbox cache *disabled* and usual the virtual inbox 
opened. Arriving message are displayed fine in the vinbox view. But 
sometimes, if go to a folder with some new messages, the messages are 
not displayed. The new mail count in the header is correct, but I see 
an old mailbox view. Also, sometimes the (thread) sorting is broken. 
Both can only be fixed by logging out and in again.
2006-05-10 20:22:42 Michael Slusarz Comment #29 Reply to this comment
I think my fixes last night cleared this last set of issues up.
2006-05-10 12:31:23 vilius (at) lnk (dot) lt Comment #28 Reply to this comment

[Show Quoted Text - 10 lines]
After CVS update an hour ago, this one went away.



In next couple of days I will see how things are going here and let 
you know if I'll find something.
2006-05-10 07:35:30 Jan Schneider Comment #27
State ⇒ Assigned
Reply to this comment
Emptying mailboxes still doesn't work for me, cache disabled.
2006-05-10 07:03:33 vilius (at) lnk (dot) lt Comment #26 Reply to this comment
I must note, that 95% of my users does not use any other mail client. 
They are using IMP only. But they see some issues too. The most 
annoying reproducable here like 9 times from 10 is this:



1. User logins to his account. He has 1 new message.

2. He checks the checkbox in message list and clicks Delete in navbar.

3. The message is marked as deleted.

4. User clicks on Purge Deleted and _all_ messages in INBOX are gone.

5. He clicks on Mail entry in sidebar, and all messages are back 
except for deleted message.



I can give you a test account on my webmail if you want.
2006-05-10 06:27:01 Michael Slusarz Comment #25
State ⇒ Resolved
Reply to this comment
Looking through here, there doesn't appear to be anything left open.   
This ticket may be reopened if new issues arise with the current 
mailbox list caching code.  Resolving for now.
2006-05-10 06:25:53 Michael Slusarz Comment #24 Reply to this comment

[Show Quoted Text - 13 lines]
This has nothing to do with mailbox list caching - this has to do with 
message caching.  This is a known limitation of our current caching 
system - it was designed so that IMP was the only MUA to access the 
mailbox.  If you can't handle that IMP may not always catch a 
non-destructive flag on a message done by a foreign MUA, then you 
should disable message caching in conf.php.
2006-05-10 06:12:48 Michael Slusarz Comment #23 Reply to this comment
I think it'd be great to log these at a DEBUG level for now - once
it's not happening anymore, we can either remove the logMessage
calls, or leave them to make sure they don't recur. :)
Unfortunately, looking at the code, adding these debug statements 
won't get us very far since we will still throw a fatal error in 
mailbox.php or message.php if the data doesn't exists and there is 
really no way around this outside of "making-up" data to pass back.   
But I personally haven't seen any of these errors for several days now 
so I think Phase 1 of the caching code is pretty close.
2006-05-07 06:44:02 adrieder (at) sbox (dot) tugraz (dot) at Comment #22 Reply to this comment
Since a cvs update this morning the deleting of messages which were 
opened from within the "Virtual Inbox" seems to work again. I do not 
see the "Fatal Error" logs.
2006-05-07 01:21:09 Chuck Hagenbuch Comment #21 Reply to this comment
I think I will try to add some error catching code in MailboxCache to
prevent these ugly PHP errors.  However, once we get the mailbox
caching sorted out, we should never hit these error messages ever.
I think it'd be great to log these at a DEBUG level for now - once 
it's not happening anymore, we can either remove the logMessage calls, 
or leave them to make sure they don't recur. :)
2006-05-06 23:50:06 Michael Slusarz Summary ⇒ Mailbox list caching tracking bug
 
2006-05-06 22:51:30 Michael Slusarz Comment #20 Reply to this comment

[Show Quoted Text - 9 lines]
These errors aren't really helpful.  They occur anytime we incorrectly 
update the current mailbox list - and this could be for a variety of 
reasons.  That's why Bug 3889 is marked as a duplicate.



I think I will try to add some error catching code in MailboxCache to 
prevent these ugly PHP errors.  However, once we get the mailbox 
caching sorted out, we should never hit these error messages ever.
2006-05-06 19:37:40 adrieder (at) sbox (dot) tugraz (dot) at Comment #19 Reply to this comment
When deleting a message which was opened from the "Virtual Inbox" 
mailboxlist I get the following errors:



[Sat May  6 21:27:52 2006] [error] PHP Notice:  Undefined property:   
seen in /usr/local/apache/htdocs/cvs_dev/horde/imp/message.php on line 
206

[Sat May  6 21:27:52 2006] [error] PHP Notice:  Undefined property:   
seen in /usr/local/apache/htdocs/cvs_dev/horde/imp/message.php on line 
214

[Sat May  6 21:27:52 2006] [error] PHP Fatal error:  Call to undefined 
function:  setstriplink() in 
/usr/local/apache/htdocs/cvs_dev/horde/imp/message.php on line 223



When deleting it directly from the "Virtual Inbox" mailboxlist without 
opening it first everything is fine.

I already reported this in bug #3889 which was marked as duplicate of 
this bug report. But I forgot to mention the two "Undefined property" 
errors above the "Fatal error"
2006-05-06 18:15:38 Michael Slusarz Comment #18 Reply to this comment
Also after update from CVS couple of hours ago, I see some new errors:

PHP Warning:  array_diff(): Argument #1 is not an array in
/var/www/html/horde/imp/lib/Message.php on line 669

It seems that imap_search not always succeed and returns false.
This issue has been fixed.
2006-05-06 18:05:45 Michael Slusarz Comment #17 Reply to this comment
OK - the important part about this issue is that you are having 
concurrent accesses to the IMAP server other than IMP.  Very 
important, and the reason I am not seeing this issue.  Now that I know 
where to look, this should be easier to track down.
2006-05-06 12:41:12 vilius (at) lnk (dot) lt Comment #16 Reply to this comment
Ok this one is reproducable 100% here:

1. Login to your account from other client and mark some messages as
deleted. Logout.
I spent debugging my IMAP server all morning trough telnet and it 
seems that my other IMAP client deleted messages and after that 
automatically expunged. So everything that is seen in IMP as marked as 
deleted is just not updated mailbox cache. This needs to be fixed.



More on this later.
2006-05-06 12:01:17 vilius (at) lnk (dot) lt Comment #15 Reply to this comment
Also after update from CVS couple of hours ago, I see some new errors:



PHP Warning:  array_diff(): Argument #1 is not an array in 
/var/www/html/horde/imp/lib/Message.php on line 669



It seems that imap_search not always succeed and returns false.
2006-05-06 11:59:19 vilius (at) lnk (dot) lt Comment #14 Reply to this comment
Ok this one is reproducable 100% here:



1. Login to your account from other client and mark some messages as 
deleted. Logout.

2. Login to IMP. You can see deleted messages. If you try to hit 
expunge they are not deleted at all. (I think this is because of these 
messages are not stored in Horde cache, and expungeMailbox() somehow 
deletes only those messages that are on server AND in cache.).



3. Check couple of those marked as deleted messages and mark them as 
deleted from IMP.



4. Click Expunge and only marked messages are deleted.
2006-05-05 17:35:38 adrieder (at) sbox (dot) tugraz (dot) at Comment #13 Reply to this comment

[Show Quoted Text - 9 lines]
fogot to mention: this was with using trash folder and caching on
2006-05-05 17:34:19 adrieder (at) sbox (dot) tugraz (dot) at Comment #12 Reply to this comment
I can also run into the described problems:



e.g.

I had one folder with 5 messages in total. All of them were unseen. I 
was deleting the first 4 from within the mailboxlist without opening 
them first. Then I was opening the remaining message and deleting it 
from the message view. After a logout and a relogin I opened the 
folder and the first 4 messages reappared but now marked as deleted 
(They should have gone into the trash folder)
2006-05-05 14:45:38 Michael Slusarz Comment #11 Reply to this comment
I should also note I have tested this with mailbox caching on/off, 
hide deleted on/off, and trash folder on/off, and in normal and search 
mailboxes.
2006-05-05 14:42:01 Michael Slusarz Comment #10
Priority ⇒ 2. Medium
Reply to this comment
I see none of this behavior.  I will need your help tracing through 
IMP_Message:: to figure out why expungeMailbox() is not being called 
correctly.



Bumping down as I can't reproduce this at all.
2006-05-05 08:41:45 vilius (at) lnk (dot) lt Comment #9 Reply to this comment
One more strange issue.



When there is only one message in a folder and you try to delete it 
from folder list screen, sometimes message is not deleted from server 
only from the cache. After relogin you can see the message.
2006-05-05 08:36:15 vilius (at) lnk (dot) lt Comment #8 Reply to this comment
Errors are gone, but I see other strange things.



I'm using Trash folder and for example if I detele messages from Spam 
folder in bunched of 20 from mailbox screen. They are deleted from a 
cache. But after logging out and logging in back I can still see them 
in spam folder marked as deleted, and no messages are moved to Trash.



Also, when you are in INBOX folder and new mail arrives, new message 
count near folder name in header increases, but you can't see them in 
the list. After relogin messages shows up.
2006-05-04 23:58:05 Michael Slusarz Comment #7
State ⇒ Feedback
Reply to this comment
Are you still seeing this after today's commits?
2006-05-04 14:41:22 vilius (at) lnk (dot) lt Comment #6 Reply to this comment
Some more errors after related delete actions:



PHP Fatal error:  Call to undefined function:  findbody() in 
/var/www/html/horde/imp/lib/IMAP/MailboxCache.php on line 585


2006-05-03 15:09:22 Michael Slusarz Comment #5
State ⇒
Priority ⇒ 3. High
Reply to this comment
I'm almost positive the issue is as follows (I've known about this for 
a week, but haven't gotten aroudn to it yet):

In IMP_Message, when performing actions on message(s) everywhere but 
the message screen, we are passing IMP_Mailbox::updateMailbox() an 
array of indices.  updateMailbox() was originally designed to only 
update the mailbox when given a single index.  I've hackishly "fixed" 
updateMailbox() to silence errors in the former case, but it isn't 
updating the mailbox status correctly.  This needs to be fixed.
2006-05-03 14:52:55 vilius (at) lnk (dot) lt Comment #4 Reply to this comment
YES: Return to the mailbox listing after deleting, moving, or copying 
a message?

YES: When deleting messages, move them to your Trash folder instead of 
marking them as deleted?

NO: Use Virtual Trash Folder?

INBOX.Trash: Trash folder.



However my users are using:



YES: Return to the mailbox listing after deleting, moving, or copying 
a message?

NO: When deleting messages, move them to your Trash folder instead of 
marking them as deleted?

NO: Use Virtual Trash Folder?



And they see this behaviour sometimes when expunging too.



I tried clearing all cache* files without a result.
2006-05-03 14:43:33 Michael Slusarz Comment #3
Assigned to Michael Slusarz
Assigned to Horde DevelopersHorde Developers
Reply to this comment
I am almost positive this is due to the new mailbox list caching code.
2006-05-03 14:37:37 Chuck Hagenbuch Comment #2
State ⇒ Feedback
Reply to this comment
I'm not using a trash folder (or hide_deleted) or anything, and I 
don't get this when marking messages as deleted. I do get some notices 
somewhat randomly on expunge, but nothing on mark. What set of 
relevant prefs are you using?
2006-05-03 12:48:07 vilius (at) lnk (dot) lt Comment #1
Type ⇒ Bug
State ⇒ Unconfirmed
Priority ⇒ 1. Low
Summary ⇒ Undefined property: msgno in /var/www/html/horde/imp/mailbox.php on line 915
Queue ⇒ IMP
Reply to this comment
When I try to empy Trash, or users withouth trash folders a trying to 
flag messages as deleted, we get:



Notice: Undefined property: msgno in 
/var/www/html/horde/imp/mailbox.php on line 915

Saved Queries