[#8805] expanded_folders preference makes user unable to login
Summary expanded_folders preference makes user unable to login
Queue IMP
Queue Version 4.3.5
Type Bug
State Resolved
Priority 2. Medium
Owners slusarz@horde.org
Requester pguiran@linagora.com
Created 2009-12-29 (4183 days ago)
Updated 2010-02-10 (4140 days ago)
Resolved 2010-02-05 (4145 days ago)
Milestone 4.3.7
Patch Yes

pguiran@linagora.com 2009-12-29 02:38:59

One of my user have problems to login into horde and it seems that the 
problem is from the 'expanded_folders' preference. In fact, its 
'expended_folder' preference in database may grown up with horde's 
upgrades, or with an unknown problem. Its 'expanded_folder' preference 
contain more than 1000 elements (old, unused, etc... elements) !

When the user login into horde, 'expanded_folders' is loaded from 
database, IMAP connects to the imap server, receives list of mailbox, 
makes modifications into 'expanded_folders' and store it into database 
and session. All those operations take lot of time, and the webserver 
connection is timeout or the web client close the connection due to 
timeout. So the user can't login.

The problem is that at each modification of 'expanded_folder', 
serialize + Prefs_sql::setValue() are called, which take lots of time. 
In fact, the call chain is : serialize -> Prefs_sql::setValue() -> 
Prefs::_setValue() -> Prefs_sql::convertToDriver() -> 
String::convertCharset() on a 150KB string :-(

I attach to this ticket a patch :
  * modify IMAP_Tree::_modifyExpandedList() -> track modification of 
$this->_expanded instead of the modification of the serialization of 
  ** add: if the value modified was in $this->_expanded, the 
Prefs_sql::isDurty( 'expanded_folders', true ) is called
  ** del: call of Prefs_sql::setValue() and of serialize()
  * modifify IMAP_Tree::_store()
  ** add: track modification of $this->_expanded and save it if necessary
  ** add: call of $this->_cleanExpandedList()
  * IMAP_Tree::_cleanExpandedList()
  ** only work when $this->_fulllist is not null, ie when the class 
has had a connection with the imap server
  ** try to clean $this->_expanded by removing useless entries.

I attach more information about tests files etc... in reply to this ticket.


ps : Sorry for my bad english :(

patrick.guiran@linagora.com 2009-12-29 02:44:27
export_sql_prefs.sql file, which was create through this query :

SELECT pref_scope, pref_name, pref_value FROM horde_prefs WHERE 
pref_uid = 'pguiran' AND ( pref_scope = 'imp' OR pref_scope = 'horde' 
) INTO OUTFILE 'export_sql_prefs.sql'

patrick.guiran@linagora.com 2009-12-29 02:51:02
horde log ouput with $conf['log']['priority'] = PEAR_LOG_NOTICE; in 

this log is made before applying the patch.
loggin time = 8min 45sec

patrick.guiran@linagora.com 2009-12-29 02:53:21
horde log ouput with $conf['log']['priority'] = PEAR_LOG_NOTICE; in 

this log is made after applying the patch.
loggin time = 17 sec

Jan Schneider <jan@horde.org> 2009-12-29 08:25:40
This is the obfuscated version of the sql dump.

patrick.guiran@linagora.com 2009-12-29 10:44:33
How to reproduce :
1/ logout from horde
2/ sed -ie 's/^pguiran/you_login/g' export_sql_prefs.sql
3/ load export_sql_prefs.sql with "load file" mysql command
4/ login to horde

CVS Commit <cvs@lists.horde.org> 2010-02-05 21:26:31

CVS Commit <cvs@lists.horde.org> 2010-02-05 21:27:03
Changes have been made in CVS for this ticket:

Bug: 8805
Merge from horde-git: a3f0c954834ed50d3cbe84863651ab82e3c0c529

Michael Slusarz <slusarz@horde.org> 2010-02-05 21:31:50
Good catch.  The proper/easiest fix is to check if the expanded 
mailbox provided to _modifyExpandedList() will change the current 
expanded list entry, and only set the value if this value changes.

Performing garbage collection on this preference value is not a good 
idea in IMP 4.x.  This is better done in IMP 5 where we can set up a 
system task to occasionally do this.  However, the expanded list 
preference may be going away in the near future since it has a major 
limitation: it doesn't work across various mail servers.  So I am not 
in any rush to create a garbage collection script in IMP 5 either.

CVS Commit <cvs@lists.horde.org> 2010-02-10 19:27:27