[#3046] Remove all DataTree usage
Summary Remove all DataTree usage
Queue Horde Framework Packages
Queue Version HEAD
Type Enhancement
State Accepted
Priority 2. Medium
Owners
Requester luc (dot) germain (at) usherbrooke (dot) ca
Created 11/25/2005 (1000 days ago)
Due
Updated 08/13/2008 (8 days ago)
Assigned
Resolved
Attachments
Milestone Horde 4.0
Patch No

History
08/13/2008 Duck Comment #14 Reply to this comment
And now remains only Trean ... #7161

07/31/2008 Chuck Hagenbuch Comment #13 Reply to this comment
This list is now:

Auth_Signup
trean

... and we'll remove admin/datatree.php in Horde 4 when this is done.
07/31/2008 Duck Comment #12 Reply to this comment
incubator/faq/
was removed along with Horde_Template
06/22/2008 Chuck Hagenbuch Comment #11
State ⇒ Accepted
Reply to this comment
The places left that only use DataTree (i.e., no other driver is 
available) are:

Auth_Signup
admin/datatree.php (doesn't really count, but for completeness)
incubator/faq/
jonah/lib/Delivery (being deprecated once Jonah is branched)
trean

... and some assorted upgrade scripts that should be fine to break in HEAD.
06/22/2008 Duck Comment #10 Reply to this comment
We have a native SQL driver in 3.2. AFAIK the only places where we 
still use DT is Threan bookmarsks and Horde_Auth_Signup.
06/15/2007 Chuck Hagenbuch State ⇒ Stalled
 
11/20/2006 Chuck Hagenbuch Comment #9
Summary ⇒ Remove all DataTree usage
Reply to this comment
Note that the only reason this isn't high priority is that Horde 4 
isn't the top priority yet - it will move up to high as soon as 3.2 
and related apps are releases and Horde 4 development is begun in 
earnest.
11/20/2006 Chuck Hagenbuch Comment #8
Priority ⇒ 2. Medium
Reply to this comment
Making this into a general "get rid of DataTree for Horde 4.0" ticket.
12/09/2005 Ben Chavet Comment #7 Reply to this comment
It does work, as long as you don't need to use anything that's in the 
horde datatree tables while using that app, like perms or groups.
12/08/2005 Jan Schneider Comment #6 Reply to this comment
Jan- have you successfully used that idea?
No, I only think it might work.
12/08/2005 Chuck Hagenbuch Comment #5
State ⇒ Accepted
Reply to this comment
Jan- have you successfully used that idea? I think it'd create 
problems if Nag were to instantiate the Perms driver, etc...
11/28/2005 Jan Schneider Comment #4 Reply to this comment
If each type of shares had its
own table (horde.shares.nag, horde.shares.kronolith), it would not
have to do the left join and generate a temporary table each time,
and it would probably better use the indexes.
That should be possible already. Try adding lines (and the according 
tables) like this to the applications' conf.php files:
$conf['datatree']['params']['table'] = 'horde_datatree_nag';
$conf['datatree']['params']['table_attributes'] = 
'horde_datatree_attributes_nag';

11/28/2005 luc (dot) germain (at) usherbrooke (dot) ca Comment #3 Reply to this comment
The permission system was one of the main reasons for writing the
DataTree backend, and we need a tree structure for permissions. Also,
now that history has been moved out, permissions is (almost) the only
thing left that uses DataTree.
When we got our performance problem, history was not activated, so 
moving out the history would not be enough. The performance problem 
with the actual permission implementation arise when there are many 
users. Our database server came to a halt after about 4000 users has 
logged in.

We found that these type of requests were the main performance problem:

SELECT c.datatree_id, c.datatree_name FROM horde_datatree c LEFT JOIN
horde_datatree_attributes a1 ON a1.datatree_id = c.datatree_id WHERE
c.group_uid = 'horde.shares.nag' AND ((a1.attribute_name = 'owner' AND
a1.attribute_value = 'myuserid') OR (a1.attribute_name = 'perm_users'
AND a1.attribute_key = 'myuserid' AND a1.attribute_value &
2) OR (a1.attribute_name = 'perm_creator' AND a1.attribute_value & 2) OR
(a1.attribute_name = 'perm_default' AND a1.attribute_value & 2))
GROUP BY c.datatree_id, c.datatree_name, c.datatree_order ORDER BY
c.datatree_order, c.datatree_name, c.datatree_id

When doing an "explain" on the mysql server, it shows that this 
request has to evaluate each time a number of entries roughly 
proportionnal to the number of users in the database (since each user 
has at least one share for each type). If each type of shares had its 
own table (horde.shares.nag, horde.shares.kronolith), it would not 
have to do the left join and generate a temporary table each time, and 
it would probably better use the indexes.

If you have another way (other than changing the database structure or 
use session caching) to improve the performance of the permission 
system with thousands of users, I'm all ears!  Right now,we are 
stucked with horde 2.2 because of this problem :(

11/25/2005 Jan Schneider Comment #2
State ⇒ Feedback
Reply to this comment
The permission system was one of the main reasons for writing the 
DataTree backend, and we need a tree structure for permissions. Also, 
now that history has been moved out, permissions is (almost) the only 
thing left that uses DataTree.
11/25/2005 luc (dot) germain (at) usherbrooke (dot) ca Comment #1
Queue ⇒ Horde Framework Packages
Summary ⇒ Move permissions data to its own table
State ⇒ New
Priority ⇒ 3. High
Type ⇒ Enhancement
Reply to this comment
Database queries for permission data in the datatree become really too 
slow when the user base reach many thousands users. Moving the 
permission data to its own table with proper indexes would greatly 
improve performance of the permission system.