Remove all DataTree usage

Database queries for permission data in the datatree become

Fri, 25 Nov 2005 20:19:07 +0000

Database queries for permission data in the datatree become really too slow when the user base reach many thousands users. Moving the permission data to its own table with proper indexes would greatly improve performance of the permission system.

The permission system was one of the main reasons for writin

Fri, 25 Nov 2005 21:20:47 +0000

The permission system was one of the main reasons for writing the DataTree backend, and we need a tree structure for permissions. Also, now that history has been moved out, permissions is (almost) the only thing left that uses DataTree.

> The permission system was one of the main reasons for writ

Mon, 28 Nov 2005 17:09:17 +0000

> The permission system was one of the main reasons for writing the > DataTree backend, and we need a tree structure for permissions. Also, > now that history has been moved out, permissions is (almost) the only > thing left that uses DataTree. When we got our performance problem, history was not activated, so moving out the history would not be enough. The performance problem with the actual permission implementation arise when there are many users. Our database server came to a halt after about 4000 users has logged in. We found that these type of requests were the main performance problem: SELECT c.datatree_id, c.datatree_name FROM horde_datatree c LEFT JOIN horde_datatree_attributes a1 ON a1.datatree_id = c.datatree_id WHERE c.group_uid = 'horde.shares.nag' AND ((a1.attribute_name = 'owner' AND a1.attribute_value = 'myuserid') OR (a1.attribute_name = 'perm_users' AND a1.attribute_key = 'myuserid' AND a1.attribute_value & 2) OR (a1.attribute_name = 'perm_creator' AND a1.attribute_value & 2) OR (a1.attribute_name = 'perm_default' AND a1.attribute_value & 2)) GROUP BY c.datatree_id, c.datatree_name, c.datatree_order ORDER BY c.datatree_order, c.datatree_name, c.datatree_id When doing an "explain" on the mysql server, it shows that this request has to evaluate each time a number of entries roughly proportionnal to the number of users in the database (since each user has at least one share for each type). If each type of shares had its own table (horde.shares.nag, horde.shares.kronolith), it would not have to do the left join and generate a temporary table each time, and it would probably better use the indexes. If you have another way (other than changing the database structure or use session caching) to improve the performance of the permission system with thousands of users, I'm all ears! Right now,we are stucked with horde 2.2 because of this problem :(

> If each type of shares had its > own table (horde.shares

Mon, 28 Nov 2005 17:24:33 +0000

> If each type of shares had its > own table (horde.shares.nag, horde.shares.kronolith), it would not > have to do the left join and generate a temporary table each time, > and it would probably better use the indexes. That should be possible already. Try adding lines (and the according tables) like this to the applications' conf.php files: $conf['datatree']['params']['table'] = 'horde_datatree_nag'; $conf['datatree']['params']['table_attributes'] = 'horde_datatree_attributes_nag';

Jan- have you successfully used that idea? I think it'd crea

Thu, 08 Dec 2005 05:44:08 +0000

Jan- have you successfully used that idea? I think it'd create problems if Nag were to instantiate the Perms driver, etc...

> Jan- have you successfully used that idea? No, I only t

Thu, 08 Dec 2005 08:24:26 +0000

> Jan- have you successfully used that idea? No, I only think it might work.

It does work, as long as you don't need to use anything that

Fri, 09 Dec 2005 15:25:37 +0000

It does work, as long as you don't need to use anything that's in the horde datatree tables while using that app, like perms or groups.

Making this into a general "get rid of DataTree for Horde 4.

Tue, 21 Nov 2006 01:01:40 +0000

Making this into a general "get rid of DataTree for Horde 4.0" ticket.

Note that the only reason this isn't high priority is that H

Tue, 21 Nov 2006 01:02:46 +0000

Note that the only reason this isn't high priority is that Horde 4 isn't the top priority yet - it will move up to high as soon as 3.2 and related apps are releases and Horde 4 development is begun in earnest.

We have a native SQL driver in 3.2. AFAIK the only places wh

Sun, 22 Jun 2008 10:35:11 +0000

We have a native SQL driver in 3.2. AFAIK the only places where we still use DT is Threan bookmarsks and Horde_Auth_Signup.

The places left that only use DataTree (i.e., no other drive

Sun, 22 Jun 2008 23:11:07 +0000

The places left that only use DataTree (i.e., no other driver is available) are: Auth_Signup admin/datatree.php (doesn't really count, but for completeness) incubator/faq/ jonah/lib/Delivery (being deprecated once Jonah is branched) trean ... and some assorted upgrade scripts that should be fine to break in HEAD.

> incubator/faq/ was removed along with Horde_Template

Thu, 31 Jul 2008 17:19:27 +0000

> incubator/faq/ was removed along with Horde_Template

This list is now: Auth_Signup trean ... and we'll rem

Thu, 31 Jul 2008 17:27:25 +0000

This list is now: Auth_Signup trean ... and we'll remove admin/datatree.php in Horde 4 when this is done.

And now remains only Trean ... #7161

Wed, 13 Aug 2008 08:45:34 +0000

And now remains only Trean ... #7161

Changes have been made in Git (master): commit ee0da6e1ba5f

Wed, 13 Nov 2013 03:29:00 +0000

Changes have been made in Git (master): commit ee0da6e1ba5ff80a794f30def8ccfe67da1d9bd4 Author: Michael M Slusarz Date: Tue Nov 12 20:28:05 2013 -0700 Request #3046: No longer need DataTree package framework/DataTree/composer.json | 25 -- framework/DataTree/doc/Horde/DataTree/COPYING | 458 ------------------------- framework/DataTree/package.xml | 94 ----- 3 files changed, 0 insertions(+), 577 deletions(-) http://git.horde.org/horde-git/-/commit/ee0da6e1ba5ff80a794f30def8ccfe67da1d9bd4

Changes have been made in Git (master): commit 5cde334a8ad7

Tue, 17 Dec 2013 11:39:03 +0000

Changes have been made in Git (master): commit 5cde334a8ad7529bb0e877ba814708431740fedf Author: Michael M Slusarz Date: Tue Nov 12 20:28:05 2013 -0700 Request #3046: No longer need DataTree package framework/DataTree/composer.json | 25 -- framework/DataTree/doc/Horde/DataTree/COPYING | 458 ------------------------- framework/DataTree/package.xml | 94 ----- 3 files changed, 0 insertions(+), 577 deletions(-) http://git.horde.org/horde-git/-/commit/5cde334a8ad7529bb0e877ba814708431740fedf

Changes have been made in Git (master): commit f4e74132cd43

Tue, 17 Dec 2013 11:39:21 +0000

Changes have been made in Git (master): commit f4e74132cd43b577c4db9243e5974742f534a893 Author: Michael M Slusarz Date: Tue Nov 12 20:28:05 2013 -0700 Request #3046: No longer need DataTree package framework/DataTree/composer.json | 25 -- framework/DataTree/doc/Horde/DataTree/COPYING | 458 ------------------------- framework/DataTree/package.xml | 94 ----- 3 files changed, 0 insertions(+), 577 deletions(-) http://git.horde.org/horde-git/-/commit/f4e74132cd43b577c4db9243e5974742f534a893

Changes have been made in Git (master): commit 00641d0bd849

Tue, 17 Dec 2013 11:39:38 +0000

Changes have been made in Git (master): commit 00641d0bd849e2e4874205083b62c08634ac0d98 Author: Michael M Slusarz Date: Tue Nov 12 20:28:05 2013 -0700 Request #3046: No longer need DataTree package framework/DataTree/composer.json | 25 -- framework/DataTree/doc/Horde/DataTree/COPYING | 458 ------------------------- framework/DataTree/package.xml | 94 ----- 3 files changed, 0 insertions(+), 577 deletions(-) http://git.horde.org/horde-git/-/commit/00641d0bd849e2e4874205083b62c08634ac0d98

Changes have been made in Git (master): commit 0e95470023c9

Tue, 17 Dec 2013 11:39:55 +0000

Changes have been made in Git (master): commit 0e95470023c9add22070504690173a1f4d009349 Author: Michael M Slusarz Date: Tue Nov 12 20:28:05 2013 -0700 Request #3046: No longer need DataTree package framework/DataTree/composer.json | 25 -- framework/DataTree/doc/Horde/DataTree/COPYING | 458 ------------------------- framework/DataTree/package.xml | 94 ----- 3 files changed, 0 insertions(+), 577 deletions(-) http://git.horde.org/horde-git/-/commit/0e95470023c9add22070504690173a1f4d009349