Tickets :: [#2298] Move history to its own table

6.0.0-git

2021-01-18

Summary	Move history to its own table
Queue	Turba
Queue Version	HEAD
Type	Bug
State	Resolved
Priority	1. Low
Owners	chuck (at) horde (dot) org
Requester	leena.heino (at) uta (dot) fi
Created	2005-07-19 (5662 days ago)
Due
Updated	2005-07-20 (5661 days ago)
Assigned	2005-07-19 (5662 days ago)
Resolved	2005-07-20 (5661 days ago)
Milestone
Patch	No

2005-07-20 21:50:46	Jan Schneider	Comment #15	Reply to this comment
... though I have to say it's not that huge a change. The History class is pretty simple and this doesn't change the API at all. The huge change is in the data storage, not in the API, and that's what admins care about, rather not about the API.

2005-07-20 21:44:05	Chuck Hagenbuch	Comment #14	Reply to this comment
... though I have to say it's not that huge a change. The History class is pretty simple and this doesn't change the API at all.

2005-07-20 21:43:01	Chuck Hagenbuch	Comment #13	Reply to this comment
I was very careful to make sure this doesn't break BC. But I'm okay with it waiting for 3.1 if we're focusing on 3.1 soon.

2005-07-20 21:08:08	Jan Schneider	Comment #12	Reply to this comment
I apparently forgot to add the upgrade script, it's there now. Why not merge it? Because it's a huge change, requiring upgrade scripts to be executed, probably breaking BC somewhere. Something we never did in a bug fix release and shouldn't start doing. We should rather concentrate on x.1 releases once the current bugfix releases are out.

2005-07-20 21:08:04	Jan Schneider	Comment #11	Reply to this comment
I apparently forgot to add the upgrade script, it's there now. Why not merge it? Because it's a huge change, requiring upgrade scripts to be executed, probably breaking BC somewhere. Something we never did in a bug fix release and shouldn't start doing. We should rather concentrate on x.1 releases once the current bugfix releases are out.

2005-07-20 18:52:56	leena (dot) heino (at) uta (dot) fi	Comment #10	Reply to this comment
As you saw in the datatree monster sql the query took something like 54 seconds to complete. And this query is run every time the user adds an address to the address book or creates a new event or task or note. I've now tested this with horde head and framework 3 versions. The task was adding a single entry to address book or a task to calendar: Horde framework 3 version with history in a datatree: 2 minutes. Horde head version with history in a separate table: 2 seconds.

2005-07-20 14:32:18	leena (dot) heino (at) uta (dot) fi	Comment #9	Reply to this comment
I apparently forgot to add the upgrade script, it's there now. Why not merge it? Is there a way to try separate history table code if framework 3. The history information in the DataTree is killing the system in terms of performance. As you saw in the datatree monster sql the query took something like 54 seconds to complete. And this query is run every time the user adds an address to the address book or creates a new event or task or note.

2005-07-20 13:22:55	Chuck Hagenbuch	Comment #8	Reply to this comment
I apparently forgot to add the upgrade script, it's there now. Why not merge it?

2005-07-20 07:07:10	Jan Schneider	Comment #7	Reply to this comment
There is no upgrade script yet, only the SQL script for creating the table. And we shouldn't merge this to 3.0.5.

2005-07-20 04:00:18	Chuck Hagenbuch	Comment #6 Version ⇒ HEAD	Reply to this comment
Only for HEAD right now, but I'll merge it to Horde 3.0.5/6 if it works well.

2005-07-20 03:59:44	Chuck Hagenbuch	Comment #5 State ⇒ Resolved	Reply to this comment
Okay, driver rewritten and upgrade script added. Let's see how that does.

2005-07-19 21:48:35	Chuck Hagenbuch	Assigned to Chuck Hagenbuch

2005-07-19 21:48:23	Chuck Hagenbuch	Comment #4 Summary ⇒ Move history to its own table State ⇒ Assigned	Reply to this comment
Yup, that's a monster query. Nice. I'm actually already working on moving Horde_History to use its own SQL table instead of using DataTree. Should help a bunch.

2005-07-19 17:46:44	leena (dot) heino (at) uta (dot) fi	Comment #3 New Attachment: datatree-monster-sql.txt	Reply to this comment
read horde/docs/PERFORMANCE, and don't post vague bug reports that don't even tell us about the query you're blaming for the problem. I've read horde/docs/PERFORMANCE, thank you for providing it. Attached is the datatree monster sql. This is from MySQL's slow query log.

2005-07-19 16:15:00	Chuck Hagenbuch	Comment #2 State ⇒ Not A Bug	Reply to this comment
read horde/docs/PERFORMANCE, and don't post vague bug reports that don't even tell us about the query you're blaming for the problem.

2005-07-19 10:09:25	leena (dot) heino (at) uta (dot) fi	Comment #1 Type ⇒ Bug State ⇒ Unconfirmed Priority ⇒ 1. Low Summary ⇒ adding addresses is abysmally slow Queue ⇒ Turba	Reply to this comment
Each adding of address to address book seems to query DataTree and if you have large localsql like address book then this SQL query is abysmally slow. This monster SQL query had something like 18000 OR and 9000 LIKE parameters. Is there some way to disable DataTree in turba with localsql like address books. Or is there some way to prune DataTree.