6.0.0-alpha14
7/2/25

[#13876] horde_histories_modseq reset to 1 by code
Summary horde_histories_modseq reset to 1 by code
Queue Synchronization
Queue Version Git master
Type Bug
State Resolved
Priority 1. Low
Owners mrubinsk (at) horde (dot) org
Requester horde (at) albasoft (dot) com
Created 02/20/2015 (3785 days ago)
Due
Updated 03/09/2015 (3768 days ago)
Assigned 02/23/2015 (3782 days ago)
Resolved 03/09/2015 (3768 days ago)
Github Issue Link
Github Pull Request
Milestone
Patch No

History
03/09/2015 04:12:05 PM Michael Rubinsky Assigned to Michael Rubinsky
 
03/09/2015 04:11:52 PM Michael Rubinsky Comment #6
State ⇒ Resolved
Reply to this comment
Seems reasonable. Committed.
03/09/2015 04:11:28 PM Git Commit Comment #5 Reply to this comment
Changes have been made in Git (master):

commit d6659821d3d656223a0504e930ee46d7d15ec5de
Author: Michael J Rubinsky <mrubinsk@horde.org>
Date:   Mon Mar 9 12:09:26 2015 -0400

     Prevent sequence reset with certain DB backends (Bug #13876).

  framework/History/lib/Horde/History/Sql.php |    4 +++-
  1 files changed, 3 insertions(+), 1 deletions(-)

http://github.com/horde/horde/commit/d6659821d3d656223a0504e930ee46d7d15ec5de
03/06/2015 09:25:43 AM horde (at) albasoft (dot) com Comment #4 Reply to this comment
I had a new history_modseq reset in the same database just some days 
ago. My current modseq should be right now over 550000, but it shows 
just about 6000.
I'm almost sure that it is some sort of "race condition" when server 
load is too high, as I explained before.

I'm trying the following patch, to prevent "horde_histories_modseq" 
table to become empty.
It really doesn't change anything, as I guess code executes deletes in 
"horde_histories_modseq" table because old data there is useless.

--- a/Horde/History/Sql.php        2015-03-06 10:10:49.075159565 +0100
+++ b/Horde/History/Sql.php        2015-03-06 10:11:20.547151406 +0100
@@ -355,7 +355,7 @@
      {
          try {
              $result = $this->_db->insert('INSERT INTO 
horde_histories_modseq (history_modseqempty) VALUES(0)');
-            $this->_db->delete('DELETE FROM horde_histories_modseq 
WHERE history_modseq <> ?', array($result));
+            $this->_db->delete('DELETE FROM horde_histories_modseq 
WHERE history_modseq < (? - 25)', array($result));
          } catch (Horde_Db_Exception $e) {
              throw new Horde_History_Exception($e);
          }


02/23/2015 06:12:38 PM horde (at) albasoft (dot) com Comment #3 Reply to this comment
I guess you make "manually inserting keys" when carrying out migration 
of databases. On a daily use, you don't have to make manual inserts in 
a sequence controlled column, as sequence is up to date by its own. 
Maybe you can check if a "manual insert" on primary key is being done, 
before calling resetPkSequence.

I think the problem with history_modseq reset must be a call to 
resetPkSequence when horde_history_modseq table was empty (no rows at 
all).
Your code would then reset primary key to '1'. Is this right?

But this table can be empty if enough processes are running at the 
same time. Function _nextModSeq at "Horde/History/Sql.php" makes 
inserts and also deletes:
...
$result = $this->_db->insert('INSERT INTO horde_histories_modseq 
(history_modseqempty) VALUES(0)');
$this->_db->delete('DELETE FROM horde_histories_modseq WHERE 
history_modseq <> ?', array($result));
...

One process may have $result==3, and will delete others row.
A second one may have $result==4, and will delete the previous one.
Now horde_history_modseq table can be empty.
And a third process can be making a call to resetPkSequence just by 
then, finding no data at all and reseting sequence to '1'.

In my database, just before the reset, there are two login errors in 
horde_histories with the very same timestamp and consecutive modseq 
values (the highest value before the reset). Maybe those events 
cleaned  horde_history_modseq table while a third process was calling 
resetPkSequence.
02/23/2015 03:14:13 PM Jan Schneider Comment #2
Priority ⇒ 1. Low
State ⇒ Feedback
Reply to this comment
02/20/2015 06:20:15 PM horde (at) albasoft (dot) com Comment #1
Priority ⇒ 2. Medium
Type ⇒ Bug
Summary ⇒ horde_histories_modseq reset to 1 by code
Queue ⇒ Synchronization
Milestone ⇒
Patch ⇒ No
State ⇒ Unconfirmed
Reply to this comment
Hi,

This week I noticed my devices were not syncing data, and I've found 
this in my logs:
2015-02-16T11:50:35+01:00 ERR: HORDE [kronolith] SQL QUERY FAILED: 
SQLSTATE[23505]: Unique violation: 7 ERROR:  duplicate key violates 
unique constraint "horde_histories_modseq_pkey"
         INSERT INTO horde_histories_modseq (history_modseqempty) 
VALUES(0) [pid 30578 on line 553 of 
"/var/www/html/php/horde5-pear/pear/php/Horde/Db/Adapter/Base.php"]
2015-02-16T11:50:35+01:00 ERR: HORDE [kronolith] SQLSTATE[23505]: 
Unique violation: 7 ERROR:  duplicate key violates unique constraint 
"horde_histories_modseq_pkey" [pid 30578 on line 354 of 
"/var/www/html/php/horde5-pear/pear/php/Horde/History/Sql.php"]

Searching in my database (postgresql), I've found at "horde_histories" 
table that "history_modseq" was reset to 1 precisely at that 
timestamp. By that time, my server was heavy loaded.
So that "INSERT" error reset "history_modseq" sequence and my devices 
requests are no longer able to find new events, contacts, etc.

I took a look at code, trying to find out why that sequence was reset. 
I have no answer yet, as I can't reproduce that event.
But I've found something rather odd in "Db/Adapter/Pdo/Pgsql.php" at 
"insert" function: after every insert at database, primary keys are 
recalculated calling resetPkSequence. I can't understand why you need 
to do that.
And the problem is that resetPkSequence operates in a way that will 
lead to "duplicate key violation" errors with sequences, as pointed by 
the example in this post: 
http://www.postgresql.org/message-id/4E5D8369.9090205@ringerc.id.au
Calling 'setval' that way while other processes may be calling 
'nextval' will generate the above errors.

Getting a modseq reset is really something to avoid, also when heavy loaded.

Thank you.

Saved Queries