6.0.0-beta1
7/7/25

[#10123] Use memcache flags to keep track of large objects
Summary Use memcache flags to keep track of large objects
Queue Horde Framework Packages
Queue Version Git master
Type Enhancement
State Resolved
Priority 1. Low
Owners slusarz (at) horde (dot) org
Requester goncalo.queiros (at) portugalmail (dot) net
Created 05/25/2011 (5157 days ago)
Due
Updated 06/04/2011 (5147 days ago)
Assigned
Resolved 05/26/2011 (5156 days ago)
Milestone
Patch No

History
06/04/2011 07:36:42 PM Git Commit Comment #12 Reply to this comment
Changes have been made in Git for this ticket:

Bug #10123: Memcache flag fixes.
pecl/Memcache version 3.0.3 upwards started to reserve the first 16 bits

Signed-off-by: Michael M Slusarz <slusarz@curecanti.org>

  2 files changed, 13 insertions(+), 7 deletions(-)
http://git.horde.org/horde-git/-/commit/cc662ba8c894b0f967c156b4f3e74156d6656142
05/30/2011 01:00:50 PM goncalo (dot) queiros (at) portugalmail (dot) net Comment #11 Reply to this comment
Michael, the previous patch is important to keep Memcache working with 
recent pecl/Memcache as expected.
05/27/2011 05:42:06 PM goncalo (dot) queiros (at) portugalmail (dot) net Comment #10
New Attachment: 0001-pecl-Memcache-version-3.0.3-upwards-started-to-reser.patch Download
Reply to this comment
It seems that pecl/Memcache is reserving the last 16 bits now (started 
on version 3.0.3 
http://pecl.php.net/package-info.php?package=memcache&version=3.0.3)

Also, according to the Memcache protocol 
(https://github.com/memcached/memcached/blob/master/doc/protocol.txt#L155) we 
can use up to 32 bits since version 1.2.1 (Nov 2006)
05/27/2011 12:33:15 AM nuno (dot) lopes (at) portugalmail (dot) net Comment #9 Reply to this comment
I've been working with Gonçalo on this so I'll answer you.

[Show Quoted Text - 12 lines]
With the last patch you made you're right. The problem here is that 
every key will cause a cache miss. This returns the platform into a 
cold cache state which isn't an expected behavior from a minor 
upgrade. We now have upwards 10G of cache data so I think you can 
understand how regenerating this data can make a performance hit.
We have made (untested yet) for ourselves a temporary patch to
convert old large objects to the new flag system. Attached the temp
patch
No need to convert these values.  Memcache, as the name itself 
implies, is a cache.  If data objects suddenly go missing, this is 
an expected occurrence.
You're very right, but we use memcache to also store sessions and that 
implies logging everyone out. I agree that memcache isn't the right 
solution to store sessions but for our case it seems the best and it's 
an horde conf.

As Gonçalo said the warning would make sysadmins aware of these 
implications and the patch only makes the transition smoother.
05/26/2011 10:31:49 PM Git Commit Comment #8 Reply to this comment
Changes have been made in Git for this ticket:

Bug #10123: More correct fix for ignoring old values
Need to completely remove the item from the results list, since it is in
the old format and won't be pieced together correctly.

  1 files changed, 14 insertions(+), 1 deletions(-)
http://git.horde.org/horde-git/-/commit/861499b90afc4686bd1b471bbaf18f74d2cd4532
05/26/2011 10:21:56 PM Michael Slusarz Comment #7
State ⇒ Resolved
Reply to this comment
Your patch did not apply cleanly, so I had to go through by hand and
make the necessary changes.  Could you test and make sure I
transferred the changes correctly?
Yep, its ok.
I think that it should be sent to dev (or the appropriate place), a 
message to warn people that starting  to use this code will bring 
unexpected results if they have large objects on memcache.
How?  If I read the code correctly, this new code will simply cause a 
cache miss.
We have made (untested yet) for ourselves a temporary patch to 
convert old large objects to the new flag system. Attached the temp 
patch
No need to convert these values.  Memcache, as the name itself 
implies, is a cache.  If data objects suddenly go missing, this is an 
expected occurrence.
05/26/2011 10:20:33 PM Git Commit Comment #6 Reply to this comment
Changes have been made in Git for this ticket:

Request #10123: part_count might be -1, which we should ignore

  1 files changed, 1 insertions(+), 1 deletions(-)
http://git.horde.org/horde-git/-/commit/09fdf8b206ceebf097ab3ce2ef8b5b2fe049eb5d
05/26/2011 07:18:04 PM goncalo (dot) queiros (at) portugalmail (dot) net Comment #5
New Attachment: tmp.patch Download
Reply to this comment
Your patch did not apply cleanly, so I had to go through by hand and 
make the necessary changes.  Could you test and make sure I 
transferred the changes correctly?
Yep, its ok.
I think that it should be sent to dev (or the appropriate place), a 
message to warn people that starting  to use this code will bring 
unexpected results if they have large objects on memcache.

We have made (untested yet) for ourselves a temporary patch to convert 
old large objects to the new flag system. Attached the temp patch
05/26/2011 05:42:04 PM Michael Slusarz Comment #4
State ⇒ Feedback
Assigned to Michael Slusarz
Reply to this comment
Your patch did not apply cleanly, so I had to go through by hand and 
make the necessary changes.  Could you test and make sure I 
transferred the changes correctly?
05/26/2011 05:38:53 PM Git Commit Comment #3 Reply to this comment
Changes have been made in Git for this ticket:

Request #10123: Use memcache flags to keep track of large objects
Signed-off-by: Michael M Slusarz <slusarz@curecanti.org>

  2 files changed, 37 insertions(+), 58 deletions(-)
http://git.horde.org/horde-git/-/commit/63dee150d1309f6909b6d3102c68a0f1b3c64d7b
05/26/2011 01:19:56 PM goncalo (dot) queiros (at) portugalmail (dot) net Comment #2
New Attachment: 0001-Use-memcache-flags-to-keep-track-of-large-objects.patch Download
Reply to this comment
Tested the patch with and without large objects, and everything seems to work
05/25/2011 05:23:04 PM goncalo (dot) queiros (at) portugalmail (dot) net Comment #1
Priority ⇒ 1. Low
State ⇒ New
Patch ⇒ No
Milestone ⇒
Queue ⇒ Horde Framework Packages
Summary ⇒ Use memcache flags to keep track of large objects
Type ⇒ Enhancement
Reply to this comment

Saved Queries