6.0.0-beta1
7/5/25

[#7086] Consider consistent hashing for Horde_Memcache
Summary Consider consistent hashing for Horde_Memcache
Queue Horde Framework Packages
Queue Version HEAD
Type Enhancement
State Resolved
Priority 1. Low
Owners
Requester chuck (at) horde (dot) org
Created 07/21/2008 (6193 days ago)
Due
Updated 07/21/2008 (6193 days ago)
Assigned
Resolved 07/21/2008 (6193 days ago)
Milestone
Patch No

History
07/21/2008 05:33:19 PM Michael Slusarz Comment #7
State ⇒ Resolved
Reply to this comment
For the way we use memcache, key redundancy doesn't makes sense.  The 
overhead in keeping cache information on multiple servers on every 
request is not worth the gains we might get in the (rare) event a 
memcache server is removed from the pool.  Since our memcache data is 
not entirely based on underlying DB/SQL data, B thrashing issues are 
reduced.
07/21/2008 05:05:34 AM CVS Commit Comment #5 Reply to this comment
07/21/2008 04:55:08 AM Michael Slusarz Comment #4 Reply to this comment
Version 3.0 also added key redundancy.
07/21/2008 04:54:02 AM Michael Slusarz Comment #3
State ⇒ Feedback
Reply to this comment
To clarify, this is internally done by the PECL memcache driver.  As 
of memcache 2.2.0, a new PHP ini setting was added:

"memcache.hash_strategy" = {standard, consistent}

However, 'consistent' did not become the default until 3.0.0.  So I 
guess it may be useful to set this automatically if using memcache 
2.2+.  No need to make it a config option however.
07/21/2008 04:37:16 AM Michael Slusarz Comment #2
State ⇒ Rejected
Reply to this comment
Memcache already has internal consistent hashing.  When adding 
server(s) to the pool, a certain percentage of keys from the existing 
servers are moved to the new server.  When removing server(s), the 
lost keys are redistributed to the remaining servers.
07/21/2008 03:30:37 AM Chuck Hagenbuch Comment #1
Priority ⇒ 1. Low
Type ⇒ Enhancement
Summary ⇒ Consider consistent hashing for Horde_Memcache
Queue ⇒ Horde Framework Packages
Milestone ⇒
Patch ⇒ No
State ⇒ Feedback
Reply to this comment
I don't know enough about how pecl/memcache handles multiple memcache 
servers to know if this is redundant, but if it's not already using 
consistent hashing, we should have at least an option to manage the 
array of memcache servers ourselves, and do that to spread keys 
around. In addition, we could use this same code to implement 
multiple-write, where we write data to 2 memcache servers instead of 
one, to decrease thrashing or cache misses if a server drops out.



A good explanation:

http://www.spiteful.com/2008/03/17/programmers-toolbox-part-3-consistent-hashing/



A potential implementation (I'll be adding one to the Horde_Support 
package hopefully within a few days as well):

http://code.google.com/p/flexihash/

Saved Queries