This was only an issue in Horde itself, not the apps.  This has been fixed.

Changes have been made in Git for this ticket:

Bug #9504: Mask might render all available options unavailable

http://git.horde.org/horde-git/-/commit/1444ec62c6688518002be1d935176a66447d6c7c

Cache themes is false, and i also turned off any cache.
I have the silver theme defined, which has the new horde favicon, but 
on the Horde login screen (and everywhere else) i keep getting the 
default (old) favicon.
I already checked that the silver/graphics/favicon.png exists and is 
web-accessible, so the problem is not that either.

Following the script logic, when i get to
Horde_Themes_Cache::get() i have $mask = 1010 (binary) that comes from 
(Horde_Themes_Cache::APP_DEFAULT | Horde_Themes_Cache::APP_THEME)..

Then this line $entry = $this->_get($item) is making $entry = 101, 
because the call to _get() finds the graphics/favicon.png for horde 
default theme (Horde_Themes_Cache::HORDE_DEFAULT) and for silver theme 
(Horde_Themes_Cache::HORDE_THEME), but then, it won't continue, 
because current app on login page is 'horde', which makes sense, 
because we already saw that favicon is present on horde silver 
theme...but then, the $entry 2nd and 4th bit will always be 0, so

getting back to Horde_Themes_Cache::get() we will find that '$entry &= 
$mask;' is equal to 101 & 1010 which will make $entry = 0, making it 
fail on all if sentences that follow..

Sorry to explain with code, but couldn't find a better way to explain 
how its working over here :-)

I don't see this.  e.g. In IMP, I copied over the favicon.ico file 
from default/graphics to themename/graphics and the favicon is now 
being served from themename/graphics.

Make sure you are not caching themes.

Digging the code, i think it has something to do with   
Horde_Themes_Cache::_get, and the "if ($this->_app != 'horde') {" part..
I don't understand the logic here 100% (so there's no patch), but my 
intuition says it has something to do with that..