perf(cache): keep host local asset and mimetype caches in the local tier - #41734
Merged
Conversation
11 tasks
DeepDiver1975
force-pushed
the
fix/ghsa-488r-findbinarypath-cache-poisoning
branch
from
July 27, 2026 14:32
6bf19ed to
b8e22c3
Compare
DeepDiver1975
force-pushed
the
perf/tier-host-local-caches
branch
from
July 27, 2026 14:32
d04f81d to
4cbb897
Compare
Base automatically changed from
fix/ghsa-488r-findbinarypath-cache-poisoning
to
master
July 27, 2026 15:44
The image paths of the active theme and the id <-> mimetype map were both stored in the distributed memory cache, although both are derived purely from the files and the database of the instance running the request. A value that only makes sense for one host does not belong behind a network socket, so both now use the host local tier via LocalCacheFactory. Neither cache had a TTL and neither is invalidated on the nodes that did not cause the change, so both got one: without it a moved installation or a repaired mimetype table would keep serving the old values indefinitely. This also bounds the staleness that comes with the local tier, where occ can only clear the cache of the node it runs on. Two coherence problems around the mimetype cache come along with it: - OC\Repair\RepairMimeTypes deletes rows from the mimetypes table and never invalidated the cache in front of it, so the id of a deleted mimetype stayed cached. It now takes an optional IMimeTypeLoader and resets it when it actually repaired something. - occ upgrade cleared the cache through ICacheFactory::create(), which only ever reaches the distributed tier. It now clears both. URLGenerator resolves its cache once in the constructor instead of on every imagePath() call. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: Thomas Müller <1005065+DeepDiver1975@users.noreply.github.com>
DeepDiver1975
force-pushed
the
perf/tier-host-local-caches
branch
from
July 27, 2026 15:49
4cbb897 to
fc71e85
Compare
phil-davis
approved these changes
Jul 28, 2026
phil-davis
left a comment
Contributor
There was a problem hiding this comment.
Looks reasonable, good for these cached things to stay in a cache only local to the node.
11 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Moves two host-local caches out of the distributed tier and into the local one, and adds the TTLs and the invalidation that makes doing so correct.
URLGenerator::imagePath()— the cache is now resolved once in the constructor instead of callingcreate()on every invocation (:162), and entries get a 24 h TTL (they were unbounded). Local tiering here is strictly better than what we had: these entries are never invalidated anywhere today and the key embeds the theme name, so staleness becomes node-scoped and dies with the worker instead of persisting cluster-wide forever. Measured 0.019 ms → 0.0027 ms per call, ~60 icons per page, so the cache is well worth keeping — just not in the shared tier.Files\Type\Loader— same tier move plus aCACHE_TTLapplied at all sixset()sites, which were unbounded. Worth keeping too: a 500-file listing costs 1.98 ms uncached vs 0.375 ms cached.reset()keeps clearing its own cache; that method exists to repair this process's view after a parallel-scan unique-constraint race (Files/Cache/Scanner.php:474, which clears its in-memory arrays right there), so local scope is the correct scope. Today's behaviour — one scanner losing a race nukes every node's mimetype map — is over-broad.Latent bug fixed here because it is what makes the tiering defensible:
Repair\RepairMimeTypesdeletes rows from*PREFIX*mimetypeswith no cache invalidation at all, so a deleted mimetype's id→name mapping survived the repair. It now takes an optionalIMimeTypeLoader(defaulted, so third-party construction keeps working) and calls->reset()at the end ofrun()— but only when something was actually repaired, so a no-op upgrade doesn't throw every node's map away. Single construction sitelib/private/Repair.php:129updated.core/Command/Upgrade.php— the blanketcreate()->clear()only ever reached the distributed tier. It now clears both, and the stale TODO is rewritten. Residual limit stated plainly in the code comment: clearing local from CLI only clears the node runningocc, so bounded staleness now comes from the new TTLs plus worker restarts. That is exactly why the TTLs are in this PR and not a follow-up.Untouched on purpose: the locking cache (2.96 ms DB vs 0.023 ms memcache per file — 130×) and everything genuinely cluster-shared.
Related Issue
Motivation and Context
Host-local values in a network-reachable, protocol-injectable backend are a standing supply of write primitives like the one in #41732. Neither of these two needs to be shared between nodes, and moving them also makes their staleness bounded rather than permanent.
How Has This Been Tested?
memcache.local => APCu+memcache.distributed => Redis— CI sets nomemcache.*and is blind to tiering. (Note for anyone reproducing:OC\Memcache\APCu::isAvailable()gates on the ini keyapc.enable_cli, notapcu.enable_cli, or the local tier silently degrades toNullCacheon CLI.)tests/lib/Files/Type/LoaderTest.php— the old->expects($this->once())->method('create')mock is replaced withFixedCacheFactory; newtestUsesTheLocalCacheTier()(assertscreateLocal('mimetypes')once,create/createDistributednever) andtestCachedEntriesExpire()(assertsset()receivesLoader::CACHE_TTL).tests/lib/UrlGeneratorTest.php— newtestImagePathIsCached()(asserts the resolved path, that the theme name is part of the key, and that a planted value is served on the second call),testImagePathsUseTheLocalCacheTier(),testImagePathThrowsForMissingImage().tests/lib/Repair/RepairMimeTypesTest.php—testMimeTypeCacheIsResetAfterRepair()(version 8.0.0.0 →reset()once) andtestMimeTypeCacheIsKeptWhenNothingIsRepaired()(version 10.0.0.0 → never).imagePathandmimetypeskeys exist only in APCu, withredis KEYSreturning 0 for both namespaces; icons and mimetypes still resolve correctly.tests/lib— 7149 tests, 40221 assertions, no new failures.make test-php-styleand phan clean.Types of changes
Checklist:
🤖 Generated with Claude Code