Skip to content

perf(cache): keep host local asset and mimetype caches in the local tier - #41734

Merged
DeepDiver1975 merged 1 commit into
masterfrom
perf/tier-host-local-caches
Jul 28, 2026
Merged

perf(cache): keep host local asset and mimetype caches in the local tier#41734
DeepDiver1975 merged 1 commit into
masterfrom
perf/tier-host-local-caches

Conversation

@DeepDiver1975

Copy link
Copy Markdown
Contributor

Description

⚠️ Stacked on #41732 (base is fix/ghsa-488r-findbinarypath-cache-poisoning, which introduces LocalCacheFactory and the FixedCacheFactory test double). Review/merge that one first; the base flips to master once it lands.

Moves two host-local caches out of the distributed tier and into the local one, and adds the TTLs and the invalidation that makes doing so correct.

URLGenerator::imagePath() — the cache is now resolved once in the constructor instead of calling create() on every invocation (:162), and entries get a 24 h TTL (they were unbounded). Local tiering here is strictly better than what we had: these entries are never invalidated anywhere today and the key embeds the theme name, so staleness becomes node-scoped and dies with the worker instead of persisting cluster-wide forever. Measured 0.019 ms → 0.0027 ms per call, ~60 icons per page, so the cache is well worth keeping — just not in the shared tier.

Files\Type\Loader — same tier move plus a CACHE_TTL applied at all six set() sites, which were unbounded. Worth keeping too: a 500-file listing costs 1.98 ms uncached vs 0.375 ms cached. reset() keeps clearing its own cache; that method exists to repair this process's view after a parallel-scan unique-constraint race (Files/Cache/Scanner.php:474, which clears its in-memory arrays right there), so local scope is the correct scope. Today's behaviour — one scanner losing a race nukes every node's mimetype map — is over-broad.

Latent bug fixed here because it is what makes the tiering defensible: Repair\RepairMimeTypes deletes rows from *PREFIX*mimetypes with no cache invalidation at all, so a deleted mimetype's id→name mapping survived the repair. It now takes an optional IMimeTypeLoader (defaulted, so third-party construction keeps working) and calls ->reset() at the end of run() — but only when something was actually repaired, so a no-op upgrade doesn't throw every node's map away. Single construction site lib/private/Repair.php:129 updated.

core/Command/Upgrade.php — the blanket create()->clear() only ever reached the distributed tier. It now clears both, and the stale TODO is rewritten. Residual limit stated plainly in the code comment: clearing local from CLI only clears the node running occ, so bounded staleness now comes from the new TTLs plus worker restarts. That is exactly why the TTLs are in this PR and not a follow-up.

Untouched on purpose: the locking cache (2.96 ms DB vs 0.023 ms memcache per file — 130×) and everything genuinely cluster-shared.

Related Issue

  • Fixes n/a — follow-up hardening from the GHSA-488r-cjpq-p3vc investigation

Motivation and Context

Host-local values in a network-reachable, protocol-injectable backend are a standing supply of write primitives like the one in #41732. Neither of these two needs to be shared between nodes, and moving them also makes their staleness bounded rather than permanent.

How Has This Been Tested?

  • test environment: PHP 8.3.32, sqlite, plus a run with memcache.local => APCu + memcache.distributed => Redis — CI sets no memcache.* and is blind to tiering. (Note for anyone reproducing: OC\Memcache\APCu::isAvailable() gates on the ini key apc.enable_cli, not apcu.enable_cli, or the local tier silently degrades to NullCache on CLI.)
  • test case 1: tests/lib/Files/Type/LoaderTest.php — the old ->expects($this->once())->method('create') mock is replaced with FixedCacheFactory; new testUsesTheLocalCacheTier() (asserts createLocal('mimetypes') once, create/createDistributed never) and testCachedEntriesExpire() (asserts set() receives Loader::CACHE_TTL).
  • test case 2: tests/lib/UrlGeneratorTest.php — new testImagePathIsCached() (asserts the resolved path, that the theme name is part of the key, and that a planted value is served on the second call), testImagePathsUseTheLocalCacheTier(), testImagePathThrowsForMissingImage().
  • test case 3: tests/lib/Repair/RepairMimeTypesTest.phptestMimeTypeCacheIsResetAfterRepair() (version 8.0.0.0 → reset() once) and testMimeTypeCacheIsKeptWhenNothingIsRepaired() (version 10.0.0.0 → never).
  • test case 4: negative verification — each new test was re-run with the source change temporarily reverted and confirmed to fail (3 failures), then restored. A test that passes either way proves nothing.
  • test case 5: real-instance A/B — after the change imagePath and mimetypes keys exist only in APCu, with redis KEYS returning 0 for both namespaces; icons and mimetypes still resolve correctly.
  • test case 6: full tests/lib — 7149 tests, 40221 assertions, no new failures. make test-php-style and phan clean.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Database schema changes (next release will require increase of minor version instead of patch)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Technical debt
  • Tests only (no source changes)

Checklist:

  • Code changes
  • Unit tests added
  • Acceptance tests added
  • Documentation ticket raised: n/a
  • Changelog item, see TEMPLATE

🤖 Generated with Claude Code

@DeepDiver1975
DeepDiver1975 requested a review from a team as a code owner July 27, 2026 14:19
@DeepDiver1975
DeepDiver1975 force-pushed the fix/ghsa-488r-findbinarypath-cache-poisoning branch from 6bf19ed to b8e22c3 Compare July 27, 2026 14:32
@DeepDiver1975
DeepDiver1975 force-pushed the perf/tier-host-local-caches branch from d04f81d to 4cbb897 Compare July 27, 2026 14:32
@DeepDiver1975 DeepDiver1975 self-assigned this Jul 27, 2026
@DeepDiver1975 DeepDiver1975 added this to the development milestone Jul 27, 2026
@DeepDiver1975
DeepDiver1975 requested a review from phil-davis July 27, 2026 14:34
Base automatically changed from fix/ghsa-488r-findbinarypath-cache-poisoning to master July 27, 2026 15:44
The image paths of the active theme and the id <-> mimetype map were both stored
in the distributed memory cache, although both are derived purely from the files
and the database of the instance running the request. A value that only makes
sense for one host does not belong behind a network socket, so both now use the
host local tier via LocalCacheFactory.

Neither cache had a TTL and neither is invalidated on the nodes that did not
cause the change, so both got one: without it a moved installation or a repaired
mimetype table would keep serving the old values indefinitely. This also bounds
the staleness that comes with the local tier, where occ can only clear the cache
of the node it runs on.

Two coherence problems around the mimetype cache come along with it:

- OC\Repair\RepairMimeTypes deletes rows from the mimetypes table and never
  invalidated the cache in front of it, so the id of a deleted mimetype stayed
  cached. It now takes an optional IMimeTypeLoader and resets it when it actually
  repaired something.
- occ upgrade cleared the cache through ICacheFactory::create(), which only ever
  reaches the distributed tier. It now clears both.

URLGenerator resolves its cache once in the constructor instead of on every
imagePath() call.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Thomas Müller <1005065+DeepDiver1975@users.noreply.github.com>
@DeepDiver1975
DeepDiver1975 force-pushed the perf/tier-host-local-caches branch from 4cbb897 to fc71e85 Compare July 27, 2026 15:49

@phil-davis phil-davis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable, good for these cached things to stay in a cache only local to the node.

@DeepDiver1975
DeepDiver1975 merged commit 7b6d573 into master Jul 28, 2026
29 checks passed
@DeepDiver1975
DeepDiver1975 deleted the perf/tier-host-local-caches branch July 28, 2026 11:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

2 participants