Skip to content

fix(webapp): retry branch environment lookups that race replica lag - #4857

Open
myftija wants to merge 1 commit into
mainfrom
fix/preview-branch-auth-replica-race-tri-13546
Open

fix(webapp): retry branch environment lookups that race replica lag#4857
myftija wants to merge 1 commit into
mainfrom
fix/preview-branch-auth-replica-race-tri-13546

Conversation

@myftija

@myftija myftija commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Background

The first deploy of a newly created preview branch could fail because of a read-replica race: the deploy creates the branch environment on the primary and immediately authenticates with it, but auth-time branch resolution reads the replica, which may not have the row yet. The request then fails with "Branch not found" or "No matching branch env" even though the branch exists.

Fix

Branch environment lookups in API auth now retry the replica once with a short jittered delay and fall back to the primary before reporting the branch missing. The retry primitive follows the same replica-retry-then-primary approach as #3889.

@changeset-bot

changeset-bot Bot commented Sep 1, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: ea07db3

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai

coderabbitai Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Team

Run ID: 6782e439-b08a-42e4-9c27-425e4889c006

📥 Commits

Reviewing files that changed from the base of the PR and between 9591de6 and ea07db3.

📒 Files selected for processing (1)
  • apps/webapp/app/models/runtimeEnvironment.server.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 7 remain after this review.

📜 Recent review details
⏰ Context from checks skipped due to timeout. (35)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (17, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (22, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (21, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (20, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (11, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (19, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (18, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (24, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (15, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (23, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (16, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (14, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (9, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (10, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (5, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (12, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (3, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (8, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (4, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (6, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (2, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (7, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (1, 24)
  • GitHub Check: webapp / 🧪 Unit Tests: Webapp (13, 24)
  • GitHub Check: obsmap / 🧪 Unit Tests: Observability Map
  • GitHub Check: fk-cascade-guard / fk-cascade-guard
  • GitHub Check: internal / 🧪 Unit Tests: Internal
  • GitHub Check: e2e-webapp / 🧪 E2E Tests: Webapp (1, 2)
  • GitHub Check: typecheck / typecheck
  • GitHub Check: e2e-webapp / 🧪 E2E Tests: Webapp (2, 2)
  • GitHub Check: runops-guard / runops-guard
  • GitHub Check: code-quality / code-quality
  • GitHub Check: audit
  • GitHub Check: Analyze (javascript-typescript)
  • GitHub Check: 🛡️ E2E Auth Tests (full)
🧰 Additional context used
📓 Path-based instructions (9)
Never use `request.signal` to detect client disconnects. Use `getRequestAbortSignal()` from `app/services/httpAsyncStorage.server.ts`, which is wired to Express response close events.

📄 CodeRabbit inference engine (apps/webapp/CLAUDE.md)

Files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
For dashboard changes, visually verify the running Remix app with Chrome DevTools MCP, using snapshots, screenshots, interaction, and console-message checks as appropriate.

📄 CodeRabbit inference engine (apps/webapp/CLAUDE.md)

Files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
**Prefer static imports over dynamic imports.** Only use dynamic `import()` when:

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
Add crumbs as you write code — not just when debugging. Mark lines with

📄 CodeRabbit inference engine (AGENTS.md)

Files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
Use zod for validation in packages/core and apps/webapp

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
Access environment variables through the `env` export of `env.server.ts` instead of directly accessing `process.env`

📄 CodeRabbit inference engine (.cursor/rules/webapp.mdc)

Files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
Use function declarations instead of default exports

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
Use types over interfaces for TypeScript

📄 CodeRabbit inference engine (.github/copilot-instructions.md)

Files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
When creating or editing OTEL metrics (counters, histograms, gauges), ensure metric attributes have low cardinality by using only enums, booleans, bounded error codes, or bounded shard IDs

📄 CodeRabbit inference engine (.cursor/rules/otel-metrics.mdc)

Files:

  • apps/webapp/app/models/runtimeEnvironment.server.ts
🔇 Additional comments (1)
apps/webapp/app/models/runtimeEnvironment.server.ts (1)

9-11: LGTM!

Also applies to: 115-128, 249-251, 272-274


Walkthrough

The change adds a shared read-replica retry helper with jittered delays, primary fallback, outcome telemetry, and tests. Runtime environment resolution, personal access token authentication, organization access token authentication, and RBAC branch resolution use the retry path for missing branch environments. A changelog entry documents the fix for first deploys on newly created preview branches.

Merge Risk: 🟡 Moderate · up to ea07d

This PR reduces preview-branch authentication failures by retrying replica lookups and falling back to the primary, but it can briefly combine fresh branch data with stale authorization state during revocations or membership changes, while invalid branch requests can add primary-database load. Required branch-local instrumentation and test placement follow-up also remain outstanding, so merge requires explicit owner acceptance or fixes.

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Description check ⚠️ Warning The description explains the background and fix, but it omits the required issue reference, checklist, Testing, Changelog, and Screenshots sections from the repository template. Add the required template sections. Include the issue reference, completed checklist, test steps and results, changelog entry, and screenshots or an explicit statement that screenshots are not applicable.
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 6 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: retrying branch environment lookups to handle read-replica lag.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/preview-branch-auth-replica-race-tri-13546

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

devin-ai-integration[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

The first deploy of a newly created branch upserts the branch environment
and immediately authenticates with it, so the auth-time replica read can
miss the just-committed row and the request fails. Branch lookups in API
auth (PAT and OAT resolution, the RBAC bearer resolver, and the legacy
API key resolver) now retry the replica once with jitter and fall back
to the primary before reporting the branch missing. Installs without a
dedicated read replica skip the retry entirely.
@myftija
myftija force-pushed the fix/preview-branch-auth-replica-race-tri-13546 branch from 9591de6 to ea07db3 Compare September 1, 2026 09:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant