Skip to content

feat: Default ingestion policy mappings merged with per-tenant mappings - #18926

Merged
salvacorts merged 2 commits into
mainfrom
salvacorts/ingestion-limits-per-policy-overrides/merge-policy-mappings-with-defaults
Aug 21, 2025
Merged

feat: Default ingestion policy mappings merged with per-tenant mappings#18926
salvacorts merged 2 commits into
mainfrom
salvacorts/ingestion-limits-per-policy-overrides/merge-policy-mappings-with-defaults

Conversation

@salvacorts

@salvacorts salvacorts commented Aug 20, 2025

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

This PR adds support for default policy stream mappings that are automatically merged with per-tenant overrides, following the same pattern used for GlobalOTLPConfig. This allows operators to define common policy stream mappings at the distributor level that are automatically applied to all tenants, while still allowing per-tenant customizations.

With this change, operators can define default mappings once and have them automatically applied to all tenants.

How does it work?

  1. Default mappings are configured in the distributor config under default_policy_stream_mappings
  2. During startup, these defaults are applied to the global limits configuration
  3. When loading tenant limits from YAML, the defaults are automatically merged with tenant-specific mappings
  4. Tenant-specific mappings take precedence over defaults, allowing for customization

Example

Default mappings in distributor config:

distributor:
  default_policy_stream_mappings:
    finance:
      - selector: '{namespace="prod", container="billing"}'
        priority: 2
    ops:
      - selector: '{namespace="prod", container="ops"}'
        priority: 1
    security:
      - selector: '{namespace="prod", container="security"}'
        priority: 3

Per-tenant overrides in runtime config:

overrides:
  tenant1:
    policy_stream_mapping:
      finance:
        - selector: '{namespace="prod", container="custom-billing"}'
          priority: 4

Result after merging (i.e. tenant1's effective policy_stream_mapping):

tenant1:
  policy_stream_mapping:
    finance:
      - selector: '{namespace="prod", container="custom-billing"}'
        priority: 4
      - selector: '{namespace="prod", container="billing"}'
        priority: 2
    ops:
      - selector: '{namespace="prod", container="ops"}'
        priority: 1
    security:
      - selector: '{namespace="prod", container="security"}'
        priority: 3

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • Documentation added
  • Tests updated
  • Title matches the required conventional commits format, see here
    • Note that Promtail is considered to be feature complete, and future development for logs collection will be in Grafana Alloy. As such, feat PRs are unlikely to be accepted unless a case can be made for the feature actually being a bug fix to existing behavior.
  • Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
  • If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR
@github-actions

github-actions Bot commented Aug 20, 2025

Copy link
Copy Markdown
Contributor

💻 Deploy preview deleted.

@salvacorts
salvacorts marked this pull request as ready for review August 21, 2025 10:17
@salvacorts
salvacorts requested a review from a team as a code owner August 21, 2025 10:17
@salvacorts
salvacorts merged commit 3b2498b into main Aug 21, 2025
70 checks passed
@salvacorts
salvacorts deleted the salvacorts/ingestion-limits-per-policy-overrides/merge-policy-mappings-with-defaults branch August 21, 2025 11:12
felix0102 pushed a commit to felix0102/loki that referenced this pull request Aug 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2 participants