Add comprehensive Feature Gates analysis system for Kubernetes

[Arunodoy18]

Implement feature gates tracking system to support data-driven decisions about eliminating beta stage for new features, as discussed in sig-architecture.

Components added:

• Feature gates SQL metric (metrics/kubernetes/feature_gates.sql)

  • Tracks feature gates across alpha/beta/GA/deprecated states
  • Associates features with SIGs and release milestones
  • Uses regex patterns to extract feature gate information from issues/PRs
  • Provides multi-dimensional aggregation for visualization

• Metrics configuration (metrics/kubernetes/metrics.yaml)

  • Multi-row single column series configuration
  • All time periods supported (d,w,m,q,y)
  • Annotations and aggregation enabled

• Grafana dashboard (grafana/dashboards/kubernetes/feature-gates-by-sig-and-state.json)

  • Feature gates by state over time visualization
  • Interactive filtering by period and states
  • Color-coded state progression (alpha=orange, beta=yellow, GA=green, deprecated=red)
  • Template variables for flexible analysis

• Comprehensive documentation (docs/FEATURE_GATES_ANALYSIS.md)

  • System overview and usage patterns
  • Decision support framework for sig-architecture
  • Implementation details and optimization recommendations

• Updated DASHBOARDS.md with feature gates dashboard entry

This system enables analysis of:

• Beta feature distribution and bottlenecks across SIGs
• Feature maturity progression patterns over time
• Release readiness assessment based on feature states
• Impact analysis for potential beta stage elimination

Supports: Kubernetes sig-architecture beta elimination discussion

Please make sure that you follow instructions from CONTRIBUTING

Specially:

• Check if all tests pass, see TESTING for deatils.
• Make sure you've added test coverage for new features/metrics.
• Make sure you have updated documentation.
• If you added a new metric, please make sure you have been following instructions about adding new metric.

[lukaszgryglicki]

This looks like a very nice feature - I would have to pull that branch and execute on the database to confirm it works correctly, then I need to import new dashboards on test and see how they look and how much time metric calculation takes for all projects, only then I can merge it (and backfill this data for all projects) - I can't do this ATM, will do on my next CNCF working. day after the more urgent migration task, so this is added to my backlog.

[Arunodoy18]

Understood. Thanks for outlining the steps — pulling the branch, validating the DB changes, testing dashboards, and checking performance all make sense. Just let me know when you start working on it; I’ll be available if you need any support or adjustments.

[lukaszgryglicki]

Thanks, but I cannot merge this - I've run the SQL on the actual Kubernetes prod DB and it was clearly never executed, it doesn't even parse:

• unnest(...) is not allowed in WHERE (Postgres rejects the query), and metrics run from the hourly sync, so merging this would crash the entire Kubernetes sync.
• Postgres uses \y for word boundary, \b is a backspace character - so all your regexps match nothing. After fixing the syntax the metric forever returns a single All_All_All = 0.
• multi_row_single_column metrics must return 2 columns: 'prefix,series_name' and value (see metrics/kubernetes/sig_mentions.sql), calc_metric adds the time itself. You return (time, series, value) which panics calc_metric.
• The "extraction" regexp just keeps every word >2 chars, so you're counting distinct English words, not feature gates. I've run a fixed variant on one week of prod data: 10041 "feature gates" - Kubernetes has a few hundred total.
• That single 1-week run took 2 minutes on an idle replica. With d,w,m,q,y + aggregate: 1,7 + annotations_ranges and data since 2014 the backfill would hammer the DB for days and hourly syncs would never finish. Metrics must complete in seconds.
• gha_issues/gha_pull_requests store one row per event, you need the latest-event dedup pattern like every other metric. Also gha_issues_events_labels.issue_id joined with a PR id never matches, so SIG is always unknown for PRs, and rollup() produces NULLs, not 'All', so your CASE branches never fire and the extra UNION ALLs double-count the same aggregates.
• The dashboard queries sfeature_gates with a series column - TSDB stores each series as its own s<name> table with (time, period, value), so every panel and the states variable would just error out. "id" must be null too.

So this needs a redesign, not fixes. Feature gates are structured data: use pkg/features/kube_features.go and/or kep.yaml files from kubernetes/enhancements (stage/milestone/SIG), or at least labels+milestones instead of regexp-scanning 11M+ bodies. Follow the sig_mentions metric format, add drop:, and add tests per TESTING.md/METRICS.md. LMK if you want to go the KEP route, I can help with the metric structure.