Conversation
- Update junit_mini_parser.py to output failing tests as structured JSON. - Update junit_mini_parser_test.py with full test coverage and typing annotations. - Update main.yaml and process_test_results to parse JSON output using jq. Tag: agy Conv: fb32e968-5c28-4711-9b18-47b6c44fd1d0
Update process_test_results action to use set -euo pipefail and improve quoting and failure output parsing. Tag: agy Conv: 497490ac-8ae1-409a-88cb-30bc08e7ffe6 Bug: 551996294
Remove error handling and empty checks from junit_mini_parser.py. Simplify test_failures.json parsing in process_test_results and main.yaml workflows to use a unified jq query without empty checks. Tag: agy Conv: ef023ce0-0476-4a39-82ad-153794531729 Bug: 546722232
Restore the command-line arguments check and logging configuration in the entry point of junit_mini_parser.py as requested. Tag: agy Conv: ef023ce0-0476-4a39-82ad-153794531729 Bug: 546722232
Update process_test_results and main.yaml to check if the first XML file exists before calling junit_mini_parser.py. This prevents crashes when no XML files match the glob and nullglob is not enabled. Tag: agy Conv: ef023ce0-0476-4a39-82ad-153794531729 Bug: 546722232
…evice tests Implement test retry logic in GitHub Actions for on-host and on-device platforms. This introduces a mechanism to track and retry specific failures in PRs while allowing full retries on push events. Update test filtering and reporting to support automated generation of retry filters and consolidated test reporting. This reduces manual developer intervention for flaky tests and optimizes data uploads to external monitoring tools by merging result sets. Tag: agy Conv: b0702142-38ef-440d-acc8-a87a139ea206 Bug: 546722232
Move test_filter.py and test_filter_test.py to cobalt/devinfra/github. Update references in main.yaml, on_host_tests, and on_device_tests workflows. Simplify generate_retry_filter action by removing unnecessary empty check. Tag: agy Conv: ef023ce0-0476-4a39-82ad-153794531729 Bug: 546722232
🤖 Gemini Suggested Commit Message💡 Pro Tips for a Better Commit Message:
|
|
There was a problem hiding this comment.
Code Review
This pull request mocks several GitHub Actions steps—including GN generation, Ninja builds, browser tests, on-host tests, and artifact archiving—by replacing their actual execution with a Python helper script (mock_step.py) and a configuration file (mock_config.json) to simulate build and test outcomes. The review feedback focuses on improving the robustness of the helper script, specifically by resolving the configuration path relative to the script location, safely handling empty directory paths during directory creation, and defensively casting configuration values to prevent type errors.
| config_path = os.path.join( | ||
| os.environ.get('GITHUB_WORKSPACE', '.'), | ||
| '.github/config/mock_config.json') |
There was a problem hiding this comment.
Using os.environ.get('GITHUB_WORKSPACE', '.') to resolve the configuration file path can fail if the script is executed from a different working directory (such as cobalt/src as configured in the build action) and GITHUB_WORKSPACE is not set or points elsewhere. Resolving the path relative to the script's own location (__file__) is more robust and ensures it works consistently across different execution environments.
| config_path = os.path.join( | |
| os.environ.get('GITHUB_WORKSPACE', '.'), | |
| '.github/config/mock_config.json') | |
| script_dir = os.path.dirname(os.path.abspath(__file__)) | |
| repo_root = os.path.abspath(os.path.join(script_dir, '..', '..')) | |
| config_path = os.path.join(repo_root, '.github/config/mock_config.json') |
| failure.text = 'Mocked failure details' | ||
|
|
||
| tree = ET.ElementTree(root) | ||
| os.makedirs(os.path.dirname(filepath), exist_ok=True) |
There was a problem hiding this comment.
If filepath is a simple filename without any directory components (e.g., "result.xml"), os.path.dirname(filepath) will return an empty string "". Calling os.makedirs("", exist_ok=True) will raise a FileNotFoundError in Python. Guarding the directory creation prevents this crash.
| os.makedirs(os.path.dirname(filepath), exist_ok=True) | |
| dirname = os.path.dirname(filepath) | |
| if dirname: | |
| os.makedirs(dirname, exist_ok=True) |
| fail_until = rule.get('fail_until_attempt', 0) | ||
| if attempt <= fail_until: |
There was a problem hiding this comment.
Defensive programming: rule.get('fail_until_attempt', 0) might return a non-integer type (such as a string or None if configured incorrectly or set to null in JSON). Comparing an integer attempt with a non-integer using <= will raise a TypeError in Python 3. Casting and handling potential conversion errors ensures the script is resilient to malformed configuration.
try:
fail_until = int(rule.get('fail_until_attempt', 0))
except (TypeError, ValueError):
fail_until = 0
if attempt <= fail_until:b400838 to
9ac7e7c
Compare
- Add cobalt/devinfra to CI_ESSENTIALS in main.yaml.
- Rename validation rollup jobs to ${{ matrix.name }}_validation and tvos_validation to avoid erroneous retries from shepherd/deflake workflows.
- Fix bash parameter expansion for test_failures in main.yaml validation step.
- Sanitize colon prefixes in test target names across test_filter.py and actions.
- Support dictionary-formatted and string-formatted test targets in on_device_tests and on_host_tests actions and on_device_tests_gateway_client.py.
- Update test_filter_test.py with coverage for colon-prefixed target names.
Tag: agy
Conv: a62819b8-a42e-437a-b0f5-972911759cd8
Bug: 546722232
Tag: agy Conv: 6ba17f35-7b52-48e7-a275-931ef58d3366 Bug: 546722232
Fix formatting violation in cobalt/devinfra/github/test_filter_test.py to satisfy pre-commit yapf hook check. Tag: agy Conv: f5adcc09-6b58-4e6a-9594-32e4a55670ba Bug: 546722232
…ion reporting Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
Add mock_step.py and actions/workflow mocking configuration to execute simulated fast builds and tests on hosted runners for CI Shepherd verification. Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
6e3fe6a to
ca8c6d7
Compare
Mock .github/actions/checkout and .github/actions/depot_tools to skip multi-gigabyte repo cloning and tool bootstrapping in sandbox verification testing. Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
ca8c6d7 to
36b7b0a
Compare
Configure deflake.json with deflake_runs=2 and fail test on shard 1 through attempt 2 to verify custom max attempts limit. Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
Ensure checkout action copies CI_ESSENTIALS into cobalt/src and build action populates test targets and dummy APK artifacts without network calls. Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
36b7b0a to
747acd5
Compare
e69c706 to
07dac0d
Compare
Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
Fail shard 1 on attempt 1, succeed on retry. Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
747acd5 to
ab6a4d0
Compare
33c6196 to
5c47f86
Compare
Verifying flaky test retry success.
Bug: 551996294