Conversation
- Update junit_mini_parser.py to output failing tests as structured JSON. - Update junit_mini_parser_test.py with full test coverage and typing annotations. - Update main.yaml and process_test_results to parse JSON output using jq. Tag: agy Conv: fb32e968-5c28-4711-9b18-47b6c44fd1d0
Update process_test_results action to use set -euo pipefail and improve quoting and failure output parsing. Tag: agy Conv: 497490ac-8ae1-409a-88cb-30bc08e7ffe6 Bug: 551996294
Remove error handling and empty checks from junit_mini_parser.py. Simplify test_failures.json parsing in process_test_results and main.yaml workflows to use a unified jq query without empty checks. Tag: agy Conv: ef023ce0-0476-4a39-82ad-153794531729 Bug: 546722232
Restore the command-line arguments check and logging configuration in the entry point of junit_mini_parser.py as requested. Tag: agy Conv: ef023ce0-0476-4a39-82ad-153794531729 Bug: 546722232
Update process_test_results and main.yaml to check if the first XML file exists before calling junit_mini_parser.py. This prevents crashes when no XML files match the glob and nullglob is not enabled. Tag: agy Conv: ef023ce0-0476-4a39-82ad-153794531729 Bug: 546722232
…evice tests Implement test retry logic in GitHub Actions for on-host and on-device platforms. This introduces a mechanism to track and retry specific failures in PRs while allowing full retries on push events. Update test filtering and reporting to support automated generation of retry filters and consolidated test reporting. This reduces manual developer intervention for flaky tests and optimizes data uploads to external monitoring tools by merging result sets. Tag: agy Conv: b0702142-38ef-440d-acc8-a87a139ea206 Bug: 546722232
Move test_filter.py and test_filter_test.py to cobalt/devinfra/github. Update references in main.yaml, on_host_tests, and on_device_tests workflows. Simplify generate_retry_filter action by removing unnecessary empty check. Tag: agy Conv: ef023ce0-0476-4a39-82ad-153794531729 Bug: 546722232
|
🤖 Gemini Suggested Commit Message💡 Pro Tips for a Better Commit Message:
|
There was a problem hiding this comment.
Code Review
This pull request replaces several real GitHub Actions steps—including GN generation, Ninja builds, browser tests, on-host tests, and artifact archiving—with mocked steps using a newly introduced helper script mock_step.py and a configuration file mock_config.json. The feedback focuses on improving the robustness of the Python script, specifically handling empty directory paths in os.makedirs, catching potential regular expression errors when matching job patterns, using os.path.splitext to safely extract filenames with multiple dots, and validating that parsed test targets are indeed lists to prevent runtime type errors.
| failure.text = 'Mocked failure details' | ||
|
|
||
| tree = ET.ElementTree(root) | ||
| os.makedirs(os.path.dirname(filepath), exist_ok=True) |
There was a problem hiding this comment.
If filepath has no directory component (e.g., it is just a filename), os.path.dirname(filepath) returns an empty string. Calling os.makedirs("", exist_ok=True) raises a FileNotFoundError. To prevent this, only call os.makedirs when the directory path is non-empty.
| os.makedirs(os.path.dirname(filepath), exist_ok=True) | |
| dir_name = os.path.dirname(filepath) | |
| if dir_name: | |
| os.makedirs(dir_name, exist_ok=True) |
| def find_rule(config, job_name): | ||
| rules = config.get('rules', []) | ||
| for rule in rules: | ||
| pattern = rule.get('job_pattern') | ||
| if pattern and re.match(pattern, job_name): | ||
| return rule | ||
| return None |
There was a problem hiding this comment.
If job_pattern in the configuration file contains an invalid regular expression, re.match raises a re.error exception, which crashes the script. Wrapping the match in a try-except block prevents crashes from malformed configuration patterns.
| def find_rule(config, job_name): | |
| rules = config.get('rules', []) | |
| for rule in rules: | |
| pattern = rule.get('job_pattern') | |
| if pattern and re.match(pattern, job_name): | |
| return rule | |
| return None | |
| def find_rule(config, job_name): | |
| rules = config.get('rules', []) | |
| for rule in rules: | |
| pattern = rule.get('job_pattern') | |
| if pattern: | |
| try: | |
| if re.match(pattern, job_name): | |
| return rule | |
| except re.error as e: | |
| print(f'Invalid regex pattern "{pattern}": {e}') | |
| return None |
| for target_path in targets: | ||
| filename = os.path.basename(target_path) | ||
| # Remove extension if any (like .exe or run_ prefix) | ||
| test_name = filename.split('.')[0] |
There was a problem hiding this comment.
| targets = [] | ||
| if args.test_targets: | ||
| try: | ||
| targets = json.loads(args.test_targets) | ||
| except Exception as e: # pylint: disable=broad-except | ||
| print(f'Error parsing test targets: {e}') |
There was a problem hiding this comment.
If args.test_targets is parsed as a non-list JSON type (such as a dictionary, integer, or boolean), iterating over targets later in the script raises a TypeError or yields unexpected keys. Validating that the parsed object is a list of strings prevents runtime errors.
| targets = [] | |
| if args.test_targets: | |
| try: | |
| targets = json.loads(args.test_targets) | |
| except Exception as e: # pylint: disable=broad-except | |
| print(f'Error parsing test targets: {e}') | |
| targets = [] | |
| if args.test_targets: | |
| try: | |
| parsed = json.loads(args.test_targets) | |
| if isinstance(parsed, list): | |
| targets = [str(t) for t in parsed] | |
| else: | |
| print(f'Warning: test targets is not a list: {parsed}') | |
| except Exception as e: # pylint: disable=broad-except | |
| print(f'Error parsing test targets: {e}') |
b400838 to
9ac7e7c
Compare
- Add cobalt/devinfra to CI_ESSENTIALS in main.yaml.
- Rename validation rollup jobs to ${{ matrix.name }}_validation and tvos_validation to avoid erroneous retries from shepherd/deflake workflows.
- Fix bash parameter expansion for test_failures in main.yaml validation step.
- Sanitize colon prefixes in test target names across test_filter.py and actions.
- Support dictionary-formatted and string-formatted test targets in on_device_tests and on_host_tests actions and on_device_tests_gateway_client.py.
- Update test_filter_test.py with coverage for colon-prefixed target names.
Tag: agy
Conv: a62819b8-a42e-437a-b0f5-972911759cd8
Bug: 546722232
Tag: agy Conv: 6ba17f35-7b52-48e7-a275-931ef58d3366 Bug: 546722232
Fix formatting violation in cobalt/devinfra/github/test_filter_test.py to satisfy pre-commit yapf hook check. Tag: agy Conv: f5adcc09-6b58-4e6a-9594-32e4a55670ba Bug: 546722232
…ion reporting Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
Add mock_step.py and actions/workflow mocking configuration to execute simulated fast builds and tests on hosted runners for CI Shepherd verification. Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
0988a97 to
1b68631
Compare
Mock .github/actions/checkout and .github/actions/depot_tools to skip multi-gigabyte repo cloning and tool bootstrapping in sandbox verification testing. Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
1b68631 to
2607f72
Compare
Configure deflake.json with deflake_runs=2 and fail test on shard 1 through attempt 2 to verify custom max attempts limit. Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
Ensure checkout action copies CI_ESSENTIALS into cobalt/src and build action populates test targets and dummy APK artifacts without network calls. Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
2607f72 to
8114ea4
Compare
e69c706 to
07dac0d
Compare
Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
Fail shard 1 persistently on all attempts to verify max retries exhaustion. Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
8114ea4 to
5214df6
Compare
33c6196 to
5c47f86
Compare
Verifying persistent test failure stops after max retries.
Bug: 551996294