Add core and missing transformers for yaml integration testing - phase 2sh #34654
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #34654 +/- ##
==========================================
- Coverage 72.31% 72.15% -0.16%
==========================================
Files 678 732 +54
Lines 99772 109302 +9530
==========================================
+ Hits 72146 78871 +6725
- Misses 26064 28869 +2805
Partials 1562 1562
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Assigning reviewers. If you would like to opt out of this review, comment R: @shunping for label python. Available commands:
The PR bot will only process comments in the main thread (not review comments). |
| @@ -542,7 +542,6 @@ class _Explode(beam.PTransform): | |||
| `cross_product` is set to `true` but only the two rows | |||
| `('a', 1)` and `('b', 2)` when it is set to `false`. | |||
| Only meaningful (and required) if multiple rows are specified. | |||
| error_handling: Whether and how to handle errors during iteration. | |||
There was a problem hiding this comment.
Do we still need this line? It seems the function parameter error_handling is still there.
There was a problem hiding this comment.
I was getting this failure when running with it:
FAILED apache_beam/yaml/integration_tests.py::ExplodeTest::test_Explode_ExternalJavaProvider_WriteToJson_ExternalJavaProvider_3 - ValueError: Error applying transform at line 101: Transform 'beam:schematransform:org.apache.beam:yaml:explode:v1' was configured with unknown fields: {'error_handling'}. Valid fields: {'fields', 'cross_product'}
There was a problem hiding this comment.
I reverted the error_handling doc string deletion and made another TODO to investigate further in conjunction with the other todo. Thanks!
|
R: @tvalentyn for final approval |
derrickaw
force-pushed
the
e2eTopTransformersIntegrationTesting
branch
from
bab6582 to
692d5e7
Compare
|
I think the two failing checks are not related to this PR. |
derrickaw
changed the title
derrickaw
changed the title
|
Run Python_Coverage PreCommit |
|
Run Python_ML PreCommit 3.10 |
|
What is the plan to continuously test these yaml examples? |
|
nvm, these are test pipelines. |
Please add a meaningful description for your change here
Fixes #34611
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.