Fix: Throw IllegalArgumentException for null IDs/Src/Dst in toGraphX (Fixes #765) #766
Conversation
|
Hi @JeganVG Thanks for this PR! To fix the linter it should be enough to run |
|
We have both scalafmt and scalafix. If you do not want to use pre-commit, just call |
|
Hi @SemyonSinchenko, My bad, I ran the comment and I didn't noticed the error after formatting was done, scalafix and scalafmt passed, I have rebuilt the PR, could you check ? |
|
updated the logic, verified test cases and precommit checks locally, could you check ? ;( |
| } | ||
|
|
||
| // 2. Null Src/Dst | ||
| val badEdges = edges.filter(col(SRC).isNull || col(DST).isNull).limit(1).count() |
There was a problem hiding this comment.
Tbh, I do not like it. It is an expensive operation. I would prefer just to raise an exception instead. If no match: an exception that graph is malformed.
There was a problem hiding this comment.
Thanks for the feedback, I have rebuilt the PR, could you check ?
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #766 +/- ##
==========================================
- Coverage 84.49% 84.08% -0.41%
==========================================
Files 66 66
Lines 3179 3312 +133
Branches 387 394 +7
==========================================
+ Hits 2686 2785 +99
- Misses 493 527 +34 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
SemyonSinchenko
left a comment
There was a problem hiding this comment.
@JeganVG Thanks for the contribution!
…(for #765)
What changes were proposed in this pull request?
Added Two TCs :
Why are the changes needed?
Fixes #765
Currently, if a GraphFrame contains null values in the ID (or src/dst) columns and toGraphX is called, the pattern matching falls, throwing a GraphFramesUnreachableException without proper exception.