Thanks to visit codestin.com
Credit goes to github.com

Skip to content

malformed validation scenarios fail silently when running for Job #431

@ghukill

Description

@ghukill

Schematron based validation scenario that has an XML error with the schematron, fails silently when applied and running for a Job.

However, when testing that scenario against a single record in Combine (via the GUI), the error is apparent, e.g.:

Opening and ending tag mismatch: rule line 33 and pattern, line 34, column 12 (file_3fb170911b114205a51b6a7ecdc5484b.sch, line 34)

Would be beneficial to inform user that a Validation Scenario was erroring for records when running for a Job, instead of failing quietly.

Thoughts/Questions:

  • perhaps the "validation" tab in a Job details page should show errors, in addition to valid/invalid?
  • should a Job get marked as invalid when errors are reported to be conservative with thinking a Job / Records are valid? (tried applying validation, didn't work, assume we don't know about the validity of the records)
  • how to bubble up errors from Spark?
    • each time a Record fails a validation, a corresponding RecordValidation instance is created
    • wonder if this could record the error that occurred during validation? treat the Record as if it failed validation (and in a sense it did), and instead of recording why the validation failed, record the error as best as possible?

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions