DOCK-2582: Better exception messages #6037
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #6037 +/- ##
=============================================
- Coverage 74.56% 74.48% -0.09%
- Complexity 5501 5515 +14
=============================================
Files 381 380 -1
Lines 19786 19810 +24
Branches 2043 2044 +1
=============================================
+ Hits 14754 14756 +2
- Misses 4056 4072 +16
- Partials 976 982 +6
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚨 Try these New Features:
|
| String message = "this is a test"; | ||
| Throwable t = makeNonJavaException(message); | ||
| assertEquals(t.getMessage(), message(t)); | ||
| assertEquals(400, status(t)); |
There was a problem hiding this comment.
Can use an existing enum like https://www.javadoc.io/doc/org.apache.httpcomponents/httpcore/4.4.4/org/apache/http/HttpStatus.html#SC_BAD_REQUEST to be more readable
Ditto below
There was a problem hiding this comment.
Can use an existing enum like https://www.javadoc.io/doc/org.apache.httpcomponents/httpcore/4.4.4/org/apache/http/HttpStatus.html#SC_BAD_REQUEST to be more readable
Ditto below
The raw values are on purpose. If both the code under test and the code doing the testing use the same constant A, if A is accidentally changed to a wrong value, the tests continue to pass, even though the code now works incorrectly. Less of a problem when using third party constants, through. I changed the test to use a different classes' http status constants.
This sounds fine. |
Regarding consistency with each other, if there was some way we could express a "constraint" so that it expanded to both
that would be great. Not sure that's doable given our current architecture, but maybe... |
This kinda relates to another thing... I wanted to add support for constraint "descriptions" in the liquibase XML, so that we could specify that constraint "xyz" meant "your XYZ can't be blank" (or whatever) in the "create constraint" liquibase clause. Didn't seem to be a way to do it, though... |
Not sure if it's what you mean, but there's a fallback method for adding in postgres-specific stuff not supported by JPA. |
Thanks! Was thinking that maybe a "description" could be added to the constraint's liquibase tag or annotation, and we could extract that metadata at build time to create a map of constraint names to descriptions. I suppose we could have a db table that represented that mapping, we'd want to read it early to make sure it is available (because when we're handling a db exception, it's probably inaccessible). Maybe someday, what's in the PR now is probably sufficient... |
dockstore-webservice/src/main/java/io/dockstore/webservice/helpers/ExceptionHelper.java
Outdated
Show resolved
Hide resolved
| return handleJavaThrowable() | ||
| .or(() -> handleConstraintViolationException()) | ||
| .or(() -> handlePersistenceException()) | ||
| .or(() -> result(HttpStatus.SC_BAD_REQUEST)) |
There was a problem hiding this comment.
Shouldn't the fallback be an internal server error? We don't know what happened; it's some sort of unexpected exception -- while it could be a bad request, we don't know for sure and should probably look at it (we'd get notified via CloudWatch alerts).
There was a problem hiding this comment.
Can see it both ways, but your take seems reasonable, let's do it. Also, we should probably handle CustomWebApplicationException by passing those messages/codes get as is. Will make those mods.
We actually have comments on some columns dockstore/dockstore-webservice/src/main/resources/import.sql Lines 5 to 21 in a95a5d5 Commenting is not a SQL standard, but is postgres specific https://www.postgresql.org/docs/current/sql-comment.html |
…plicationException message/status
|
david4096
left a comment
There was a problem hiding this comment.
Great, and very thorough description
Description
This PR changes the webservice to improve the quality of messages that are generated when an exception is thrown.
Rather than patch the particular problem noted in the issue, I took the opportunity to refactor the exception message generation code...
The core change is a new
ExceptionHelperclass, which generates some information (currently, a recommended message and an HTTP status code) for a given exception. The basic concept is that, down the line, we can put much of our exception handling logic intoExceptionHelperand use it everywhere, and, as a start to that process, in this PR, we wire it into the push processing code and our customExceptionMappers.Eventually,
ExceptionHelpercould do other things, like help us to log information about the exception, etc.Yes, it's called
ExceptionHelpereven though it handles allThrowabless. This is a common practice, see apache'sExceptionUtilsfor an example...Currently,
ExceptionHelperhandles throwables as follows:javaorjavaxpackages, use the throwable's message and status code 500.ConstraintViolationException, map the constraint name to a message if possible, status code 409.PersistenceException, use a generic "update failed"-type message, status code 500.Throwable.getMessage()and status code 400.There's a
switchthat maps the constraint names and handles a few constraint violations that I thought were mostly likely to happen, given a list of constraints (via\d <tablename>,select conname from pg_catalog.pg_constraint;, etc) and my non-comprehensive understanding of the code. If the constraint name can't be mapped to a specific message, we say the constraint's name and that it has been violated. There are a ton of different constraints that could be violated, doesn't seem practical to map all of them by hand. If we were more systematic about naming them, we could handle a lot more via some relatively simple code...Opinionated Take: Database constraints are a last line of defense to maintain db consistency. We should strive to check and fail early on conditions that would violate a constraint, because the later in a request that we fail, the more chance that we've updated other state that can't be rolled back (ES, discuss, etc).
Previously, we'd implemented three
ExceptionMappers, two of which were handling subclasses ofPersistenceException. We refactored these to a single mapper, and moved their logic intoExceptionHelper.Originally, I tried to use a method from apache's
ExceptionUtilto generate a list of throwables for a given exception. I happened to look at the code, and noticed it had average case runtime O(N^2). So, it could be attacked by creating an exception with a very large number of chained causes. This could happen: imagine a tag parser that wrapped a thrown exception at each enclosing tag level, and a malicious input file that had tags nested millions deep.Checkstyle had some issues with this code. It didn't like
LinkedHashSetappearing in the var declaration, but it has to be there to ensure the performance problem is remedied. Also, it got confused by theswitchexpression.Leaking information via exception/error messages is always a concern. In certain situations, this PR will put the name of the violated db constraint name into the message. At other times, the message stored in the
Exception(retrieved viaException.getMessage) is used. The net effect isn't much different from how the code works now, but something to consider...There's a unit test that does some basic checks, and I user tested the scenario in the ticket for both workflows and notebooks.
For fun, I wrote
ExceptionHelperto beif-statement-free, and it ended up being fairly intelligible, maybe more so than if it hadifs...Review Instructions
Try to reproduce the bug as detailed in the ticket, and confirm that the message in the App Logs has changed to something like "a workflow with the same name exists".
Issue
https://ucsc-cgl.atlassian.net/browse/DOCK-2582
#5997
Security and Privacy
See note above about information leakage via exception messages.
e.g. Does this change...
Please make sure that you've checked the following before submitting your pull request. Thanks!
mvn clean install@RolesAllowedannotation