Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@guizmaii
Copy link
Contributor

@guizmaii guizmaii commented Mar 18, 2018

Pull Request Checklist

Helpful things

Fixes

#5782

Purpose

It's been a long time now we're waiting for Play to stop send HTML content when an error occurs during a REST call.

The goal of this PR is to provide a minimal JsonDefaultHttpErrorHandler and use it if the charset is application/json in order to answer correctly to REST calls.

Background Context

Minimal approach.

I know there's a big discussion about a refactoring of Play error handling here: #6171 but the discussion is too old and nothing seems to move since 2016.

I want to make a small step forward here, not the big refactoring.

For now I just implemented the JsonDefaultHttpErrorHandler. There's still work to do but I want to know the Play team opinion on this proposal before going farer.

References

override protected def onDevServerError(request: RequestHeader, exception: UsefulException): Future[Result] =
Future.successful(InternalServerError(error(Json.obj(
"id" -> request.id,
"message" -> exception.id,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe this should be id and the request id should be requestId?

Copy link
Contributor Author

@guizmaii guizmaii Mar 19, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok why not. Done

))))

override protected def onProdServerError(request: RequestHeader, exception: UsefulException): Future[Result] =
Future.successful(InternalServerError(error(Json.obj("id" -> request.id, "message" -> exception.id))))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should omit the request ID here. I don't think it's useful for anything to the user. Typically in production a user would report the exception ID and then you can search the logs to find other request information.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok done

@gmethvin
Copy link
Member

@guizmaii Can you add a test for this?

"exception" -> Json.obj(
"title" -> exception.title,
"description" -> exception.description,
"cause" -> Json.obj(
Copy link
Member

@gmethvin gmethvin Mar 20, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There could be a multiple levels of exceptions (i.e. this cause could have a cause), so this might actually miss the root cause, which could be really confusing to the developer. If we want to provide all information this should probably be a recursive structure.

Copy link
Contributor Author

@guizmaii guizmaii Mar 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Instead of a recursive structure, I chose to use an Array. We can discuss that. If you prefer a recursive structure, I can change the code.

I chose the Array because, IMO, it'll look like a stacktrace we have in a console.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'm not sure what the best thing is here. I'd like to make it clear how to find the "root cause" exception though. Using a list seems strange since it's a chain of causes. What was your motivation for using an array?

I guess it doesn't matter that much in the end, since this is just a dev mode tool. Maybe it would be enough to include the entire recursive stack trace in the initial stackTrace and not list causes at all?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is a dev tool, which should make life easier, then I would favor a format that is easily consumable by developers. Maybe we just need the complete stack trace as a String field? I mean, it looks like it would be easier to read than a recursive structure or a List.

Moreover, we can make it extensible so that users can easily replace the error reporting format if they want/need.

WDYT?

Copy link
Member

@gmethvin gmethvin Mar 28, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'm fine with having the complete recursive stack trace in the exception message.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(the test should also verify this)

Copy link
Contributor Author

@guizmaii guizmaii Mar 31, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First,

Maybe we just need the complete stack trace as a String field

Yes, I agree.

Do you think that it's possible to use the Logger to format this string ?

IMO, the best default format we can provide to the user is the one he configured. He could already had configured how errors are reported in its Log4j configuration.

For example, the user could have chosen to log with the root cause first thanks to the "%rEx" option of Logback: https://logback.qos.ch/manual/layouts.html#rootException

It could be a good idea to use this configuration.

WDYT ?

Second,

Moreover, we can make it extensible so that users can easily replace the error reporting format if they want/need.

Good idea. It'll be done

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After some research, I didn't find any way to use the Logger to format the exception.

Copy link
Member

@gmethvin gmethvin Apr 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can get the same output as printStackTrace by doing:

val errors = new StringWriter
exception.printStackTrace(new PrintWriter(errors))
errors.toString

Copy link
Member

@marcospereira marcospereira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for having tests here. Also, we need some documentation about it.

@guizmaii
Copy link
Contributor Author

guizmaii commented Mar 31, 2018

Hi,

I'll try to add tests and documentation this weekend. Thanks for your patience. :)

Jules

Future.successful(res)
}

def formatDevServerErrorException(expcetion: Throwable): JsValue = JsString(ExceptionUtils.getStackTrace(expcetion))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: expcetion -> exception

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. Thanks

@guizmaii
Copy link
Contributor Author

guizmaii commented Apr 2, 2018

Should I add some documentation in the Play documentation ? If yes, could you give me a pointer to where should I add this documentation please ?

Other question:
There's a new version of commons-lang3 (v3.7, Play currently using v3.6), library I use in this PR. Are you interested by an update of this lib ? If yes, do you prefer a new PR or in this one is ok ?

Thanks

"exception" -> Json.obj(
"title" -> exception.title,
"description" -> exception.description,
"cause" -> formatDevServerErrorException(exception.cause)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think what we were suggesting is to display the full stack trace of the original exception like "stacktrace" -> formatDevServerErrorException(exception), right @marcospereira?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gmethvin yes, that is right.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But if I undestood the code correctly, it is already doing that. We only need to change from cause to stacktrace.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code formats exception.cause and I think we want to format exception.

Copy link
Contributor Author

@guizmaii guizmaii Apr 6, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want to format exception which is a UsefulException ?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh I see, the UsefulException is already wrapping the "real" cause exception, which is what we want the stacktrace for.

Copy link
Contributor Author

@guizmaii guizmaii Apr 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly. That's why I think it's better to do formatDevServerErrorException(exception.cause) than formatDevServerErrorException(exception).
The UsefulException doesn't provide any useful information in the stacktrace.

But I can change "cause" -> to "stacktrace" -> if you prefer.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah is stacktrace ok with you? Seems more clear.

@marcospereira
Copy link
Member

Should I add some documentation in the Play documentation ? If yes, could you give me a pointer to where should I add this documentation please ?

This looks like the right place to add the docs:

https://github.com/playframework/playframework/blob/master/documentation/manual/working/scalaGuide/main/http/ScalaErrorHandling.md

You can create a new section there to explain when and how to use/extend the new HttpErrorHandler.

@guizmaii
Copy link
Contributor Author

guizmaii commented Apr 28, 2018

Hi @gmethvin and @marcospereira,

Before writing the documentation, I would argue that the "stacktrace" value is IMO better when it's an Array and not a String.

Here's a version with an Array:

capture d ecran 2018-04-28 18 21 04

And here's the (currently commited) version with String:

capture d ecran 2018-04-28 17 26 15

What are your opinions ?

I should like to remind you that this "stacktrace" field is only present in Dev mode.
So, it should be optimized for human readability and, always IMO, the Array version is better for that purpose.

* In Prod mode, they will not be rendered.
*
* You could override how exceptions are rendered in Dev mode by extending this class and overriding
* the [[formatDevServerErrorException]] method.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrong Javadoc link, must be {@link ...}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A small detail to solve after rebasing.

Thanks for reviewing, @PromanSEW.

@gmethvin
Copy link
Member

gmethvin commented Apr 28, 2018

If the goal really is to optimize for human readability, there's a pretty easy way to do that: send as HTML. Actually, maybe in dev mode this should default to the old behavior if an Accept header is passed and text/html is preferred to application/json? That way, if you're viewing from a browser, you get a nice pretty error message, and if you're making an API request, you'll still get a JSON response.

For example, when my browser (Firefox) loads a page it sends:

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8

In other words, the browser would prefer you send it HTML, XHTML, or XML, but will also accept anything else if you have it.

If you're making an HTTP request from JavaScript to an API returning JSON, you'd usually set:

Accept: application/json

So that provides a very simple way to determine whether the request is coming from a client that needs/prefers a JSON response versus something else.

@gmethvin
Copy link
Member

Actually, maybe we should just make DefaultHttpErrorHandler return either JSON or HTML based on the preferences of the client?

@marcospereira
Copy link
Member

Hey @guizmaii, sorry for taking so long to move this forward. I think it looks good and we can merge and possibly backport to 2.6.x but there are some conflicts now. Could you please rebase with master?

}

/**
* Format a [[Throwable]] as a JSON value.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrong Javadoc link, must be {@link ...}

* Override this method if you want to change how exceptions are rendered in Dev mode.
*
* @param exception an exception
* @return a JSON representation of the passed expcetion
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: expcetion

/**
* Invoked in prod mode when a server error occurs.
*
* Override this rather than [[onServerError]] if you don't want to change Play's debug output when logging errors
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrong Javadoc link, must be {@link ...}

@guizmaii
Copy link
Contributor Author

Hi everyone,

Sorry for the late answer.

If the goal really is to optimize for human readability, there's a pretty easy way to do that: send as HTML.

I do have to disagree. When you enable an "API mode" in a framework, you never want HTML, because you're maybe not testing with a browser but just with curl, for example.

Actually, maybe we should just make DefaultHttpErrorHandler return either JSON or HTML based on the preferences of the client?

It was one of the initial goal of this PR:

The goal of this PR is to provide a minimal JsonDefaultHttpErrorHandler and use it if the charset is application/json in order to answer correctly to REST calls.

(#8299 (comment))

But I'll not implement the second part of this sentence. I don't have the time and it's seems not to be so simple to do.

Writing a default JsonDefaultHttpErrorHandler was the first step toward this goal.
When this PR will be finished (soon), someone else can implement the second step.

Are you ok with that ?

@gmethvin
Copy link
Member

gmethvin commented Jul 27, 2018

I do have to disagree. When you enable an "API mode" in a framework, you never want HTML, because you're maybe not testing with a browser but just with curl, for example.

If you're testing with curl, you'd either explicitly set the Accept header or curl would set Accept: */* by default. So you'd want a way to configure what to do when JSON and HTML are equally preferred (in "API mode", return JSON). Otherwise it should be fine to decide based on Accept.

Writing a default JsonDefaultHttpErrorHandler was the first step toward this goal.
When this PR will be finished (soon), someone else can implement the second step.

That sounds reasonable. So I guess the ideal solution we discussed would involve configuring an ordered list of (content type, error handler) pairs and deciding which one to call based on the request, whenever someone wants to pick that up.

* @param request The request that was bad.
* @param message The error message.
*/
override protected def onBadRequest(request: RequestHeader, message: String): Future[Result] =
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about just extending HttpErrorHandler instead and removing these extra methods? You could also factor out the dev/prod logic for onServerError into a helper so all you need here is onClientError and onServerError. It seems like it would be less code and easier to understand. For the API use case you're describing I can't see much value in overriding this. Actually I think it'd be nicer to have fewer things to override, so it's easier to make sure your error format is consistent.

Copy link
Contributor Author

@guizmaii guizmaii Aug 11, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @gmethvin,

I wasn't sure to understand what you want here, so I implemented what I understood in a second PR you'll find here: https://github.com/guizmaii/playframework/pull/1

I'm not sure that you'll like it that much.
It's not very retrocompatibility friendly.

But it's the only way I see to not duplicate the onClientError and onServerError code.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I implemented the same thing as in https://github.com/guizmaii/playframework/pull/1 but in a more OOP way here: https://github.com/guizmaii/playframework/pull/2

It could be more compliant with the way Play is implemented.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eager to have your feedback on that.

I think that the second solution is nice.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, that's not exactly what I was suggesting.

JsonHttpErrorHandler#onClientError can simply be:

override def onClientError(request: RequestHeader, statusCode: Int, message: String): Future[Result] = {
  Future.successful(Results.Status(statusCode)(error(Json.obj("requestId" -> request.id, "message" -> message))))
}

The implementation is basically the same for all these helpers (onBadRequest, onForbidden, etc.). For a JSON API I don't see any need to single out these error codes and give them their own methods, so I think we should just extend HttpErrorHandler and implement it once in onClientError.

onServerError is a bit trickier to simplify, but it'd be fine to do something like:

def onServerError(request: RequestHeader, exception: Throwable): Future[Result] = {
  try {
    val usefulException = HttpErrorHandlerExceptions.throwableToUsefulException(
      sourceMapper,
      !config.showDevErrors,
      exception
    )
    logServerError(request, usefulException)
    Future.successful(
      InternalServerError(
        if (config.showDevErrors) {
          devServerErrorJson(request, usefulException)
        } else {
          serverErrorJson(request, usefulException)
        }
      )
    )
  } catch {
    case NonFatal(e) =>
      Logger.error("Error while handling error", e)
      Future.successful(InternalServerError)
  }
}

protected def devServerErrorJson(request: RequestHeader, exception: UsefulException): JsValue = {
  error(Json.obj(
    "id" -> exception.id,
    "requestId" -> request.id,
    "exception" -> Json.obj(
      "title" -> exception.title,
      "description" -> exception.description,
      "stacktrace" -> formatDevServerErrorException(exception.cause)
    )
  ))
}

protected def serverErrorJson(request: RequestHeader, exception: UsefulException): JsValue = {
  error(Json.obj("id" -> exception.id))
}

I was thinking we could do something for the common code for onServerError between the HTML and JSON error handlers, but it's not really that much duplicated code. The more complex we make the implementation, the more we have to worry about backwards compatibility issues now and in the future. These error handlers are just meant to be convenient defaults; we don't have to consider every case for customization.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also we can avoid adding a useless => Option[Router] dependency if we avoid extending DefaultHttpErrorHandler.

@guizmaii
Copy link
Contributor Author

guizmaii commented Aug 6, 2018

@gmethvin @marcospereira

I'm using commons-lang3 in this PR which has been removed in that PR: #8455

First, I'm really surprised that copying code (without its tests) from an open-source library is something preferred over just using this lib and benefit from the maintenance and the tests of this lib.

Finally, what should I do? Should I copy the code of the ExceptionUtils.getStackFrames somewhere in Play or should I re-put the dependency and use it ?

EDIT: Solution chosen: I copied ExceptionUtils.getStackFrames methods

@gmethvin
Copy link
Member

@guizmaii I think this would be a great feature for Play 2.7. Do you think you'll have time to consider the other suggested changes? If not I can help since I'm interested in getting this into 2.7. This would be great to show in an example project with PlayService as the recommended setup for a JSON API on Play.

@guizmaii
Copy link
Contributor Author

Hi @gmethvin,

I can try to finish that today and/or tomorrow.

@guizmaii
Copy link
Contributor Author

Done but the doc is still missing. 😕

Copy link
Member

@gmethvin gmethvin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. I agree some docs would be nice :)

* the [[formatDevServerErrorException]] method.
*/
@Singleton
class JsonDefaultHttpErrorHandler(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor thing, but I wonder if we should remove the "default" in the name (i.e. JsonHttpErrorHandler), since this isn't actually the default.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree

Future.successful(InternalServerError)
}

protected def onDevServerError(request: RequestHeader, exception: UsefulException): JsValue = {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getDevErrorJson/getProdErrorJson?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

devErrorJson / prodErrorJson, I don't like the get

@guizmaii
Copy link
Contributor Author

guizmaii commented Sep 26, 2018

@gmethvin,

While writing the documentation and an example project, I saw that the use of the Scala version could be simplified. So I changed its constructor in this commit 1742323

Furthermore, it aligns the Java and Scala implementations

WDYT ?

@guizmaii guizmaii changed the title Implement a JsonDefaultHttpErrorHandler WIP: Implement a JsonDefaultHttpErrorHandler Sep 26, 2018
@guizmaii
Copy link
Contributor Author

guizmaii commented Sep 26, 2018

I have three example projects:

I can transfer them to the playframework github organization if you want ?
Add a pointer to them in the doc ?

@guizmaii guizmaii changed the title WIP: Implement a JsonDefaultHttpErrorHandler WIP: Implement a JsonHttpErrorHandler Sep 26, 2018
@guizmaii
Copy link
Contributor Author

I added a minimal documentation. Do you want more details ?

@guizmaii guizmaii changed the title WIP: Implement a JsonHttpErrorHandler Implement a JsonHttpErrorHandler Sep 26, 2018
Copy link
Member

@marcospereira marcospereira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Thanks for contributing and for the patience, @guizmaii.

@marcospereira marcospereira merged commit 12440f3 into playframework:master Sep 26, 2018
Copy link
Member

@gmethvin gmethvin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @guizmaii. I had a few more suggestions, mainly on the documentation (notably the Java docs are missing).


To use that `HttpErrorHandler` implementation, you should configure the `play.http.errorHandler` configuration property in `application.conf` like this:

play.http.errorHandler = play.http.JsonHttpErrorHandler
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is referring to the Java class. I'd assume Scala users would want to use the Scala version of the error handler (especially if they decide to extend it later).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed here: #8641

* You could override how exceptions are rendered in Dev mode by extending this class and overriding
* the [[formatDevServerErrorException]] method.
*/
@Singleton
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to make this a singleton.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed here: #8641

*/
@Singleton
class JsonHttpErrorHandler(
environment: Environment,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I slightly prefer having a separate config class since it makes it more explicit if you want to force prod behavior in a non-prod mode. This is something I'll sometimes do in tests.

Copy link
Contributor Author

@guizmaii guizmaii Sep 28, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, I disagree because:

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a case class so it's easy to get an instance for compile-time DI. If we wanted to make it easier we could make a trait for it. We actually handle this in DefaultHttpErrorHandler using an alternate constructor.

My original thinking here was actually that Environment contains things we don't need for the error handler. I don't feel that strongly though.


The interface through which Play handles these errors is [`HttpErrorHandler`](api/scala/play/api/http/HttpErrorHandler.html). It defines two methods, `onClientError`, and `onServerError`.

## Handling errors in a JSON API
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add this section to the Java doc as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed here: #8641

## Handling errors in a JSON API

By default, Play returns errors in a HTML format.
For a JSON API, it could be more interesting to return errors in JSON.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not just more interesting, it's more consistent with what most users expect from a JSON API. If you normally return JSON as responses, your errors should also be in JSON so they can be easily parsed and interpreted by clients.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed here: #8641

@guizmaii guizmaii mentioned this pull request Sep 28, 2018
8 tasks
* the [[formatDevServerErrorException]] method.
*/
@Singleton
class JsonHttpErrorHandler(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd add @Inject here since some runtime DI frameworks require the @Inject on the constructor even if there's only one.

@guizmaii guizmaii deleted the json_default_http_error_handler branch November 16, 2018 00:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants