Awesome. @wsargent should probably review.

Hi @schmitch -- so the documentation is good on its own terms -- the issue is that we'd like to be able to send WSPart without having to import org.asynchttpclient classes directly, so there needs to be a shell on top for both Scala and Java APIs.

The other thing is that we'd want to attach a Streams API if possible to form upload, and the FeedableBodyGenerator (at least last time I looked) didn't implement backpressure.

@wsargent are case classes which are wrapper around Part okay? And what About java should it have his own wrapper? Actually i'm unsre about the streams api. If i would create it i would've either Need to materialize it before sending or provide a custom multipart body generator. Actualiy file upload materializes before sending as i remember so that wouldn't backpressure. I thought about the streams api, bunt implementing that with asynchttpclient is way harder than just using Akka-Http .

Actually there is a way to insert a Streambody however i would Need to create a custom multipart Generator on top of the streams api.

@wsargent I added wrapper classes for scala now, are something like that sufficant? Actually what I could change would be instead of passing a Seq(WSPart) I could actually pass a WSPartHolder which consists of a Seq(WSFileLikePart) and a Map[String, String] the Map[String, String] will later be converted to multiple StringParts while the WSFileLikePart is actually either a WSByteArrayPart or a WSFilePart

Also I further looked into the Streams API. Something that I came up recently is, that if you want a 100% streamable file upload API with mutlipart/form data you either just use akka-http or you pretty much need to build a lot from scratch.
The only thing to have a Stream Like API Akka-Streams to Ahc would be the setBody(Publisher) method. which uses FeedableBodyGenerator.

However since the current Implementation of Ahc's Multipart/form data is actually built around Netty ByteBuf and WritableByteChannel you would've need to reimplement nearly everything to support a stream's api for multipart. (to push a multipart to the setBody method.)
The other way would be creating a SourcePart and materialize it into a ByteArrayPart before sending, however that wouldn't backpressure and I would call that 'cheating', cause of this issue's I actually thought about just creating a play-ws2 or play-akka-http which would've been built on top of akka-http. However some people would be against that. Even that akka-http isn't as full featured as Ahc it's still is somewhat better in the underlying api. I mean bringing a stream api to multipart/form data in Ahc is way out of scope.

TL/DR: at the current stage of ahc I wouldn't implement a streams api on top of ahc's multipart/form data.

Edit: I'm really unsure about the Java counterparts if creating proxy like objects.

I think there should be only two types of WSPart:

1. DataPart. This is a simple in memory part, equivalent to the MVC multipart form data API. It would conceptually by equivalent to a text input field. It would have a name and a value, both of type String.
2. FilePart. This is conceptually equivalent to a file input field, and would have a name, filename, optional contentType, and most importantly, a content field with a type of Source[ByteString, _]. Files are not the only things that you might want to send, in fact very commonly you will want to stream a request body from the server through to the client without it having to touch the disk.

Also, while a helper may exist to allow passing a List[WSPart], the main API should allow passing a Source[WSPart, _].

In practical terms, implementing all this actually means we don't need to make any fundamental changes to the WS API - since it can all be implemented using the setBody(Source[ByteString, _]), with an appropriate Flow[WSPart, ByteString, _] that converts the Source[WSPart, _] to the formatted body. That said, some helper methods will need to be created since a boundary needs to be generated, which would mean setting the content type to have that boundary in it.

It also means that we don't use the underlying AHC multipart form data support, instead, we implement the formatting of the multipart body using Akka streams. The advantage here is that the same code that formats the WS multipart/form-data bodies can also be used when implementing tests for actions that accept multipart/form-data bodies.

Finally, I think ideally I'd rather not introduce a new API for multipart/form-data when one already exists in Play - in play.api.mvc.MultipartFormData.Part for Scala, and Http.MultipartFormData for Java. These APIs may not be quite ready for consumption by the WS API, but it shouldn't be hard to modify them so they are. This would also aid in the dual purpose of using the same code for WS formatting as is used in formatting the bodies in testing. In general, we should be aiming for more unification between the play.api.mvc APIs and the play.api.ws APIs, and so introducing a new feature like this is a perfect opportunity to do so.

Update:
-- Scala
Actually I could reuse MultipartFormData.FilePart and MultiPartFormData.DataPart
Then I would have:

• MultipartFormData.FilePart[Source[ByteString, _]]
• MultiPartFormData.DataPart

I could give post, put, patch a Seq[MultiPartFormData.Part[Source[ByteString, _]]] or a MultiPartFormData.Part[Source[ByteString, _]] I'm unsure about some things and I actually don't make use of a Flow, everything is explicit. Still working my way towards streams.

Thanks to FileIO.fromFile everybody could easily send files and other stuff. I'm unsure about some things still, I've created a file full of helpers that makes use of Source.combine and I'm unsure if that is a great way to combine the multipart related ByteString to the body parts.

-- Java
still on my todo

@jroper I added a new way of creating ws multipart requests under scala it's not merge ready (not near) could you take a look if that would fit better? It makes use of DataPart and FilePart[Source[ByteString, _]] actually it's just a quick prototype and I used code from akka-http-core.

The good thing, this could be used with Play 2.5 without a official integration if I wouldn't make everything private[ws] it just plugs into StreamedBody.

@schmitch A bit off-topic - anyway:
I haven't really looked into this, but would a fix for #2289 and #2125 (which look like duplicates) be similiar to this PR? What do you think?

@schmitch That's exactly what I was thinking - and starting with the code from akka-http is good, that's exactly what I did for the multipart/form-data parser.

I imagine we should be able to put this into a point release of Play 2.5 if it doesn't make 2.5.0.

A few things:

• BodyPartHelper should be moved to Play core, it is going to be useful to more than just ws. One possibility is to put it in play.core.parsers.Multipart, though it's not really a parser, so maybe play.core.formatters.Multipart. I don't have a strong opinion here.
• Formatters should be slimmed down (remove anything we don't need) and put into the Multipart object as a provite class. If this API is going to be useful elsewhere, then we should force whoever wants to use it make a decision then of where it should live, rather than just make it available for adhoc usage.
• The transform method should take a Source[Part], rather than a Seq[Part]. The first thing the method does is convert the Seq to a Source anyway, Source should be the lowest level API, and helpers that take a Seq can be easily built on top of that.
• Similar to the previous point, the BodyPartHelper could provide a method that looks like this:

def format(boundary: String /* and maybe other params, charset etc */): Flow[Part[...], ByteString, NotUsed]

An API like this would be suitable to make public and provides the highest level of flexibility. With that, the implementation of transform (which now takes a Source) just needs to do body.via(flow(..)) to convert the source.

@jroper I did what you wrote, hopefully this takle's everything you told me.
Actually I hope I could do similar on the Java side.

Edit:
For the Java parts I could actually reuse everything except the transform interface, there would be one that converts everything to a MultipartFormData.Part[Source[ByteString, _]] however Java doesn't have public interfaces to use, I mean the FilePart could be set via play.mvc.MultipartFormData.FilePart<Source<ByteString, _>> however there is no way of defining a DataPart also FilePart has no interface so I would need to add one. Should I just add a public static class DataPart to play.mvc.MultipartFormData and create a interface for FilePart<A> and my DataPart and then write a wrapper that will transform everything to the scala counterpart and run .asJava to get a javadsl.Source or do you have a better idea?

Also I thought about a backport to 2.4 however it looks impossible since even the Enumerator class throws a NotImplementedError:

case class StreamedBody(bytes: Enumerator[Array[Byte]]) extends WSBody {
  throw new NotImplementedError("A streaming request body is not yet implemented")
}

Btw. the idea about Source[ByteString, _] was great since actually we won't use a FileObject and even if we would we could create transform it via FileIO.fromFile which directly returns a Source[ByteString]

@mkurz Actually they are related but unfortunately my quick look at those issue's prevent me from saying that this PR would fix those. Actually my PR will add an easy way to add a MultipartBody to everything that has a Source[ByteString] interface, which actually RequestBuilder has not / not yet, I'm not sure if there will be one. However when you have a Source[ByteString] you could easily materialize that into a Future[ByteString] so that these people could actually add bodies with .asRaw, not sure if I should provide some helper for that on the Java part, I actually start tackling that as soon as the Scala implementation is fine.

Edit2: Cleaned Up everything so that there is only the Scala part now. Could I remove * WSdoes not support streaming body upload. In this case, you should use theFeedableBodyGenerator provided by AsyncHttpClient., too? I mean my PR will fullfil that. Is there a File to upload under the documentation so that I can provide a good documentation for File upload? When not, where can I put it?

Edit3: I added a 'demo' of the Java API which converts everything internally, however I'm not sure if this is a good idea, since type erasure can't check the inner things.
I also used Scala for the transformation since the Java Counterpart will quickly get even more ugly.
Any hints to do it better?

Rebased and added Copyright Header, could somebody look at comment and help me out with the Java API?

/cc @gmethvin @jroper

I rebased, squashed it and fixed the Java API, however there is a IntelliJ error caused by type erasure in the MultipartFormatter, however compiling, etc works fine.

What's the state of this PR? Is it ready to merge?

Actually it's ready to review.

Build is green. @gmethvin Maybe we could get this in 2.5.1?

lgtm but I'd like to see a few tests. You could add a few to play.it.libs.WSSpec.

@gmethvin I added some tests.
Actually it was good that I made some tests since the Java implementation needed a small change.

LGTM, besides the issue mentioned by @mkurz.

I removed the part inside JavaWS.md.

Edit:
Updated Thanks to @wsargent

@gmethvin Build is green

Backport: #5887

@schmitch Thanks a lot for your time and providing this fix! I am quite happy to finally have this working out of the box.
Can I ask you a favor? As you are familiar with the multipart stuff, API, RFC's, etc. now could you have a look at #2289 and #2125 (which look like they are duplicates anyway) and maybe provide a fix for them as well if you find the time? It would be really great to get this long standing issues fixed finally.

@mkurz for #2289 a PR is planned soon (probably I take a look on sunday or on easter since I have some holidays there), and withFile is different, that means sending a file directly. However I could tackle that aswell.
Also I'm mostly fine with multipart stuff, but some RFC's are difficult to read, however there are great people who even helped me on this like wsargent.

@schmitch Great to hear! Looking forward to it 😄 Thanks a lot - I really appreciate it!

So... can we close #2125 now?

No #2125 is only sending a file, however I would tacke that in the upcoming week. The question is only, do we really need a withFile(java.io.File)? I mean with bodyRaw you could already sent files when you read them to a Array[Byte] or ByteString and Streaming the file isn't possible since the route() functions of testing don't provide a streaming interface.

What would be the difference between a withFile method and using a FilePart?

From my point of understanding, withFile would send a file directly without encoding, while the FilePart that goes through withMultipart will actually be a multipart body, but that's what I said withFile is useless, since you could just use bodyRaw after you have your file in a Array[Byte] or ByteString, since the RequestBuilder would need to convert it internally anyway.

Thanks, @schmitch. I also don't see the point of having withFile method. bodyRaw even looks like a better name to me.