Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Fix ZStream.fromInputStream #3678

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 21, 2020
Merged

Fix ZStream.fromInputStream #3678

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b6a4a10
Fix ZStream.fromInputStream
iravid May 21, 2020
c94f068
Add warning to Chunk.fromArray
iravid May 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions core/shared/src/main/scala/zio/Chunk.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -767,6 +767,8 @@ object Chunk {

/**
* Returns a chunk backed by an array.
*
* WARNING: The array must not be mutated after creating the chunk.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it would make sense to have a naming convention to serve the same purpose as this warning. Chunk.fromArray could copy by default and then the alternate "unsafe" one would be the fast version. Depends if there is already a precedent for that in the ZIO codebase.

Also when seeing this, I also thought about if there might be a use for a Chunk.fromArray[A](array: Array[A], start: Int, len: Int) version.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@reibitto I like the slicing version! Could be a nice PR if you have the cycles.

Re the naming: I think a warning is sufficient because we're not going to add the other variant. The caller can always defensively copy if required.

*/
def fromArray[A](array: Array[A]): Chunk[A] =
(if (array.isEmpty) Empty
Expand Down
22 changes: 15 additions & 7 deletions streams-tests/shared/src/test/scala/zio/stream/ZStreamSpec.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package zio.stream

import java.io.ByteArrayInputStream
import java.util.concurrent.TimeUnit

import scala.concurrent.ExecutionContext
Expand Down Expand Up @@ -2661,13 +2662,20 @@ object ZStreamSpec extends ZIOBaseSpec {
assertM(ZStream.fromEffectOption(fa).runCollect)(equalTo(List()))
}
),
testM("fromInputStream") {
import java.io.ByteArrayInputStream
val chunkSize = ZStream.DefaultChunkSize
val data = Array.tabulate[Byte](chunkSize * 5 / 2)(_.toByte)
def is = new ByteArrayInputStream(data)
ZStream.fromInputStream(is, chunkSize).runCollect map { bytes => assert(bytes.toArray)(equalTo(data)) }
},
suite("fromInputStream")(
testM("example 1") {
val chunkSize = ZStream.DefaultChunkSize
val data = Array.tabulate[Byte](chunkSize * 5 / 2)(_.toByte)
def is = new ByteArrayInputStream(data)
ZStream.fromInputStream(is, chunkSize).runCollect map { bytes => assert(bytes.toArray)(equalTo(data)) }
},
testM("example 2") {
checkM(Gen.small(Gen.listOfN(_)(Gen.anyByte)), Gen.int(1, 10)) { (bytes, chunkSize) =>
val is = new ByteArrayInputStream(bytes.toArray)
ZStream.fromInputStream(is, chunkSize).runCollect.map(assert(_)(equalTo(bytes)))
}
}
),
testM("fromIterable")(checkM(Gen.small(Gen.listOfN(_)(Gen.anyInt))) { l =>
def lazyL = l
assertM(ZStream.fromIterable(lazyL).runCollect)(equalTo(l))
Expand Down
3 changes: 1 addition & 2 deletions streams/js/src/main/scala/zio/stream/platform.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,14 +18,13 @@ trait ZStreamPlatformSpecificConstructors { self: ZStream.type =>
ZStream {
for {
done <- Ref.make(false).toManaged_
buf <- Ref.make(Array.ofDim[Byte](chunkSize)).toManaged_
capturedIs <- Managed.effectTotal(is)
pull = {
def go: ZIO[Any, Option[IOException], Chunk[Byte]] = done.get.flatMap {
if (_) Pull.end
else
for {
bufArray <- buf.get
bufArray <- UIO(Array.ofDim[Byte](chunkSize))
bytesRead <- Task(capturedIs.read(bufArray))
.refineToOrDie[IOException]
.mapError(Some(_))
Expand Down
3 changes: 1 addition & 2 deletions streams/jvm/src/main/scala/zio/stream/platform.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,14 +164,13 @@ trait ZStreamPlatformSpecificConstructors { self: ZStream.type =>
ZStream {
for {
done <- Ref.make(false).toManaged_
buf <- Ref.make(Array.ofDim[Byte](chunkSize)).toManaged_
capturedIs <- Managed.effectTotal(is)
pull = {
def go: ZIO[Blocking, Option[IOException], Chunk[Byte]] = done.get.flatMap {
if (_) Pull.end
else
for {
bufArray <- buf.get
bufArray <- UIO(Array.ofDim[Byte](chunkSize))
bytesRead <- blocking
.effectBlocking(capturedIs.read(bufArray))
.refineToOrDie[IOException]
Expand Down