Non-blocking writes and chunked output #2282

richardp345 · 2025-10-08T11:25:46Z

richardp345
Oct 8, 2025

I would like to write an HTTP server using Mojolicious that performs non-blocking I/O and chunked transfer encoding.

Looking at the example at https://docs.mojolicious.org/Mojolicious/Guides/Rendering#Chunked-transfer-encoding

This doesn't seem to run:

[Mojolicious-9.41]$ cat ../t1.pl
use Mojolicious::Lite -signatures;

get '/' => sub ($c) {
  $c->write_chunk('<html><head><title>Example</title></head>' => sub ($c) {
    $c->finish('<body>Example</body></html>');
  });
};

app->start;
[Mojolicious-9.41]$ strace -e read,write -t perl -I ./lib ../t1.pl daemon -l http://*:8080

-%-
$ curl http://localhost:8080/
curl: (18) transfer closed with outstanding read data remaining

-%-

09:48:44 write(2, "[2025-10-08 09:48:44.37899] [234"..., 91[2025-10-08 09:48:44.37899] [2347840] [trace] [Yu0qirQXp7x2] 200 OK (0.003426s, 291.886/s)
) = 91
09:48:44 write(4, "HTTP/1.1 200 OK\r\nDate: Wed, 08 O"..., 157) = 157
09:48:44 write(2, "Mojo::Reactor::Poll: I/O watcher"..., 155Mojo::Reactor::Poll: I/O watcher failed: Too many arguments for subroutine 'main::__ANON__' (got 2; expected 1) at lib/Mojolicious/Controller.pm line 328.
) = 155
09:49:14 write(2, "[2025-10-08 09:49:14.38171] [234"..., 84[2025-10-08 09:49:14.38171] [2347840] [trace] Inactivity timeout (see FAQ for more)
) = 84

With my own code, nothing is sent to the client until the sub returns:

Mojolicious-9.41]$ cat ../t2.pl 
#! /usr/bin/perl

use strict;
use warnings;

use Mojolicious::Lite -signatures;

get '/' => sub ($c) {
  $c->write_chunk("hello\n");
  sleep(5);
  $c->finish;
};

app->start();

$ strace -s 1024 -t -e recvfrom,sendto curl -s http://localhost:8080/
09:29:27 sendto(5, "GET / HTTP/1.1\r\nHost: localhost:8080\r\nUser-Agent: curl/8.6.0\r\nAccept: */*\r\n\r\n", 77, MSG_NOSIGNAL, NULL, 0) = 77
09:29:32 recvfrom(5, "HTTP/1.1 200 OK\r\nDate: Wed, 24 Sep 2025 09:29:32 GMT\r\nServer: Mojolicious (Perl)\r\nTransfer-Encoding: chunked\r\n\r\n6\r\nhello\n\r\n0\r\n\r\n", 102400, 0, NULL, NULL) = 128
hello
09:29:32 +++ exited with 0 +++
$ 

[Mojolicious-9.41] $ strace -e read,write -t perl -I ./lib ../t2.pl daemon -l http://*:8080
09:29:27 read(4, "GET / HTTP/1.1\r\nHost: localhost:"..., 131072) = 77
09:29:27 write(2, "[2025-09-24 09:29:27.87982] [195"..., 69[2025-09-24 09:29:27.87982] [1952780] [trace] [vTTBdoMlu1Gp] GET "/"
) = 69
09:29:27 write(2, "[2025-09-24 09:29:27.88117] [195"..., 83[2025-09-24 09:29:27.88117] [1952780] [trace] [vTTBdoMlu1Gp] Routing to a callback
) = 83
09:29:27 write(2, "[2025-09-24 09:29:27.88212] [195"..., 91[2025-09-24 09:29:27.88212] [1952780] [trace] [vTTBdoMlu1Gp] 200 OK (0.002261s, 442.282/s)
) = 91
09:29:32 write(4, "HTTP/1.1 200 OK\r\nDate: Wed, 24 S"..., 128) = 128

The example I referred to at https://docs.mojolicious.org/Mojolicious/Guides/Rendering#Chunked-transfer-encoding states
"A common use would be to send the head section of an HTML document to the browser in advance and speed up preloading of referenced images and stylesheets."

How exactly do you achieve this with Mojolicious, please?

I saw a similar question at #2260 but without a direct solution.

Answered by guest20

Oct 8, 2025

It looks like something got lost when creating this discussion - this line from strace:

09:48:44 write(2, "Mojo::Reactor::Poll: I/O watcher"..., 155Mojo::Reactor::Poll: I/O watcher failed: Too many arguments for subroutine 'main::__ANON__' (got 2; expected 1) at lib/Mojolicious/Controller.pm line 328.

... shows that your mojo app is crashing out with an exception,
and that the exception is being printed to STDERR, but it isn't in the output you pasted.

When I take your example and remove the prototype from off the callback to write_chunk,
that error goes away...
and I get the head and body text back with
carton exec -- perl ./your-software get /.

@kraih Looking at the code, this might ac…

View full answer

guest20 · 2025-10-08T14:38:58Z

guest20
Oct 8, 2025

It looks like something got lost when creating this discussion - this line from strace:

09:48:44 write(2, "Mojo::Reactor::Poll: I/O watcher"..., 155Mojo::Reactor::Poll: I/O watcher failed: Too many arguments for subroutine 'main::__ANON__' (got 2; expected 1) at lib/Mojolicious/Controller.pm line 328.

... shows that your mojo app is crashing out with an exception,
and that the exception is being printed to STDERR, but it isn't in the output you pasted.

When I take your example and remove the prototype from off the callback to write_chunk,
that error goes away...
and I get the head and body text back with
carton exec -- perl ./your-software get /.

@kraih Looking at the code, this might actually be a doc bug?

The docs at/around here

mojo/lib/Mojolicious/Controller.pm

Line 897 in 75508c2

$c = $c->write_chunk($bytes => sub ($c) {...});

... shows the sub with a one-item prototype: sub ($c) {...}

but write_chunk at/around here
https://github.com/mojolicious/mojo/blob/75508c2df11e0961b3254989495df0bbbb31ef2a/lib/Mojolicious/Controller.pm#L326C5-L326C16
calls that callback as a method, with @_, which contains $chunk and $cb :

sub write_chunk {
  my ($self, $chunk, $cb) = @_;
  $self->res->content->write_chunk($chunk, $cb ? sub { shift; $self->$cb(@_) } : ());
  return $self->rendered;
}

which is effectively:

$cb->($self, $chunk, $cb)

and that's not compatible with a $cb prototyped as sub($c){ ... }.

... so maybe the docs shouldn't prototype that code-ref?

@richardp345 Also, your use of sleep in your second example is no good if you're trying to do async IO, since sleep is very-sync.

0 replies

richardp345 · 2025-10-08T20:59:05Z

richardp345
Oct 8, 2025
Author

Thank you! It does look like a mistake in the docs. WIthout the proto the code runs.

$ cat t1.pl 
#! /usr/bin/perl

use strict;
use warnings;

use Mojolicious::Lite -signatures;

sub ss {
 my ($c, $data) = @_;

 my $d = shift @$data;
 if (defined $d) {
   sleep(2);
   $c->write_chunk($d => sub { ss($c, $data) });
 } else {
   $c->finish("done");
 }
}

my @data = @INC;
#my @data = ('a') x 100_000;

get '/' => sub ($c) {
  $c->write_chunk('<html><head><title>Example</title></head>' => sub { ss($c, [ @data ]) });
};

app->start;


[Mojolicious-9.41]$ strace -e read,write -t perl -I ./lib ../t1.pl daemon -l http://*:8080
-%-
20:49:42 write(4, "HTTP/1.1 200 OK\r\nDate: Wed, 08 O"..., 157) = 157
20:49:44 write(4, "\r\n5\r\n./lib", 10) = 10
20:49:46 write(4, "\r\n1b\r\n/usr/local/lib64/perl5/5.3"..., 33) = 33
20:49:48 write(4, "\r\n1b\r\n/usr/local/share/perl5/5.3"..., 33) = 33
20:49:50 write(4, "\r\n1c\r\n/usr/lib64/perl5/vendor_pe"..., 34) = 34
20:49:52 write(4, "\r\n1c\r\n/usr/share/perl5/vendor_pe"..., 34) = 34
20:49:54 write(4, "\r\n10\r\n/usr/lib64/perl5", 22) = 22
20:49:56 write(4, "\r\n10\r\n/usr/share/perl5", 22) = 22
20:49:56 write(4, "\r\n4\r\ndone\r\n0\r\n\r\n", 16) = 16
20:49:56 read(4, "", 131072)            = 0

$ strace -s 1024 -t -e recvfrom,sendto curl -s http://localhost:8080/
20:49:42 sendto(5, "GET / HTTP/1.1\r\nHost: localhost:8080\r\nUser-Agent: curl/8.6.0\r\nAccept: */*\r\n\r\n", 77, MSG_NOSIGNAL, NULL, 0) = 77
20:49:42 recvfrom(5, "HTTP/1.1 200 OK\r\nDate: Wed, 08 Oct 2025 20:49:42 GMT\r\nServer: Mojolicious (Perl)\r\nTransfer-Encoding: chunked\r\n\r\n29\r\n<html><head><title>Example</title></head>", 102400, 0, NULL, NULL) = 157
20:49:44 recvfrom(5, "\r\n5\r\n./lib", 102400, 0, NULL, NULL) = 10
20:49:46 recvfrom(5, "\r\n1b\r\n/usr/local/lib64/perl5/5.38", 102400, 0, NULL, NULL) = 33
20:49:48 recvfrom(5, "\r\n1b\r\n/usr/local/share/perl5/5.38", 102400, 0, NULL, NULL) = 33
20:49:50 recvfrom(5, "\r\n1c\r\n/usr/lib64/perl5/vendor_perl", 102400, 0, NULL, NULL) = 34
20:49:52 recvfrom(5, "\r\n1c\r\n/usr/share/perl5/vendor_perl", 102400, 0, NULL, NULL) = 34
20:49:54 recvfrom(5, "\r\n10\r\n/usr/lib64/perl5", 102400, 0, NULL, NULL) = 22
20:49:56 recvfrom(5, "\r\n10\r\n/usr/share/perl5", 102400, 0, NULL, NULL) = 22
20:49:56 recvfrom(5, "\r\n4\r\ndone\r\n0\r\n\r\n", 102400, 0, NULL, NULL) = 16
<html><head><title>Example</title></head>./lib/usr/local/lib64/perl5/5.38/usr/local/share/perl5/5.38/usr/lib64/perl5/vendor_perl/usr/share/perl5/vendor_perl/usr/lib64/perl5/usr/share/perl5done20:49:56 +++ exited with 0 +++

N.B. the sleep is a contrived example of a synchronous call. The real use case is to read from a DB.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Non-blocking writes and chunked output #2282

Uh oh!

{{title}}

Uh oh!

Replies: 2 comments

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Non-blocking writes and chunked output #2282

Uh oh!

richardp345 Oct 8, 2025

Replies: 2 comments

Uh oh!

guest20 Oct 8, 2025

Uh oh!

richardp345 Oct 8, 2025 Author

richardp345
Oct 8, 2025

guest20
Oct 8, 2025

richardp345
Oct 8, 2025
Author