Thanks to visit codestin.com
Credit goes to github.com

Skip to content
This repository was archived by the owner on Jan 20, 2025. It is now read-only.

Webserver does not always serve files correctly #979

Closed
TheKaese opened this issue May 1, 2021 · 26 comments
Closed

Webserver does not always serve files correctly #979

TheKaese opened this issue May 1, 2021 · 26 comments
Labels

Comments

@TheKaese
Copy link

TheKaese commented May 1, 2021

I'm no networking expert by anymeans. But I'm running into an issue that looks like only part of an http response is being sent. For example:
image

Note in this screenshot, the tail end of my template index.html is shown. The rest of the content does not appear to ever show up.

This is not always occurring, but I would say 75% of the time my ESP boots up I am unable to access the interface due to this issue.
Full code can be found under /src/WebClient.cpp in https://github.com/TheKaese/IP-ESP32-CAM

@MoJo2600
Copy link

I think I can confirm this issue. My ESP8266 is serving a list of animated .gif files from LittleFS. On some of the files, the transfer just stops. Sometimes the Webserver stops responding completely.
asyncfiles

@jendakol
Copy link

Guys @MoJo2600 @TheKaese, see #770 - it might help you.

@MoJo2600
Copy link

I will test it and report back. But it sounds like it is the same issue.

@TheKaese
Copy link
Author

I took a look at the "changes", and I'm not really understanding what @jendakol is saying in the updated readme.
The example for "Specifying Template Processor callback" doesn't appear to actually do..anything? It just appears to look for a file?
Anyways, the first example seems like a good starting point... though I'd be curious for a rundown as to what's different about this vs. what I was doing in my example. I mean, it seems like my example is just shorthand for the same thing without the extra header stuff. Maybe there's more going on in the background. Very curious what you find @MoJo2600.

@MoJo2600
Copy link

Ok, I'm not a c++ expert, but I tried to implement the changes.

My code for serving static files before:

void handleFileRead(AsyncWebServerRequest *request, String path)
 { 
   // send the right file to the client (if it exists)
   Serial.println("[WEBSERVER] handleFileRead: " + path);
   if (path.endsWith("/"))
     path += "index.html";                    // If a folder is requested, send the index file
   String contentType = getContentType(path); // Get the MIME type
   request->send(LittleFS, path, contentType);
   Serial.println(String("[WEBSERVER] Sent file: ") + path);
 }

Now it looks like this:

bool handleStaticFile(AsyncWebServerRequest *request, String path) {
  
  if (path.endsWith("/")) path += F("index.html");
  
  Serial.println(String("[WEBSERVER] handleFileRead: ") + path);

  String contentType = getContentType(path);
  String pathWithGz = path + ".gz";
  
  if (fileSystem->exists(pathWithGz) || fileSystem->exists(path)) {
    bool gzipped = false;
    if (fileSystem->exists(pathWithGz)) {
        gzipped = true;
        path += ".gz";
    }
      
    Serial.println(String("[WEBSERVER] opening file: ") + path);
    File file = fileSystem->open(path, "r"); // e.g. SPIFFS.open(path, "r"); 

    AsyncWebServerResponse *response = request->beginResponse(
      contentType,
      file.size(),
      [file](uint8_t *buffer, size_t maxLen, size_t total) mutable -> size_t {
        int bytes = file.read(buffer, maxLen);
          
        // close file at the end
        if (bytes + total == file.size()) file.close();

        return max(0, bytes); // return 0 even when no bytes were loaded
      }
    );

    if (gzipped) {
      response->addHeader(F("Content-Encoding"), F("gzip"));
    }

    request->send(response);
    Serial.println(String("[WEBSERVER] sent file: ") + path);
    
    return true;
  }
  
  return false;
}

But sadly, the result is the same. I still have the issue, that files are not served:

error

@zekageri
Copy link

Maximum allowed TCP socket connection on an ESP32 is 10 at once. You must controll the requests on client side to not exceed this request limit at once.

@MoJo2600
Copy link

@zekageri Thank you for this input. Mine is a ESP8266. I implemented some throttling into my application, but still the issue persists.

error_new

I think that I will revert to the default ESP webserver. Not only because of this bug, but because of another issue - I'm interfacing a WS2812B LED strip. Since I moved to the async webserver the first pixel is flickering red, which is usually a sign that the CPU is under too much load to get the timing correct. But this is nothing that this library has to care for :)

@zekageri
Copy link

I belive, on the esp8266 the max allowed tcp connections is like 5 or 6

@jendakol
Copy link

Too bad it didn't help you guys 😞

@TheKaese What I'm trying to say is that there is a problem with serving too big files from the ESP and this code can help with it. However, it looks like you are suffering because you have too many files, not too big files.

@TheKaese
Copy link
Author

Maximum allowed TCP socket connection on an ESP32 is 10 at once. You must controll the requests on client side to not exceed this request limit at once.

I appreciate the feedback, however this is not the problem. I have a single client connected.

Too bad it didn't help you guys 😞

@TheKaese What I'm trying to say is that there is a problem with serving too big files from the ESP and this code can help with it. However, it looks like you are suffering because you have too many files, not too big files.

Definitely appreciate the effort and work you put into your issue. However, I'm just serving two or three small files.. I'm not serving a ton. I guess I have to drop ASyncWebserver 😭

@jendakol
Copy link

I have a single client connected.

I think that doesn't say anything about how many connections does the client creates (assuming you mean client like a web browser).

I like this library but it seems quite dead. My PR is hanging there for more than a year, with no reaction at all. WS client doesn't work, issues exist, no reaction... So sad this is abandoned. Unfortunately, I don't understand it well enough to take it over...

@Pablo2048
Copy link

I appreciate the feedback, however this is not the problem. I have a single client connected.

It doesn't matter how many clients you have connected - it's all about browser behavior. And modern browsers do parallel download for the elements found in HTML. I see two options here:

  1. do lazy loading of images
  2. use one big sprite image

@zekageri
Copy link

zekageri commented Jun 18, 2021

Yes, as Pablo2048 said it's not about the clients. It's about how many requests are mode from one client at a time.

If you have 11 static image elements on one page if you are lucky the esp didn't crash and just not serving the 11th image. Meanwhile if you have other script file calls on the head or anything else, it get's overwhelming for the esp.

For example:

<!-- THE DOCUMENT REQUEST ITSELF IS ONE REQUEST -->

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Document</title>
     
    <!-- HERE YOU INITIATE ONE MORE REQUEST -->
    <link rel="stylesheet" href="someCSS.css" />

    <!-- HERE YOU INITIATE ONE MORE REQUEST -->
    <script defer type="text/javascript" src="someScript.js"></script>
</head>
<body>
    
</body>
</html>

This little document will initiate 4 request.
Why 4?

  • Document itself
  • one css file
  • one js file
  • one favicon.ico --> basic icon for the web, i think every browser will request this.

And it's get's worse when inside your js file, you request things at document load event with a get request, and it's just one client. On an esp32 the ASYNCWEB will let you connect 8 websocket clients. If those clients will request the same page on the same time, it causes chaos.

What can be done about that?

  • defer your script requests on the head. It will make the requests after the page has been loaded.
  • make your scripts as small as you can. You can minify the JS scripts ( there are sites for that, even with APIs ) and gzip them and serve them like this. ( note that if you serve your gzip files with .gz extension the safari browser will not know what the F you just sent to it )
  • try to make a small DOM at pageload.

@jendakol
Copy link

For me, it was better to have few huge files after all, than more small ones. That's a problem with images though, I didn't have any (almost)...

@zekageri
Copy link

There is a problem with huge files too. If you try to serve a file that is more than 130KB some magic happens and your file is stuck half way there. So far nobody knows why. It looks like the problem is with the AsyncWebServer libary. :/

@MoJo2600
Copy link

I can test the 5 connection limit as I already implemented throttling.

But if it is true and this library is no longer maintained I see no purpose in going on with it. I started using it because of the library Elegant OTA which uses this library.

@zekageri
Copy link

It is true. Sadly the dev left this lib a long time ago for some reason. But still i think there is no web server lib out there that is good enough like this. It has problems but it has a lot of benefits over the ones outthere.

@jendakol
Copy link

@zekageri

There is a problem with huge files too

I agree, but that is what I've succeeded to solve - see the #770. It works for me on the ESP32.

@stale
Copy link

stale bot commented Aug 18, 2021

[STALE_SET] This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Aug 18, 2021
@MoJo2600
Copy link

no reaction from maintainer

@stale
Copy link

stale bot commented Aug 18, 2021

[STALE_CLR] This issue has been removed from the stale queue. Please ensure activity to keep it openin the future.

@stale stale bot removed the stale label Aug 18, 2021
@stale
Copy link

stale bot commented Jan 3, 2022

[STALE_SET] This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Jan 3, 2022
@MoJo2600
Copy link

no reaction from maintainer

@stale
Copy link

stale bot commented Jan 13, 2022

[STALE_CLR] This issue has been removed from the stale queue. Please ensure activity to keep it openin the future.

@stale stale bot removed the stale label Jan 13, 2022
@stale
Copy link

stale bot commented Mar 30, 2022

[STALE_SET] This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Mar 30, 2022
@stale
Copy link

stale bot commented Apr 16, 2022

[STALE_DEL] This stale issue has been automatically closed. Thank you for your contributions.

@stale stale bot closed this as completed Apr 16, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

5 participants