Implement a useful subset of `wifi` and `socketpool` modules on 🥧🐮 #6933

jepler · 2022-09-21T21:58:28Z

Also supported:

board.LED can be used with DigitalInOut and works

Not (yet) supported:

Various getters and setters of wifi.radio
listen, bind, accept are not implemented (no servers)

Also these modules/features are not supported or enabled yet:

ssl (important, but may not be part of this PR)
mdns (probably will not be part of this PR)
web workflow (probably will not be part of this PR)

At this point, it passes a couple of very simple tests (ntp and plaintext http with adafruit_requests), but is probably not particularly robust.

A big shout out to pico-sdk for providing a great basis for work, and micropython for modlwip, large portions of which were copied verbatim into CircuitPython, with attribution.

Todo:

implement remaining unimplemented methods
more testing
rebase the commit history

It's my view that this can probably be merged independent of ssl/mdns/web workflow, though requests is not too useful in 2022 without ssl support.

Test script `do_dns_ping.py`:

import os
import wifi
import ipaddress
import socketpool

wifi.radio.connect(os.getenv('WIFI_SSID'), os.getenv('WIFI_PASSWORD'), timeout=8)

pool = socketpool.SocketPool(wifi.radio)
info = pool.getaddrinfo("google.com", 80)
addr = info[0][4][0]
print(f"Resolved google as {addr}")
ipv4 = ipaddress.ip_address(addr)

for _ in range(10):
    result = wifi.radio.ping(ipv4)
    if result:
        print("Ping google.com [%s]: %.0f ms" % (addr, result*1000))
    else:
        print("No response")

Test script: `do_ntp`:

import os
import wifi
import socketpool
import adafruit_ntp
import time
wifi.radio.connect(os.getenv('WIFI_SSID'), os.getenv('WIFI_PASSWORD'))


pool = socketpool.SocketPool(wifi.radio)
ntp = adafruit_ntp.NTP(pool, tz_offset=0)

while True:
    print(ntp.datetime)
    time.sleep(1)

Test script: `do_requests_http.py`:

import os
import wifi
import socketpool
import adafruit_requests

TEXT_URL = "http://wifitest.adafruit.com/testwifi/index.html"
wifi.radio.connect(os.getenv('WIFI_SSID'), os.getenv('WIFI_PASSWORD'))

pool = socketpool.SocketPool(wifi.radio)
requests = adafruit_requests.Session(pool)



print("Fetching text from %s" % TEXT_URL)
response = requests.get(TEXT_URL)
print("-" * 40)

print("Text Response: ", response.text)
print("-" * 40)
response.close()

Closes: #6558
Closes: #6628

urjeetpatel · 2022-09-22T18:49:50Z

i am getting an error trying to get the patch compiled

(.venv) ~/projects/circuitpython/ports/raspberrypi$ make -j4 BOARD=raspberry_pi_pico_w V=1
GEN build-raspberry_pi_pico_w/genhdr/mpversion.h
make: cmake: No such file or directory
make: *** [Makefile:95: build-raspberry_pi_pico_w/pioasm/pioasm/pioasm] Error 127
make: *** Waiting for unfinished jobs....

DeadSix27 · 2022-09-22T19:01:33Z

i am getting an error trying to get the patch compiled

(.venv) ~/projects/circuitpython/ports/raspberrypi$ make -j4 BOARD=raspberry_pi_pico_w V=1
GEN build-raspberry_pi_pico_w/genhdr/mpversion.h
make: cmake: No such file or directory
make: *** [Makefile:95: build-raspberry_pi_pico_w/pioasm/pioasm/pioasm] Error 127
make: *** Waiting for unfinished jobs....

Do you have cmake installed?

urjeetpatel · 2022-09-22T19:06:26Z

i am getting an error trying to get the patch compiled

(.venv) ~/projects/circuitpython/ports/raspberrypi$ make -j4 BOARD=raspberry_pi_pico_w V=1
GEN build-raspberry_pi_pico_w/genhdr/mpversion.h
make: cmake: No such file or directory
make: *** [Makefile:95: build-raspberry_pi_pico_w/pioasm/pioasm/pioasm] Error 127
make: *** Waiting for unfinished jobs....

Do you have cmake installed?

whoops... i thought that it was a part of build-essential. but i guess not. Installed and the compile works now :)

jepler · 2022-09-22T20:24:33Z

cmake is a new requirement. It's used to build pioasm from pico-sdk, which wasn't needed before.

anecdata · 2022-09-23T15:28:14Z

This is great, actually useful already! Do you want to hear about issues yet at this very early stage, and if so... where?
e.g.,
• doing socket operations (NTP, HTTP) in a loop every minute or two works fine, but if the loop delay is longer, say 5 minutes... OSError: [Errno 118] EHOSTUNREACH (NTP) ...device still has an IP address
• intermittent [CYW43] got unexpected packet -9 pop up, even when code is not running (just chillin' at the REPL prompt)
• IP address is sometimes 0.0.0.0 instead of None as on espressif port

jepler · 2022-09-28T14:24:42Z

Result of overnight ping DNS & ping test:

 55864 Resolved google as 142.250.191.206
 55864 Ping google.com [142.250.191.206]: 47 ms
 55865 Resolved google as 142.250.191.206
 55865 Ping google.com [142.250.191.206]: 40 ms
 55866 Resolved google as 142.250.191.206
 55866 No response
 55867 Resolved google as 142.250.191.206
[...]
 55901 Resolved google as 142.250.191.206
 55901 No response
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "do_dns_ping.py", line 14, in <module>
OSError: [Errno 2] No such file/directory

I suspect that this test stopped overnight when my wifi router resets, not because of a problem with the pico w itself.

This is a revised version of do_dns_ping that does the DNS query before each ping.

The gpio changes took away enough code space that it doesn't fit.

My pings go out, and then they come back ```py import os import wifi import ipaddress wifi.radio.connect(os.getenv('WIFI_SSID'), os.getenv('WIFI_PASSWORD')) ipv4 = ipaddress.ip_address("8.8.4.4") print("Ping google.com: %f ms" % (wifi.radio.ping(ipv4)*1000)) ```

``` >>> s = socketpool.SocketPool(wifi.radio) >>> s.getaddrinfo("google.com", 80) [(0, 0, 0, '', ('142.250.81.206', 80))] ```

jepler · 2022-09-28T15:15:42Z

I rebased this to have a prettier history (intermediate commits may still not build, I didn't test this). The original history is preserved as https://github.com/jepler/circuitpython/compare/picow-v1?expand=1

I fixed a bug which caused open sockets to not be correctly tracked. As a consequence, things would stop working after 8 sockets had been opened, until reboot.

Tests performed:

dns & ping test (>50000 iterations)
ntp test (>500 iterations)
http test (>800 iterations)

This is ready for review, and we can get "the rest" in future PRs.

DeadSix27 · 2022-09-28T16:19:28Z

thanks for all your hard work its greatly appreciated! Next is ssl and server support I presume?

dhalbert

This looks very nice! I didn't look at the network code in great detail.

Submodules that need updating to what is in adafruit/circuitpython main, it It looks like:

frozen/circuitpython-stage
ports/espressif/esp32-camera

mpy-cross/Makefile

ports/raspberrypi/Makefile

ports/raspberrypi/boards/raspberry_pi_pico_w/board.c

ports/raspberrypi/common-hal/microcontroller/Pin.c

ports/raspberrypi/common-hal/wifi/Radio.c

ports/raspberrypi/common-hal/wifi/__init__.c

shared-bindings/wifi/Radio.c

shared-bindings/digitalio/DigitalInOut.c

this was verbatim-copied from micropython

dhalbert

Great! One of us can merge after the builds finish.

anecdata

Although the connection to the wifi AP is maintained, there is usually a disruption of the network after several minutes. Trying a socket operation in a loop with 5-minute delay gives an exception, whereas I have CircuitPython code that's been running since yesterday that pings every minute to keep the network alive but does HTTP only once an hour.

code:
cp_test.py.txt

output:
cp_out.txt

MicroPython has similar behavior:

code:
up_test.py.txt

output:
up_out.txt

Not definitive, but if one of those mystery packets come in:
[CYW43] got unexpected packet -9
this will delay when the network becomes inaccessible.

Also, initial testing with the MicroPython wlan.config(pm = 0xa11140) toggle described here seems to extend the duration of network accessibility. A test underway is still running after 90-minute interval between network operations. So I suspect the disruptions may be due to the power-saving mode:

3.6.3. Power-saving mode
By default the wireless chip will active power-saving mode when it is idle, which might lead it to being less responsive. If you are running a server or need more responsiveness, you can change this by toggling the power mode.

jepler · 2022-09-28T22:32:34Z

@anecdata I also reproduced something similar on circuitpython. In my case, I got to 300 but failed on the next run.

>>> import do_anecdata
    0 200 OK b'it works\n'
   60 200 OK b'it works\n'
  120 200 OK b'it works\n'
  180 200 OK b'it works\n'
  240 200 OK b'it works\n'
  300 200 OK b'it works\n'
[Errno 2] No such file/directory

Since you first encountered this and now reproduced it micropython and circuitpython, can you open an issue with both us and them, and mention the micropython bug in the circuitpython bug so they're linked together? On the other hand if you're not fully comfortable filing issues on micropython let me know and I'll do it.

Ideally, micropython will be able to resolve the problem or make an effective bug report 'upstream' to raspberry pi since they have a more official standing it seems like. It seems like it may be some kind of bug in the power saving code inside the wifi module, and we (adafruit) don't have access to that code to investigate or modify it.

anecdata · 2022-09-28T22:39:32Z

@jepler I goofed with my math in the earlier comment... MicroPython ran for 9 minutes (not 90!) between intervals, but got OSError: -2 after 10 minutes with no network activity, and Thonny doesn't recover the connection to the device after it's reset, so more uP testing is needed. I'll open a CP issue, and if / when I have enough data, I'll open a uP issue.

It's possible, though not intuitive, that it's by design in uP and there's some command needed to "wake" the network from the local side before a new operation can happen after a delay, even with power-save off. I should also be able to test whether the local side can be kept alive from incoming activity from an external device.

b-blake · 2022-10-03T19:07:50Z

With do_ntp I keep getting the error from tm = ntp.datetime

  File "adafruit_ntp.py", line 80, in datetime
OSError: [Errno 116] ETIMEDOUT

after varying numbers of calls every 30 seconds.
Bruce
CP 8.0.0 Beta 1

jepler · 2022-10-03T19:17:57Z

ntp uses udp which is an unreliable protocol. packets can get lost. if you wrap this in a try/except, can your program continue and have success on the next request?

jepler mentioned this pull request Sep 21, 2022

Add support for Raspberry Pico W(ifi) #6558

Closed

jepler added 7 commits September 28, 2022 10:05

disable rainbowio on this arduino_nano_33_iot

77fcb8a

The gpio changes took away enough code space that it doesn't fit.

allow a board to override link.ld

9caa65c

Reindent a block

22b04ae

cyw43 basic gpio support, hwaddr in boot_out

346fff2

Implement DNS resolution

a7a1bd7

``` >>> s = socketpool.SocketPool(wifi.radio) >>> s.getaddrinfo("google.com", 80) [(0, 0, 0, '', ('142.250.81.206', 80))] ```

Implement enough of socketpool to do ntp and non-https requests

ff77314

jepler force-pushed the branch from 5d9439f to ff77314 Compare September 28, 2022 15:08

jepler marked this pull request as ready for review September 28, 2022 15:08

dhalbert requested changes Sep 28, 2022

View reviewed changes

jepler added 11 commits September 28, 2022 14:38

comment why not actually reset wifi

4380292

comment why it's this way

bc63f93

revert CFLAGS change

e100981

Add CYW43 guards to more things

edf1efd

Add missing NotImplementedErrors

6189156

Remove FIXME

09d4fbc

this was verbatim-copied from micropython

implementations not needed

3d76aa0

restore checking hostname format on esp32 where it was checked before

dbd7ce9

restore submodule commits

72bce51

don't use regcomp to check hostname validity

d59774d

better to explain why this file has no useful content

2dd6df9

dhalbert approved these changes Sep 28, 2022

View reviewed changes

anecdata reviewed Sep 28, 2022

View reviewed changes

dhalbert merged commit db065a2 into adafruit:main Sep 28, 2022

This was referenced Sep 28, 2022

Raspberry Pi Pico W network becomes inaccessible when not used for some time #6958

Closed

Raspberry Pi Pico W network becomes inaccessible when not used for some time micropython/micropython#9455

Open

BjornTheProgrammer mentioned this pull request Sep 29, 2022

Pico W out! dbisu/pico-ducky#80

Closed

jepler deleted the 🥧🐮 branch October 3, 2022 19:18

anecdata mentioned this pull request Oct 11, 2022

espressif: fails to resolve FQDNs with trailing "." #7032

Closed

ZodiusInfuser mentioned this pull request Jan 10, 2023

Support for Pimoroni board with IO Expanders #7440

Closed

Implement a useful subset of wifi and socketpool modules on 🥧🐮 #6933

Implement a useful subset of wifi and socketpool modules on 🥧🐮 #6933

Uh oh!

Conversation

jepler commented Sep 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test script do_dns_ping.py:

Test script: do_ntp:

Test script: do_requests_http.py:

Uh oh!

urjeetpatel commented Sep 22, 2022

Uh oh!

DeadSix27 commented Sep 22, 2022

Uh oh!

urjeetpatel commented Sep 22, 2022

Uh oh!

jepler commented Sep 22, 2022

Uh oh!

anecdata commented Sep 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jepler commented Sep 28, 2022

Uh oh!

jepler commented Sep 28, 2022

Uh oh!

DeadSix27 commented Sep 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dhalbert left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dhalbert left a comment

Choose a reason for hiding this comment

Uh oh!

anecdata left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jepler commented Sep 28, 2022

Uh oh!

anecdata commented Sep 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

b-blake commented Oct 3, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jepler commented Oct 3, 2022

Uh oh!

Uh oh!

Implement a useful subset of `wifi` and `socketpool` modules on 🥧🐮 #6933

Implement a useful subset of `wifi` and `socketpool` modules on 🥧🐮 #6933

jepler commented Sep 21, 2022 •

edited

Loading

Test script `do_dns_ping.py`:

Test script: `do_ntp`:

Test script: `do_requests_http.py`:

anecdata commented Sep 23, 2022 •

edited

Loading

DeadSix27 commented Sep 28, 2022 •

edited

Loading

anecdata left a comment •

edited

Loading

anecdata commented Sep 28, 2022 •

edited

Loading

b-blake commented Oct 3, 2022 •

edited

Loading