Thanks to visit codestin.com
Credit goes to github.com

Skip to content

installer: rewrite wheels config, fix pycountry#4180

Merged
bastimeyer merged 1 commit into
streamlink:masterfrom
bastimeyer:installer/fix-pycountry
Nov 17, 2021
Merged

installer: rewrite wheels config, fix pycountry#4180
bastimeyer merged 1 commit into
streamlink:masterfrom
bastimeyer:installer/fix-pycountry

Conversation

@bastimeyer

@bastimeyer bastimeyer commented Nov 17, 2021

Copy link
Copy Markdown
Member

Resolves #4179

This fixes the broken pycountry import in the Windows installer's Python environment by building the pycountry wheel locally in the makeinstaller.sh script and including it instead of the source tarball:
#4179 (comment)

A 3.0.1 release needs to be published right after testing/confirming and merging this.

This fix isn't an ideal solution. All I want is to resolve this as quickly as possible. I will take a look at rewriting this properly afterwards, similar to the appimages build config, which uses a package "lockfile" (which we don't have here) with hashes. Pynsist unfortunately doesn't support this and since we're now using pip for building wheels locally anyway, we can utilize that and switch to the local_wheels config completely (via pip download $WHEEL_DEPS and pip wheel $SRC_DEPS).

$ ./script/makeinstaller.sh
[makeinstaller.sh] Setting up clean build directories
[makeinstaller.sh] Building streamlink-3.0.0+1.g132d37c package
running build
running build_py
UPDATING build/lib/streamlink/_version.py
set build/lib/streamlink/_version.py to '3.0.0+1.g132d37c'
[makeinstaller.sh] Creating empty plugin files
[makeinstaller.sh] Creating icons
[makeinstaller.sh] Locally building missing dependency wheels
Collecting pycountry==20.7.3
  Downloading pycountry-20.7.3.tar.gz (10.1 MB)
     |████████████████████████████████| 10.1 MB 1.0 MB/s            
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: pycountry
  Building wheel for pycountry (setup.py) ... done
  Created wheel for pycountry: filename=pycountry-20.7.3-py2.py3-none-any.whl size=10746883 sha256=6db926a1bbf100cf484c3120595f756546ab8aad0cd62f8a4bc4e58899c5c4d0
  Stored in directory: /tmp/pip-ephem-wheel-cache-pvl9c2j6/wheels/de/cb/0e/b40fff1168704e2498630eee7ad70a78b458fe4a902179ae2c
Successfully built pycountry
[makeinstaller.sh] Configuring installer
[makeinstaller.sh] Preparing assets
/home/basti/repos/streamlink/build/cache/ffmpeg-n4.4.1-win32-gpl-4.4.zip: OK
[makeinstaller.sh] Assembling files directory
Archive:  /home/basti/repos/streamlink/build/cache/ffmpeg-n4.4.1-win32-gpl-4.4.zip
   creating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/LICENSE.txt  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/BUILDINFO.txt  
   creating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/bin/
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/bin/ffplay.exe  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/bin/ffprobe.exe  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/bin/ffmpeg.exe  
   creating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/ffplay.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/ffmpeg-protocols.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/mailing-list-faq.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/faq.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/ffmpeg-formats.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/libavdevice.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/platform.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/libavcodec.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/ffprobe.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/fate.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/libavfilter.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/ffmpeg-codecs.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/general.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/nut.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/ffmpeg.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/ffmpeg-bitstream-filters.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/ffplay-all.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/ffmpeg-resampler.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/developer.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/ffmpeg-devices.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/ffmpeg-filters.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/git-howto.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/ffmpeg-scaler.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/ffprobe-all.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/ffmpeg-utils.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/libswresample.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/libavformat.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/ffmpeg-all.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/libswscale.html  
  inflating: /tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/doc/libavutil.html  
install: creating directory '/home/basti/repos/streamlink/build/files/ffmpeg'
'/tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/bin/ffmpeg.exe' -> '/home/basti/repos/streamlink/build/files/ffmpeg/ffmpeg.exe'
'/tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/LICENSE.txt' -> '/home/basti/repos/streamlink/build/files/ffmpeg/LICENSE.txt'
'/tmp/tmp.1nSkP7NsJn/0/ffmpeg-n4.4.1-win32-gpl-4.4/BUILDINFO.txt' -> '/home/basti/repos/streamlink/build/files/ffmpeg/BUILDINFO.txt'
[makeinstaller.sh] Building streamlink-3.0.0_1.g132d37c installer
Unpacking Python...
Copying packages into build directory...
Using cached wheel: /home/basti/.cache/pynsist/pypi/certifi/2021.5.30/certifi-2021.5.30-py2.py3-none-any.whl
Using cached wheel: /home/basti/.cache/pynsist/pypi/charset-normalizer/2.0.4/charset_normalizer-2.0.4-py3-none-any.whl
Using cached wheel: /home/basti/.cache/pynsist/pypi/idna/3.2/idna-3.2-py3-none-any.whl
Using cached wheel: /home/basti/.cache/pynsist/pypi/isodate/0.6.0/isodate-0.6.0-py2.py3-none-any.whl
Using cached wheel: /home/basti/.cache/pynsist/pypi/lxml/4.6.4/lxml-4.6.4-cp39-cp39-win32.whl
Using cached wheel: /home/basti/.cache/pynsist/pypi/pycryptodome/3.10.1/pycryptodome-3.10.1-cp35-abi3-win32.whl
Using cached wheel: /home/basti/.cache/pynsist/pypi/PySocks/1.7.1/PySocks-1.7.1-py3-none-any.whl
Using cached wheel: /home/basti/.cache/pynsist/pypi/requests/2.26.0/requests-2.26.0-py2.py3-none-any.whl
Using cached wheel: /home/basti/.cache/pynsist/pypi/six/1.16.0/six-1.16.0-py2.py3-none-any.whl
Using cached wheel: /home/basti/.cache/pynsist/pypi/urllib3/1.26.6/urllib3-1.26.6-py2.py3-none-any.whl
Using cached wheel: /home/basti/.cache/pynsist/pypi/websocket-client/1.2.1/websocket_client-1.2.1-py2.py3-none-any.whl
Collecting wheel file: pycountry-20.7.3-py2.py3-none-any.whl (from: /tmp/tmp.Se2qzPr8Y2/*.whl)
Writing NSI file to nsis/installer.nsi

~~~ Running makensis ~~~
Processing config: /etc/nsisconf.nsh
Processing script file: "nsis/installer.nsi" (UTF8)

Processed 1 file, writing output (x86-unicode):

Output: "/home/basti/repos/streamlink/dist/streamlink-3.0.0_1.g132d37c.exe"
Install: 6 pages (384 bytes), 5 sections (2 required) (163960 bytes), 3430 instructions (96040 bytes), 1663 strings (58828 bytes), 1 language table (354 bytes).
Uninstall: 1 page (128 bytes), 1 section (32792 bytes), 185 instructions (5180 bytes), 112 strings (3268 bytes), 1 language table (198 bytes).
Datablock optimizer saved 48300 bytes (~0.1%).

Using lzma compression.

EXE header size:              134144 / 102912 bytes
Install code:                  20367 / 238110 bytes
Install data:               39629435 / 120274517 bytes
Uninstall code+data:           26561 / 38745 bytes
CRC (0x080A95BA):                  4 / 4 bytes

Total size:                 39810511 / 120654288 bytes (32.9%)
Installer written to /home/basti/repos/streamlink/dist/streamlink-3.0.0_1.g132d37c.exe
[makeinstaller.sh] Success!

$ streamlink --loglevel debug
[cli][debug] OS:         Windows 10
[cli][debug] Python:     3.9.8
[cli][debug] Streamlink: 3.0.0+1.g132d37c
[cli][debug] Requests(2.26.0), Socks(1.7.1), Websocket(1.2.1)
[cli][debug] Arguments:
[cli][debug]  --loglevel=debug
[cli][debug]  --ffmpeg-ffmpeg=C:\Program Files (x86)\Streamlink\ffmpeg\ffmpeg.exe
usage: streamlink [OPTIONS] <URL> [STREAM]

Use -h/--help to see the available options or read the manual at https://streamlink.github.io

@bastimeyer bastimeyer force-pushed the installer/fix-pycountry branch from 132d37c to 8a7b779 Compare November 17, 2021 10:15
@bastimeyer

bastimeyer commented Nov 17, 2021

Copy link
Copy Markdown
Member Author

If there's no review within the next two hours, I'm going to merge and release 3.0.1, because 3.0.0 is broken and shouldn't stay like this.

@bastimeyer bastimeyer force-pushed the installer/fix-pycountry branch from 8a7b779 to 0b2ef88 Compare November 17, 2021 13:31
@bastimeyer bastimeyer changed the title installer: fix pycountry installer: rewrite wheels config, fix pycountry Nov 17, 2021
@bastimeyer

bastimeyer commented Nov 17, 2021

Copy link
Copy Markdown
Member Author

I've updated the PR with a wheel lockfile, similar to the appimage build config.

As mentioned, pynsist doesn't support wheel hash information, and since we need to build the pycountry wheel locally and include it via the local_wheels pynsist directive anyway, we can easily download the remaining wheels via pip too and set the hashes.

This by the way also updates some dependency versions.

Check the installer build log for the wheel lists:

A quick validation check from my locally built installer (built on Linux, copied into W10 VM):

$ streamlink -l debug
[cli][debug] OS:         Windows 10
[cli][debug] Python:     3.9.8
[cli][debug] Streamlink: 3.0.0+1.g0b2ef88
[cli][debug] Requests(2.26.0), Socks(1.7.1), Websocket(1.2.1)
[cli][debug] Arguments:
[cli][debug]  --loglevel=debug
[cli][debug]  --ffmpeg-ffmpeg=C:\Program Files (x86)\Streamlink\ffmpeg\ffmpeg.exe
usage: streamlink [OPTIONS] <URL> [STREAM]

Use -h/--help to see the available options or read the manual at https://streamlink.github.io

@bastimeyer bastimeyer merged commit b19458c into streamlink:master Nov 17, 2021
@bastimeyer bastimeyer deleted the installer/fix-pycountry branch November 17, 2021 14:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

bug Windows installer

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3.0.0 - The 'pycountry' distribution was not found and is required by the application

1 participant

@bastimeyer