Thanks to visit codestin.com
Credit goes to github.com

Skip to content
This repository was archived by the owner on Feb 14, 2023. It is now read-only.

[MRG] Creates nightly builds for Python >= 3.6 on all platforms #17

Merged
merged 59 commits into from
Feb 22, 2019

Conversation

thomasjpfan
Copy link
Collaborator

@thomasjpfan thomasjpfan commented Feb 18, 2019

Resolves scikit-learn/scikit-learn#9485

To enable daily builds, it has to be enabled with travis-ci's cron-job and appveyor's scheduled builds.

@thomasjpfan
Copy link
Collaborator Author

This PR has been updated to only build for 3.6/3.7 on osx, linux, windows(64 bit). This is a total of 6 daily builds.

@ogrisel
Copy link
Contributor

ogrisel commented Feb 21, 2019

If we want the osx nightly build, we can merge scikit-learn/scikit-learn#12651 into master.

I would rather just finish the review of the new K-Means implementation ( scikit-learn/scikit-learn#11950 ) which I believe would fix this unstable test on macOS. @jeremiedbb let me know if I am wrong.

@jeremiedbb
Copy link
Contributor

I do think it fixes the test. However, originally I made this test in this PR and then moved it in a separate PR to ease reviews. So it would not be absurd to remove this test from master and put it back in my PR.

Copy link
Contributor

@ogrisel ogrisel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM besides the following comments:

@adrinjalali
Copy link
Collaborator

thanks for doing all of these in parallel @thomasjpfan. Do you think if we move to microsoft CI, we should also move these to that CI?

@thomasjpfan
Copy link
Collaborator Author

@adrinjalali It is possible to move our build to azure. We would need to adapt https://github.com/matthew-brett/multibuild to run on azure.

For now, I think it is best to keep this running on Travis and AppVeyor.

Copy link
Contributor

@ogrisel ogrisel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The container name should be updated in the appveyor configuration.

BTW I have created the sklearn-nightly container on rackspace files and added a CNAME entry so that its content should be available under http://nightly.scikit-learn.org/ at some point (it does not seem to work yet). I don't know how to get https for this though.

Co-Authored-By: thomasjpfan <[email protected]>
@amueller
Copy link
Contributor

The owner of this web site has set X-Container-Meta-Web-Index: index.html. However, this file is not found. @ogrisel is that what you saw? Looks like it requires some config on the server, not just the cname.

# Maybe get and clean and patch source
- clean_code $REPO_DIR $BUILD_COMMIT
- build_wheel $REPO_DIR $PLAT
# Maybe get and clean and patch source
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

patch? for the wheel building?

@amueller amueller merged commit 89029a7 into MacPython:master Feb 22, 2019
@amueller
Copy link
Contributor

Ah, wheelhouse_uploader creates the index, so all is good, never mind.

@amueller
Copy link
Contributor

enabled for travis, no idea how to do it for appveyor, maybe need to log in with the appveyor admin account?
Also this: https://www.appveyor.com/blog/2017/11/08/sheduled-builds-for-free-accounts/

@amueller amueller mentioned this pull request Feb 22, 2019
@thomasjpfan
Copy link
Collaborator Author

thomasjpfan commented Feb 22, 2019

@ogrisel The first windows build has been uploaded here: http://68bea3902784d69c0aa8-ef5ddc96fef712c596c71d053dc2ad71.r77.cf2.rackcdn.com

The rest of the builds say they are uploaded to the same location. The index file has not been updated on the CDN because the TTL was set to 3 days. You can set this to 15 minutes by:

screen shot 2019-02-22 at 3 46 31 pm

screen shot 2019-02-22 at 3 48 22 pm

We can set the TTL to one hour after we confirm that everything works.

@ogrisel
Copy link
Contributor

ogrisel commented Feb 25, 2019

I changed the TTL, deleted the old sklearn-nightly container that I created in the North Virginia datacenter (IAD) instead of Chicago (ORD), and updated the CNAME record for http://nightly.scikit-learn.org/ and it seems to work now.

@rth
Copy link
Contributor

rth commented Feb 25, 2019

Great work on making nightly wheels available!

@ogrisel
Copy link
Contributor

ogrisel commented Feb 25, 2019

I checked an unfortunately it's not possible to configure HTTPS on CNAME entries that point to rackspace cloud files / CDN.

@ogrisel
Copy link
Contributor

ogrisel commented Feb 25, 2019

I tried:

pip install --pre -f http://nightly.scikit-learn.org scikit-learn

and it works (with a warning).

@amueller
Copy link
Contributor

@ogrisel the https is a rackspace issue? Should we look into finding an alternative? Or replace the issue on the sklearn issue tracker with one that asks for https hosting?

@ogrisel
Copy link
Contributor

ogrisel commented Feb 25, 2019

The fact that we cannot setup https for CNAME -based URLs (e.g. nightly.scikit-learn.org in our case) is a limitation of rackspace CDN.

If we use the raw CDN URL, it should work with HTTPS but apparently the certificate is not valid:

$ pip install -U --pre -f https://68bea3902784d69c0aa8-ef5ddc96fef712c596c71d053dc2ad71.r77.cf2.rackcdn.com scikit-learn
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
Looking in links: https://68bea3902784d69c0aa8-ef5ddc96fef712c596c71d053dc2ad71.r77.cf2.rackcdn.com
Certificate did not match expected hostname: 68bea3902784d69c0aa8-ef5ddc96fef712c596c71d053dc2ad71.r77.cf2.rackcdn.com. Certificate: {'crlDistributionPoints': (u'http://crl3.digicert.com/ssca-ecc-g1.crl', u'http://crl4.digicert.com/ssca-ecc-g1.crl'), 'subjectAltName': (('DNS', 'a248.e.akamai.net'), ('DNS', '*.akamaihd.net'), ('DNS', '*.akamaized-staging.net'), ('DNS', '*.akamaihd-staging.net'), ('DNS', '*.akamaized.net')), 'notBefore': u'Oct 18 00:00:00 2018 GMT', 'caIssuers': (u'http://cacerts.digicert.com/DigiCertECCSecureServerCA.crt',), 'OCSP': (u'http://ocsp.digicert.com',), 'serialNumber': u'04E528087A9D1BCDC05BECFC68C569D5', 'notAfter': 'Oct 18 12:00:00 2019 GMT', 'version': 3L, 'subject': ((('countryName', u'US'),), (('stateOrProvinceName', u'Massachusetts'),), (('localityName', u'Cambridge'),), (('organizationName', u'Akamai Technologies, Inc.'),), (('commonName', u'a248.e.akamai.net'),)), 'issuer': ((('countryName', u'US'),), (('organizationName', u'DigiCert Inc'),), (('commonName', u'DigiCert ECC Secure Server CA'),))}
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(CertificateError("hostname '68bea3902784d69c0aa8-ef5ddc96fef712c596c71d053dc2ad71.r77.cf2.rackcdn.com' doesn't match either of 'a248.e.akamai.net', '*.akamaihd.net', '*.akamaized-staging.net', '*.akamaihd-staging.net', '*.akamaized.net'",),)': /

so indeed we could try to find an alternative solution...

@thomasjpfan
Copy link
Collaborator Author

thomasjpfan commented Feb 25, 2019

The $0.10/GB is the standard CDN charge, for the first 10TB, so we should be able to turn on https without much more fees: https://support.rackspace.com/how-to/rackspace-cdn-secure-delivery-options/ (Using the SAN certificate)

(There is an additional $0.010 per 10,000 requests for HTTPS)

@thomasjpfan
Copy link
Collaborator Author

https://sklearn-nightly.scdn8.secure.raxcdn.com works now

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants