Add CI Build & Deployment pipeline #2

tnation14 · 2018-01-05T19:02:39Z

Overview

Add a TravisCI pipeline that runs tox tests and builds/deploys packages to pypi.

Notes

rasterio dependency issues: due to example on numpy.histogram2d is broken (Trac #1830) numpy/numpy#2423, numpy must already be installed in the virtualenv before trying to build pystac distributions, or setting up rasterio will fail. This is also true when installing pystac from source. Numpy doesn't need to be pre-installed when installing via wheel.
- numpy and tox: since tox uses separate virtualenv from Travis, i also had to include numpy as a dependency in tox.ini.
flake8 errors: I took care of some of the most obvious flake8 errors, but flake8 still fails in CI. I disabled that testing step in order to force builds to pass.

Testing

See Travis CI output
Install pystac==0.1.1 via pip

$ pip install pystac==0.1.1

Fixes azavea/raster-foundry-platform#206

hectcastro · 2018-01-08T00:25:18Z

.travis.yml

+addons:
+  apt:
+    packages:
+      - libgdal-dev

 language: python
 python:


We should add configuration to cache pip packages:

cache: pip

hectcastro · 2018-01-08T00:25:59Z

.travis.yml

+    secure: kjKWOTwTm2eaa10Pp4/ll480TfFxP5Lb+/9cuxC1gZNZ1JuuJ/y11RQTXCKKP5CxQV0JQ8zs1RAeg0ODXsX4f1edC714oWQx/2dl3BMb5rLo2Pbk5AkVQ50gN/P9d6cSVQrlv6U+CClr8w9FNWTZ/yIeL6bs2AbK9dUyTGx/i7GBOfkAcyW23kY4vYcemonwiTKtql1Pez9rPcwX3Q+3hyiU9rX0psK3H9vdH+VzrFR4+R9URtXoaYBrNDkO3IBy62qBLhMUOwrTDkV2IGjZ3O0w9sGmON3KieuwmbII4P40FRx5OLUnmSYXhQTkhXQ1OU4LnU62y4QYCw0B94tbM4gKJdLapOVO4SjqdCmVqBLV84A+DqdMxhYQYgtP8dUwDmmyOr4jS3AVpZY75Vgmd4V1tpMxaAJwBKAcXOueXExg4GaoT/1cSpZcPPojWcujNnJX2AnrTDHctJiKPWpkV9a0MQyQ3szgwgetl4YVbUoiyKjTPwQyoeTG+gCI5czwPZj35mqFeO+OCkJq3vvkIJ4K1f1sHcO541AJa24v8Akdp/VXPxQOG75/kK3EXoH/BfWkyJF4QH8GA+dksTQGozv9dtpTa7zpNh7FkJ8VFjxSvMcB3LjfzggvQ5N1CZNnWDIZ8JWbwMbNNCdn/mWEfMZ0/ChXt+U5BGdQUeqnCe0=
+  on:
+    tags: true
+    distributions: "sdist bdist_wheel"


This needs to be at the same level as provider and user; it should not be under on.

hectcastro · 2018-01-08T00:26:41Z

.travis.yml

+  user: rasterfoundry
+  password:
+    secure: kjKWOTwTm2eaa10Pp4/ll480TfFxP5Lb+/9cuxC1gZNZ1JuuJ/y11RQTXCKKP5CxQV0JQ8zs1RAeg0ODXsX4f1edC714oWQx/2dl3BMb5rLo2Pbk5AkVQ50gN/P9d6cSVQrlv6U+CClr8w9FNWTZ/yIeL6bs2AbK9dUyTGx/i7GBOfkAcyW23kY4vYcemonwiTKtql1Pez9rPcwX3Q+3hyiU9rX0psK3H9vdH+VzrFR4+R9URtXoaYBrNDkO3IBy62qBLhMUOwrTDkV2IGjZ3O0w9sGmON3KieuwmbII4P40FRx5OLUnmSYXhQTkhXQ1OU4LnU62y4QYCw0B94tbM4gKJdLapOVO4SjqdCmVqBLV84A+DqdMxhYQYgtP8dUwDmmyOr4jS3AVpZY75Vgmd4V1tpMxaAJwBKAcXOueXExg4GaoT/1cSpZcPPojWcujNnJX2AnrTDHctJiKPWpkV9a0MQyQ3szgwgetl4YVbUoiyKjTPwQyoeTG+gCI5czwPZj35mqFeO+OCkJq3vvkIJ4K1f1sHcO541AJa24v8Akdp/VXPxQOG75/kK3EXoH/BfWkyJF4QH8GA+dksTQGozv9dtpTa7zpNh7FkJ8VFjxSvMcB3LjfzggvQ5N1CZNnWDIZ8JWbwMbNNCdn/mWEfMZ0/ChXt+U5BGdQUeqnCe0=
+  on:


We need to add python: "3.6" under on so that only the 3.6 wheel gets published.

hectcastro · 2018-01-08T00:29:07Z

MANIFEST.in

@@ -3,9 +3,12 @@ include CONTRIBUTING.rst
 include HISTORY.rst
 include LICENSE
 include README.rst
+include Makefile


Not sure this file needs to be included in the distribution.

hectcastro · 2018-01-08T00:29:22Z

MANIFEST.in

@@ -3,9 +3,12 @@ include CONTRIBUTING.rst
 include HISTORY.rst
 include LICENSE
 include README.rst
+include Makefile
+
+exclude .editorconfig requirements_dev.txt

 recursive-include tests *


See graft comment in azavea/django-amazon-ses@f4903b2.

hectcastro · 2018-01-08T00:44:49Z

setup.py

-        join(dirname(__file__), *names),
-        encoding=kwargs.get('encoding', 'utf8')
-    ).read()
-
 requirements = [


I think that this should just be inline below with install_requires.

hectcastro · 2018-01-08T00:45:04Z

setup.py

+    'Click>=6.0',
+    'geojson==2.3.0',
+    'requests==2.18.0',
+    'typing==3.6.2;python_version<"3.5"'


Interesting; was not aware of this conditional syntax.

hectcastro · 2018-01-08T00:45:21Z

setup.py

+    'Click>=6.0',
+    'geojson==2.3.0',
+    'requests==2.18.0',
+    'typing==3.6.2;python_version<"3.5"'
 ]

 setup_requirements = [


We should just get rid of this variable.

hectcastro · 2018-01-08T00:45:27Z

setup.py

 ]

 setup_requirements = [
-    # TODO(notthatbreezy): put setup requirements (distutils extensions, etc.) here
+    # NOQA TODO(notthatbreezy): put setup requirements (distutils extensions, etc.) here
 ]

 test_requirements = [


hectcastro · 2018-01-08T00:47:28Z

setup.py

    long_description=readme + '\n\n' + history,
    author="Raster Foundry",
    author_email='[email protected]',
-    url='https://github.com/notthatbreezy/pystac',
+    url='https://github.com/rasterfoundry/pystac',
    packages=find_packages(),
    py_modules=[splitext(basename(path))[0] for path in glob('pystac/*.py')],
    entry_points={


Couldn't comment on the appropriate line, but I think that it would be good to ask in the channel for some appropriate keywords before publishing.

As far as keywords go, not sure really what would be good, maybe the following:

imagery

raster

catalog

At least as a start.

hectcastro

👍

hectcastro · 2018-01-08T18:18:52Z

tox.ini

+commands =
+    check-manifest --ignore tox.ini
+    {envpython} setup.py check -m -r -s
+    #    flake8 .


I think that we should edit this command (even if left commented-out) to:

flake8 --exclude ".svn,CVS,.bzr,.hg,.git,__pycache__,.tox,.eggs,*.egg,docs"

This helps remove a bunch of stuff that has little to do with the project source code.

As far as improving this, I would add __init__.py and build as well (at least helps reduce issues for local development):

flake8 --exclude "__init__.py,build,.svn,CVS,.bzr,.hg,.git,__pycache__,.tox,.eggs,*.egg,docs"

notthatbreezy · 2018-01-08T19:20:30Z

tox.ini

-;     -r{toxinidir}/requirements.txt
+[flake8]
+exclude = .tox,*.egg,build,data
+select = E,W,F


Can you add max-line-length: 100 -- the default is pretty small (80) and as a team that's what we default too

notthatbreezy

Looks good to me -- I think you can re-enable flake8 again -- I pushed a commit that fixes it once you add the options @hectcastro and I suggested

- Build with python 3.4-4.6 - Deploy to PyPi

tnation14 self-assigned this Jan 5, 2018

tnation14 requested review from hectcastro and notthatbreezy January 5, 2018 19:02

hectcastro reviewed Jan 8, 2018

View reviewed changes

hectcastro approved these changes Jan 8, 2018

View reviewed changes

notthatbreezy reviewed Jan 8, 2018

View reviewed changes

notthatbreezy approved these changes Jan 8, 2018

View reviewed changes

tnation14 force-pushed the feature/tnation/cibuild-cipublish branch from 0a74be5 to 6958e14 Compare January 8, 2018 21:38

tnation14 added 3 commits January 8, 2018 16:56

Add configuration for TravisCI build & deployment pipeline

31a8b29

- Build with python 3.4-4.6 - Deploy to PyPi

Check-manifest fixes

1762c8c

Flake8 fixes

38ca085

tnation14 force-pushed the feature/tnation/cibuild-cipublish branch from 6958e14 to 38ca085 Compare January 8, 2018 21:57

tnation14 merged commit 9f71e4a into develop Jan 8, 2018

lossyrob deleted the feature/tnation/cibuild-cipublish branch November 1, 2019 01:31

emmanuelmathot added a commit to emmanuelmathot/pystac that referenced this pull request May 26, 2021

WIP stac-utils#2

73ebfc2

Add CI Build & Deployment pipeline #2

Add CI Build & Deployment pipeline #2

Uh oh!

Conversation

tnation14 commented Jan 5, 2018

Overview

Notes

Testing

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hectcastro left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

notthatbreezy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!