Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@sritchie
Copy link
Contributor

This PR gets us started with an approach for custom base images; this is inspired by @sagravat's work in #20, but I had changed so much about the project organization in the meantime that it felt appropriate to whip up this example.

@sagravat, would love your thoughts on how this feels to use!

The user can supply a "base_image" entry into .calibanconfig.json that can either be:

  • a string, or
  • a dict with "cpu" and "gpu" keys. Optional is fine, of course.

Any of the DLVM images will automatically expand. I took slight liberty by forcing the job mode into the name and adding a prefix. So we now have these DLVM options:

dlvm:pytorch-cpu
dlvm:pytorch-cpu-1.0
dlvm:pytorch-cpu-1.1
dlvm:pytorch-cpu-1.2
dlvm:pytorch-cpu-1.3
dlvm:pytorch-cpu-1.4
dlvm:tf-cpu
dlvm:tf-cpu-1.0
dlvm:tf-cpu-1.13
dlvm:tf-cpu-1.14
dlvm:tf-cpu-1.15
dlvm:tf2-cpu
dlvm:tf2-cpu-2.0
dlvm:tf2-cpu-2.1
dlvm:tf2-cpu-2.2
dlvm:pytorch-gpu
dlvm:pytorch-gpu-1.0
dlvm:pytorch-gpu-1.1
dlvm:pytorch-gpu-1.2
dlvm:pytorch-gpu-1.3
dlvm:pytorch-gpu-1.4
dlvm:tf-gpu
dlvm:tf-gpu-1.0
dlvm:tf-gpu-1.13
dlvm:tf-gpu-1.14
dlvm:tf-gpu-1.15
dlvm:tf2-gpu
dlvm:tf2-gpu-2.0
dlvm:tf2-gpu-2.1
dlvm:tf2-gpu-2.2

Let me know what you think!

@sritchie sritchie requested a review from ajslone July 15, 2020 18:38
@sritchie
Copy link
Contributor Author

@ajslone @sagravat one thought... we COULD allow the user to provide a format string, like

"dlvm:tf-{}-1.14"

And then we could splice in the job mode at runtime. Is this good? Then it's easier to specify this from the command line.

{"base_image": "dlvm:tf-{}-1.14"}

or

--base_image "dlvm:tf-{}-1.14"

Thoughts?

Copy link
Contributor

@ajslone ajslone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, this will enable many use cases that have been painful or impossible before.

base_image_fn = base_image_id

base_image = base_image_fn(job_mode)
base_image = c.base_image(caliban_config, job_mode) or base_image_id(job_mode)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

much better using config

@sagravat
Copy link

LGTM, +1

@sritchie sritchie mentioned this pull request Jul 15, 2020
@codecov
Copy link

codecov bot commented Jul 16, 2020

Codecov Report

Merging #39 into master will increase coverage by 1.00%.
The diff coverage is 96.66%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #39      +/-   ##
==========================================
+ Coverage   49.51%   50.51%   +1.00%     
==========================================
  Files          28       28              
  Lines        2872     2894      +22     
==========================================
+ Hits         1422     1462      +40     
+ Misses       1450     1432      -18     
Impacted Files Coverage Δ
caliban/platform/notebook.py 45.00% <ø> (ø)
caliban/docker/build.py 31.84% <50.00%> (+0.35%) ⬆️
caliban/config/__init__.py 100.00% <100.00%> (+28.07%) ⬆️
caliban/platform/cloud/types.py 89.47% <0.00%> (-0.66%) ⬇️
caliban/platform/gke/util.py 71.92% <0.00%> (+0.38%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f641c9d...4d3ed9e. Read the comment docs.

@sritchie sritchie merged commit f95b955 into master Jul 16, 2020
@sritchie sritchie deleted the sritchie/custom_base_image branch July 16, 2020 04:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants