Add pack option to the builder options for cloud native buildpacks #916
Conversation
| buildpacks, | ||
| "-t", config.absolute_image, | ||
| "-t", config.latest_image, | ||
| "--env", "BP_IMAGE_LABELS=service=#{config.service}", |
There was a problem hiding this comment.
Kamal expects there to be a service label, this automatically adds the label via the paketo-buildpacks/image-labels buildpack.
| end | ||
|
|
||
| def buildpacks | ||
| (pack_buildpacks << "paketo-buildpacks/image-labels").map { |buildpack| ["--buildpack", buildpack] } |
There was a problem hiding this comment.
Adding this buildpack automatically so that we can label the image for Kamal
|
|
||
| # Buildpack configuration | ||
| # | ||
| # The build configuration for using pack to build a Cloud Native Buildpack image. |
There was a problem hiding this comment.
Add mention of project.toml to set your excluded options. https://buildpacks.io/docs/for-app-developers/how-to/build-inputs/use-project-toml/
There was a problem hiding this comment.
As I was thinking about this and removing context: "." it doesn't matter as much since it's using the git clone. The exclusion list is really only relevant when you're using "." as your build context.
|
hey @nickhammond and @dhh, does this change resolve the custom build issue like using the builder of choice. e.g. docker build cloud? |
Oh, thanks for the awareness. I would be glad if you could help me make a PR for that since I don't know how to build gems or modify it for now |
|
This is fascinating work, @nickhammond. I'm surprised by how unobtrusive it is! But I'd like to understand the whole flow better. I'm not sure this is going to be all that relevant for Rails apps that now already come with well-optimized Dockerfiles out of the box, but I could see how that may well be different if you're doing a Sinatra app or some app from another framework that doesn't provide that. Could you show how the entire flow would go with, say, a Sinatra app, using buildpacks, and deploying on something like Digital Ocean? Want to make sure that this isn't tied to any one company or platform. |
|
Hey @dhh, thanks for taking a look! I think adding support for buildpacks will be great for the adoption of Kamal but you can always still reach for the sharper tool(a full Dockerfile) when needed. I built out a few hello world examples, the main thing is just making sure your app boots on port 80 for kamal-proxy or just ensuring that you set your Here are the hello world apps that I built and tested out on Digital Ocean and wrote a more detailed overview for the whole process as well.
|
|
@nickhammond thanks for all your investigations and opening this PR. ❤️
@dhh 👋 It's been a while. As Cloud Native Buildpacks (CNB) maintainer I'm biased and would love to see this supported in kamal. :) Nick touches on this in his blog, but if it's any assurance CNB as an upstream project is a CNCF Incubation project which pushes for not being a single vendor OSS project. In fact, the project was started from the get go by two companies, Heroku & Pivotal. It's really about bringing that Heroku magic to container image building, transforming your app source code into an OCI image (No |
|
Started on the docs in this kamal-site PR basecamp/kamal-site#117. |
djmb
left a comment
There was a problem hiding this comment.
@nickhammond - I noticed in your sample apps, that you've set the context for the builder to ., which avoids using the git clone for building.
Is that just a preference or is there any reason it would be required?
lib/kamal/commands/builder/base.rb
Outdated
|
|
||
| def inspect_builder | ||
| docker :buildx, :inspect, builder_name unless docker_driver? | ||
| docker :buildx, :inspect, builder_name unless docker_driver? || pack? |
There was a problem hiding this comment.
Maybe we could extract a buildx? method here?
def buildx?
!docker_driver? && !pack?
end
There was a problem hiding this comment.
@djmb We could also run pack builder inspect which returns a bunch of information about the default builder. It's a lot of information but might be useful to help triage if you're not sure what builder you're using. The Pack CLI lets you set your default builder so I have mine set to heroku/builder:24 via pack config default-builder heroku/builder:24
There was a problem hiding this comment.
@nickhammond Does this mean I can now pass my builder name to kamal?
Buildx cloud_builder
There was a problem hiding this comment.
@alexohre No, I don't think there's a PR open for that, just the discussion here #914 (comment)
| "-t", config.absolute_image, | ||
| "-t", config.latest_image, | ||
| "--env", "BP_IMAGE_LABELS=service=#{config.service}", | ||
| *argumentize("--env", secrets, sensitive: true), |
There was a problem hiding this comment.
Is using environment variables the standard way to get secrets into a buildpack?
There was a problem hiding this comment.
@djmb Yes, they only have the --env flag.
I just tested building with a few secrets because I was concerned they'd end up in the final image but they don't.
I just found this in the docs site though. TLDR; It's just a build-time env var, they're not available at image runtime. So they're naturally "secret", neat.
https://buildpacks.io/docs/for-platform-operators/how-to/integrate-ci/pack/cli/pack_build/#options
-e, --env stringArray Build-time environment variable, in the form 'VAR=VALUE' or 'VAR'.
When using latter value-less form, value will be taken from current
environment at the time this command is executed.
This flag may be specified multiple times and will override
individual values defined by --env-file.
Repeat for each env in order (comma-separated lists not accepted)
NOTE: These are NOT available at image runtime.
|
@edmorley Adding the pack CLI runThis branch has been updated with the latest from main as well. I'm currently using this branch to deploy 2 apps that I'm actively working on. |
|
The |
|
I've updated this with the latest from main. I've been using this branch to deploy a few projects for a few months now and haven't had any issues. Let me know if I can tweak or update anything. |
|
The test matrix run for Ruby 3.4.0-preview2 with the current Gemfile is failing for some reason, it's working fine with the same version and the Rails edge gemfile. It's passing for me locally with that test, I'll try to revisit it this week at some point. Test failure output |
|
Hey @nickhammond - I'm keen to get this into Kamal - I'll be away next week, but when I'm back I'll do a final review. Thanks for your patience! |
| end | ||
|
|
||
| def export(export_action) | ||
| return unless export_action == "registry" |
There was a problem hiding this comment.
With the output option changes from #1357, support needed to be added for an export_action but it's a little different than docker buildx output behavior. I went with supporting a build and push behavior as the default and then if you specify anything other than registry it just doesn't push the image to the registry.
I don't believe there's additional options for the output format since pack build is just outputting the OCI image but @edmorley might have some ideas here.
Docker buildx output options: https://docs.docker.com/reference/cli/docker/buildx/build/#output
There was a problem hiding this comment.
Hi! I'm not sure I quite follow the question (I've only skimmed the comments via email notifications) - but in case it helps, this is the only Pack CLI option relating to how the generated asset is handled:
--publish Publish the application image directly to the container registry specified in <image-name>, instead of the daemon. The run image must also reside in the registry.
|
@djmb No worries, enjoy your time off next week! Ideally buildpacks/pack#2268 that @edmorley opened gets merged in so we don't have to inject the image-labels pack but that looks like it's still a WIP. |
|
The Paketo buildpack CLI also recommends adding labels via the image labels buildpack so we might just want to leave the labelling as-is. https://paketo.io/docs/howto/configuration/#applying-custom-labels |
|
@djmb Are there any tests or verification steps that I can add to make this easier for you to review? I'm using this branch to deploy 5 different apps now, a few weekly and the others at least once a month. |
|
Hey @nickhammond - no nothing needed from me. I'm happy to include this in Kamal 2.7 when that is ready to go out |
|
@djmb Sounds good, I'll take a look at the kamal-site docs PR again as well this week. |
|
@djmb I updated all of the demo apps and redeployed with the latest using this branch, all went well. The build context was removed Let me know if I can provide insight on anything else, excited to get people using this! |
|
@nickhammond tried this for a spring boot app: Got this error: Using Kamal 2.7.0 If you had insight into how to get this working for a Spring boot app that would be great :) |
|
@gregjotau It looks like you've defined a builder but you also need a buildpack. I haven't built a spring app but this looks like the one mentioned in the Spring docs(https://docs.spring.io/spring-boot/reference/packaging/container-images/cloud-native-buildpacks.html) https://github.com/paketo-buildpacks/spring-boot builder:
pack:
# This is the official Paketo builder based on Ubuntu Jammy.
# It includes buildpacks for Java, Maven, Gradle, and more.
builder: paketobuildpacks/builder-jammy-base
buildpacks:
- paketo-buildpacks/spring-boot |
|
@nickhammond never got it to work, so started a separate dicussion instead: #1637 Thanks for the feature! Would be great if we managed to use it 🙏 |
This PR introduces Cloud Native Buildpacks to the list of builder options for Kamal.
This opens up the option to utilize buildpacks instead of writing a Dockerfile from scratch for each app that you want to deploy with Kamal. The end result is still an OCI-compliant docker image that you can run the same as the currently built docker images with Kamal.
You can also use any buildpacks or builders that you'd like so if you prefer some of the Paketo buildpacks instead you can use those too. The example below is utilizing Heroku's builder with the ruby and procfile buildpack which gives you the familiar Heroku build process when you deploy your application. Auto-detection of bundler, cache management of gem and asset installation, and various other features that come along with those.
With this PR you'd need to have pack installed as well as Docker and then within your deploy.yml change your builder to:
The default process that the buildpack tries to boot is the web process, you can add a Procfile for this:
And lastly, buildpacks don't bind to a default port so you'll either need to set
proxy.app_port(Kamal 2.0 / kamal-proxy) to your application port or set your app to use port 80 which is the defaultkamal-proxyport.Option 1 (Assuming your app uses port 3000):
Option 2 (Rails app running Puma that supports setting $PORT):
Buildpacks work in a detect and then build flow. The detect step looks for common files or checks that indicate it is indeed a ruby application by looking for a Gemfile.lock for instance. If the detect step passes then it triggers the build phase which essentially triggers a bundle install in this example.
With heroku/builder:24 so far I've found that the image size is about the same, it's only 2mb off for a 235mb image. Build time is typically faster with pack but depends on how well you've optimized your Dockerfile. The win though is not having to think about how to cache your gem installs, node installs or any other package manager installs that have a buildpack. It's also following the common conventions for building containers and various stumbling blocks that Heroku and others have been blazing through over the years.
Kamal discussion: #795
Heroku discussion: heroku/buildpacks#6
Heroku official buildpacks: https://devcenter.heroku.com/articles/buildpacks
Heroku 3rd party buildpacks: https://elements.heroku.com/buildpacks
Full setup overview: https://www.fromthekeyboard.com/deploying-a-rails-app-with-kamal-heroku-style/
Todos:
Discuss potential for a remote pack optionOut of scopekamal build createdo when using buildpacks? Just point to the install docs? https://buildpacks.io/docs/for-platform-operators/how-to/integrate-ci/pack/ - Since you don't typically callkamal build createand it's instead called within a build I'm going to close this one out.kamal build detailsto runpack version && pack builder inspectDoesSince we're not creating a build context like you normally would with Docker there's nothing to actually remove. Is there anything in the Kamal lifecycle though that we at least need a no-op method for this?kamal build removeneed to do anything?Demo applications: