go2nix - because Go and Nix are both amazing

For Nixers - packaging Go applications

Concept

go2nix provides an autmatic way to create Nix derivations for Go applications.

Start with app sources than can be built on your machine with go build. It means that you need to get all dependencies into current GOPATH.
Run go2nix save in application source dir where main package lives. This will create 2 files default.nix and deps.nix that can be moved into its own directory under nixpkgs.

Example

If you are not sure how to organize your directory structure, read this official guide first.

Our project will be called influxdb-demo and this demo will be using the influxdb client library.

But first, prepare the project directory:

 mkdir example
 cd example
 mkdir bin pkg src

Change into a shell environment which contains GO and GIT:
```
 nix-shell -p go git go2nix
```

Note: Make sure go2nix is at least version 1.1.1

Then set the GOPATH:

 export GOPATH=`pwd`
 mkdir -p src/github.com/qknight/influxdb-demo

Prepare src/github.com/qknight/influxdb-demo/influxdb-client.go:

 package main

 import (
     "log"
     "time"

     "github.com/influxdata/influxdb/client/v2"
 )

 const (
     MyDB = "square_holes"
     username = "bubba"
     password = "bumblebeetuna"
 )

 func main() {
     // Make client
     c, err := client.NewHTTPClient(client.HTTPConfig{
         Addr: "http://localhost:8086",
         Username: username,
         Password: password,
     })

     if err != nil {
         log.Fatalln("Error: ", err)
     }

     // Create a new point batch
     bp, err := client.NewBatchPoints(client.BatchPointsConfig{
         Database:  MyDB,
         Precision: "s",
     })

     if err != nil {
         log.Fatalln("Error: ", err)
     }

     // Create a point and add to batch
     tags := map[string]string{"cpu": "cpu-total"}
     fields := map[string]interface{}{
         "idle":   10.1,
         "system": 53.3,
         "user":   46.6,
     }
     pt, err := client.NewPoint("cpu_usage", tags, fields, time.Now())

     if err != nil {
         log.Fatalln("Error: ", err)
     }

     bp.AddPoint(pt)

     // Write the batch
     c.Write(bp)
 }

Note: We use go influxdb client as an external library example.

Create a GIT repository

 cd src/github.com/qknight/influxdb-demo
 git init
 git add influxdb-client.go
 git commit -m 'initial commit'

Also create the repository on github.com, in this example it would be github.com/qknight/influxdb-demo

 git remote add origin [email protected]:qknight/influxdb-demo.git
 git push -u origin master

Note: go2nix requires a GIT repository to retrieve the commit hash and the remote so it can generate the values as name, version, rev and goPackagePath in the default.nix file.

Download the dependency the go-way
```
 go get
```

Note: go get will populate src/github.com/influxdata/influxdb for you.

Building the project
```
 go build
```
Generate default.nix/deps.nix

If go was able to build the binary, use go2nix to derive the default.nix and deps.nix:

  go2nix save

Note: This gives you a default.nix and a deps.nix which can be put into nixpkgs but, if you want to use it with nix-shell, you need some tiny adaptions.

The default.nix file we generated

 # This file was generated by go2nix.
 { stdenv, buildGoPackage, fetchgit, fetchhg, fetchbzr, fetchsvn }:

 buildGoPackage rec {
   name = "influxdb-demo-${version}";
   version = "20161030-${stdenv.lib.strings.substring 0 7 rev}";
   rev = "718c85cd733bca964abf03f5371c939d19845f72";

   goPackagePath = "github.com/qknight/influxdb-demo";

   src = fetchgit {
     inherit rev;
     url = "[email protected]:qknight/influxdb-demo.git";
     sha256 = "0csbqcnncklimysgcbxlj190bynx1ppvyxvl5viz40fvbcj4l8xb";
   };

   goDeps = ./deps.nix;

   # TODO: add metadata https://nixos.org/nixpkgs/manual/#sec-standard-meta-attributes
   meta = {
   };
 }

Note: Update the fetchgit url to use https instead of ssh or nix-build won't be able to fetch the software later.

Note: If you want to use this default.nix in nixpkgs you should extend the meta section and correct the goPackagePath.

The deps.nix we generated

# This file was generated by go2nix.
[
  {
    goPackagePath = "github.com/influxdata/influxdb";
    fetch = {
      type = "git";
      url = "https://github.com/influxdata/influxdb";
      rev = "bcb48a8ff2e9c118d5cfa04f03a62134b9c414c7";
      sha256 = "1f4jp7p2xxlsxhqbvglcl95y7fhab0w6zsvyqmqglmi0g1c5v21z";
    };
  }
]

Building the project using nix-build

nix-build -E 'with import <nixpkgs> { };  callPackage ./default.nix {}'

This will print something like this:

    these derivations will be built:
      /nix/store/8naxqswyhsmhqhjzskz5q0nf33nvyzsf-influxdb-demo-718c85c.drv
      /nix/store/p9bgjqn683xkmns36fpw2ipz5k57srl3-influxdb-bcb48a8.drv
      /nix/store/k0q9k2wfq0ccdsy04pyg4l01iqh5j1aa-go1.7-influxdb-demo-20161030-718c85c.drv
    building path(s) ‘/nix/store/45jjx5jakvlgwzq358pivr5x8xpgdsck-influxdb-bcb48a8’
    building path(s) ‘/nix/store/pqlqq2wjgixvf5j6qmrv5qyjh1bc6i8q-influxdb-demo-718c85c’
    exporting https://github.com/influxdata/influxdb (rev bcb48a8ff2e9c118d5cfa04f03a62134b9c414c7) into /nix/store/45jjx5jakvlgwzq358pivr5x8xpgdsck-influxdb-bcb48a8
    exporting https://github.com/qknight/influxdb-demo.git (rev 718c85cd733bca964abf03f5371c939d19845f72) into /nix/store/pqlqq2wjgixvf5j6qmrv5qyjh1bc6i8q-influxdb-demo-718c85c
    Initialized empty Git repository in /nix/store/45jjx5jakvlgwzq358pivr5x8xpgdsck-influxdb-bcb48a8/.git/
    Initialized empty Git repository in /nix/store/pqlqq2wjgixvf5j6qmrv5qyjh1bc6i8q-influxdb-demo-718c85c/.git/
    remote: Counting objects: 3, done.        
    remote: Compressing objects: 100% (2/2), done.        
    remote: Total 3 (delta 0), reused 3 (delta 0), pack-reused 0        
    From https://github.com/qknight/influxdb-demo
    * branch            HEAD       -> FETCH_HEAD
    Switched to a new branch 'fetchgit'
    removing `.git'...
    remote: Counting objects: 540, done.        
    remote: Compressing objects: 100% (506/506), done.        
    remote: Total 540 (delta 34), reused 179 (delta 6), pack-reused 0        
    Receiving objects: 100% (540/540), 1.44 MiB | 0 bytes/s, done.
    Resolving deltas: 100% (34/34), done.
    From https://github.com/influxdata/influxdb
    * branch            HEAD       -> FETCH_HEAD
    Switched to a new branch 'fetchgit'
    removing `.git'...
    building path(s) ‘/nix/store/9k4v7rhs5606fyia8mb341k71m3yrcbq-go1.7-influxdb-demo-20161030-718c85c-bin’, ‘/nix/store/av17zlfgnppl704wwxnh5pjpkcxac9k3-go1.7-influxdb-demo-20161030-718c85c’
    unpacking sources
    unpacking source archive /nix/store/pqlqq2wjgixvf5j6qmrv5qyjh1bc6i8q-influxdb-demo-718c85c
    source root is influxdb-demo-718c85c
    patching sources
    configuring
    grep: Invalid range end
    unpacking source archive /nix/store/45jjx5jakvlgwzq358pivr5x8xpgdsck-influxdb-bcb48a8
    building
    github.com/influxdata/influxdb/pkg/escape
    github.com/influxdata/influxdb/models
    github.com/influxdata/influxdb/client/v2
    github.com/qknight/influxdb-demo
    installing
    /tmp/nix-build-go1.7-influxdb-demo-20161030-718c85c.drv-0/go /tmp/nix-build-go1.7-influxdb-demo-20161030-718c85c.drv-0
    /tmp/nix-build-go1.7-influxdb-demo-20161030-718c85c.drv-0
    post-installation fixup
    shrinking RPATHs of ELF executables and libraries in /nix/store/9k4v7rhs5606fyia8mb341k71m3yrcbq-go1.7-influxdb-demo-20161030-718c85c-bin
    shrinking /nix/store/9k4v7rhs5606fyia8mb341k71m3yrcbq-go1.7-influxdb-demo-20161030-718c85c-bin/bin/influxdb-demo
    stripping (with flags -S) in /nix/store/9k4v7rhs5606fyia8mb341k71m3yrcbq-go1.7-influxdb-demo-20161030-718c85c-bin/bin 
    patching script interpreter paths in /nix/store/9k4v7rhs5606fyia8mb341k71m3yrcbq-go1.7-influxdb-demo-20161030-718c85c-bin
    shrinking RPATHs of ELF executables and libraries in /nix/store/av17zlfgnppl704wwxnh5pjpkcxac9k3-go1.7-influxdb-demo-20161030-718c85c
    patching script interpreter paths in /nix/store/av17zlfgnppl704wwxnh5pjpkcxac9k3-go1.7-influxdb-demo-20161030-718c85c
    /nix/store/9k4v7rhs5606fyia8mb341k71m3yrcbq-go1.7-influxdb-demo-20161030-718c85c-bin

Note: The resulting binary will be in /nix/store/9k4v7rhs5606fyia8mb341k71m3yrcbq-go1.7-influxdb-demo-20161030-718c85c-bin.

Adapting default.nix to be used with nix-shell

In order to use nix-shell you need to pass the newly created buildDeps to deps.nix:

diff --git a/default.nix b/default.nix
index 8fa7d91..d3dd797 100644
--- a/default.nix
+++ b/default.nix
@@ -1,5 +1,8 @@
-# This file was generated by go2nix.
-{ stdenv, buildGoPackage, fetchgit, fetchhg, fetchbzr, fetchsvn }:
+with import <nixpkgs> { };
+
+let
+  buildDeps = with pkgs; [stdenv buildGoPackage fetchgit fetchhg fetchbzr fetchsvn];
+in

buildGoPackage rec {
  name = "influxdb-demo-${version}";
@@ -8,15 +11,5 @@ buildGoPackage rec {

  goPackagePath = "github.com/qknight/influxdb-demo";

-  src = fetchgit {
-    inherit rev;
-    url = "[email protected]:qknight/influxdb-demo.git";
-    sha256 = "0csbqcnncklimysgcbxlj190bynx1ppvyxvl5viz40fvbcj4l8xb";
-  };
-
-  goDeps = ./deps.nix;
-
-  # TODO: add metadata https://nixos.org/nixpkgs/manual/#sec-standard-meta-attributes
-  meta = {
-  };
+  goDeps = with buildDeps; ./deps.nix;
}

After you updated the default.nix you can now use nix-shell:

nix-shell
these paths will be fetched (47.89 MiB download, 248.86 MiB unpacked):
  /nix/store/28wl3f34vfjpw0y5809bgr6382wqdscf-bash-4.3-p48
  /nix/store/7bdn99q835l042d8kcrn7yk61zkcqw6y-go1.7-govers-20150109-3b5f175
  /nix/store/aql31436x2fgr3rdj4fwbrgkvzw2kqf4-iana-etc-2.30
  /nix/store/bb2njjq32bh1wl2nl1zss0i8w1w2jgrz-tzdata-2016f
  /nix/store/dp2nf60lqzy1kbhd78ndf5nm3fb3qicd-gcc-wrapper-5.4.0
  /nix/store/drfmgb3kd3clxzi919mjrq5cnkw575w0-stdenv
  /nix/store/fkzyjrmrxv1zgswdq327h11ifbxdfpf9-go1.7-govers-20150109-3b5f175-bin
  /nix/store/g1pacy3pmx9r846z87xysfs29jz2b42q-parallel-20160722
  /nix/store/g4f1bms36hgg5abfd8xc4bj6sfzsy61d-bash-4.3-p48-info
  /nix/store/i76bwv541a7zj576hkdia62169r2nl0z-bash-4.3-p48-doc
  /nix/store/jqa9bh404crcxnyxa2lkffag7kw3yxw3-go-1.7.1
  /nix/store/jx5fyfd6mkpib1dmgy5rjirxff5vaczm-procps-3.3.11
  /nix/store/kx62kzlxcmzxgnxz1c95x7gln72iqd90-ncurses-6.0
  /nix/store/njn6il2fwaycjkd0jxbfj22gb25nd5df-perl-5.22.2
fetching path ‘/nix/store/i76bwv541a7zj576hkdia62169r2nl0z-bash-4.3-p48-doc’...
fetching path ‘/nix/store/g4f1bms36hgg5abfd8xc4bj6sfzsy61d-bash-4.3-p48-info’...
fetching path ‘/nix/store/aql31436x2fgr3rdj4fwbrgkvzw2kqf4-iana-etc-2.30’...
fetching path ‘/nix/store/bb2njjq32bh1wl2nl1zss0i8w1w2jgrz-tzdata-2016f’...
...

After the dependencies were built you can now use: go build to build your software as usual!

Note: You can also arrange a default.nix so that it can be used by nixpkgs and nix-shell but this is not covered here.

Example Leaps

Some projects are built using go get github.com/jeffail/leaps/cmd/... and for these you need to identify the src directory to run go2nix save from. So here is another example:

nix-shell -p go2nix git go
cd mktemp -d
export GOPATH=pwd
go get github.com/jeffail/leaps/cmd/...
cd src/github.com/jeffail/leaps/cmd/leaps
go2nix save

Note: The resulting default.nix and deps.nix are created in the directory you are currently in.

For Gophers - reproducible development and build environments

stay tuned

Installation

The preferred way of installing go2nix is to use nix like nix-env -iA go2nix or using it declaratively.

But you can also use go get github.com/kamilchm/go2nix.

Name		Name	Last commit message	Last commit date
Latest commit History 87 Commits
templates		templates
test		test
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
LICENSE		LICENSE
README.md		README.md
assets.go		assets.go
circle.yml		circle.yml
cli.go		cli.go
default.nix		default.nix
deps.nix		deps.nix
depsjson2nix.nix		depsjson2nix.nix
go2nix.1		go2nix.1
hash.go		hash.go
hash_test.go		hash_test.go
nix.go		nix.go
requireGo16.go		requireGo16.go
save.go		save.go
version.go		version.go

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Repository files navigation

go2nix - because Go and Nix are both amazing

For Nixers - packaging Go applications

Concept

Example

Example Leaps

For Gophers - reproducible development and build environments

Installation

About

Uh oh!

Releases 4

Packages

Contributors 12

Uh oh!

Languages

License

kamilchm/go2nix

Folders and files

Latest commit

History

Repository files navigation

go2nix - because Go and Nix are both amazing

For Nixers - packaging Go applications

Concept

Example

Example Leaps

For Gophers - reproducible development and build environments

Installation

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 4

Packages 0

Contributors 12

Uh oh!

Languages

Packages