Navigating Nix

Today, I’m diving into Docker, Nix, and the quest for the trouble-free (or maybe BIG trouble once, yet never again. Like installing Arch Linux) project setup.

What’s Up with Nix?

First off, Nix is a powerful package manager for Linux and other Unix systems that makes package management reliable and reproducible. It provides atomic upgrades and rollbacks, side-by-side installation of multiple versions of a package, multi-user package management, and easy setup of build environments.

In other words is like that smart friend who knows exactly what your project needs to run, no matter where it’s going. It’s all about keeping your development and production stages looking like identical twins.

But even smart friends can bring complexities into our lives, right?

The quest for the base image

Spoiler: a base image is NOT needed at all. But I’ll mantain this part because it is a compelling intelectual exercise.

I initially picked NixOS (remember Nix Package Manager is one thing and the OS is another!) as the base image. But that is like using a sledgehammer for a nail when it comes to Docker images. It’s a full-blown OS! Imagine packing your entire house when you just need a vacation suitcase.

Nix shines in reproducibility, but NixOS can change over time. It’s like trying to freeze a river - sure, it’s still water, but it’s always flowing and changing.

Alternatives to NixOS

So, I embarked on a quest for alternatives, seeking something lighter yet robust. Why not go for something more streamlined like Alpine Linux? Indeed it is way lighter, yet it is still an OS!

So, I went for the Tiny Python Docker Image.

ARG PYTHON_VERSION=3.11

FROM alpine as builder
ARG PYTHON_VERSION

RUN apk add --no-cache python3~=${PYTHON_VERSION}
WORKDIR /usr/lib/python${PYTHON_VERSION}
RUN python -m compileall -o 2 .
RUN find . -name "*.cpython-*.opt-2.pyc" | awk '{print $1, $1}' | sed 's/__pycache__\///2' | sed 's/.cpython-[0-9]\{2,\}.opt-2//2' | xargs -n 2 mv
RUN find . -name "*.py" -delete
RUN find . -name "__pycache__" -exec rm -r {} +

FROM scratch
ARG PYTHON_VERSION

COPY --from=builder /usr/bin/python3 /
COPY --from=builder /lib/ld-musl-x86_64.so.1 /lib/ld-musl-x86_64.so.1
COPY --from=builder /usr/lib/libpython${PYTHON_VERSION}.so.1.0 /usr/lib/libpython${PYTHON_VERSION}.so.1.0
COPY --from=builder /usr/lib/python${PYTHON_VERSION}/ /usr/lib/python${PYTHON_VERSION}/

ENTRYPOINT ["/python3"]

In the first stage, named “builder,” it installs this specific Python version on Alpine, then compiles Python files into optimized bytecode. This compilation step enhances performance by generating bytecode files for the Python interpreter. The script proceeds to reorganize these compiled files for efficiency, removes the original .py source files to save space, and cleans up cache directories.

In the second phase, starting from a “scratch” image (which is essentially empty), it copies the compiled Python and necessary libraries from the “builder” stage. Therefore, the final image contains only the Python and its direct dependencies, but not a complete installation of Alpine Linux. This is done to ensure that the image is as lean as possible, containing only the essentials to run Python applications.

After building it, I pushed to docker hub.

Oooopppsss…!!!

After building the container, I analyzed it using a tool called Dive and realized that the Python environment was pulled a second time by Nix itself. So I thought, “Hmm, maybe a base image isn’t needed at all, since all the necessary dependencies will be pulled automatically.” And I was right.

Exploring this container image turned out to be a valuable learning experience, as it introduced me to the recommended practice of multi-stage builds.

Using Nix to setup the application container

The nix file acts as the essential blueprint for my application’s environment. While a Dockerfile could have been used to achieve similar ends, this method provides a vivid demonstration of the contrasts between imperative and declarative programming paradigms. The imperative approach, as illustrated by the Dockerfile, meticulously details every step and ingredient necessary to “bake the cake.” In contrast, the declarative approach merely specifies the desired outcome, the cake itself, without getting bogged down in the procedural details. Furthermore, the reproducibility offered by Nix significantly exceeds that achievable with Docker.


{ pkgs ? import <nixpkgs> {} 
, pkgsLinux ? import <nixpkgs> { system = "x86_64-linux"; }
}:

let

  pypdftk = pkgs.python3Packages.buildPythonPackage rec {
    pname = "pypdftk";
    version = "0.5";
    src = pkgs.python3Packages.fetchPypi {
      inherit pname version;
      sha256 = "sha256-tvfwABM0gIQrO/n4Iiy2vV4rIM9QxewFDy5E2ffcMpY";
    };
    doCheck = false;
  };

in

pkgs.dockerTools.buildImage {
  name = "ac2";
  tag = "dev";
  contents = [
    pkgs.python311Packages.django
    pkgs.python311Packages.django-extensions
    pkgs.python311Packages.django-crispy-forms
    pkgs.python311Packages.django-crispy-bootstrap4
    pkgs.bash
    pkgs.coreutils
  ];/localhost:1313/

# copying prebuilt pdftk binary with runtime dependencies 
  runAsRoot = ''
    mkdir -p /app
    cp -r ${./.}/* /app/
    rm -rf /app/chroot-pdftk
    chmod +x /app/startup.sh
  '';

  config = {    
    WorkingDir = "/app";
    ExposedPorts = {
      "8000/tcp" = {};/localhost:1313/
    };
    Cmd = [ "/bin/sh" "-c" 
    "python manage.py makemigrations && python manage.py migrate && python manage.py runserver 0.0.0.0:8000" 
    ];   
  };

  diskSize = 5000;
  buildVMMemorySize = 5120;
}
  • Setting the Stage: I start by importing nixpkgs to provide access to the nix/localhost:1313/-repository.
  • Crafting the Potion: Then, I mix up a custom environment with Django, and Crispy. The Python environment is pulled automatically.
  • Customizing further: pypdftk is not available in the Nix default set of 80.000 packages, so I had to build it specifically. Pypdftk provides the link between python commands to the PDF Toolkit.

Finally the container is build with:

nix-build default.nix

And here there’s a essential additional step not needed when using simply docker, load the container:

docker load < result

Then it is time to run the container and expose the port. I mounted the working directory to reflect on the fly the changes made during development:

docker run -v $(pwd):/app -p 8000:8000 {container tag}

The total container size is 546MB before installing Pdftk from the nixpkgs. After it /localhost:1313/are minimum.

So I decided to use pre-built binaries which “allegedly” don’t have any dependencies. But when running from the container I got:

bash-5.2# pdftk
bash: /usr/bin/pdftk: cannot execute: required file not found

So it was time to check for dynamic dependencies on run time:

$ ldd pdftk
	linux-vdso.so.1 (0x00007fffdf5c3000)
	libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f3c52e00000)
	libm.so.6 => /usr/lib/libm.so.6 (0x00007f3c52d13000)
	libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f3c5308e000)
	libdl.so.2 => /usr/lib/libdl.so.2 (0x00007f3c53089000)
	libz.so.1 => /usr/lib/libz.so.1 (0x00007f3c52cf9000)
	librt.so.1 => /usr/lib/librt.so.1 (0x00007f3c53084000)
	libc.so.6 => /usr/lib/libc.so.6 (0x00007f3c52b17000)
	/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007f3c530bc000)
	libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007f3c52af2000)

As it is evident, the documentation was wrong.

Signing Off

Choosing the right base image in Docker isn’t just a technical decision; it’s a strategic move in your project’s journey. While NixOS brought me wisdom in control and isolation, I’ve found that a lighter touch might better serve the speed and adaptability my Django project needs.

So, fellow travelers, as you navigate the seas of containerization and environment configurations, keep your eyes open for the tools and choices that best fit your journey. Share your stories and learn from each other.

Until next time, keep coding and stay curious!

comments powered by Disqus