From Vagrant to NixOps

Avatar von Hendrik Schaeidt

I have been following the development of NixOps for some months. NixOps is a cloud deployment tool using nix, the functional package manager for unix systems. Nix makes it very intuitive to define absolute package dependencies. No more thinking and guessing about required runtime dependencies.
NixOps supports deploying to different platforms. Bare-metal, cloud, and even virtual environments like virtualbox work out of the box. I have worked in many projects using vagrant. Out of curiosity I migrated an existing vagrant project using wasted (Web Application STack for Extreme Development) to nix and NixOps.
This post is a walkthrough to configure a symfony2 project with nginx, mysql, and php-fpm from scratch.

Demo

In the demo project you can see the final setup using the code from this blog post.

Demo link: https://github.com/hschaeidt/cbase

Preparations

Before we get started we need following tools:

Also ensure the following settings in virtualbox:

  • open general settings
  • navigate to „Network“ section
  • in „Network“ switch to „Host-only Networks“ tab
  • if no network with the name „vboxnet0“ exists, create one

We are ready to go now.

Overview

The first setup we want to achieve is development machine for developers. The next goal will be to have the exact same configuration for production deployments using the exact same tool, NixOps, but with another provider – maybe ec2, maybe hetzner bare-metal, maybe azure, … – that depends on your use case. But this will be too much for this post, so let’s first focus on our development machine using virtualbox.

What we have:

  • a local symfony project on the host machine (the computer you are hacking on right now)
  • NixOps installed

Well, that’s all we actually need.

What our final setup will look like:

  • the host machine’s symfony2 project is mounted in the virtual machine provisioned by NixOps
  • all the project’s dependencies are available within the virtualbox development machine
    • php
    • composer
    • mysql
    • nginx
  • it is not required to install those tools on the host machine

Hands on

Now let’s start by creating a new folder for our new configuration. Let’s say ./server.
We start by declaring a virtualbox configuration file in nix. I will paste the entire configuration in here as it is pretty easy to follow up. If you are not yet familiar enough with the nix language, try out a tour of nix: https://nixcloud.io/tour/?id=1 – it’s concise, straightforward, and afterwards there are no more surprises in the config syntax.

Let’s say we have a file called ./server/cbase-vbox.nix

let
  cbase = # section 1
    { config, pkgs, ... }:
    { deployment.targetEnv = "virtualbox"; # section 2
      deployment.virtualbox = {
        memorySize = 1024;
        headless = true;
      };

      virtualisation.virtualbox.guest.enable = true; # section 3

      deployment.virtualbox.sharedFolders = { # section 4
        cbase = {
          hostPath = "/Users/hschaeidt/Projects/github/hschaeidt/cbase";
          readOnly = false;
        };
      };

      fileSystems."/var/www/cbase" = { # section 5
        device = "cbase";
        fsType = "vboxsf";
        options = [ "uid=33" "gid=33" ];
      };
    };
in
{
    network.enableRollback = true; # section 6
    inherit cbase; # section 7
}

From here on I will call my virtual development machine the „target machine“. Also my local laptop will be called „host machine“ from here on.

I split the configuration in 7 sections I will describe below:

section 1

cbase = { config, pkgs, ... }:

Defines the server name exposed to NixOps. This name will be used to ssh into the target machine later on.

section 2

deployment.targetEnv = "virtualbox";
deployment.virtualbox = {
  memorySize = 1024;
  headless = true;
};

Defines the deployment target – here virtualbox – below some virtualbox specific settings like the memory available to the target machine.

section 3

virtualisation.virtualbox.guest.enable = true;

Virtualbox guest additions are required on the target machine in order to properly mount the folder in section 4 and section 5.

section 4

deployment.virtualbox.sharedFolders = {
  cbase = {
    hostPath = "/Users/hschaeidt/Projects/github/hschaeidt/cbase";
    readOnly = false;
  };
};

Declaring the virtualbox shared folders. The key of the set is cbase, note that this must equal the device name used in section 5.
The host machine path points to the checked out git project, change it to your needs.
Read-only is set to false because we want to do some operations like composer install within the target machine etc.

section 5

fileSystems."/var/www/cbase" = {
  device = "cbase";
  fsType = "vboxsf";
  options = [ "uid=33" "gid=33" ];
};

Defining the filesystem mount from the previously declared sharedFolder from the host machine.
/var/www/cbase will be mounted on the target machine using the virtualbox shared folder containing the git symfony project.
Note: The uid and gid will be the one of the www-data user that will be created in the next step. This makes sure symfony can write its caches.

section 6

network.enableRollback = true;

Enabling the possibility to roll back to a previous configuration using nixops rollback command. This can also be omitted for the development machine.

section 7

inherit cbase;

Basically this is the same as writing cbase = cbase;
Apply the declared variable cbase to the nix config.

Now we have defined our target machine we want to use for development. Let’s keep going by creating another file configuring this machine.

Setting up php, nginx and mysql

Let’s get started by creating a new file called ./server/cbase.nix. This file is way bigger, but I think it’s important to paste it entirely first and going through it in a second step.

{
  network.description = "mtg card database";

  cbase = { config, pkgs, ... }: # section 1
  let
    fcgiSocket = "/run/phpfpm/nginx";
    projectName = "cbase";
    realPathRoot = "/var/www/cbase";
    runUser = "www-data";
    runGroup = "www-data";
  in
  {
    networking.firewall.allowedTCPPorts = [ 80 443 ]; # section 2

    environment.systemPackages = with pkgs; [ # section 3
      ag
      vim
      php
      phpPackages.composer
    ];

    services.phpfpm.poolConfigs.nginx = '' # section 4
      listen = ${fcgiSocket}
      listen.owner = $${runUser}
      listen.group = ${runGroup}
      listen.mode = 0660
      user = ${runUser}
      pm = dynamic
      pm.max_children = 75
      pm.start_servers = 10
      pm.min_spare_servers = 5
      pm.max_spare_servers = 20
      pm.max_requests = 500
    '';

    services.nginx = { # section 5
      enable = true;
      recommendedOptimisation = true;
      recommendedTlsSettings = true;
      recommendedGzipSettings = true;
      recommendedProxySettings = true;
      user = runUser;
      group = runGroup;
      virtualHosts."localhost" = {
        extraConfig = ''
          root ${realPathRoot}/web;

          location / {
            try_files $uri /app.php$is_args$args;
          }

          # DEV
          location ~ ^/(app_dev|config)\.php(/|$) {
            # this links to the defined upstream in 'appendHttpConfig'
            fastcgi_pass phpfcgi;
            fastcgi_split_path_info ^(.+\.php)(/.*)$;
            include ${pkgs.nginx}/conf/fastcgi_params;
            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
            fastcgi_param DOCUMENT_ROOT $document_root;
          }

          # PROD
          location ~ ^/app\.php(/|$) {
            # this links to the defined upstream in 'appendHttpConfig'
            fastcgi_pass phpfcgi;
            fastcgi_split_path_info ^(.+\.php)(/.*)$;
            include ${pkgs.nginx}/conf/fastcgi_params;
            fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
            fastcgi_param DOCUMENT_ROOT $document_root;
            internal;
          }

          location ~ \.php$ {
            return 404;
          }
        '';
      };
      appendHttpConfig = ''
        upstream phpfcgi {
            server unix:${fcgiSocket};
        }
      '';
    };

    services.mysql = { # section 6
      enable = true;
      package = pkgs.mysql;
      initialDatabases = [ { name = "cbase"; schema = ./cbase.sql; } ];
    };

    users.extraUsers."$${runUser}" = { # section 7
      uid = 33;
      group = "${runGroup}";
      home = "/var/www";
      createHome = true;
      useDefaultShell = true;
    };
    users.extraGroups."${runUser}".gid = 33;
  };
}

The configuration is again split into 7 sections we will go through now.

section 1

cbase = { config, pkgs, ... }:
let
  fcgiSocket = "/run/phpfpm/nginx";
  projectName = "cbase";
  realPathRoot = "/var/www/cbase";
  runUser = "www-data";
  runGroup = "www-data";
in
{
  ...
}

The cbase variable here again corresponds to the server name exposed to nixops. It has to correspond to the one defined in the previous configuration file (see section 1 from the previous configuration).
Additionally, we define some other variables we will use for convenience within the let in block.

section 2

networking.firewall.allowedTCPPorts = [ 80 443 ];

We open the ports 80 and 443 on our application server. This is necessary for the nginx to be available to the host machine.

section 3

environment.systemPackages = with pkgs; [
  ag
  vim
  php
  phpPackages.composer
];

Declaring the packages available for execution from all target machines users (including www-data user). We actually only need the php and composer packages for development. Adapt to your needs on the target machine.

section 4

services.phpfpm.poolConfigs.nginx = ''
  listen = ${fcgiSocket}
  listen.owner = $${runUser}
  listen.group = ${runGroup}
  listen.mode = 0660
  user = ${runUser}
  ...
'';

Enables the php-fpm service on the target machine. The user should be the same as for nginx. For detailed pool configuration options refer to http://php.net/manual/en/install.fpm.configuration.php

section 5

services.nginx = {
  enable = true;
  recommendedOptimisation = true;
  recommendedTlsSettings = true;
  recommendedGzipSettings = true;
  recommendedProxySettings = true;
  user = "$${runUser}";
  group = "${runGroup}";
  virtualHosts."localhost" = {
    extraConfig = ''
      root ${realPathRoot}/web;

      location / {
        try_files $uri /app.php$is_args$args;
      }

      # DEV
      location ~ ^/(app_dev|config)\.php(/|$) {
        # this links to the defined upstream in 'appendHttpConfig'
        fastcgi_pass phpfcgi;
        fastcgi_split_path_info ^(.+\.php)(/.*)$;
        include ${pkgs.nginx}/conf/fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param DOCUMENT_ROOT $document_root;
      }

      # PROD
      location ~ ^/app\.php(/|$) {
        # this links to the defined upstream in 'appendHttpConfig'
        fastcgi_pass phpfcgi;
        fastcgi_split_path_info ^(.+\.php)(/.*)$;
        include ${pkgs.nginx}/conf/fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param DOCUMENT_ROOT $document_root;
        internal;
      }

      location ~ \.php$ {
        return 404;
      }
    '';
  };
  appendHttpConfig = ''
    upstream phpfcgi {
        server unix:${fcgiSocket};
    }
  '';
};

Enables and configures the nginx service. Please note the variables used within the configuration file. It is mostly the recommended minimal nginx configuration file as described in https://www.nginx.com/resources/wiki/start/topics/recipes/symfony/

section 6

services.mysql = {
  enable = true;
  package = pkgs.mysql;
  initialDatabases = [ { name = "cbase"; schema = ./cbase.sql; } ];
};

Enables the mysql service. Note the inital database configuration. The minimal required configuration in order to get the databases available after deployment is:

--
-- Current Database: <code>{{EJS12}}</code>
--

/*!40000 DROP DATABASE IF EXISTS <code>{{EJS13}}</code>*/;

CREATE DATABASE /*!32312 IF NOT EXISTS*/ <code>{{EJS14}}</code> /*!40100 DEFAULT CHARACTER SET utf8 */;

section 7

users.extraUsers."$${runUser}" = {
  uid = 33;
  group = "${runGroup}";
  home = "/var/www";
  createHome = true;
  useDefaultShell = true;
};
users.extraGroups."${runUser}".gid = 33;

Creates the www-data user on the target machine. We enable the default shell here, because we will work with this user within our target machine. Note that the gid and uid should correspond to the ones defined in section 5 of the previous configuration file.

That covers all we need to do to describe our server setup.

Deploying the development machine

nixops deployment

Okay here we go. We just finished our minimal symfony configuration file. It’s time to test this setup now. Just a few steps to go.

First we have to create the nixops deployment configuration.

nixops create --deployment cbase ./server/cbase-vbox.nix ./server/cbase.nix

Now we can test if the machine was created succesfully

nixops list


# Should output following similar output
# +------+---------+------------------------+------------+------------+
# | UUID | Name    | Description            | # Machines |    Type    |
# +------+---------+------------------------+------------+------------+
# | ...  | cbase   | mtg card database      |          1 | virtualbox |
# +------+---------+------------------------+------------+------------+

And finally deploying it to virtualbox

nixops deploy --deployment cbase

Now we have a virtualbox able to serve our symfony application.

nixops info --deployment cbase

# Should output following similar output
# +-------+-----------------------+------------+------------------+----------------+
# | Name  |         Status        | Type       | Resource Id      | IP address     |
# +-------+-----------------------+------------+------------------+----------------+
# | cbase | Starting / Up-to-date | virtualbox | nixops-...-cbase | 192.168.56.101 |
# +-------+-----------------------+------------+------------------+----------------+

symfony setup

We need to execute composer install from within the target machine to download all dependencies.

# ssh into the target machine
nixops ssh --deployment cbase cbase

# change to www-data user
su - www-data

# navigate to document root
cd /var/www/cbase

# install dependencies
composer install

Testing the setup

Now all we have to do is look up the IP-address from the previously executed nixops info --deployment cbase command, which in my case is 192.168.56.101.
And navigating to http://192.168.56.101/app_dev.php we can see our symfony application running.

Happy hacking!

Avatar von Hendrik Schaeidt

Kommentare

22 Antworten zu „From Vagrant to NixOps“

  1. Reddit/p: From Vagrant to Nixops https://t.co/iOePKGmlt5

  2. From #Vagrant to #NixOps

    #DevOps https://t.co/GOa1j4ZEN9

  3. From Vagrant to NixOps
    https://t.co/SKd9awEhxt
    #nixops #php #vagrant

  4. From Vagrant to NixOps
    https://t.co/SKd9awEhxt
    #nixops #php #vagrant

  5. […] Nixops is a deployment tool for the NixOS operating system using the nix package manager. In this post I describe how to replace your old vagrant configuration with a nixops deployment configuration for development machines by example of a symfony php application. – Read full story at Hacker News […]

  6. […] Error loading HTMLAuthentic Article […]

Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert


Für das Handling unseres Newsletters nutzen wir den Dienst HubSpot. Mehr Informationen, insbesondere auch zu Deinem Widerrufsrecht, kannst Du jederzeit unserer Datenschutzerklärung entnehmen.