Rafał Łasocha / swistak35

For “Daj się poznać” contest I’ve decided to make a tool which is close to pinboard and pinterest, but with features I was missing. When I’ve chosen frameworks which I’ll gonna use, and both of them came with some default tests, I’ve decided to setup continuous integration using CircleCI. This post describes how to setup CircleCI for repository with two apps inside (so automatic CircleCI language recognition doesn’t work) and one of them is haskell, which is not very popular and there aren’t too much resources how to do it.

Javascript frontend

That part was easy. Asuumption is that your javascript application is located in frontend directory and the tests are run by running npm run test command.

To run our tests, we needed only one simple command in circle.yml:

test:
  post:
    - cd frontend && npm run test

If we want npm run test command to work, we need firstly to install our dependencies, which is done by this part of circle.yml config:

dependencies:
  override:
    - cd frontend && npm install

This should already work, however we don’t want to wait each time to install all node dependencies, but rather we’d like to cache them between builds. This can be achieved by following code:

dependencies:
  cache_directories:
    - "frontend/node_modules"

Haskell backend

That part is more tricky.

In the beginning I’ve tried to achieve this by running cabal install and cabal test commands on test container, however I had some problems with installing all necessary dependencies. As on my development machine I am using stack to manage dependencies (and more), I was curious that maybe it would be easier to install this tool which is on a higher level than cabal, but probably more straightforward to run.

I’ve found that someone already tried to do this but it looks that without success. Using part of this repository I’ve managed to install stack on CircleCI without any difficulties:

dependencies:
  cache_directories:
    - "~/.stack"

  override:
    - wget -q -O- https://s3.amazonaws.com/download.fpcomplete.com/ubuntu/fpco.key | sudo apt-key add -
    - echo 'deb http://download.fpcomplete.com/ubuntu/precise stable main' | sudo tee /etc/apt/sources.list.d/fpco.list
    - sudo apt-get update && sudo apt-get install stack -y

As you can see, I’ve also added ~/.stack directory to cache haskell packages.

Then, similarly like in javascript frontend, we want to install dependencies:

dependencies:
  override:
    - ...
    - cd backend && stack setup && stack build --test --only-dependencies

That is the part when things went a little bit weird. Initially, I was using only stack build, but it looks that stack didn’t compile test dependencies. stack build --test on the other hand, was building test dependencies, but it was instantly running all the tests. That’s why I end up using stack build --test --only-dependencies command.

In the end, we want to run the tests:

test:
  post:
    - cd backend && PGUSER=ubuntu PGDATABASE=circle_test stack test

After that change, it almost works.

Yesod ignoreEnv vs useEnv

When you firstly use yesod, it has config/settings.yml file which keeps information about database:

database:
  user:     "_env:PGUSER:symmetrical_chainsaw_user"
  password: "_env:PGPASS:scpasswd"
  host:     "_env:PGHOST:localhost"
  port:     "_env:PGPORT:5432"
  database: "_env:PGDATABASE:symmetrical_chainsaw_db"
  poolsize: "_env:PGPOOLSIZE:10"

You can probably guess that these _env:PGUSER strings are there to allow developer to replace this values by using environment variables. For test environment there’s different file, config/test-settings.yml in which you can override some values from config/settings.yml. So, if you want to replace this values for test container, but you also want to have some reasonable default for your development machine, you can end up with following contents of config/test-settings.yml file:

database:
  database: "_env:PGDATABASE:symmetrical_chainsaw_db_test"

Then it should work, right? Nope. At least not in automatically generated version by stack yesod website. By default, for some reason, env variables in config/test-settings.yml file don’t work. You have to change one argument in test/TestImport.hs file. By default it’s

settings <- loadAppSettings
  ["config/test-settings.yml", "config/settings.yml"]
  []
  ignoreEnv

and you should change it to

settings <- loadAppSettings
  ["config/test-settings.yml", "config/settings.yml"]
  []
  useEnv

You can also check relevant commmit in Symmetrical Chainsaw project.

Final config

Finally I’ve end up with following circle.yml:

machine:
  ghc:
    version: 7.10.2
  node:
    version: 5.1.0

dependencies:
  cache_directories:
    # Frontend app javascript modules
    - "frontend/node_modules"

    # Backend app haskell modules
    - "~/.stack"

  override:
    # Build haskell dependencies
    - wget -q -O- https://s3.amazonaws.com/download.fpcomplete.com/ubuntu/fpco.key | sudo apt-key add -
    - echo 'deb http://download.fpcomplete.com/ubuntu/precise stable main' | sudo tee /etc/apt/sources.list.d/fpco.list
    - sudo apt-get update && sudo apt-get install stack -y
    - cd backend && stack setup && stack build --test --only-dependencies

    # Build javascript dependencies
    - cd frontend && npm install

test:
  post:
    - cd backend && PGUSER=ubuntu PGDATABASE=circle_test stack test
    - cd frontend && npm run test

After this, my setup of continuous integration was ready to defend me from my mistakes.