For those that don’t know, a few of us are working on a set of tools for building for Swift. As part of that work, I’ve been thinking about how some unit tests could be done in a much simpler way. D does something interesting for unit tests; it allows you to define them inline and have them runnable at build time. Pretty cool, though D’s implementation is a bit limited.

class Sum
{
    int add(int x, int y) { return x + y; }

    unittest
    {
        Sum sum = new Sum;
        assert(sum.add(3,4) == 7);
        assert(sum.add(-2,0) == -2);
    }
}

If we had the ability to create custom attributes in Swift (ok… this feature really requires custom attributes and compiler plug-ins), I was thinking that I could build something like this:

class Sum {
    func add(x: Int, _ y: Int) -> Int { return x + y }
}

@test("Sum", "add(_:_)", "checkin") {
    let sum = Sum()
    assert(sum.add(4, 5) == 9, "Math is hard!")
    assert(sum.add(-3, 3) == 0)
}

The intent is that this provides us with more functionality than what D offers, namely the ability to filter test cases by a number of factors including type, function names, test type (e.g. checkin), or any other text-based qualifier you want. Also, since it was an attribute, we could easily strip out these code paths if a flag, say -enable-testing, wasn’t used.

So, to run all of the checkin tests, you’d do something like this (assume we had some tool run-tests that is magical for now):

$ run-tests -match "checkin"

This would let us find all of the @test items with checkin as part of the metadata and run them.

Ok… that’s great, but Swift doesn’t allow us to create these attributes… so all hope is lost, right?

Nope, we can hack around to get what we want. =)

Instead, let’s do this:

class Sum {
    func add(x: Int, _ y: Int) -> Int { return x + y }
}

func __test_sum_add_checkin() throws {
    let sum = Sum()
    assert(sum.add(4, 5) == 10, "Math is hard!")
    assert(sum.add(-3, 3) == 0)
}

The idea is fairly simple:

1. Build a static library of your module that you wish to test; make sure the -enable-testing flag is set. 2. For each Swift file with methods following our convention (top-level functions that start with __test_), create an executable that calls that function. 3. Run the executable.

Boom! Integrated unit tests.

Digging In

I’m using our build tool, but you can probably do something similar with Swift’s Package Manager.

Here’s the contents of my build file:

(package
  :name "IntegratedUnitTests"

  :tasks {
    :build {
      :tool "atllbuild"
      :sources ["Sum.swift"]
      :name "math"
      :output-type "static-library"
      :publish-product true
      :compile-options ["-enable-testing"]
    }

    :test {
      :dependencies ["generate-test-file"]
      :tool "atllbuild"
      :sources ["sum_test.swift"]
      :name "sum_test"
      :output-type "executable"
      :publish-product true
      :link-with ["math.a"]
    }

    :generate-test-file {
      :dependencies ["build"]
      :tool "shell"
      :script "echo '@testable import math' > sum_test.swift && xcrun -sdk macosx swiftc -print-ast Sum.swift | grep __test | sed 's/internal func/try/g' | sed 's/throws//g' >> sum_test.swift"
    }

    :run {
      :dependencies ["test"]
      :tool "shell"
      :script "./bin/sum_test"
    }
  }
)

The build task is responsible for creating the math.a static library. The test task is responsible for creating the test executable. The generate-test-file task actually does creates the source code for the test executable. It does the following:

1. Creates a new file named sum_test.swift 2. Appends @testable import math to it 3. Examines the AST for Sum.swift and adds the calls for our test methods.

The final file looks like this:

@testable import math
try __test_sum_add_checkin() 

And when you run it:

assertion failed: Math is hard!: file Sum.swift, line 7

Yay! Inlined test code.

This is just a preview. I plan on flushing this out some more, but I thought it was interesting enough to post about. =)