Functional JSON

About a week ago, Chris Eidhof posted this in response to my JSON parsing blogs. Essentially, it's a functional approach at parsing JSON and solving the problem in a slightly different way.

Chris is very familiar with the ins and outs of functional programming, and if you are not as adept as he, you might look at this implementation and feel a bit lost and asking yourself, "uh… how is this easier?". I'll try to walk through it a bit more and apply the pattern to my strongly-typed JSON library.

Part of the confusion I see a lot with functional programming examples is simply this: they often fail to show the ugliness of the code they are fixing. Even with the nice, strongly-typed JSON library, you will still end up with code that looks like this:

extension Blog {
    static func parse(json: JSON?) -> FailableOf<Blog> {
        if let json = json {
            if let id = json["id"].number {
                if let name = json["name"].string {
                    if let needspassword = json["needspassword"].bool {
                        if let url = json["url"].string {
                            let blog = Blog(
                                id: Int(id),
                                name: name,
                                needsPassword: needspassword,
                                url: NSURL(string:url))
                            return FailableOf(blog)
                        }
                    }
                }
            }
        }

        let error = Error(code: 101, domain: "com.kiadsoftware.json", userInfo: nil)
        return FailableOf(error)
    }
}

Note: I'm using a type FailableOf<T> that I wrote about here: Error Handling in Swift.

: .info

The code kinda sucks.

Ok, it really sucks.

But since every access into the JSON data can gives us JSValue.Invalid, we need to guard against it. So there isn't really much more we can do from an imperative coding stand-point to make this better, especially given Swift's if-let construct for optional values.

Now, there is something that is lacking from Chris' implementation: validation of the JSON structure itself. Chris' implementation is not actually a JSON parser, it's a dictionary parser with some JSON semantics. That's a problem because there is no validation of the data that is going in and coming out can properly be transported as JSON in the next steps of our program (storing the NSURL as an object, integer overflow, etc…).

So, I think we can merge the two worlds and make something a bit more usable while still having complete validation of the JSON structure.

Givens

1. The strongly typed, parsed JSON data:

let json: JSON = [
  "stat": "ok",
  "blogs": [
    "blog": [
      [
        "id" : 73,
        "name" : "Bloxus test",
        "needspassword" : true,
        "url" : "http://remote.bloxus.com/"
      ],
      [
        "id" : 74,
        "name" : "Manila Test",
        "needspassword" : false,
        "url" : "http://flickrtest1.userland.com/"
      ]
    ]
  ]
]

2. The Blog structure we want to store the data in:

struct Blog {
    let id: Int
    let name: String
    let needsPassword : Bool
    let url: NSURL
}

3. The desire for full error information captured throughout the JSON parsing and Blog creation.

Approach

Now, we already have a pretty decent way to get an actual raw blog item:

let blog = json["blogs"]["blog"][0]

NOTE: We do not need a "functional" way to parse the JSON structure, though that would be an interesting topic for another day, as JSON is our type that we are going to be processing in a functional manner.

: .info

What we need though, is the ability to take that last dictionary and turn it into our Blog structure.

First, let's start out by adding some functional helpers to the JSValue class and putting the full FailableOf<T> construct in place. I've also update the members on JSValue to return FailableOf<T> instead of Optional<T> (e.g. json["id"].number now returns FailableOf<Double>).

func string(value: FailableOf<JSValue>, key: String) -> FailableOf<String>
func string(value: FailableOf<JSValue>, index: Int) -> FailableOf<String>
func number(value: FailableOf<JSValue>, key: String) -> FailableOf<Double>
func number(value: FailableOf<JSValue>, index: Int) -> FailableOf<Double>
func bool(value: FailableOf<JSValue>, key: String) -> FailableOf<Bool>
func bool(value: FailableOf<JSValue>, index: Int) -> FailableOf<Bool>

These provide us some handy helpers for pulling out an item from a dictionary or array from a JSON structure. These helpers also make it much easier to work with FailableOf<JSValue> and Optional<JSValue> types as there is no need for our usage code to check the error states; these methods simply chain that information back to the caller.

Next, let's start out by making our imperative solution somewhat better. For that, we need to move away from init() methods and into static creation methods that can return our error type: FailableOf<T>.

If Swift ever allows us to return error information on init(), then we can move away from static creation methods. Until then… we do what we have to.

: .info

This looks like this:

static func create(id: FailableOf<Int>,
    name: FailableOf<String>,
    needsPassword: FailableOf<Bool>,
    url: FailableOf<NSURL>) -> FailableOf<Blog>
{
    if let error = id.error {
        return FailableOf(error)
    }

    if let error = name.error {
        return FailableOf(error)
    }

    if let error = needsPassword.error {
        return FailableOf(error)
    }

    if let error = url.error {
        return FailableOf(error)
    }

    let blog = Blog(
        id: id.value!,
        name: name.value!,
        needsPassword: needsPassword.value!,
        url: url.value!)
    return FailableOf(blog)
}

This is already much better than what we had for our init() methods, though I'm not a fan of forcing the value out with id.value! and the others. The alternative, however, is more nesting.

The usage would look like:

let int: FailableOf<Int> = FailableOf(Int(number(json, "id").value!))
let url: FailableOf<NSURL> = FailableOf(NSURL(string: string(json, "url").value!))

let blog = create(
    id: int,
    name: string(json, "name"),
    needsPassword: bool(json, "needsPassword"),
    url: url)

Ok, this is still somewhat cumbersome. One of the issues is dealing with conversions from one type to another, for instance, converting from a Double to an Int. We could provide an int overload on JSValue, but that seems to be mixing our JSON parsing with our Blog parsing. So let's create a function coalesce to do the job for us:

func coalesce<T, U>(input: FailableOf<T>, convert: T -> U?) -> FailableOf<U> {
    if input.failed {
        return FailableOf(input.error!)
    }

    if let converted = convert(input.value!) {
        return FailableOf(converted)
    }
    else {
        let message = "Unable to convert value from type T to type U."
        let error = Error(
            code: 0,
            domain: "com.kiadsoftware.failable",
            userInfo: [LocalizedDescriptionKey: message])
        return FailableOf(error)
    }
}

Note that this function, coalesce is essentially equivalent to the >>= operator that Chris used, though his implementation differed because of some structural choices. We can do the same and it looks like this:

func >>=<T, U>(input: FailableOf<T>, convert: T -> U?) -> FailableOf<U> {
    return coalesce(input, convert)
}

So let's take a look at how things are shaping out now:

let blog = create(
    id: number(json, "id") >>= Blog.toInt,
    name: string(json, "name"),
    needsPassword: bool(json, "needsPassword"),
    url: string(json, "url") >>= Blog.toURL)

This is starting to look really good now. Just to recap what we have so far:

A functional way to get out values from a JSValue backed by a dictionary or array
A graceful way to convert those values, while maintaining the FailableOf<T> error state, to a value of any type.

At this point in the excercise, I'm pretty happy with what we have. But, we can still go one more step into the land of curried functions and partial function application like Chris showed; so let's do it. After all, what's the point in number() if we were going to stop here.

Curried Functions

Chris created a set of curry functions that he built up to allow him to parse out the JSON structure. That's one way to build up the function. Another is to simply let the compiler do it for us:

static func make(id: FailableOf<Int>)
                (name: FailableOf<String>)
                (needsPassword: FailableOf<Bool>)
                (url: FailableOf<NSURL>) -> FailableOf<Blog>
{
    if let error = id.error { return FailableOf(error) }
    if let error = name.error { return FailableOf(error) }
    if let error = needsPassword.error { return FailableOf(error) }
    if let error = url.error { return FailableOf(error) }

    return FailableOf(Blog(id: id.value!, name: name.value!, needsPassword: needsPassword.value!, url: url.value!))
}

That is our curried make function that we'll be using as the creation method for our Blog structure. Notice the multiple sets of (); that is how Swift knows this is a curried function. You can read more here.

Next, we are going to create that nice operator that Chris' made (he used <*>) in order to chain these together in a beautifully functional way, though, I'm a sucker for unicode operators, so I'll be using: ‚áí

infix operator ‚áí { associativity left precedence 150 }
func ‚áí <A, B>(lhs: (A -> B)?, rhs: A?) -> B? {
    if let lhs = lhs {
        if let rhs = rhs {
            return lhs(rhs)
        }
    }

    return nil
}

First, I want to call out that there is no use of FailableOf<T> here as this is a completely generic curry operator. In fact, this entire section has been generically applicable thus far.

This is the code we want to write in the end:

let blog = make ‚áí (number(json, "id") >>= Blog.toInt)
                ‚áí string(json, "name")
                ‚áí bool(json, "needspassword")
                ‚áí (string(json, "url") >>= Blog.toURL)

Remember that our curry operator returns an B? so to use that nicely parsed blog type, you will have to unwrap it.

if let blog = blog {
    // Blog is actually a FailableOf<Blog> now.
}

Extra Deep Dive

I want to point out something now that there is curried function support above: the >>= and ‚áí operators are almost identical. Take a look again:

func >>=<T, U>(input: FailableOf<T>, convert: T -> U?) -> FailableOf<U>
func ‚áí <A, B>(lhs: (A -> B)?, rhs: A?) -> B?

Both operators are really just transforming one item to another, the only real difference is which side the transformation function sits on. Instead of calling our ‚áí operator the "currying" operator we called it the "apply" operator, we can remove the >>= and reduce our operators down to a single one: ‚áí.

func ‚áí <A, B>(lhs: (A -> B)?, rhs: A?) -> B? {
    if let lhs = lhs {
        if let rhs = rhs {
            return lhs(rhs)
        }
    }

    return nil
}

func ‚áí <A, B>(lhs: A?, rhs: (A -> B)?) -> B? {
    if let lhs = lhs {
        if let rhs = rhs {
            return rhs(lhs)
        }
    }

    return nil
}

This allows us to write the following code instead:

// Change Blog.toInt to the following implementation:
static func toInt(number: FailableOf<Double>) -> FailableOf<Int> {
    if let error = number.error {
        return FailableOf(error)
    }

    return FailableOf(Int(number.value!))
}

// Change Blog.toURL to the following implementation:
static func toURL(string: FailableOf<String>) -> FailableOf<NSURL> {
    if let error = string.error {
        return FailableOf(error)
    }

    return FailableOf(NSURL(string: string.value!))
}

let blog = make ‚áí (number(json, "id") ‚áí Blog.toInt)
                ‚áí string(json, "name")
                ‚áí bool(json, "needspassword")
                ‚áí (string(json, "url") ‚áí Blog.toURL)

Ok, we are almost there now! Remember that stupid unboxing of the Optional<FailableOf<Blog>> we had to do above? Well, with a couple more operator overloads, that problem goes away as well.

public func ‚áí <A, B>(lhs: A -> B, rhs: A) -> B { return lhs(rhs) }
public func ‚áí <A, B>(lhs: A, rhs: A -> B) -> B { return rhs(lhs) }

Now the blog in the code above will simply be of type: FailableOf<Blog>.

So there we have it… a completely functional way to write a parser the JSValue type. I think this neatly merges both worlds. And, there is full error reporting support as well.

Again, just to show where we went from to where we ended up, take a look here:

// Before we went functional
static func parse(json: JSON?) -> FailableOf<Blog> {
    if let json = json {
        if let id = json["id"].number.value {
            if let name = json["name"].string.value {
                if let needspassword = json["needspassword"].bool.value {
                    if let url = json["url"].string.value {
                        let blog = Blog(
                            id: Int(id),
                            name: name,
                            needsPassword: needspassword,
                            url: NSURL(string:url))
                        return FailableOf(blog)
                    }
                }
            }
        }
    }

    return FailableOf(Error(code: 101, domain: "com.kiadsoftware.json", userInfo: nil))
}

// After we went functional
static func fparse(json: JSValue?) -> FailableOf<Blog> {
    return make ‚áí (number(json, "id") ‚áí Blog.toInt)
                ‚áí string(json, "name")
                ‚áí bool(json, "needspassword")
                ‚áí (string(json, "url") ‚áí Blog.toURL)
}

Resources

The full source for the JSON parser can be found here: https://github.com/owensd/json-swift.

The full source for the sample app is below. Simply copy it into a project and link the json-swift project.

import Foundation
import JSONLib

struct Blog {
    let id: Int
    let name: String
    let needsPassword : Bool
    let url: NSURL
}

func printBlog(blog: FailableOf<Blog>)
{
    if let blog = blog.value {
        println("id: \(blog.id)")
        println("name: \(blog.name)")
        println("needspassword: \(blog.needsPassword)")
        println("url: \(blog.url)")
    }
    else {
        let message = blog.error!.userInfo[LocalizedDescriptionKey]
        println("Error parsing blog: \(message)")
    }
}

var json: JSON = [
    "stat": "ok",
    "blogs": [
        "blog": [
            [
                "id" : 73,
                "name" : "Bloxus test",
                "needspassword" : true,
                "url" : "http://remote.bloxus.com/"
            ],
            [
                "id" : 74,
                "name" : "Manila Test",
                "needspassword" : false,
                "url" : "http://flickrtest1.userland.com/"
            ]
        ]
    ]
]

extension Blog {
    static func parse(json: JSON?) -> FailableOf<Blog> {
        if let json = json {
            if let id = json["id"].number.value {
                if let name = json["name"].string.value {
                    if let needspassword = json["needspassword"].bool.value {
                        if let url = json["url"].string.value {
                            let blog = Blog(
                                id: Int(id),
                                name: name,
                                needsPassword: needspassword,
                                url: NSURL(string:url))
                            return FailableOf(blog)
                        }
                    }
                }
            }
        }

        return FailableOf(Error(
            code: 101,
            domain: "com.kiadsoftware.json",
            userInfo: nil))
    }

    static func fparse(json: JSValue?) -> FailableOf<Blog> {
        return make ‚áí (number(json, "id") ‚áí Blog.toInt)
                    ‚áí string(json, "name")
                    ‚áí bool(json, "needspassword")
                    ‚áí (string(json, "url") ‚áí Blog.toURL)
    }
}

extension Blog {
    static func make(id: FailableOf<Int>)
                    (name: FailableOf<String>)
                    (needsPassword: FailableOf<Bool>)
                    (url: FailableOf<NSURL>) -> FailableOf<Blog>
    {
        if let error = id.error { return FailableOf(error) }
        if let error = name.error { return FailableOf(error) }
        if let error = needsPassword.error { return FailableOf(error) }
        if let error = url.error { return FailableOf(error) }

        return FailableOf(Blog(
            id: id.value!,
            name: name.value!,
            needsPassword: needsPassword.value!,
            url: url.value!))
    }

    static func toInt(number: FailableOf<Double>) -> FailableOf<Int> {
        if let error = number.error {
            return FailableOf(error)
        }

        return FailableOf(Int(number.value!))
    }
    static func toURL(string: FailableOf<String>) -> FailableOf<NSURL> {
        if let error = string.error {
            return FailableOf(error)
        }

        return FailableOf(NSURL(string: string.value!))
    }
}

printBlog(Blog.parse(json["blogs"]["blog"][0]))
println("--------------")
printBlog(Blog.fparse(json["blogs"]["blog"][0]))

owensd.io

A builder of things.

Functional JSON

Givens

Approach

Curried Functions

Extra Deep Dive

Resources

Givens

Approach

Curried Functions

Extra Deep Dive

Resources

Share this: