Generators Need a current value

When you build a parser, you need the ability to scan through your tokens to build up your output. Swift offers us a few different constructs for iteration; I needed the one that would be the fastest, after-all, I'm building a parser!

I wrote a small test suite to test the various iteration types, which essentially boils down to two options for Strings:

  1. The tradition for-loop that uses an index value
  2. The GeneratorType based approach

Index-based for-loop Here's the code for this one:

var string = ""

let scalars = self.largeJSON.unicodeScalars
self.measureBlock() {
    for var idx = scalars.startIndex; idx < scalars.endIndex; idx = idx.successor() {
        let scalar = scalars[idx]
        scalar.writeTo(&string)
    }
}

It's pretty straight-forward; simply start at startIndex and traverse your way through the string until you hit endIndex. There are couple of gotchas though, the most significant being that Swift doesn't allow Int-based indexing – all of the types have their own special indexing type. The thing to watch out for, not all of them are bi-directional.

GeneratorType Approach

var string = ""

self.measureBlock() {
    for scalar in self.largeJSON.unicodeScalars {
        scalar.writeTo(&string)
    }
}

This one is fairly simple as well: simply loop through all of the unicode values. We can also write this loop in a slightly different way:

var string = ""

self.measureBlock() {
    var generator = self.largeJSON.unicodeScalars.generate()
    for var scalar = generator.next(); scalar != nil; scalar = generator.next() {
        scalar?.writeTo(&string)
    }
}

In my testing, I found that the GeneratorType-based approach was about 18% faster. This is significant enough for me to use it. =)

Implementing the Parsing

Next up is actually parsing the JSON string. The basic idea is to look for specific tokens and call into one these methods:

  1. parseObject – used to parse out a JSON object (e.g. dictionary)
  2. parseArray – used to parse an array
  3. parseNumber – used to parse a number value
  4. parseString – used to parse out a string, also used when parsing keys from a dictionary
  5. parseTrue – used to parse the boolean value true
  6. parseFalse – used to parse the boolean value false
  7. parseNull – used to parse the literal value null

I think that about covers the basics of what I need. And here comes the problem… each of these a one more or pieces of the information:

  1. The current generator value so that increments can be done
  2. The current character the generator is pointing to
  3. The character used at the start of the parse call

When we look at the API for GeneratorType, we find that it only supports next(). Hmm… that's not going to be sufficient. So now we are left with two choices:

  1. Pass the current unicode token around with the our generator instance, or
  2. Package up the generator and the current unicode token into a single class

To me, this is a no-brainer. As soon as we introduce this coupling, it is best to package up the dependencies and maintain that state with a single value.

Ideally, we would simply be able to extend the GeneratorType instance for String.UnicodeScalarView, however, we cannot extend types with stored properties, so we are left with creating an entirely new type to box this functionality.

: .info

To work around this limitation, I created the following type:

struct UnicodeScalarParsingBuffer {
    var generator: String.UnicodeScalarView.Generator
    var current: UnicodeScalar? = nil

    init(_ generator: String.UnicodeScalarView.Generator) {
        self.generator = generator
    }

    mutating func next() -> UnicodeScalar? {
        self.currentUnicodeScalar = generator.next()
        return self.currentUnicodeScalar
    }
}

I find this to be a deficiency in the current implementation of GeneratorType. While it may be the case that you are always working in the same scope, it is also necessary at times to pass this context around. Once you start doing that, you're going to need that current value, otherwise you need to pass both generator and current – no one really wants to do that.

The full source for the json-swift library can be found over on GitHub; the parsing code is here.

Generators Need a current value

Combining Characters

In my JSON Parsing post, I talked about an issue I was having with a particular character set:

let string = "\"\u{aaef}abcd"
countElements(string)           // 5
countElements(string.utf8)      // 8

Well, it turns out that \u{aaef} is a unicode combining character that modifies the character before it. There are some combining characters that create a single character, but there are also combining characters that still result in multiple visible characters, as seen above.

However, it seems there is a view into the string that gave me what I wanted:

let string = "\"\u{aaef}abcd"
countElements(string.unicodeScalars)     // 6

If we take a look at a few other examples, we can see that the unicodeScalars seems to give us the full make-up of the unicode values that are making up the string.

let single = "è"    // \u{e8}
for scalar in single.unicodeScalars {
    println("\(scalar) (\(scalar.value))")      // prints: è
}

let combined = "e\u{300}"
for scalar in combined.unicodeScalars {
    println("\(scalar) (\(scalar.value))")      // prints: e, `
}

Notice the difference in the two: the first is an single unicode value, the second is the letter "e" combined with the accent grave (`).

The only downside that I've run into with this approach is that it seems to be significantly slower the UT8-based approach I was using earlier.

Combining Characters

JSON Parsing

As part of my diving into Swift I've been using JSON as one of my learning projects: json-swift. Continuing on that tract, I took a look at what it would take to create a JSON parser using only Swift and no ObjC bridging. The results: ok, but lots of room for improvement.

One of the nice things about Swift is that it tries to abstract away all of the unicode information from you and create a nice, simple API for you to work with. Well, that's nice when it works, but there are cases that I ran into where it seemed to simply not be doing what I expected.

One such example:

let string = "\"Í´Øabcd"
countElements(string)           // 5
countElements(string.utf8)      // 8

// The raw bytes:
34      // "
234     // makes up "Í´Ø
171
175
97      // b
98      // c
99      // d
100     // e

In case that character isn't showing up, this is: unicde character, not sure the name of it.

I do not know enough about unicode so I don't know all of the ins and outs of why the " is attached to the unicode character (probably something to do with it being only three bytes), so some of you might be: duh!. That's ok. =)

: .info

That's not the worse part though, evident the value "Í´Ø is a single character, that is treated somewhat like a quote so you have to escape it, hence: \"Í´Øabcd.

I really wanted to try the String class out, so I had to use String.UTF8View as the mapping and compare each of the bytes and build up my strings manually using UnsafePointer and String.fromCString:

static func parseString(string: String.UTF8View, inout startAt index: String.UTF8View.Index, quote: UInt8) -> FailableOf<JSValue> {
    var bytes = [UInt8]()

    index = index.successor()
    for ; index != string.endIndex; index = index.successor() {
        let cu = string[index]
        if cu == quote {
            // Determine if the quote is being escaped or not...
            var count = 0
            for byte in reverse(bytes) {
                if byte == Token.Backslash.toRaw() { count++ }
                else { break }
            }

            if count % 2 == 0 {     // an even number means matched slashes, not an escape
                index = index.successor()

                bytes.append(0)
                let ptr = UnsafePointer<CChar>(bytes)
                return FailableOf(JSValue(JSBackingValue.JSString(String.fromCString(ptr)!)))
            }
            else {
                bytes.append(cu)
            }
        }
        else {
            bytes.append(cu)
        }
    }

    let info = [
        ErrorKeys.LocalizedDescription: ErrorCode.ParsingError.message,
        ErrorKeys.LocalizedFailureReason: "Error parsing JSON string."]
    return FailableOf(Error(code: ErrorCode.ParsingError, domain: JSValueErrorDomain, userInfo: info))
}

Since I do not know enough about unicode, I'm not sure if these are Swift bugs, limitations, or simply a lack in my own understanding.

Some of the problems I ran into:

  1. Unable to take a character from String and figure out the bytes that made it, so I have to use String.UTF8View.
  2. The String.UTF8View.Index is not Comparable, though it Equatable; I was doing index < endIndex initially.
  3. The String.UTF8View.Index is forward indexing only; originally, part of my algorithm would also step backwards through part of the string.
  4. Performance of my algorithm is about 3x slower than the NSJSONSerialization algorithm: 0.12s vs. 0.4s to parse a roughly 688KB file. I still need to investigate the memory usage.

I'm going to keep playing with it, especially as the later betas come out. I may try a purely functional based algorithm as well and see how that plays out in Swift.

JSON Parsing

Implicit Chaining and Context

If you’ve been following along, I’ve been struggling with the optional chaining syntax.

The Swift team has been fairly active in helping people with their troubles and confusion, and Chris Lattner responded to mine here (login required).

I won’t quote all of it (mainly because I don’t think I’m supposed to), but the gist of of it was this: if Swift supported implicit optional chaining, then it would be unclear which code was executed in the chain and which was not (this is my paraphrase, not Chris’ exact words).

An example:

No implicit chaining

my.delegate?.can().call()?.some(stuff())
foo.bar(baz())

With implicit chaining

my.delegate.can().call().some(stuff())
foo.bar(baz())

Chris’ argument is that with implicit chaining, the code above is ambiguous. In line #1, it’s explicit that there are many potential breaks along the code so it’s clear that everything to the right of a ? has the potential of not being called. For line #2, it’s clear there no breaks. However, with implicit chaining, lines #1 and lines #2 do not carry that information and it’s unclear if stuff() or baz() are ever called.

I understand his point; afterall, it is clear (in any editing/reading environment) that there are multiple failure (or maybe more accurately, short-circuiting) points along the way when we use ?. But, it still didn’t sit right with me.

I was asking myself why… I think I know why it didn’t sit well: I think the example is void of any and all context that could potentially already answer that question for us.

For example, if we know that Swift supports implicit optional chaining, then we already know that member lookups can potentially fail. Now, you might argue that I’ve simply made every member lookup ambiguous. But I don’t think that is the case either.

You see, the code samples above are all taken out of context. Code doesn’t live in isolation, but it participates in the context around it.

Here’s a snippet of code from a JSON tests:

func testValidateSingleValueNumberUsagePatternOptionalChaining() {
    var json: JSValue = 123

    let value = json.number?.distanceTo(100) ?? 0
    XCTAssertEqual(value, -23)
}

If I had written:

func testValidateSingleValueNumberUsagePatternOptionalChaining() {
    var json: JSValue = 123

    let value = json.number.distanceTo(100) ?? 0
    XCTAssertEqual(value, -23)
}

Or even:

func testValidateSingleValueNumberUsagePatternOptionalChaining() {
    var json: JSValue = 123

    let value = json.number.distanceTo(distanceToMoon()) ?? 0
    XCTAssertEqual(value, distanceToMoon() - 123)
}

The code carries no loss in value. Even without that the aid of coding tools, the ?? operator already tells me I’m working with a left-hand side that is Optional<T> and a righ-hand side this of type T. Context is also how I know that the type of value is Double (not an Int like you might be expecting) because JSValue.number holds a Double?.

I think it’s easy to take a code snippet, or worse, make up a line of code that also has no intrinsic meaning, and make a good case for why it has the potential to cause ambiguity. My argument is not that it’s not possible write ambiguous code with implicit chaining, but rather, that the context of your code ensures that the ambiguity is seldom, if ever, there.

In the end, I think it simply comes down to this:

Writing code is like writing in any language – often times we have constructs and words that look the same but are different, such as “I read a book yesterday” and “I read every morning”. The usage of “read”, when taking out of the context of its environment carries too little information to reveal its full meaning. However, once we provide the surrounding context, the meaning is made explicitly clear.

As a corrolary, we shouldn’t also then say that because I can write, “I read in the morning”, that the usage of “read” should not be allowed because it creates ambiguity: do I read every morning, or did I read this morning. Rather, we say that the sentence is ambiguous when taking out of its full context and we should add more context, or, if the statement is meant to stand on its own, make the disambiguate the meaning.

I think pragmatically, the use of Optional<T> is the same: context reveals the explicit meaning of the code.

Implicit Chaining and Context

The Case for Implicit Optional Chaining

Update #2, Chris Lattner has stated this isn't a good idea because it can lead to ambiguity in code that may have side effects, such as foo.bar(baz()). The question is, does baz() get called or not. With foo?.bar(baz()) it is clear that there is a potential for baz() to not get called.

I understand what he is saying, though, I'm not sure I'm in agreement yet. Anyhow, it's nice to have this dialoge with the fine folks on the Swift team.

: .info

I've been wrestling with the verbosity of Optional Chaining for quite some time now. It started with my first JSON Parsing article and really hasn't gone away since then. Much of the work there was done to explicity hide the fact that a lookup could fail. Why do we need that? The language already has a construct to help us with this: Optionals.

Here's the example code that's essentially in the Swift Programming Language guide linked above:

class Residence {
    var numberOfRooms: Int = 1
}

class Person {
    var residence: Residence?
}

var john = Person()
// john.residence = Residence()  // uncomment this to toggle with if-else branch you get

if let roomCount = john.residence?.numberOfRooms {
    println("John's residence has \(roomCount) room(s).")
} else {
    println("Unable to retrieve the number of rooms.")
}

The above, the lone ? isn't that big of a deal. Though, I do think it's superfluous.

Let's get the cat out the bag: Swift is all about type inference. We can seldom look at code out of context and reason the type that is store in any given variable. For instance, what is the type for numberOfRooms? It's like an Int, but it could be Int?, Double, a UInt, maybe an Int8, or whatever.

There is only way to know: look at the definition.

A handy way to do that: ‚å• + Click

Showing type interference for 'let roomCount' is Int?

However, I'm also going to assert this: we do know the type while we are authoring the code. We know because we are going to use it and we need to know. This is subtley different from above but just as important. The context we are in gives us clarity over what it is.

This is what I think the code should look like:

if let roomCount = john.residence.numberOfRooms {
    println("John's residence has \(roomCount) room(s).")
} else {
    println("Unable to retrieve the number of rooms.")
}

That's it, just remove the ?. Wait, doesn't that add confusion? How did numberOfRooms end up as an Int? requiring us to use if-let? Well, we know because at the time of authoring, we knew that residence is backed by Residence?. It doesn't matter when we come to modify the code how roomCount became an Int?, it only matters that it is one and you need to work with it as one.

The only time you will care why roomCount became an Int? is when you need to modify that variable itself, but then you are going to need to understand the entire chain john.residence.numberOfRooms at that point anyway. We've actually lost very little here and gained clarity in syntax.

Let's change the example above to contain a little richer data:

class RoomInfo {
    var description: String = ""
    var width: Double = 0
    var depth: Double = 0
}

class Residence {
    var rooms: [RoomInfo]? = nil
}

class Person {
    var residence: Residence?
}

var john = Person()

if let roomCount = john.residence?.rooms?.count {
    println("John's residence has \(roomCount) room(s).")
} else {
    println("Unable to retrieve the number of rooms.")
}

This is where things are string to get really ugly for me. Why are there two ?? Yes, I know that both residence and rooms are an Optional<T>, but I don't really care. The purpose of the code is this: retrieve the number of rooms at John's residence. The ? do not help me get there. Further, I'll add that if you really care about the ?, you should be checking for which of the items, residence or rooms is nil, but the simple truth is this: we do not actually care; we only care about the end result in the chain.

So, instead, simply make the code what we want:

if let roomCount = john.residence.rooms.count {
    println("John's residence has \(roomCount) room(s).")
} else {
    println("Unable to retrieve the number of rooms.")
}

The code above tells me one simple truth: when I get a roomCount, it's going to be an Optional<T>, because something down the chain failed; I do not care what failed, just that it failed and I'm going to handle that.

The code with the ?. tells me something different. It tells me that on your way to roomCount the members residence or rooms could have returned nil. I can get that info from the type declaration too, if I really wanted it. Don't make me repeat myself, especially when I'm doing so and it is not even important enough for me to do something about.

Update: August 13th, 2014

: .info

There was an interesting question asked about Optional<T> methods and extensions that collide with the type T. Here's an example (current Swift syntax; no implicit optional chaining):

extension Optional: Printable {
    public var description: String {
        return "a message for you"
    }
}

// Add the extension for String:
extension String: Printable {
    public var description: String {
        return "printable: string"
    }
}

let string: String? = nil
let s1 = string.description     // s1: String
let s2 = string?.description    // s2: String?

Today that is not ambiguous, we know which to call. With implicit chaining:

let string: String? = nil
let s1 = string.description

So, do we call the description on String or on Optional<T>?

I'm going to assert the following:

  1. We want to optimize for the non-Optional<T>; after all, that's why I want implicit chaining.
  2. The Optional<T> is already a special construct in the language and I'm ok with adding more rules in certain cases for when we need to deal with it directly as I think those cases are the exception, not the rule.

This means that I want the String.description version called.

If we really want the Optional<T> version, we have a way to do so:

let s1 = (string as Optional<String>).description

In the rare case where there is a collision, the compiler could do the following:

  1. Create an error or warning (I'd prefer a warning) that this is ambiguity, and
  2. Provide two "fix-it" options for the two valid cases for you to disambiguate, or
  3. Do nothing… I tend to think this is actually the right answer.

We could also (as mentioned by Wallacy Freitas on the devforums) simply invert the ? usage to treat it as explicitly working with the optional:

let s1 = string?.description

Again, I think this is simply an explicit example of the rare case. With the ? today, we need to always defend against this possibility. I would rather optimize for the normal case and provide a way around the corner case.

The Case for Implicit Optional Chaining

Fixed Enum Layout Sizes

WARNING: Turns out there are some nasty side-effects, see the update below.

: .warning

If you have every attempted to create a generic enum, you have most certainly encountered the following error message:

error: unimplemented IR generation feature non-fixed multi-payload enum layout

: .warning

Yikes! Well, it turns out that Swift (whether by-design or a current limitation) needs to know the full layout size of the enum at compile time. So code like this:

enum FailableOf<T> {
  case Success(T)
  case Failure(Error)

  init(_ value: T) {
    self = .Success(value)
  }

  init(_ error: Error) {
    self = .Failure(error)
  }

  var failed: Bool { /* ... */ }
  var error: Error? { /* ... */ }

  var value: T? {
    switch self {
    case .Success(let value):
      return value

    default:
      return nil
    }
  }
}

Is just not possible to write. I had worked around this by using a wrapper class FailableWrapper<T> (see Error Handling in Swift).

However, Rob Napier tweeted a much more elegant solution.

//platform.twitter.com/widgets.js
When I saw that, I thought, "well duh!". So much better than my workaround.

Here's the source for my full FailableOf<T> class; no more need for the wrapper class.

public enum FailableOf<T> {
    case Success(@autoclosure() -> T)
    case Failure(Error)

    public init(_ value: T) {
        self = .Success(value)
    }

    public init(_ error: Error) {
        self = .Failure(error)
    }

    public var failed: Bool {
        switch self {
        case .Failure(let error):
            return true

        default:
            return false
        }
    }

    public var error: Error? {
        switch self {
        case .Failure(let error):
            return error

        default:
            return nil
        }
    }

    public var value: T? {
        switch self {
        case .Success(let value):
            return value()

        default:
            return nil
        }
    }
}

UPDATE August 6th: Thanks for John Vasileff for pointing out, what should have been, an obvious design flaw with this approach.

: .info

//platform.twitter.com/widgets.js
Yep… he's right. =)

Back to my wrapper class for now.

Fixed Enum Layout Sizes

Functional JSON

About a week ago, Chris Eidhof posted this in response to my JSON parsing blogs. Essentially, it's a functional approach at parsing JSON and solving the problem in a slightly different way.

Chris is very familiar with the ins and outs of functional programming, and if you are not as adept as he, you might look at this implementation and feel a bit lost and asking yourself, "uh… how is this easier?". I'll try to walk through it a bit more and apply the pattern to my strongly-typed JSON library.

Part of the confusion I see a lot with functional programming examples is simply this: they often fail to show the ugliness of the code they are fixing. Even with the nice, strongly-typed JSON library, you will still end up with code that looks like this:

extension Blog {
    static func parse(json: JSON?) -> FailableOf<Blog> {
        if let json = json {
            if let id = json["id"].number {
                if let name = json["name"].string {
                    if let needspassword = json["needspassword"].bool {
                        if let url = json["url"].string {
                            let blog = Blog(
                                id: Int(id),
                                name: name,
                                needsPassword: needspassword,
                                url: NSURL(string:url))
                            return FailableOf(blog)
                        }
                    }
                }
            }
        }

        let error = Error(code: 101, domain: "com.kiadsoftware.json", userInfo: nil)
        return FailableOf(error)
    }
}

Note: I'm using a type FailableOf<T> that I wrote about here: Error Handling in Swift.

: .info

The code kinda sucks.

Ok, it really sucks.

But since every access into the JSON data can gives us JSValue.Invalid, we need to guard against it. So there isn't really much more we can do from an imperative coding stand-point to make this better, especially given Swift's if-let construct for optional values.

Now, there is something that is lacking from Chris' implementation: validation of the JSON structure itself. Chris' implementation is not actually a JSON parser, it's a dictionary parser with some JSON semantics. That's a problem because there is no validation of the data that is going in and coming out can properly be transported as JSON in the next steps of our program (storing the NSURL as an object, integer overflow, etc…).

So, I think we can merge the two worlds and make something a bit more usable while still having complete validation of the JSON structure.

Givens

1. The strongly typed, parsed JSON data:

let json: JSON = [
  "stat": "ok",
  "blogs": [
    "blog": [
      [
        "id" : 73,
        "name" : "Bloxus test",
        "needspassword" : true,
        "url" : "http://remote.bloxus.com/"
      ],
      [
        "id" : 74,
        "name" : "Manila Test",
        "needspassword" : false,
        "url" : "http://flickrtest1.userland.com/"
      ]
    ]
  ]
]

2. The Blog structure we want to store the data in:

struct Blog {
    let id: Int
    let name: String
    let needsPassword : Bool
    let url: NSURL
}

3. The desire for full error information captured throughout the JSON parsing and Blog creation.

Approach

Now, we already have a pretty decent way to get an actual raw blog item:

let blog = json["blogs"]["blog"][0]

NOTE: We do not need a "functional" way to parse the JSON structure, though that would be an interesting topic for another day, as JSON is our type that we are going to be processing in a functional manner.

: .info

What we need though, is the ability to take that last dictionary and turn it into our Blog structure.

First, let's start out by adding some functional helpers to the JSValue class and putting the full FailableOf<T> construct in place. I've also update the members on JSValue to return FailableOf<T> instead of Optional<T> (e.g. json["id"].number now returns FailableOf<Double>).

func string(value: FailableOf<JSValue>, key: String) -> FailableOf<String>
func string(value: FailableOf<JSValue>, index: Int) -> FailableOf<String>
func number(value: FailableOf<JSValue>, key: String) -> FailableOf<Double>
func number(value: FailableOf<JSValue>, index: Int) -> FailableOf<Double>
func bool(value: FailableOf<JSValue>, key: String) -> FailableOf<Bool>
func bool(value: FailableOf<JSValue>, index: Int) -> FailableOf<Bool>

These provide us some handy helpers for pulling out an item from a dictionary or array from a JSON structure. These helpers also make it much easier to work with FailableOf<JSValue> and Optional<JSValue> types as there is no need for our usage code to check the error states; these methods simply chain that information back to the caller.

Next, let's start out by making our imperative solution somewhat better. For that, we need to move away from init() methods and into static creation methods that can return our error type: FailableOf<T>.

If Swift ever allows us to return error information on init(), then we can move away from static creation methods. Until then… we do what we have to.

: .info

This looks like this:

static func create(id: FailableOf<Int>,
    name: FailableOf<String>,
    needsPassword: FailableOf<Bool>,
    url: FailableOf<NSURL>) -> FailableOf<Blog>
{
    if let error = id.error {
        return FailableOf(error)
    }

    if let error = name.error {
        return FailableOf(error)
    }

    if let error = needsPassword.error {
        return FailableOf(error)
    }

    if let error = url.error {
        return FailableOf(error)
    }

    let blog = Blog(
        id: id.value!,
        name: name.value!,
        needsPassword: needsPassword.value!,
        url: url.value!)
    return FailableOf(blog)
}

This is already much better than what we had for our init() methods, though I'm not a fan of forcing the value out with id.value! and the others. The alternative, however, is more nesting.

The usage would look like:

let int: FailableOf<Int> = FailableOf(Int(number(json, "id").value!))
let url: FailableOf<NSURL> = FailableOf(NSURL(string: string(json, "url").value!))

let blog = create(
    id: int,
    name: string(json, "name"),
    needsPassword: bool(json, "needsPassword"),
    url: url)

Ok, this is still somewhat cumbersome. One of the issues is dealing with conversions from one type to another, for instance, converting from a Double to an Int. We could provide an int overload on JSValue, but that seems to be mixing our JSON parsing with our Blog parsing. So let's create a function coalesce to do the job for us:

func coalesce<T, U>(input: FailableOf<T>, convert: T -> U?) -> FailableOf<U> {
    if input.failed {
        return FailableOf(input.error!)
    }

    if let converted = convert(input.value!) {
        return FailableOf(converted)
    }
    else {
        let message = "Unable to convert value from type T to type U."
        let error = Error(
            code: 0,
            domain: "com.kiadsoftware.failable",
            userInfo: [LocalizedDescriptionKey: message])
        return FailableOf(error)
    }
}

Note that this function, coalesce is essentially equivalent to the >>= operator that Chris used, though his implementation differed because of some structural choices. We can do the same and it looks like this:

func >>=<T, U>(input: FailableOf<T>, convert: T -> U?) -> FailableOf<U> {
    return coalesce(input, convert)
}

So let's take a look at how things are shaping out now:

let blog = create(
    id: number(json, "id") >>= Blog.toInt,
    name: string(json, "name"),
    needsPassword: bool(json, "needsPassword"),
    url: string(json, "url") >>= Blog.toURL)

This is starting to look really good now. Just to recap what we have so far:

  1. A functional way to get out values from a JSValue backed by a dictionary or array
  2. A graceful way to convert those values, while maintaining the FailableOf<T> error state, to a value of any type.

At this point in the excercise, I'm pretty happy with what we have. But, we can still go one more step into the land of curried functions and partial function application like Chris showed; so let's do it. After all, what's the point in number() if we were going to stop here.

Curried Functions

Chris created a set of curry functions that he built up to allow him to parse out the JSON structure. That's one way to build up the function. Another is to simply let the compiler do it for us:

static func make(id: FailableOf<Int>)
                (name: FailableOf<String>)
                (needsPassword: FailableOf<Bool>)
                (url: FailableOf<NSURL>) -> FailableOf<Blog>
{
    if let error = id.error { return FailableOf(error) }
    if let error = name.error { return FailableOf(error) }
    if let error = needsPassword.error { return FailableOf(error) }
    if let error = url.error { return FailableOf(error) }

    return FailableOf(Blog(id: id.value!, name: name.value!, needsPassword: needsPassword.value!, url: url.value!))
}

That is our curried make function that we'll be using as the creation method for our Blog structure. Notice the multiple sets of (); that is how Swift knows this is a curried function. You can read more here.

Next, we are going to create that nice operator that Chris' made (he used <*>) in order to chain these together in a beautifully functional way, though, I'm a sucker for unicode operators, so I'll be using: ‚áí

infix operator ‚áí { associativity left precedence 150 }
func ‚áí <A, B>(lhs: (A -> B)?, rhs: A?) -> B? {
    if let lhs = lhs {
        if let rhs = rhs {
            return lhs(rhs)
        }
    }

    return nil
}

First, I want to call out that there is no use of FailableOf<T> here as this is a completely generic curry operator. In fact, this entire section has been generically applicable thus far.

This is the code we want to write in the end:

let blog = make ‚áí (number(json, "id") >>= Blog.toInt)
                ‚áí string(json, "name")
                ‚áí bool(json, "needspassword")
                ‚áí (string(json, "url") >>= Blog.toURL)

Remember that our curry operator returns an B? so to use that nicely parsed blog type, you will have to unwrap it.

if let blog = blog {
    // Blog is actually a FailableOf<Blog> now.
}

Extra Deep Dive

I want to point out something now that there is curried function support above: the >>= and ‚áí operators are almost identical. Take a look again:

func >>=<T, U>(input: FailableOf<T>, convert: T -> U?) -> FailableOf<U>
func ‚áí <A, B>(lhs: (A -> B)?, rhs: A?) -> B?

Both operators are really just transforming one item to another, the only real difference is which side the transformation function sits on. Instead of calling our ‚áí operator the "currying" operator we called it the "apply" operator, we can remove the >>= and reduce our operators down to a single one: ‚áí.

func ‚áí <A, B>(lhs: (A -> B)?, rhs: A?) -> B? {
    if let lhs = lhs {
        if let rhs = rhs {
            return lhs(rhs)
        }
    }

    return nil
}

func ‚áí <A, B>(lhs: A?, rhs: (A -> B)?) -> B? {
    if let lhs = lhs {
        if let rhs = rhs {
            return rhs(lhs)
        }
    }

    return nil
}

This allows us to write the following code instead:

// Change Blog.toInt to the following implementation:
static func toInt(number: FailableOf<Double>) -> FailableOf<Int> {
    if let error = number.error {
        return FailableOf(error)
    }

    return FailableOf(Int(number.value!))
}

// Change Blog.toURL to the following implementation:
static func toURL(string: FailableOf<String>) -> FailableOf<NSURL> {
    if let error = string.error {
        return FailableOf(error)
    }

    return FailableOf(NSURL(string: string.value!))
}

let blog = make ‚áí (number(json, "id") ‚áí Blog.toInt)
                ‚áí string(json, "name")
                ‚áí bool(json, "needspassword")
                ‚áí (string(json, "url") ‚áí Blog.toURL)

Ok, we are almost there now! Remember that stupid unboxing of the Optional<FailableOf<Blog>> we had to do above? Well, with a couple more operator overloads, that problem goes away as well.

public func ‚áí <A, B>(lhs: A -> B, rhs: A) -> B { return lhs(rhs) }
public func ‚áí <A, B>(lhs: A, rhs: A -> B) -> B { return rhs(lhs) }

Now the blog in the code above will simply be of type: FailableOf<Blog>.

So there we have it… a completely functional way to write a parser the JSValue type. I think this neatly merges both worlds. And, there is full error reporting support as well.

Again, just to show where we went from to where we ended up, take a look here:

// Before we went functional
static func parse(json: JSON?) -> FailableOf<Blog> {
    if let json = json {
        if let id = json["id"].number.value {
            if let name = json["name"].string.value {
                if let needspassword = json["needspassword"].bool.value {
                    if let url = json["url"].string.value {
                        let blog = Blog(
                            id: Int(id),
                            name: name,
                            needsPassword: needspassword,
                            url: NSURL(string:url))
                        return FailableOf(blog)
                    }
                }
            }
        }
    }

    return FailableOf(Error(code: 101, domain: "com.kiadsoftware.json", userInfo: nil))
}

// After we went functional
static func fparse(json: JSValue?) -> FailableOf<Blog> {
    return make ‚áí (number(json, "id") ‚áí Blog.toInt)
                ‚áí string(json, "name")
                ‚áí bool(json, "needspassword")
                ‚áí (string(json, "url") ‚áí Blog.toURL)
}

Resources

The full source for the JSON parser can be found here: https://github.com/owensd/json-swift.

The full source for the sample app is below. Simply copy it into a project and link the json-swift project.

import Foundation
import JSONLib

struct Blog {
    let id: Int
    let name: String
    let needsPassword : Bool
    let url: NSURL
}

func printBlog(blog: FailableOf<Blog>)
{
    if let blog = blog.value {
        println("id: \(blog.id)")
        println("name: \(blog.name)")
        println("needspassword: \(blog.needsPassword)")
        println("url: \(blog.url)")
    }
    else {
        let message = blog.error!.userInfo[LocalizedDescriptionKey]
        println("Error parsing blog: \(message)")
    }
}

var json: JSON = [
    "stat": "ok",
    "blogs": [
        "blog": [
            [
                "id" : 73,
                "name" : "Bloxus test",
                "needspassword" : true,
                "url" : "http://remote.bloxus.com/"
            ],
            [
                "id" : 74,
                "name" : "Manila Test",
                "needspassword" : false,
                "url" : "http://flickrtest1.userland.com/"
            ]
        ]
    ]
]

extension Blog {
    static func parse(json: JSON?) -> FailableOf<Blog> {
        if let json = json {
            if let id = json["id"].number.value {
                if let name = json["name"].string.value {
                    if let needspassword = json["needspassword"].bool.value {
                        if let url = json["url"].string.value {
                            let blog = Blog(
                                id: Int(id),
                                name: name,
                                needsPassword: needspassword,
                                url: NSURL(string:url))
                            return FailableOf(blog)
                        }
                    }
                }
            }
        }

        return FailableOf(Error(
            code: 101,
            domain: "com.kiadsoftware.json",
            userInfo: nil))
    }

    static func fparse(json: JSValue?) -> FailableOf<Blog> {
        return make ‚áí (number(json, "id") ‚áí Blog.toInt)
                    ‚áí string(json, "name")
                    ‚áí bool(json, "needspassword")
                    ‚áí (string(json, "url") ‚áí Blog.toURL)
    }
}

extension Blog {
    static func make(id: FailableOf<Int>)
                    (name: FailableOf<String>)
                    (needsPassword: FailableOf<Bool>)
                    (url: FailableOf<NSURL>) -> FailableOf<Blog>
    {
        if let error = id.error { return FailableOf(error) }
        if let error = name.error { return FailableOf(error) }
        if let error = needsPassword.error { return FailableOf(error) }
        if let error = url.error { return FailableOf(error) }

        return FailableOf(Blog(
            id: id.value!,
            name: name.value!,
            needsPassword: needsPassword.value!,
            url: url.value!))
    }

    static func toInt(number: FailableOf<Double>) -> FailableOf<Int> {
        if let error = number.error {
            return FailableOf(error)
        }

        return FailableOf(Int(number.value!))
    }
    static func toURL(string: FailableOf<String>) -> FailableOf<NSURL> {
        if let error = string.error {
            return FailableOf(error)
        }

        return FailableOf(NSURL(string: string.value!))
    }
}

printBlog(Blog.parse(json["blogs"]["blog"][0]))
println("--------------")
printBlog(Blog.fparse(json["blogs"]["blog"][0]))
Functional JSON