What’s new in Vapor 3?

Learn the main changes in this server-side Swift framework

Paul Hudson    April 2nd 2018    @twostraws

The development of server-side Swift continues to accelerate thanks to the imminent release of Vapor 3: an almost-complete rewrite of the Vapor framework designed to take full advantage of Swift’s power features such as Codable, keypaths, conditional conformances, and more.

I’ve been using Vapor 3 since the very first alpha because I was writing a book on it, and it’s been a heck of a ride. The early alphas were already huge improvements over Vapor 2, but as work progressed more and more code became simpler to read and write. The final book (available here) ended up being about 30 pages shorter than earlier drafts thanks to these changes – it was a fantastic experience, and certainly taught me a lot.

If you used Vapor 2 previously, you will pretty much need to forget everything you learned and start from scratch. But that’s OK: you’ll find vast swathes of your code can now be deleted, because Vapor leans heavily on Swift to provide advanced functionality almost for free.

If you have not used Vapor before, this article isn’t really for you. I’ve gone into specific technical detail here about things that will affect Vapor 2 developers who want to upgrade, and honestly it will probably just put you off if you haven’t used Vapor before! Instead, start with my free online tutorial: Get started with Vapor 3 for free.

With that in mind, let’s take a look at the core differences in Vapor 3, diving in at the deep end with what is almost certainly the most complex: futures.

Sponsor Hacking with Swift and reach the world's largest Swift community!

The future is futures

Vapor 3 uses a non-blocking architecture to allow greater performance with fewer threads. On the one hand this will allow folks who have extremely limiting environments to work at high speed, but on the other hand it does cause headaches for developers.

A non-blocking architecture means that any operation that takes more than a tiny amount of time to run – for example a network request, a database request, or even rendering templates – is now run asynchronously. That is, your network request code returns immediately even though the work itself has not completed.

This approach allows your code to continue on to do other work without having to wait for the slow code to complete. You could in theory fire off multiple pieces of asynchronous work at the same time, all without pausing your main code.

While that all sounds nice, it comes with a problem: a lot of the time we need to check the results of these asynchronous operations, and manipulate those results somehow. We might even need to do more asynchronous work after manipulating those results, and check those results somehow.

If this sounds confusing, it’s because it is confusing: we’re dealing with futures here, which are containers for work that has not yet completed. Futures are similar to optionals in that they contain values, and can be passed around like regular types and have their own properties and methods. Also like optionals, futures have map() and flatMap() methods that let you manipulate their contents – in this case, they let you add work to be run when the future completes, i.e. when its asynchronous work finishes and you have a real result to work with.

Let’s look at an example in code. Vapor 3 has a new Client protocol that makes web requests on our behalf. In code you might use it like this:

let response = req.client().get("https://www.hackingwithswift.com")

That will fetch the contents of the Hacking with Swift homepage, but as it’s all asynchronous you won’t get back a string, a Data object, or any other such primitive type. Instead, you’ll get back an EventLoopFuture<Response> – a container that will contain a Response at some point, but may or may not actually contain one right this second.

Now, what we don’t want to do is something like this pseudocode:

while !response.isCompleted {
    sleep(1)
}

That kind of thing would effectively toss away all the performance advantage of having a non-blocking architecture in the first place.

Instead, we need to use either map() or flatMap() to add some work to happen when the future completes. Which of the two methods you use depends on what your work will return: if you want to return a future you should flatMap(), otherwise you should use map().

The difference is identical to using map() and flatMap() with optionals. If you use map() on an optional and your closure returns an optional, you’ll end up with an optional optional – the kind of thing no one wants to see. Whereas if you use flatMap() on an optional and your closure returns an optional, flatMap() collapses the optional optional into a single optional – either the whole thing exists or nothing exists.

This is identical to working with futures. If you use map() on a future and your closure returns a future, you’ll end up with a future future – again, the kind of “burn it with fire” type that shows something has probably gone wrong with your architecture. On the other hand, if you use flatMap() on a future and your closure returns a future, flatMap() collapses the future future into a single future.

Again, don’t worry if you find this confusing – it is hard, at least at first. The problem here is that unlike JavaScript ES2017 and C# there is no support for true async/await functionality in the Swift language. Instead, Swift implements its own futures on top of SwiftNIO, using map(), flatMap(), and other methods.

Using map() and flatMap() allows us to chain together asynchronous work, however it can easily result in rather complex code. For example:

1. You use the Client protocol above to fetch some data from the web.
2. When its information comes back as JSON you decode it using Codable.
3. You then use that data to make a database query.
4. Render the whole thing using a templating language such as Leaf.

That code makes a web request, decodes content, runs a database query, and generates pages from templates – all operations that return a future. As a result, you might find you have to use flatMap() multiple times.

However, Vapor provides some helper functionality that can sometimes help reduce the complexity, and it’s a good idea to lean on them as much as you can if you want to keep your code clean.

1. You can return futures directly from routes. Vapor will automatically wait for the future to complete before sending back its contents. So, when you’re rendering a Leaf template to generate a View object, just send back the EventLoopFuture<View> directly and let Vapor figure it out.
2. When decoding data using Codable you can use special synchronous methods. Decoding is asynchronous by default because some browsers send large requests in chunks rather than as one lump. If you know that isn’t the case – if you’re dealing with content that isn’t big or is being supplied by your own server – then you can decode synchronously without a performance hit.
3. If you’re accepting some data from the user from a form or similar, you can have Vapor decode it for you using Codable before your route code is even run using POST helper methods. This means by the time your own code is run you have a real object to work with, so that’s one less flat map.
4. Rather than write one flat map inside another inside another, you can write them as chains: a.flatMap { }.flatMap { }.
5. If you need two asynchronous operations to complete before you can run some work, Vapor provides a global variadic flatMap() function that can wait for up to five futures to complete before running further work.
6. If you’re writing tests for your Vapor code, you can use a special wait() method to wait for a future to complete. This will crash if you call it inside a Vapor route – it’s really not meant for production code.

Each of these help you avoid pyramids of maps and flat maps, and so are highly recommended.

Content is Codable

Codable was introduced in Swift 4 and then enhanced further in Swift 4.1, but that doesn’t mean it’s always used effectively – it’s common to see Swift 3 code that uses Codable here and there, when really it’s able to do a heck of a lot.

Vapor 3 is a project that has taken Codable to its heart – literally it sits at the heart of how Vapor works, to the point where it’s almost invisible.

Working with databases, sending JSON back from routes, passing data into templates, reading user form data, and more – all are powered by the Codable protocol. Codable is so central to Vapor 3 that you don’t even know it’s working half the time because its use is implied when you use other Vapor 3 protocols.

To demonstrate this, here’s a simple struct:

struct Message {  
    var id: Int?
    var title: String
    var body: String
}

If we want to create one of those and send it back from a Vapor route, we just need to conform to one protocol: Content. This automatically brings Codable with it, and you may use Message instances in routes however you need. For example:

router.get("test") { request in
    return Message(id: 1, title: "Hello, world", body: "Vapor 3 rocks!")
}

That will automatically send back our Message instance encoded as JSON with no further work from us.

This Content protocol can also be used when sending content out of Vapor. Earlier we looked at the Client protocol for fetching data over the web – it has a post() method too, for sending data. This uses the same Content protocol, meaning that your data will automatically be serialized to JSON before being sent over the wire.

When you put this together with automatic decoding of content, the Codable protocol effectively disappears. On one side you can send data like this:

req.client().post(someURL) { postRequest in
    try postRequest.content.encode(message)
}

And on the other side you can automaticaly decode it as part of a route like this:

router.post(Message.self, at: "some", "route", "here") { }

The end result is that you rarely handle JSON or Codable directly – you leave it up to Vapor. This is a huge change from the Node type of previous releases.

But wait, there’s more…

Fluent is back and better than ever

Of all the things Vapor was known for, I think Fluent was probably the best-known – it made database access easy and efficient, and was often almost transparent.

Well, in Vapor 3 Fluent got taken to the next level. Using a slick combination of associated types, generics, and protocol extensions, Fluent is now almost invisible. Most of the time it takes only three or four lines of code to configure, and from then on you can pretty much forget your database exists – Fluent abstracts it all away neatly.

Even better, Fluent is built upon Codable. This means your data model objects are serialized and deserialized seamlessly to content in your database, and tie perfectly into the Content protocol covered above.

To demonstrate this, here’s our original Message struct:

struct Message {
    var id: Int?
    var title: String
    var body: String
}

If we want to send back instances of Message as JSON, we need to make it conform to Content like this:

struct Message: Content {
    var id: Int?
    var title: String
    var body: String
}

And if we want to read and write instances of those structs using MySQL? Just add the MySQLModel protocol on top, like this:

struct Message: Content, MySQLModel {
    var id: Int?
    var title: String
    var body: String
}

So now if we want to write a route to query our database and return all messages we have stored, the entire route is this:

router.get("messages") { request in
    return Message.query(on: request).all()
}

Querying the database like that will return an EventLoopFuture<[Message]> – a future that will contain an array of messages at some point. Because Message conforms to Content, we can hand it directly over to Vapor: it will complete the database request, fetch the results, convert them all to JSON, and send them back to the caller on our behalf.

That’s easy enough, but how do you manage creating the database in the first place? Well, thanks to the magic of protocol extensions – and lots of hard work from the Vapor team – you just need to add a third protocol to your types:

struct Message: Content, MySQLModel, Migration {
    var id: Int?
    var title: String
    var body: String
}

When you then configure your database connection, you can instruct Vapor to automatically create a messages table based on the properties of the Message struct. If you need to override specific things, such as using TEXT rather than VARCHAR for a field, you can create a custom migration.

You’ve seen how loading data returns a future, and the same is also true of saving data. Our Message struct has an optional id property, which means it can be created with a nil identifier. Fluent automatically detects this when we save the new message, fills in the ID number for us, and returns a new Message instance with the correct ID number.

This is all done using futures: saving a Message will return an EventLoopFuture<Message>, and when that completes you’ll have your new Message with the correct ID number. If you need to send that back from your route, you can return the future directly as usual – your caller will get back the message in JSON form, complete with its new ID number.

Part of the reason Fluent is so simple now is its extensive use of Swift 4 keypaths. I’ve talked about these previously (see How Swift keypaths let us write more natural code), but in practice a lot of folks don’t understand how they work or really what they are for.

Well, Vapor 3 is pretty much the holotype of smart keypath usage. When we said that our Message model conformed to the MySQLModel protocol, Fluent automatically looks for a property called id that stores an optional integer. If you wanted to use a UUID for your identifier instead, you would just use MySQLUUIDModel. Both of these models specify MySQL as the database and one of two types of identifier, but if you wanted to use a different property entirely you can do that:

struct Message: Content, Model, Migration {
    typealias Database = MySQLDatabase
    static let idKey = \Message.postIdentifier

    var postIdentifier: String?
    var title: String
    var body: String
}

If you’re using SQLite there’s even an SQLiteStringModel protocol for string primary keys – perhaps that will spread to MySQL with a future update.

Fluent’s use of keypaths also extends to sorting and filtering so that you specify properties in your structs when you want to adjust your queries. For example, this will pull out all messages in our database, sorted by their ID number descending:

return try Message.query(on: req).sort(\Message.id, .descending).all()

Before moving on, it would be remiss of me if I didn’t add that Vapor’s MySQL support is now written in pure Swift – it’s blazing fast, and has no external dependencies!

At your service

Last but certainly not least, I want to look at how Vapor 3 has revamped the way we work with shared resources. Vapor is designed to launch multiple worker threads at launch, with each one handling many requests asynchronously.

That sounds like a recipe for race conditions and other thread problems, but Vapor 3’s careful architecture avoids the problems entirely: rather than trying to create objects that you pass between routes, you instead register services when your app starts and ask Vapor to provide instances of them as you need them.

For example, a common service you’ll want to register is LeafProvider – this handles rendering of Leaf templates. If you’re making a front-end website you’ll need to render views in almost every route, but rather than worry about creating and re-using Leaf renderers you just start by registering the service when your app is being configured:

try services.register(LeafProvider())
config.prefer(LeafRenderer.self, for: ViewRenderer.self)

You can then ask Vapor to find or create a template renderer for you inside your route, and pass it a Leaf template to render:

let view = try req.view().render("error")

It won’t surprise you to learn that returns a future because views take time to load and render!

The same approach is taken with database requests: you configure your connection once, then more or less forget about it – the rest of your code doesn’t reference MySQL or SQLite directly. For example, you can connect to a local MySQL database like this:

try services.register(FluentMySQLProvider())
let databaseConfig = MySQLDatabaseConfig(hostname: "127.0.0.1", port: 3306, username: "swift", password: "swift", database: "swift")
services.register(databaseConfig)

Once that’s done, any database requests in your app will automatically be sent to that database. For example:

return Message.query(on: request).all()

Even serving up static files from a public folder is done using services, this time grouped inside a general MiddlewareConfig struct:

var middleware = MiddlewareConfig.default()
middleware.use(FileMiddleware.self)
services.register(middleware)

This approach guarantees thread safety, and also maximizes your performance because Vapor becomes responsible for creating and managing instances of these workers on your behalf.

All set? Dive in!

Now is the perfect time to dive in to Vapor 3 – the code is stable at last, and the long process of tagging for final release is under way. Any projects you start with the final release candidate will automatically upgrade to the 3.0 release as each component is tagged.

I wrote a whole book about using Vapor 3 to build websites and APIs, teaching routing, templates, form handling, sessions, database access, authentication, and much more. It’s written from scratch for Vapor 3 and comes with free Swift updates for life. Click here to buy it now!

Sponsor Hacking with Swift and reach the world's largest Swift community!