Bernat Rafales in Coding 5 minutes

Svelte and the importance of tooling

Today we are going to talk about Svelte, a gem we built here at notonthehighstreet.com to help us being more efficient with the integration of all the new shiny services we are developing in here.

We are also announcing the release of Svelte as an open source project under the MIT License as part of our continued efforts to support and contribute to the FOSS cause.

Standards

We build microservices. And we need to communicate with them. We do this through HTTP APIs. And in order to ensure we are all in the same page, we decided to set some standards up. One of these standards is to document our APIs using Swagger. Swagger is a powerful and standard API documentation system. Whatever language you are using to implement your API, documenting it with Swagger will let other developers know everything about it. Their operations, the formats they support, the request and response parameters and method signatures, the API location itself…

It also has support for the most used languages out there, and even provides some tools for generating static consumers for a given API.

On top of that, it’s widely used and the community around it is huge.

Ruby

Turns out we use Ruby a lot in here. In fact, what we are actually splitting into services is a Monolithic Rails application.

When we first started an integration with a service that used Swagger, we tried to figure out the best way to approach the problem. We could have built an ad-hoc client for that specific API. But then another service integration would follow, and another ad-hoc client would have to be built for that. That approach quickly leads to a maintenance pain. Another option could’ve been to use one of the Swagger built in code generators. A much better approach, without doubt, but we still felt that’d lead to an unnecessary degree of manual intervention when adding new services or updating the current ones to API changes. So we thought: “wouldn’t it be great to be able to feed a Swagger JSON spec to a tool and have it automagically build you a dynamically generated client for it”?

And then we just went and built that tool. Because it would solve a problem for us, because maybe other people have the same problem as us, so it was a great opportunity to open source it and let anyone benefit from it, and last, but not least, because it’d be lots of fun! We called it Svelte.

The end goal of Svelte is the same as any other code generation tool. We just decided to take a slightly different approach on how to achieve such goal. We believe that this was a perfect example of metaprogramming done right.

Svelte

Let’s have a closer look at what the gem does. We will use the emblematic PetStore API for the examples throughout. You can find the raw JSON at http://petstore.swagger.io/v2/swagger.json, and you can have a look at the docs generated by Swagger at http://petstore.swagger.io/.

Now make sure you have the Svelte gem installed:

$ gem install 'svelte'

Let’s fire up an IRB session and play a bit:

Svelte.create(
  url: 'http://petstore.swagger.io/v2/swagger.json',
  module_name: 'PetStore') # Svelte::Service::PetStore
response = Svelte::Service::PetStore::Pet.get_pet_by_id(petId: 1) # <Faraday::Response:0x007f91a1bb5a00 ...>
response.status # 200
response.body # {"id"=>1, "name"=>"a", "photoUrls"=>[], "tags"=>[], "status"=>"available"}

So what did exactly happen here? We asked Svelte to create a client for the API located at a specific URL. We told it to build that client with a particular constant name PetStore. And that is exactly what we got back.

Then we made a request to an operation on that API, passing in the parameter petId.

The operation we requested is in the /pet/{petId} path of the Swagger API, more precisely the get request, which has the getPetById as its operation identifier.

Note how Svelte automatically created a module hierarchy to represent the different paths available. We sent the message get_pet_by_id to the module Svelte::Service::PetStore::Pet. If we had wanted to query about an order in the store, we would have sent this message instead:

response = Svelte::Service::PetStore::Store::Order.get_order_by_id(
  orderId: 1)
response.status # 404
response.body # {"code"=>1, "type"=>"error", "message"=>"Order not found"}

The path /pet/{petId} creates a Pet module on top of PetStore, whereas /store/order/{orderId} will create Store::Order module instead.

Method names will also be generated based on the Swagger spec values. Svelte takes the operationId attribute and generates a Ruby friendly version of it. getPetById generates a get_pet_by_id method, deleteOrder creates a delete_order method, and so on.

Now let’s try to make the whole thing crash:

response = Svelte::Service::PetStore::Pet.get_pet_by_id
# Svelte::ParameterError: Required parameter `petId` missing

Svelte is smart enough to know that there is a required parameter that you are not sending as part of the request, and will tell you accordingly. This feature works only for URL parameters for now, but full parameter validation will be added in the near future.

Models

But that’s not the end of it! For easy requests like the ones shown so far, building the JSON payload for is quite easy. But things get more complex quickly. Let’s look at the PetStore API again. We see the operation placeOrder receives a body parameter which schema is a reference to #/definitions/Order. When we go and look at what this definition looks like we see this:

{
  "type": "object",
  "properties": {
    "id": {
      "type": "integer",
      "format": "int64"
    },
    "petId": {
      "type": "integer",
      "format": "int64"
    },
    "quantity": {
      "type": "integer",
      "format": "int32"
    },
    "shipDate": {
      "type": "string",
      "format": "date-time"
    },
    "status": {
      "type": "string",
      "description": "Order Status",
      "enum": [
        "placed",
        "approved",
        "delivered"
      ]
    },
    "complete": {
      "type": "boolean",
      "default": false
    }
  },
  "xml": {
    "name": "Order"
  }
}

We could easily build the request manually in Ruby, but Svelte can also help us with this. Let’s have a look:

require 'svelte'
require 'net/http'
require 'json'

class PetStoreModels
  extend Svelte::ModelFactory

  define_models JSON.parse(Net::HTTP.get(URI('http://petstore.swagger.io/v2/swagger.json')))
end

order = PetStoreModels::Order.new
order.petId = 1
order.quantity = 3
order.shipDate = '2015-08-21'
order.status = 'placed'
order.complete = false

# Is this a valid model?
order.valid? # true

order.petId = 'Not an integer'
order.status = 'funny status'

# And now?
order.valid? # false
order.validate # {"petId"=>"Invalid parameter: Expected valid integer, but was \"Not an integer\"", "status"=>"Invalid parameter: Expected one of [\"placed\", \"approved\", \"delivered\"], but was \"funny status\""}

# Back to normality
order.petId = 1
order.status = 'placed'

order.as_json # {:petId=>1, :quantity=>3, :shipDate=>"2015-08-21", :status=>"placed", :complete=>false}

As we can see, it’s really easy to generate the payloads for our requests. And more importantly, the models generated by Svelte provide validation out of the box, so you can always check your payloads before you send them to the API.

At the moment the define_models class method only supports a Hash parameter (hence why we make an HTTP request and parse its response), but we will add support to do this automatically based on an URL if needed.

Conclusion

The morale of the story is that every time we need to integrate any Ruby service or application to a Swagger API, we just need to provide Svelte with a JSON and we get an API client and a few helpers for free. Had we decided to take one of the other approaches mentioned earlier, doing this would now be more laborious and tedious, and it would very quickly become a chore that very few people would enjoy doing.

Does this mean that the first integration with a service took longer than usual because building Svelte was more complex than building a simple ad-hoc REST HTTP client? Yes. But the benefits it has provided in the long term are definitely much higher than the costs of that initial workload. Remember it’s better to build things solidly from the start, even if it means a slower kickstart. You will make your future self and colleagues much happier in the long term when having to maintain the codebase. A better codebase leads to happier developers. Happier developers are more likely to keep delivering high quality work, and high quality work means everyone’s happy.

So never underestimate the need for good tools. We use them all the time, sometimes we take them for granted. And sometimes you need to build them yourself. Be always looking for opportunities to build tools like this one, and equally important, be always looking at how you can build them in a way that everyone, including people from outside of your organisation, can benefit from them. The open source community has given so much to all of us, it’s only fair to contribute when we have the opportunity.