Automating Shape Verification in Elixir APIs

I run a React Native app with an Elixir backend. I picked Elixir for a few reasons:

There are some realtime features that play nicely with the BEAM's concurrency primitives
I like writing functional languages
Many people had told me they wished they could work in Elixir but the jobs are hard to find

One thing I generally try to set up right away in a project is strong typing across the API boundary. The shape I like is:

Backend generates an OpenAPI spec
Run a codegen step to create a frontend client for the API
Use the client and associated types throughout the frontend

Architecture diagram showing OpenAPI spec flowing to an autogenerated client and backend API controllers backed by business logic — The target setup: an OpenAPI spec drives a generated frontend client, which talks to typed API controllers backed by business logic.

With something like FastAPI in Python, this is really easy to do and more or less comes for free out of the box.

After reading some docs and poking around, I decided to define schemas in Elixir with OpenApiSpex.

A simple user object:

defmodule UserOut do
  @moduledoc false
  require OpenApiSpex
  alias OpenApiSpex.Schema

  OpenApiSpex.schema(%Schema{
    title: "UserOut",
    description: "information about the user",
    type: :object,
    properties: %{
      id: %Schema{type: :integer, description: "User ID"},
      username: %Schema{type: :string, description: "User username"}
    },
    required: [:id, :username]
  })
end

Then you can use these specs in a controller:

defmodule MyProject.UserProfileController do
  use MyProject, :controller
  use OpenApiSpex.ControllerSpecs

  alias Db.User
  alias MyProject.Schemas.UserSchemas.{UserOut, ErrorResponse}

  operation(:get_user_profile,
    summary: "Get user profile",
    description: "Get a user's profile",
    parameters: [
      user_id: [in: :path, description: "User ID", type: :integer, required: true],
    ],
    responses: [
      ok: {"User profile retrieved successfully", "application/json", UserOut},
    ]
  )
  ...

So with this we can generate a fully typed API client for the link between the backend API and the frontend client. That said, we still don't have any compile time checks, linting, or tests that confirm the objects we return from the business logic actually match the shape of what we promise to return from the API.

Diagram highlighting the gap: the typed frontend/API layer in green versus the unverified business logic in red — The gap: the frontend and API layer are typed (green), but nothing verifies that the business logic actually returns what the contract promises (red).

For example, I may say at the contract level that I'm going to return an object with an id field, yet not actually include it:

def build_user do
  %{public_id: 1, username: "hew"} # oops, "id" is missing
end

In a language with a decent typing story (Haskell or Python), you would likely try to encapsulate this in the types and expect a blunder to be caught by a type check. In a language like Ruby, you would probably write tests to make sure the fields you expect are actually being returned.

I considered trying to use Dialyzer (Elixir's type annotations), but opted to try to establish a pattern for this logic that was both testable and operationally easy to work with.

The pattern defines a constructor (Builder) for each schema we may return from the API:

defmodule MyProject.Builders.UserBuilder do

  @behaviour MyProject.Builders.VerifiedShapeBehavior

  def schema, do: MyProject.Schemas.UserOut
  def base_db_model, do: MyProject.User

  def build(user) do
    %{
      "id" => user.id,
      "username" => user.username,
    }
  end
end

Diagram showing the backend API's GetUser endpoint connected to business logic using UserOutBuilder.build(user) — The builder sits between the API endpoint and the database model, ensuring the returned shape is always constructed the same way.

This says that the builder expects a User (as defined by our database schema) and is expected to return a UserOut.

The behaviour just requires that every builder module defines schema, base_db_model, and build:

defmodule MyProject.Builders.VerifiedShapeBehavior do
  @callback build(map()) :: map()
  @callback schema() :: module()
  @callback base_db_model() :: module()
end

With this in place we can auto-generate tests for every builder. We know what object to pass the builder, and we can assert against the expected return type:

def assert_shape(module) do
  base_model = base_db_to_struct(module)
  data = module.build(base_model)
  schema = module.schema()

  case OpenApiSpex.cast_value(data, schema) do
    {:ok, _} ->
      :ok

    {:error, errors} ->
      flunk("""
      Schema validation failed for \#{inspect(module)}

      Data:
      \#{inspect(data, pretty: true)}

      Errors:
      \#{format_errors(errors)}
      """)
  end
end

This is, of course, not perfect. If you don't actually use the builders to construct the objects you return from the API, they'll do you no good. You can still easily write code that breaks the API contract and passes all the tests. But as a project with a small number of developers (1), it's pretty easy to just follow the convention. Also, localizing the logic around building your transfer objects makes a lot of code a bit cleaner and easier to read.

If you made it this far, hopefully that was interesting! And if I'm missing a better pattern, please reach out and let me know!