Introduction

This is part of an ongoing series about the Rescript programming language. I’ve been taking notes as I build a toy “Recipe Book” app to explore the Rescript ecosystem. The toy app has evolved into a fairly complete offline app at this point, and it’s now time to add sync to turn it into a progressive web app.

In earlier articles, I created an offline-enabled “recipe book” entirely in Rescript. More recently, I created a graphql express server and set it up to have the same data model as the frontend app.

This article will (finally) marry the two with graphql endpoints to allow RxDB to sync the frontend with the backend.

Patreon

This series takes a lot of time to write and maintain. The only thing that has kept me invested in it is the support I’ve received on my Patreon account. I thank you for your support. I am not expecting this to pay for my time investment, but it means so much to know people are placing value on my content.

As a bonus for patrons, articles will be published there at least two weeks before they make it to my public blog.

While I originally set up the Patreon for individuals to support me, I’d like to point out that if your company is invested in the Rescript ecosystem, my work is probably helping you recruit and onboard new hires. Supporting my efforts makes the ecosystem stronger.

Other ways to show support include sharing the articles on social media, commenting, or a quick thank you on the Rescript forum.

With over a dozen articles and counting, I’ve created a table of contents listing all the articles in this series in reading order.

Let’s begin

We’ll be extending the rescript-express-recipes repo I’ve used in a couple previous articles. It has a basic graphql endpoint in place, but with little more than a hello world schema.

If you haven’t been following along with these articles, you can git checkout the refactor branch to have the same starting point.

If you want to keep up with the commits for this article, they are in the offline-sync branch. Each commit roughly maps to a section in this article.

Defining the GraphQL Schema

The RxDB docs have a page describing what the graphql endpoints needs to look like to sync a collection. The endpoints are per collection, so we need two endpoints for each collection: a query and a mutation.

The query needs to be sortable on the updatedAt field we defined previously, and documents must never be deleted, but instead must have a deleted flag set to true.

The server endpoint for each collection needs to return an array of documents in that collection that are more recent than a given element. So the query for a recipe would contain the id and updatedAt of the most recent document and return any newer documents in order. We can also supply a limit to behave as pagination.

So let’s start by replacing the schema definition in our server’s Schema.res with the following:

type Query {
  recipeRxDbFeed(id: String!, minUpdatedAt: Float!, limit: Int!): [Recipe!]!

  taggedRecipesRxDbFeed(
    tag: String!
    minUpdatedAt: Int!
    limit: Int!
  ): [TaggedRecipes!]!
}

Of course, that depends on the existence of two more graphql types, Recipe and TaggedRecipes. We can port those over from the db model in Store.res. Update the schema to contain these two types:

type Recipe {
  id: String!
  title: String!
  ingredients: String!
  instructions: String!
  tags: [String]!
  updatedAt: Float!
  deleted: Boolean!
}

type TaggedRecipes {
  tag: String!
  recipes: [String]!
  updatedAt: Float!
  deleted: Boolean!
}

The schema also needs mutations. RxDB expects a straightforward mutation that just accepts a single instance of the document being modified:

type Mutation {
  setRecipe(recipe: RecipeInput!): Recipe!
  setTaggedRecipes(taggedRecipes: TaggedRecipesInput!): TaggedRecipes!
}

But, once again, we need to define the RecipeInput and TaggedRecipesInput types. Our inputs won’t allow setting the updatedAt field, but will otherwise be identical to the query types:

input RecipeInput {
  id: String!
  title: String!
  ingredients: String!
  instructions: String!
  tags: [String]!
  deleted: Boolean!
}

input TaggedRecipesInput {
  tag: String!
  recipes: [String]!
  deleted: Boolean!
}

Update the Rescript query schema types

We changed the schema passed into buildSchema, but we haven’t told Rescript about the new models. Let’s change our schema types to use using the “real” models we have defined in Store.res.

Replace the greetByNameArgs and rootValue at the top of Schema.res with a new rootValue type:

type rootValue = {
  recipeRxDbFeed: recipesRxDbFeedInput => array<Store.recipe>,
  taggedRecipesRxDbFeed: taggedRecipesRxDbFeedInput => array<Store.taggedRecipes>,
  setRecipe: recipeInput => Store.recipe,
  setTaggedRecipes: taggedRecipesInput => Store.taggedRecipes,
}

The feed input types are records with the three argument values:

type recipesRxDbFeedInput = {id: Store.id, minUpdatedAt: float, limit: int}
type taggedRecipesRxDbFeedInput = {tag: Store.tag, minUpdatedAt: float, limit: int}

The two mutation Input types are a bit trickier because the type of the argument value is essentially a “recipe without an updatedAt field”. But rescript has no way to model this. I toyed with the idea of just using the Store.recipe and Store.taggedRecipes type and intentionally choose to ignore the updatedAt field. However, I decided that would eventually bite me in the ass.

So I chose a more verbose, but type-safe solution instead. I have a new recipeInputFields type for the type of the thing being passed in, and then a recipeInput type to contain that type. Do the same for taggedRecipesInputFields and it looks like this:

type recipeInputFields = {
  id: Store.id,
  title: Store.title,
  ingredients: Store.ingredients,
  instructions: Store.instructions,
  tags: array<Store.tag>,
  deleted: bool,
}

type taggedRecipesInputFields = {
  tag: Store.tag,
  recipes: array<Store.id>,
  deleted: bool,
}

type recipeInput = {recipe: recipeInputFields}
type taggedRecipesInput = {taggedRecipes: taggedRecipesInputFields}

type rootValue = {
  recipeRxDbFeed: recipesRxDbFeedInput => array<Store.recipe>,
  taggedRecipesRxDbFeed: taggedRecipesRxDbFeedInput => array<Store.taggedRecipes>,
  setRecipe: recipeInput => Store.recipe,
  setTaggedRecipes: taggedRecipesInput => Store.taggedRecipes,
}

For what it’s worth, if I had been writing Typescript, I would have used the Omit<Recipes, 'id'> type here, but Rescript doesn’t have anything like that (with good reason).

Bolierplate resolvers

Our Resolvers.res file is no longer compiling because we don’t have greetByNameArgs anymore, and the rootValue doesn’t satisfy the new Schema.rootValue type. Let’s add some dummy endpoints to make it all compile again. Replace the entire Resolvers.res with the following:

let recipeRxDbFeed = ({id, minUpdatedAt, limit}: Schema.recipesRxDbFeedInput) => {
  []
}

let taggedRecipesRxDbFeed = ({tag, minUpdatedAt, limit}: Schema.taggedRecipesRxDbFeedInput) => {
  []
}

let setRecipe = ({recipe}: Schema.recipeInput): Store.recipe => {
  id: recipe.id,
  title: recipe.title,
  ingredients: recipe.ingredients,
  instructions: recipe.instructions,
  tags: recipe.tags,
  deleted: recipe.deleted,
  updatedAt: Js.Date.now(),
}

let setTaggedRecipes = ({taggedRecipes}: Schema.taggedRecipesInput): Store.taggedRecipes => {
  tag: taggedRecipes.tag,
  recipes: taggedRecipes.recipes,
  deleted: taggedRecipes.deleted,
  updatedAt: Js.Date.now(),
}

let rootValue: Schema.rootValue = {
  recipeRxDbFeed: recipeRxDbFeed,
  taggedRecipesRxDbFeed: taggedRecipesRxDbFeed,
  setRecipe: setRecipe,
  setTaggedRecipes: setTaggedRecipes,
}

These don’t actually do anything, but it compiles and is enough to run a couple of quick queries in your local graphiql to make sure that it’s not crashing or anything. I used these queries:

query Query {
  recipeRxDbFeed(id: "John", minUpdatedAt: 1.0, limit: 5) {
    id
  }
  taggedRecipesRxDbFeed(tag: "John", minUpdatedAt: 1.0, limit: 5) {
    tag
  }
}

mutation Mutation {
  setRecipe(recipe: {id: "0xabcdef", title: "fried eggs", ingredients: "eggs", instructions: "fry", tags: [], deleted: false}) {
    id
    title
    updatedAt
  }

  setTaggedRecipes(taggedRecipes:{tag: "shells", recipes:["0xabcdef"], deleted: false}) {
    tag,
    recipes,
    updatedAt
  }
}

The Recipe feed

Let’s first implement the recipeRxDbFeed function. We can use a structure similar to the one chose for the REST endpoints when we first set up the express server in an earlier article. However, we’ll have to do some additional filtering and sorting to return the correctly requested subset. This is going to require a fair bit of data structure manipulation, so the first thing I did was put open Belt at the top of the file. This gives us slightly safer array’s and immutable maps in our namespace, among other things.

We’ll be operating on the state.Recipes map where state was returned by Store.Reducer.getState(). The first thing we need is the list of values in that map as an array, which is easily extracted by piping it into Map.String.valuesToArray:

  Store.Reducer.getState().recipes
  ->Map.String.valuesToArray

We are only interested in recipes that have been updated more recently than the timestamp supplied as an argument. So let’s add a filtering pass to the pipeline. The equivalent of filter from Javascript is called keep in Rescript’s Belt.Array module. It accepts a function that returns a boolean for each element in the array:

  Store.Reducer.getState().recipes
  ->Map.String.valuesToArray
  ->Array.keep(r => r.updatedAt > minUpdatedAt)

The resulting array needs to be sorted by updatedAt, falling back to sorting by id if the updatedAt is the same to ensure consistent ordering of two distinct recipes with the same timestamp.

Belt provides a SortArray module for performing sort operations on an array. This module contains a few functions to check if an array is sorted or to perform binary search. We are currently interested specifically in the stableSortBy function. In addition to the array itself, passed in using pipe, it requires a callback function in the form of a standard comparator: Given two values, return an integer indicating what order they should come in. It’s a large conditional, but it looks quite elegant:

    ->SortArray.stableSortBy((r1, r2) => {
      if r1.updatedAt > r2.updatedAt {
        1
      } else if r1.updatedAt < r2.updatedAt {
        -1
      } else if r1.id > r2.id {
        1
      } else if r1.id < r2.id {
        -1
      } else {
        0
      }
    })

If this looks surprising, remember that an if statement is an expression, which means it has a return value. The return value is the last line of whichever arm gets executed.

Finally, we need to implement the ‘limit’ argument, which just means slicing off the front of the sorted array.

  ->Array.slice(~offset=0, ~len=limit)

However, the Rescript compiler has issued a warning that highlights a problem I would have totally missed otherwise. We aren’t using the id variable at all. Looking back at the RxDB docs to see why they needed it, I discovered that the filtering condition needs to check the id as well as the updatedAt.

In retrospect, this is obvious, since the filter should be using the same criteria as the sort. Thank you for finding that bug, Rescript compiler! Here’s the entire recipeRxDbFeed function, now with the correct filtering in place:

let recipeRxDbFeed = ({id, minUpdatedAt, limit}: Schema.recipesRxDbFeedInput) => {
  Store.Reducer.getState().recipes
  ->Map.String.valuesToArray
  ->Array.keep(r => {
    if r.updatedAt == minUpdatedAt {
      r.id > id
    } else {
      r.updatedAt > minUpdatedAt
    }
  })
  ->SortArray.stableSortBy((r1, r2) => {
    if r1.updatedAt > r2.updatedAt {
      1
    } else if r1.updatedAt < r2.updatedAt {
      -1
    } else if r1.id > r2.id {
      1
    } else if r1.id < r2.id {
      -1
    } else {
      0
    }
  })
  ->Array.slice(~offset=0, ~len=limit)
}

You can test this function by first adding several recipes using the REST endpoints we implemented in a previous article, then issuing the appropriate queries in graphiql. Because I was testing this quite a few times, I created the following Python script to hit the addRecipe endpoint several times and print out the ids:

import requests
import json

for i in range(8):
    print(
        requests.post(
            "http://localhost:3000/addRecipe",
            data=json.dumps(
                {"title": f"{i}", "instructions": "blah", "ingredients": "blat"}
            ),
            headers={"content-type": "application/json"},
        ).json()
    )

If you’re wondering why I didn’t use Rescript; I knew how to do it in Python without looking anything up. Pragmatic is better than pedantic.

The taggedRecipes feed resolver

We should be able to pretty much copy-paste the recipes resolver for taggedRecipesRxDbFeed. Indeed, all I had to do was substitute tag for id and the endpoint was complete:

let taggedRecipesRxDbFeed = ({tag, minUpdatedAt, limit}: Schema.taggedRecipesRxDbFeedInput) => {
  Store.Reducer.getState().tags
  ->Map.String.valuesToArray
  ->Array.keep(r => {
    if r.updatedAt == minUpdatedAt {
      r.tag > tag
    } else {
      r.updatedAt > minUpdatedAt
    }
  })
  ->SortArray.stableSortBy((r1, r2) => {
    if r1.updatedAt > r2.updatedAt {
      1
    } else if r1.updatedAt < r2.updatedAt {
      -1
    } else if r1.tag > r2.tag {
      1
    } else if r1.tag < r2.tag {
      -1
    } else {
      0
    }
  })
  ->Array.slice(~offset=0, ~len=limit)
}

Conclusion

The article is getting long and I’m a little short on time, so I’m going to defer the mutations for next week. They will require new actions and reducers, so I want to make sure I take the time to get it right.

Other than the SortArray module, we didn’t encounter much new Rescript knowledge in this article. It’s good to feel like I’m reaching a steady state with the language, and I hope you are feeling it, too! There’s still plenty to cover, of course, but I’m easily able to do daily coding in the language now, and it continues to be the most pleasant coding experience I’ve encountered.

I’ll be honest with you: I’m not actually sure if this is going to work! I haven’t actually finished this and hooked it up to RxDB yet. I’m learning as much as you are here, though I hope my experience distilling information is making it a little easier for you than it is for me!

I’m getting busy now, but I look forward to writing these articles and hope to continue churning them out for my Patrons. Please Join them if you want to support my efforts or preview articles before they are published. I have a healthy backlog of articles available to paying subscribers that will continue to roll out on a one to two week schedule.