Introduction

This is part of an ongoing series about the Rescript programming language. I’ve been taking notes as I build a toy “Recipe Book” app to explore the Rescript ecosystem. The toy app has evolved into a fairly complete offline app at this point, and I’ve also written a graphql server in rescript and express. The most recent articles are hooking the two up to create a true Progressive Web App.

In the previous article, we hooked up queries to sync changes from the graphql server to RxDB in the frontend. In this one, we’ll hook up mutations to go in the other direction.

Patreon

This series takes a lot of time and dedication to write and maintain. The main thing that has kept me invested in this writing this series is support from my Patrons.

Other ways to show support include sharing the articles on social media, commenting on them here, or a quick thank you on the Rescript forum.

With over a dozen articles and counting, I’ve created a table of contents listing all the articles in this series in reading order.

Let’s begin

This article references two repositories that I’ve been working on in earlier articles. The server is in the rescript-express-recipes. You can git checkout the frontend-hookup branch if you want to start from the same place I am in this article. I make one additional change on that branch later in this article.

Most of our edits will happen in the rescript-offline repository. You can start from the syncing branch.

If you want to follow along with the changes in this article, I try to make a separate commit for each section. You can find them in the mutations branch.

The plan

We’ve already written bindings to the RxDB plugin for graphql replication. The next thing we’ll have to do is set up a querybuilder (similar to the one we used for pull replication) for pushing. This will build a mutation, but the idea is much the same.

The RxDB docs for graphql push replication tell us how to set up the query. We already have the mutations in place on the server, so all we have to do is pass that stuff forward.

Refactoring queryBuilder

In Javascript, the Query builder for pull and push is identical, in that both accept a RxDB document and return a query and some variables.

However, they differ slightly in our current Rescript implementation because the type of the variables that are passed with the query are different. Recall, we specified the queryParams as follows:

type queryParams = {
  query: string,
  variables: {"id": Model.id, "minUpdatedAt": float, "limit": int},
}

But our push builder will be passing in a different variable (probably the doc itself). Let’s explore our options for solving this:

  • We could give up and put everything in a {% raw tag. This is, of course silly, but I like to start with the silliest option I can come up with when I’m enumerating options to solve an architecture problem. It puts me in a more creative mindset and makes it more likely that I will come up with an innovative solution.
  • We could make variables entirely generic in the queryParams type. Then we would just return whatever variables we want from each queryBuilder and let Rescript blindly pass it on to Javascript. This would remove a certain amount of typechecking.
  • We could make variables generic in queryParams, but make queryBuilder have a second generic argument. This would solve the strong type safety issue in the previous option, but would make everything more verbose.
  • We could create an entirely new set of bindings for the push query builder. This would be extra verbose, but would allow us to be very explicit. I can imagine maintenance issues in the future. Generally, duplicating code is a code smell, although I’m sometimes willing to forgive it if it is providing build-time safety (e.g: unit tests, type checking).

I think the second option is the sanest. You may recall that I actually made a conscious decision to not model variables generically in the previous article and expressly asserted that I would always use that format.

Turns out I was wrong! But that’s ok; it will be an easy change to make, and this highlights another of my basic architecture principles: Do the simplest thing first, but make it easy to add complexity only when it is needed.

First, let’s add a new type for pullQueryVariables to the Sync.res file. We can extract the type from the existing queryParams as follows:

type pullQueryVariables = {"id": Model.id, "minUpdatedAt": float, "limit": int}

type queryParams = {
  query: string,
  variables: pullQueryVariables,
}

This, of course, doesn’t change anything, and your compiler should still be happy. However, our goal is to make queryParams generic, like this:

type queryParams<'variables> = {
  query: string,
  variables: 'variables,
}

Making this change will cause some compiler errors. I decided to make some on-the-fly changes to the design while I was fixing them.

First, the queryBuilder type needs a second generic parameter for the variables:

type queryBuilder<'document, 'variables> = Js.Nullable.t<'document> => queryParams<'variables>

Second, I think it makes sense to immediately add a new pullQueryBuilder type that specializes this new generic parameter:

type pullQueryBuilder<'document> = queryBuilder<'document, pullQueryVariables>

Then it’s a matter of replacing the existing queryBuilder references with pullQueryBuilder. There are three of them in these locations:

  • The pullOptions type
  • The recipeQueryBuilder definition
  • The taggedRecipesQueryBuilder definition

Your code should compile when this is all over. The cool thing is, it doesn’t make any changes to the compiled Javascript file. These changes were all strong typing guarantees that are eliminated with zero runtime cost by the Rescript compiler.

Modeling pushQueryVariables

The example GraphQL mutation in the RxDB docs looks like this:

  mutation CreateHuman($human: HumanInput) {
      setHuman(human: $human) {
          id,
          updatedAt
      }
  }

where HumanInput was described as:

input HumanInput {
    id: ID!,
    name: String!,
    lastName: String!,
    updatedAt: Int!,
    deleted: Boolean!
}

We’ll me pushing recipes and tags instead of humans (pushing people is considered rude) but the takeaway is that a push query has only one variable. More to the point, that variable looks suspiciously like the document being modelled. So we can actually create a new pushQueryVariables type as follows (I put earch of these near the equivalent pull types in Sync.res):

type pushQueryVariables<'document> = {"document": 'document}

and a new pushQueryBuilder that looks like this:

type pushQueryBuilder<'document> = queryBuilder<'document, pushQueryVariables<'document>>

Next, we can mimic the existing pullOptions and create a new pushOptions type:

type pushOptions<'document> = {queryBuilder: pushQueryBuilder<'document>}

Finally, we need to add these push options to the existing syncGraphQLOptions type:

type syncGraphQLOptions<'document> = {
  url: string,
  pull: pullOptions<'document>,
  push: pushOptions<'document>,
  deletedFlag: string,
  live: bool,
}

This will break the build for a while because the calls to syncGraphQL in App.res don’t include this new property. I’m not going to worry about that just yet because I need to digress.

Hold on! (non-nullable push queries)

Pull queries accept a Js.Nullable type (you may recall that tripped us up in the previous article). But Push queries always accept a non-null type. I worried this was going to be difficult to model, but it turned out to be as simple as moving the Nullable definition from queryBuilder to pullQueryBuilder. The three queryBuilder types now look like this:

type queryBuilder<'document, 'variables> = 'document => queryParams<'variables>

type pullQueryBuilder<'document> = queryBuilder<Js.Nullable.t<'document>, pullQueryVariables>
type pushQueryBuilder<'document> = queryBuilder<'document, pushQueryVariables<'document>>

The Recipes push query builder

Now that we have all the types in place, we can craft the query to push changes to the recipes collection. As a refresher, the related schema in our server looks like this:

  input RecipeInput {
    id: String!,
    title: String!,
    ingredients: String!,
    instructions: String!,
    tags: [String]!,
    deleted: Boolean!,

  }

  input TaggedRecipesInput {
    tag: String!,
    recipes: [String]!,
    deleted: Boolean!,
  }

  type Mutation {
    setRecipe(recipe: RecipeInput!): Recipe!
    setTaggedRecipes(taggedRecipes: TaggedRecipesInput!): TaggedRecipes!
  }

The associated push mutation looks very similar to RxDB’s example setHuman mutation, except we’ll be calling setRecipe instead. Add a new recipesMutationBuilder like so:

let recipeMutationBuilder: pushQueryBuilder<Model.recipe> = recipe => {
  let query = `
    mutation CreateRecipe($document: RecipeInput!) {
      setRecipe(recipe: $document) {
          id,
          updatedAt
      }
    }
  `
  let variables = {"document": recipe}
  {query: query, variables: variables}
}

The TaggedRecipes push query builder

That was simple enough that I’m just going to copy-paste it and make the taggedRecipesMutationBuilder as well:

let taggedRecipesMutationBuilder: pushQueryBuilder<Model.taggedRecipes> = taggedRecipes => {
  let query = `
    mutation CreateTaggedRecipes($document: TaggedRecipesInput!) {
      setTaggedRecipes(taggedRecipes: $document) {
          tag,
          updatedAt
      }
    }
  `
  let variables = {"document": taggedRecipes}
  {query: query, variables: variables}
}

Note: If you are copy-pasting like I did, don’t forget to change the id selector to tag inside the setTaggedRecipes mutation. (I did!)

Hook it all up

Now open App.res and fix the compiler errors where syncGraphQL is called (inside the useEffect3). You’ll need to add a push: attribute to each of them, formatted according to pushOptions. The two calls should now look like this:

  db.recipes->Db.RxCollection.syncGraphQL({
    url: "/graphql",
    pull: {queryBuilder: Sync.recipeQueryBuilder},
    push: {queryBuilder: Sync.recipeMutationBuilder},
    deletedFlag: "deleted",
    live: true,
  })

  db.tags->Db.RxCollection.syncGraphQL({
    url: "/graphql",
    pull: {queryBuilder: Sync.taggedRecipesQueryBuilder},
    push: {queryBuilder: Sync.taggedRecipesMutationBuilder},
    deletedFlag: "deleted",
    live: true,
  })

The compiler should be happy now.

Test it out (looks like there’s a bug)

I fired up both the express server and react devserver and immediately saw 500 errors showing up in the console.

Looking in the network tab of the chrome dev console, I see that the server is returning some useful information: Field \"updatedAt\" is not defined by type \"RecipeInput\"."

Urgh! RxDB is sending us a Model.recipe, just as it is supposed to, but our server is expecting a Model.recipe with no updatedAt tag. Once again, there are a few ways we can fix this:

  • Add updatedAt to RecipeInput on the server. The server will overwrite this flag, but that’s fine. This is the easiest thing to do.
  • Have pushQueryBuilder accept a completely generic type for variables.document instead of the 'document passed in. I don’t care for this idea because it means I need to add a new recipesInput type to the model that I really don`t care about outside of this sync code.
  • Have pushQueryBuilder return Js.Object instead of a specific type. This would be super easy, but we lose a bit of type safety.

I was leaning towards the last option, but then I checked the schema RxDB used as an example in their documentation. Turns out, they include updatedAt on the HumanInput. I’m just going to go ahead and follow their example: Option 1 it is!

Fixing the schema on the server

Open Schema.res in the server repository. Modify both the RecipeInput and TaggedRecipesInput to include the additional updatedAt field:

  input RecipeInput {
    id: String!,
    title: String!,
    ingredients: String!,
    instructions: String!,
    tags: [String]!,
    deleted: Boolean!,
    updatedAt: Float

  }

  input TaggedRecipesInput {
    tag: String!,
    recipes: [String]!,
    deleted: Boolean!,
    updatedAt: Float
  }

Notice that I did not mark the new updatedAt field as required (using a !) as I do for the other fields. I did this for two reasons. First, the server really doesn’t care what updatedAt is and is going to discard it regardless, so I want to leave it optional for other clients. More importantly, if those hypothetical other clients already exist, making updatedAt optional does not break any client queries.

This is all that needs to change. The server code is fine; we just had to modify the query so the client could send unnecessary data if it wanted!

Rebuild and restart the server.

Test it out again

Reload your React app and add a recipe. Have a look in the network tab to make sure that the graphql endpoint was really queried when you submitted it. Also add a tag to the recipe from the UI.

Then head to the Graphiql endpoint to query the server and see if the data was really added. You can run a query like this:

query {
  recipeRxDbFeed(id: "", minUpdatedAt: 0, limit:5) {
    id
    title
    instructions
    ingredients
    deleted
    updatedAt
  }
  
}

You should see the recipe you just added as well as any other recipes that you had in your client app. This is notable because the server as currently implemented stores everything in RAM. When you restart it, the client has to populate it all over.

Try opening a new profile in Chrome. Navigate to the tags page in both browsers. Wait a few seconds for RxDB to catch up; you should see the tags show up in the new window.

Then add a new recipe, and add a tag to it in one of the windows. Don’t touch the other browser profile. It might take up to ten seconds, but you should see the newly tagged recipe show up in the other browser.

Conclusion

So it seems to be working! An offline enabled RxDB database that syncs with a graphql server with live updates, all written in Rescript.

I am really delighted with how well the code has come together. I had to do a few refactors and modifications through this article, but the type checking and instant compile time made it very easy to find what needed to change. I was actually more skeptical that RxDB would do it’s job than I was of Rescript or GraphQL itself!

I admit I still don’t trust that RxDB will do the right thing if I stop and restart the server or do things in the browser in offline mode. But if there are problems there, it’s likely a bug in RxDB rather than in our code! Sync is a very tricky problem to solve, and I’m happy to leave the heavy lifting to libraries like RxDB.

I was hoping to make this respond in realtime using GraphQL subscriptions. However, I have never worked with subscriptions before and didn’t realize it would require additional changes on the server side. I may do that eventually, but if you want to tackle it on your own, apollo has a nice article on the subject. Hooking up the websockets in rescript express might require some interesting modelling, but I am sure you can make it work.

For now, I’m going to consider this RxDB project “complete”. In the next article, I want to focus on writing some unit tests in Rescript. See you then!