Version Control for Dashboards: How We Built It

What if you could git blame your dashboard? Track who changed that broken metric, revert a bad visualization, and collaborate on dashboards the same way you collaborate on code. In the past few weeks I have been working on setting up a smooth 2-way version control system where changes made in the UI are merged seamlessly with changes made locally in the yaml files that describe dashboards.

Most BI tools either lack version control entirely or bolt it on as an afterthought. Having built hundreds of dashboards, I know the pain: who changed this metric? When? Why? Even basic accountability is a nightmare. And don't get me started on editing, you open a dashboard, find unpublished changes from who-knows-when, and now you're stuck figuring out whether to publish someone else's half-finished work or trash it.

High-level our current dashboard yaml structure has the following format:

dashboard:
  id: uuid
  title: string
  sheets:
    - id: string
      name: string
      elements:
        - element_id: uuid
          type: table
          name: string
          grid: { x, y, w, h }
          properties:
            columns:
              - id: string
                expression: string
                shown: { label, width, format, ... }
                sort: { order: asc | desc }
            dataSource:
              mode: sql
              connectionId: uuid
              dataset: { sql, columns }
            display: { rowBanding, density, ... }

Having a file format means that we can use version control tools to manage changes to the yaml files. This allows us to track changes, revert to previous versions, and collaborate with other team members on the same dashboard. But it also allows users to use their favorite LLM to generate and edit yaml files for dashboards.

For engineers this is a dream. Clone the repo, open your editor and start building. But it's not just engineers who want to edit dashboards, your marketing lead needs to make edits too. We need to provide a way for them to create and edit dashboards without having to write yaml files. So they get a powerful UI that allows them to create and edit dashboards without having to write yaml files.

So we have two interfaces: YAML files for the power users, a visual editor for everyone else. The challenge was making them work together seamlessly, without constantly stepping on each other's toes. Here's how we solved it. At the core, we use real Git repositories to support all of the above, which simplifies our setup but also means we are now hosting a powerful system that can be used maliciously.

In total I went through 3 iterations before I was happy with the setup:

1. Use go-git with an S3 backend — go-git is great, but it was also missing features at the point that I was implementing a POC with it, it did not support worktrees (it looks like they do now), and storing everything in S3 ended up not being a good idea as everything becomes a separate http request. It's more efficient to use a local filesystem.
2. Use go-git/git shell with EFS — my second proof of concept still leveraged go-git, with a fall back on git shell for missing features such as worktrees. Although I was very happy with the feature set I could achieve with this, I ended up spending too much time thinking about and removing security concerns, and I still did not trust the final output I realized I had to go with something more battle tested.
3. Use gitea — I ended up leveraging gitea and its API to host the git repositories. This allowed us to leverage gitea's API to bring safe git interactions to Thelix.

With Gitea handling the heavy lifting, we could focus on what matters: exposing git's most useful features in a way that feels native to dashboard building.

Branches

Branches are first citizens in the UI, you can create them, merge them, etc... all standard operations are supported through the UI. The nice thing is that if the UI doesn't support it you can always fallback on using the git shell.

There are some existing BI tools like Looker, Evidence.dev etc... that support branches, we felt it was table stakes to have this available at your fingertips. The downside is that most dashboard viewers won't care about branches, they just want to look at the main branch. We solved this by having an IDE environment where you can build your dashboards, and plan to build a separate viewer UI that takes away some of the complexity.

Protected Main Branch

The main branch is protected, within the Thelix IDE you can only view dashboards, not edit them. Protecting main keeps production safe. But what about the messy middle—when two people are iterating on the same dashboard at the same time?

Collaboration on dashboards

Do you want to collaborate with others in the IDE on a dashboard in the same branch? We got you covered, any changes you make are synced instantly between users via CRDT, in the backend we use PartyKit/Yjs for this.

Saving / Committing changes

When you are ready to commit your changes to version control, there are basically two paths:

1. Happy Path — This is the clean commit path, there are no conflicting commits on your branch. Add your commit message, and you are good to go.
2. Conflicts Detected — It can happen that between the start of your first change and commit in the UI, changes from other branches or remote commits were pushed to your editing branch, in that case we show a resolve conflict modal.

Publish to Main Branch

In git it's pretty standard to merge all the changes in a branch into the main branch, the UI allows you to publish your branch easily.

But a lot of workflows around dashboards are centered around editing a single dashboard and once done, wanting to publish them to the main branch. We made this really easy, open your dashboard, click Publish, and you are done! In the backend we cherrypick the changes to main.

Auto-update from Main

A neat little feature that we added, is that you can set your branch to auto-update from main, anytime the main branch gets updated it will push its changes to your branch. As long as there is no conflict of course.

Bi-directional sync

As a lot of our users have their git repositories hosted centrally, you can also sync your Thelix repository with a remote repository. As long as your host allows for https access to your git repos using an access token, you are good to go. Services such as GitHub, Gitlab, Bitbucket, Gitea etc... all support this pattern.

What we're not building

We're a dashboarding tool, working hard to be the absolute best at dashboarding, which means we're not GitHub. We cover 80% of the git workflows you actually need for dashboard development. For the other 20%? Sync your repo and use the tools you already love.