Jake Donham > Technical Difficulties > Slate Explorer

Slate Explorer, a tool for exploring the Slate API

2021-02-26

I have been programming for a pretty long time—let's just say that this is not an unfamiliar object. Insofar as I've gotten better at it in that time, it's been a byproduct of doing programming (for work and for fun); but I never set myself a goal to improve, or thought about how to improve. Now at the Recurse Center I'm surrounded by people who are focused on becoming better programmers, so I've been thinking about what that means for me.

One thing I noticed about myself is that I often do things the hard way: type something out rather than copy and paste; manually change several occurrences rather than use search and replace or multicursor; retype the same commands at a shell or REPL rather than make a script or build a tool. It's a weird form of laziness—do more work to avoid the work needed to do less work—but I think it makes psychological sense: Breaking out of a groove takes mental effort; and it can feel risky if you're anxious about keeping focus. It's often not clear that the "easier" way will turn out to be easier in the end.

I think this is a way I could improve: try to notice when I'm doing things the hard way, and push myself a little more to stop and figure out an easier way. (This doesn't always pay off! But I think I err on the side of doing it too little, for psychological reasons.)

Now I want to talk about a tool I made to make things easier for myself, but first some background about

Slate

For Programmable Matter I'm using a framework for building rich-text editors called Slate. It is very cool! The basic idea is that editor state is represented as a tree, the UI is rendered from the tree (using React), and actions in the UI are expressed as tree manipulations. You can write most of your editor code in terms of trees, without thinking about the concrete UI. In particular it's really easy to write tests: Slate comes with a library for writing editor state as a JSX expression, like so:

<editor>
  <h1>Text</h1>
  <ul>
    <li><anchor/>plain<focus/>; or</li>
    <li><text bold="true">bold</text>; or
    </li>
    <li><text italic="true">italic</text>
    </li>
  </ul>
</editor>

You can write a test for an editor action by giving an initial editor state, performing the action (manipulating the tree), then comparing the result to an expected final editor state. This is very compact and doesn't deal with the concrete UI at all. Nice!

Slate is very flexible about what trees look like, so in this example the editor and text nodes are built in, but h1, ul, and li are application-specific. The application can render them as HTML, but that's not predefined; you're free to use whatever nodes you like and render them however you like. The anchor and focus nodes represent the endpoints of the selected region (so here the word "plain" is selected). Managing the selection is really important for making a usable editor: actions should apply in a sensible way to the current selection, and after an action the selection should be left in a sensible place. So having an easy way to write tests for it is hugely useful.

There's a lot more to say about Slate but I will save it for a future post; let's move on to

Understanding the Slate API

Slate is amazing, but there are some wrinkles: the API is pretty complicated, it's not very well-documented, and I've run across several confusing bugs. For example, here is the signature of a function that splits nodes:

Transforms.splitNodes: (
  editor: Editor,
  options?: {
    at?: Location
    match?: (node: Node, path: Path) => boolean
    mode?: 'highest' | 'lowest'
    always?: boolean
    height?: number
    voids?: boolean
  }
) => void

The documentation is thin; it's hard to know what these arguments do and how they interact. Here are some ways I have tried to understand it:

Read the code. I find this is a good approach when I'm trying to understand a specific bug (in my code or in Slate) but it's hard to get a general sense of what a function is supposed to do. It is a lot of work, and I tend to forget the details I learned pretty quickly.
Write a test. This is useful when I have a specific hypothesis about how something works: I can write a test and confirm or refute the hypothesis. It's easy to get started, since I already have a file of tests set up and I can usually cut and paste to get started; and it's a useful as a way to capture my understanding, since the tests are part of my project code. But it's hard to get a general sense without writing dozens of tests, and sometimes writing and running a test feels like a lot of overhead.
Experiment in the console. require the Slate module, make an editor, populate it with some nodes, try some API calls, see what happens. It's a little bit painful to get started, but it's easy to try lots of things quickly (compared to setting up a bunch of tests) to get a general sense. Working in the console is a pain: there's no JSX, so I construct trees manually, and inspect them with JSON.stringify. It's hard to match up the selection with where it goes in the tree. When I'm doing this I often feel like I should figure out a better way.

To make things easier for myself, I decided that I would spend some time making a tool to explore the Slate API, called

Slate Explorer

The box at the upper left is a Slate editor widget. It is basically the stock editor; I added rendering for a few HTML tags (h1, h2, h3, p, ul, li), and text styles (bold and italic), but didn't customize the editing behavior at all. So there is no UI for setting text styles or inserting headers or lists.

The box at the upper right is an XML rendering of the editor state; as you edit or move the selection in the Slate editor you'll see updates to the state. You can also edit the XML, and changes will be reflected back to the Slate editor. You can add the supported HTML tags by editing the XML. (I'm using a stock XML parser here, and XML is not exactly JSX: you can't embed Javascript expressions in curly braces, so use e.g. bold='true' rather than bold={true}; and the whitespace handling is slightly different, to avoid losing whitespace in the editor state when going Slate -> XML -> Slate.)

The box in the middle contains arbitrary Javascript, with editor and some Slate modules (Editor, Element, Node, Path, Range, Text, and Transforms) available in the environment. Put whatever you like here to transform the editor state. The boxes at the bottom show the result of the transformation, rendered by Slate and as XML.

We can see in the example that calling splitNodes(editor) with no options splits the innermost node at the cursor position, and leaves the cursor at the beginning of the new node. Some other things to try:

what happens if a range of text is selected? (surprising but OK)
what does mode: 'highest' do? (makes sense)
if you give a location to split at, e.g. at: { path: [1, 0, 0], offset: 3 }, what happens to the selection? (nice)

You can try Slate Explorer here, and see the code here. Take a look at the Slate docs for functions to explore, especially the Editor interface and Transforms.

Did it pay off?

HARD YES!

I spent about 3 days building Slate Explorer, which might seem like a lot for an ad-hoc tool, but I have definitely burned days trying to understand the Slate API in the more arduous ways above. (Also I polished it more than I might have for a personal tool, because I want to show it to RC and Slate people.) I learned about CSS grid layout along the way, so that's a bonus.

I don't think I will ever experiment with Slate in the console again; it's now very easy to set up editor state and try a bunch of API calls. I will probably write more Slate tests (I use them in part to capture bugs and surprising behavior), and I can now repro them in Slate Explorer then copy the editor state into a test. I will certainly read the Slate code again, and it will be really helpful to use Slate Explorer for support.

Here I want to call out AST Explorer, a great tool for seeing the parse trees that result from various parsers, which I have used a ton while working on Programmable Matter, and which inspired the name Slate Explorer. Figuring out parse trees is another thing I used to burn a lot of time doing in the console. I hope that people building on Slate will find Slate Explorer as useful.

Please email me with comments, criticisms, or corrections.