Jake Donham > Technical Difficulties > Slate Explorer

Slate Explorer, a tool for exploring the Slate API

2021-02-26

I have been programming for a pretty long time—let's just say that this is not an unfamiliar object. Insofar as I've gotten better at it in that time, it's been a byproduct of doing programming (for work and for fun); but I never set myself a goal to improve, or thought about how to improve. Now at the Recurse Center I'm surrounded by people who are focused on becoming better programmers, so I've been thinking about what that means for me.

One thing I noticed about myself is that I often do things the hard way: type something out rather than copy and paste; manually change several occurrences rather than use search and replace or multicursor; retype the same commands at a shell or REPL rather than make a script or build a tool. It's a weird form of laziness—do more work to avoid the work needed to do less work—but I think it makes psychological sense: Breaking out of a groove takes mental effort; and it can feel risky if you're anxious about keeping focus. It's often not clear that the "easier" way will turn out to be easier in the end.

I think this is a way I could improve: try to notice when I'm doing things the hard way, and push myself a little more to stop and figure out an easier way. (This doesn't always pay off! But I think I err on the side of doing it too little, for psychological reasons.)

Now I want to talk about a tool I made to make things easier for myself, but first some background about

Slate

For Programmable Matter I'm using a framework for building rich-text editors called Slate. It is very cool! The basic idea is that editor state is represented as a tree, the UI is rendered from the tree (using React), and actions in the UI are expressed as tree manipulations. You can write most of your editor code in terms of trees, without thinking about the concrete UI. In particular it's really easy to write tests: Slate comes with a library for writing editor state as a JSX expression, like so:

<editor>
<h1>Text</h1>
<ul>
<li><anchor/>plain<focus/>; or</li>
<li><text bold="true">bold</text>; or
</li>
<li><text italic="true">italic</text>
</li>
</ul>
</editor>

You can write a test for an editor action by giving an initial editor state, performing the action (manipulating the tree), then comparing the result to an expected final editor state. This is very compact and doesn't deal with the concrete UI at all. Nice!

Slate is very flexible about what trees look like, so in this example the editor and text nodes are built in, but h1, ul, and li are application-specific. The application can render them as HTML, but that's not predefined; you're free to use whatever nodes you like and render them however you like. The anchor and focus nodes represent the endpoints of the selected region (so here the word "plain" is selected). Managing the selection is really important for making a usable editor: actions should apply in a sensible way to the current selection, and after an action the selection should be left in a sensible place. So having an easy way to write tests for it is hugely useful.

There's a lot more to say about Slate but I will save it for a future post; let's move on to

Understanding the Slate API

Slate is amazing, but there are some wrinkles: the API is pretty complicated, it's not very well-documented, and I've run across several confusing bugs. For example, here is the signature of a function that splits nodes:

Transforms.splitNodes: (
editor: Editor,
options?: {
at?: Location
match?: (node: Node, path: Path) => boolean
mode?: 'highest' | 'lowest'
always?: boolean
height?: number
voids?: boolean
}
) => void

The documentation is thin; it's hard to know what these arguments do and how they interact. Here are some ways I have tried to understand it:

To make things easier for myself, I decided that I would spend some time making a tool to explore the Slate API, called

Slate Explorer

The box at the upper left is a Slate editor widget. It is basically the stock editor; I added rendering for a few HTML tags (h1, h2, h3, p, ul, li), and text styles (bold and italic), but didn't customize the editing behavior at all. So there is no UI for setting text styles or inserting headers or lists.

The box at the upper right is an XML rendering of the editor state; as you edit or move the selection in the Slate editor you'll see updates to the state. You can also edit the XML, and changes will be reflected back to the Slate editor. You can add the supported HTML tags by editing the XML. (I'm using a stock XML parser here, and XML is not exactly JSX: you can't embed Javascript expressions in curly braces, so use e.g. bold='true' rather than bold={true}; and the whitespace handling is slightly different, to avoid losing whitespace in the editor state when going Slate -> XML -> Slate.)

The box in the middle contains arbitrary Javascript, with editor and some Slate modules (Editor, Element, Node, Path, Range, Text, and Transforms) available in the environment. Put whatever you like here to transform the editor state. The boxes at the bottom show the result of the transformation, rendered by Slate and as XML.

We can see in the example that calling splitNodes(editor) with no options splits the innermost node at the cursor position, and leaves the cursor at the beginning of the new node. Some other things to try:

You can try Slate Explorer here, and see the code here. Take a look at the Slate docs for functions to explore, especially the Editor interface and Transforms.

Did it pay off?

HARD YES!

I spent about 3 days building Slate Explorer, which might seem like a lot for an ad-hoc tool, but I have definitely burned days trying to understand the Slate API in the more arduous ways above. (Also I polished it more than I might have for a personal tool, because I want to show it to RC and Slate people.) I learned about CSS grid layout along the way, so that's a bonus.

I don't think I will ever experiment with Slate in the console again; it's now very easy to set up editor state and try a bunch of API calls. I will probably write more Slate tests (I use them in part to capture bugs and surprising behavior), and I can now repro them in Slate Explorer then copy the editor state into a test. I will certainly read the Slate code again, and it will be really helpful to use Slate Explorer for support.

Here I want to call out AST Explorer, a great tool for seeing the parse trees that result from various parsers, which I have used a ton while working on Programmable Matter, and which inspired the name Slate Explorer. Figuring out parse trees is another thing I used to burn a lot of time doing in the console. I hope that people building on Slate will find Slate Explorer as useful.

Please email me with comments, criticisms, or corrections.