Writing the same library twice

Motivation

An SDK is a library that wraps somebody else's system so you can use it from your own language. The system in this case was a blockchain, and what it accepts is a fixed set of messages: send these coins, stake this amount, submit this vote. The SDK's job is to make each of those available as a function in a language a programmer already knows, along with all the surrounding machinery for building a message, signing it, sending it, and understanding what came back.

None of that is hard in the way a single interesting problem is hard. It is hard because there are hundreds of pieces and no obvious order to build them in. Every piece is small, every piece is tedious, and on any given morning there is no answer to the question of which one to do next. I built one of these in Python at the start of 2020, then a second in TypeScript, then started a third in Java. Doing it three times is what turned "which piece next" from a daily annoyance into a question with an answer.

Starting in the code

A note from March, written after the Python SDK shipped, is about beginnings. What started that library was not a design document. It was writing simple serializer and deserializer classes, meaning the boring code that turns a message into bytes and bytes back into a message. The commencement, as I put it, was getting directly into the code as soon as possible without worrying whether it was best practice at the beginning, because I could always update it later.

The line I wrote under that has held up better than anything else in the entry: sometimes the minimum viable product is not what you think it is, and if you are not doing it, then maybe it is not the right one to be striving for. Not doing it is data. A first step you keep not taking is usually not the first step, and the way to find the real one is to notice which small boring thing you would actually be willing to start this morning, and start there.

Finding the API shape by using it

The first decision was where the shape of the interface should come from. It could come from the system underneath, which is the default, because the system already has a structure and copying it requires no thought. What I wrote down instead, in April, was to figure out how the JavaScript SDK was going to be used by writing some applications with it.

Building the consumer before the producer is a small inversion with a large effect. When the library's structure comes from the system, you get an accurate wrapper that is unpleasant to hold, because the system was organised around how it stores things and the programmer is organised around what they are trying to do. When it comes from a real application you wrote against it, the awkward parts show up as awkwardness in your own code, three days before anyone else has to live with them.

Depth, breadth, and tests

The part I keep is a note I wrote in May, and I wrote it because I was afraid.

I had finished the JavaScript SDK the week before and was starting the Java one, and the dread arriving on the second day was recognisably the same dread from the first day of the first one. Rather than push through it again, I wrote down what had worked, and it comes out as three ways to move across a large surface:

• Depth is how complete one class is. Take the type that represents a coin, implement every method it should have, and finish it. This is building from the bottom up.
• Breadth is covering the basics of every core class before finishing any of them. This is building from the top down.
• Tests are neither, and they are the part that matters. Writing tests is important because they mark milestones that you have completed something.

Depth against breadth is the ordinary tradeoff and either answer works. The third item is the one that solves the actual problem, which was never sequencing. On a surface with hundreds of pieces you can work hard for four days and be unable to say what is finished, because "finished" has no edge anywhere in the work. Nothing looks different at the end of the day. A passing test is the smallest object in that landscape with a definite boundary: it did not exist this morning, it exists now, and it will tell you tomorrow if you break it. The tests were load-bearing for morale before they were load-bearing for correctness.

I do not think this is unique to libraries. Any surface too large to hold in your head has the same shape, and the fix is the same, which is to manufacture things that can be visibly done.

There is one thing I would go back and copy. In September I was reading CosmJS, the JavaScript library for Cosmos chains, and wrote down that it deals with numbers better than mine did. Chain balances are integers too large for JavaScript's native number type, so every library has to decide how to carry them and how to stop a caller accidentally rounding one, and theirs had thought about it harder than mine had. That is the kind of thing depth-first would have caught and breadth-first did not, because it only shows up when you finish a type instead of sketching it.