My book-writing workflow

By request: what’s my workflow for writing books? Steps, tools, etc.

Writing with the Leanpub platform

A long time ago I noticed that testing-advocate Chris Hartjes published his books on Leanpub. When I had the idea of writing a book about the Symfony framework, I tried this platform and it was a good match. You can write your book in Markdown, commit the manuscript to a GitHub repository, and push the button to publish a new version. Leanpub will generate EPUB and PDF versions for you. Readers can be updated about releases via email or a notification on the website.

While publishing multiple books like this, I kept running into these limitations:

• Even though writing a book like this feels a lot like writing code (make a change, push, deploy), what’s missing is a continuous integration step where we can verify that the book is “correct”.
• With the Leanpub Markdown setup you can include .md chapter files from your main Book.txt file, but you can’t include files from those .md files. This makes it hard to create a nested folder structure for chapters, sections, and code samples.
• If you include source code in a book, it has to be formatted to be readable and to fit on a page, and parts have to be skipped or abbreviated. Still, you want the code to be correct. For this you want to run tests on the code, but you can’t run tests on incomplete or abbreviated code samples. I no longer wanted to copy/paste working code and format it manually for presentation purposes.
• The code samples should be tested, but they should also be analyzed with PHPStan, and they should be upgrade-able to newer PHP versions. The go-to tool for this is Rector. I want to have these tools available in my book projects by default.
• When I include the output of tools like PHPUnit in the book, I don’t want to copy/paste it from the terminal either. When I make a change to the code, the output should be regenerated automatically and included in the book.

A manuscript pre-processor

To overcome all these problems I created a pre-processor tool that allows me to do things that aren’t possible with Leanpub’s Markdown (or Markua, their variation on Markdown), but that eventually result in a manuscript directory that Leanpub can turn into an actual book. This means that nowadays I write in “Matthias-Flavored Markua” ;) And that I have a continously-integrated book-writing process which uses PHPUnit for tests, PHPStan for static analysis, Rector for automated refactoring, and ECS for the coding standard. The pre-processor tool is hand-written in PHP. Given that it has to process Markua, it contains a Markua parser which leverages the awesome Parsica library.

Some of the things the tool can do:

• You can add special comments to code samples that influence the final look when included in the manuscript, e.g. // skip-start and // skip-end to create an ellipsis (// ...) or // crop-start and // crop-end to remove lines from the beginning and end of an included file.
• You can create a cover image with Gimp, and the tool will convert the .xcf file to a .png file that matches Leanpub’s expectations.
• You can include .md files from any other .md file. In the end, everything will be compiled into a single book.md file.
• When you regenerate this manuscript, it’s easy to see which parts have been changed, which allows you to spot parts that were changed by mistake, or that no longer work because of a different output from PHPUnit, etc.
• You can collect all external links and generate a link registry for it, so the book will never have broken links.
• You can include code from a package in vendor/ and reformat it according to the style used for the other code samples.
• And so on! The project is yet another example of the amazing powers that will be unleashed if you have access to an Abstract Syntax Tree for a language, in this case Markua.

The path towards releasing a new book

For me, a new book always starts with an idea, which I immediately judge for its capability of becoming a book (as opposed to a blog post, or series of blog posts). In a sense, every idea can become a book, but it has to be interesting, I should be able to build on my own experience in the subject area, and at the same time there should be plenty to learn or explore, so the process of writing will be interesting for me as well. The result of this thinking is often a title. Strictly speaking it’s a working title; the final title is often (slightly) different:

• “More than a month with Symfony” became “A year with Symfony”
• “Principles of Package Design” was used as-is
• “Microservices for everyone” idem
• “A field guide to object design” became “Style Guide for Object Design” and when republished with Manning “Object Design Style Guide”
• “Rector book” became “Rector - The power of automated refactoring”
• “PHP for the web”, “Advanced Web Application Architecture”, and “Recipes for decoupling” had these titles from the start.

With a (working) title, you can set up a new project on Leanpub, and a new Git repository on GitHub. I usually start working on a cover first, looking for abstract stock images that somehow match the book’s topic. The cover image tends to change later on, which makes sense. The personality of the book only becomes apparent after writing the biggest part of it. To match the personality the image needs to be modified.

For quite some time I write quietly, and I don’t announce the book in public. I want to be certain of its viability before making any kind of “promise” about it. There’s the 100 page milestone, at which point I like to share the landing page, that shows the cover image and a description, and allows interested readers to sign up to be notified about the release.

When most of the chapters are written (but not yet good enough to publish), I start revising the first chapters. In the case of “Recipes for Decoupling”, my approach changed radically during the writing process. Revising the first chapters to match the style of the later chapters took quite a bit of time.

When the first chapters are good enough to be published, I push the button and release the first ~80 pages to the public. My partner still thinks it’s weird to release an incomplete book, but I do it for several reasons:

• I hope that what I wrote can already be helpful for some readers today, and it would be a shame to hide it from them.
• I hope to receive some feedback that helps me improve what has been published, but also what is still to be written.
• Knowing that people are buying and reading the book gives me an incredible boost: I really want to finish it and make sure that it doesn’t take too long.
• Publishing early prevents me from getting stuck in “perfection mode”. I want high quality, but also pragmatism (in book writing and in programming).

At this point I also add a banner to my website, so visitors may notice that a new book is available.

To stay in the “release flow” I declare that I’ll publish a new chapter every two weeks (or every week sometimes). Steadily releasing new chapters is great, but I guess it only works if there is some kind of linearity to the book. So far, I haven’t felt the need to make big changes to chapters that have already been released. I feel like this is taken care of by the big revision phase before the release of the first few chapters. Still, the chapters have to be somewhat modularized, self-contained, and I’m always looking a natural progression from one topic to the next, where the next one seamlessly builds on top of the previous one.

Then, after many hours, the final version is released.

Time investment

I kept track of all the hours that went into writing, revising, project management, etc. (except the hours for “A Year With Symfony” because I wasn’t a freelancer back then):

• Principles of Package Design: 137 (Leanpub) + 111 (Apress)
• Microservices for everyone: 342
• Object Design Style Guide: 127 (Leanpub) + 137 (Manning)
• Advanced Web Application Architecture: 422 hours
• PHP for the web: 98 hours
• Rector: 180 hours + 50 hours (book tool)
• Recipes for Decoupling: 194 hours + 20 hours (book tool)

This amounts to about 1 hour per page on average. That’s pretty efficient, but it’s not really hard to achieve this average because code samples take up a lot more space than regular paragraphs. How much time has to be spent also depends on how much research and trail and error is involved. If it’s a subject I know a lot about, it won’t take that much time. Comparing with the lifetime sales of these books, it amounts to a ~$50/hour income. Which is quite alright if you ask me, although development or training work pays more. Anyway, it doesn’t really make sense to value a book by the calculated hourly income. Personally I like to earn a living with other work, thereby earning time to write books and do other things I like to do.

Beyond a certain number of pages/chapters I find that a book becomes almost unmanagable, i.e. too big for my brain to deal with. So keeping it below 300 pages is a good idea. For “Recipes for Decoupling” I even aimed for 150 pages, and it would’ve been nice to write a smaller book this time, but the topic didn’t fit in such a small number of pages, so it’s a ~300 page book again…

The print edition

After releasing the e-book some work has to be done to make it ready for print. Preparing a book for print-on-demand via Lulu is quite easy. Leanpub has a built-in option for generating a print-ready PDF version. There used to be some problems with this PDF. Code snippets inside regular paragraphs weren’t properly line-wrapped, which would lead to text being printed too far into the page margins. Nowadays, there are no such issues. Lulu accepts the format as-is. I use the common “Crown Quarto” format, which is supported by both Lulu and Leanpub. Most of the work goes into remaking the e-book cover into something that works for a book. Lulu provides a very useful template PDF for the cover that can be imported into Gimp and the likes. This template already takes into account the number of pages and the thickness of the paper.

Before you can release the print edition, you have to buy a copy of the book for yourself, to check if everything is good. After receiving it you can activate the book’s landing page on Lulu.

Publicity

I don’t do much in the marketing area. Leanpub collects email addresses of interested readers that you can use only for the release notification. Of course, it would be useful if I could notify readers of previous books too, but I also think that there is a large overlap between my book audience and people following me on Twitter, or regularly reading my blog posts, so I’m pretty sure I’m able to reach my existing audience via these channels. This remains guesswork though.

I’ve heard from other writers that building up a mailing list works really well for them. To make it work you have to collect people’s email addresses, offer them a regular email with some truly interesting things in it. As a reward you can notify thousands of people about a new book, which is likely resulting in some sales. Something about this has always felt a bit weird to me. The newsletter is not really a way to share useful information, it’s a way to indirectly secure more income. Also, it uses the “spam” concept of targeting many, in the hope that a few will buy. This just isn’t a good fit for me, although many have recommended this approach to me, and all books about self-publishing authors advocate it. Also, everybody does it. I hope it’s clear that I’m not saying you shouldn’t do it, just that I’d rather not do it. Although I know I miss out on some money by making this decision.

What does work well for me is the monthly Leanpub sale, which regularly offers my books with a 10-15% discount. Also setting up bundles of books that can be bought together with a big discount turned out to be a good idea.

Conclusion

I hope this article has given you some useful background information about my book-writing process, the workflow I use, and what tools are involved. To be honest, I don’t think it should be taken as a tutorial for future tech book writers. It’s just what I do and what works for me, and there are probably many more factors in successful book writing that have not been covered here. However, if you were thinking about writing a book, I can recommend taking a similar lean approach to book-writing. Just get your words out there!

Would you like to have more information? Please leave a comment.

• Writing
• Books