How we tried in Docs as code and lost
What is Docs as Code famously described in the article Docs as Code: an introduction to the subject.
In a nutshell: this is maintaining documentation in a markup language (Markdown, AsciiDoc) and storing it in a repository.
The perks are all that come from working with the repository.
The disadvantages are in this article.
At the fall Analyst Days last year, a good quarter of the reports were devoted to the topic of Docs as Code. At that time, a conclave of analysts at my previous place of work had already been deciding for 9 months whether the project needed fashionable-stylish-youth Docs as Code or whether to remain in the dying Confluence. I don’t know what they came to. At our new place of work, we implemented Docs as Code in a week – there was a clear understanding of the problem, the approach seemed like a winning solution. We use it for six months.
What were the requirements for the documentation system?
Mandatory
Deployment in the inner circuit. Due to corporate secrecy, external clouds were excluded.
Possibility of parallel work on a document.
Automatic display of delta changes.
Automatic PDF generation for presentation to the customer.
The ability to reuse specific blocks from one document to another (include).
Possibility of approval of documentation by specific people.
Display documentation in fully compiled form.
Desirable
Automatic linking of documentation changes to development tasks in the task management system.
Access to developer documentation directly from the IDE.
Saving existing documents that were written in Markdown and AsciiDoc.
Opensource for the possibility of independent modification.
Using PlantUML.
Docs as Code covers all these needs. And this is how we did it.
Solution
The project uses:
Version control system Gitea.
Project and task management system Redmine.
Previously, documentation was maintained in Wiki.js. This wiki supports Markdown; some colleagues preferred to use it.
What did you choose for Docs as Code?
Markup language AsciiDoc. Unlike Markdown, it supports include and variables out of the box.
IDE PyCharm. It has a free version, official plugins for AsciiDoc and PlantUML, and at the same time a good built-in spelling and punctuation checker for the Russian language. Subjectively, it is more friendly to non-advanced users than VS Code.
Hugo to display full documentation.
To use PlantUml diagrams inside AsciiDoc documents, servers were deployed in the inner circuit Kroki And PlantUML.
Collects documentation after a commit Drone.
How did you organize the storage of documentation?
The project is service architecture. Each service has its own development team. A documentation repository was created for each service and the appropriate developers were given access and rights to approve pull requests.
We created a separate architecture repository. It stores documentation that affects several services: an architectural diagram, a list of message queues, top-level descriptions of inter-service interactions, proto-models (we use gRPC), some general rules, etc.
We also created repositories for business requirements, organizational and user documentation. All team members were given access to them and the architecture.
How the work process was organized
Initially, the idea was that the repository with documentation for the service would be pulled into the root of the repository of the corresponding service, that is, the developer does not need to separately download the repository of documentation.
Also, initially there was an idea to maintain two permanent documentation branches:
prod – with a description of the functionality of the system in prod state;
dev – with a description of the functionality of the system in the state of implemented but not yet released improvements.
Documentation, taking into account individual improvements, should have been located in temporary branches corresponding to epics for business requests.
It was assumed that for now the documentation on the feature would be merged into the dev branch simultaneously with the code being merged into the general branch, and into prod at the same time as the functionality was moved to prod.
We quickly abandoned these ideas.
Firstly, it was important to support intra-document references. Moreover, these links had to work in Hugo. Accordingly, we began to indicate in documents links to the intended document page in Hugo, and often at that moment the page in Hugo did not exist, since both documents were created within the same task.
Accordingly, if a developer decided to read the documentation in the IDE, the first link would take him to Hugo. If only the target document were old! And if new, then the link would lead to page 404, because Setting up a Hugo build for each epic branch is impractical.
Secondly, changes in requirements are necessary coordinate with developers before the start of implementation. Only pull requests can be approved. If we wanted to see consistent documentation in the epic branches, we would have to bud branches from them too.
Third, during solution design, the analyst should take into account all current requirements. Long stay of documentation in epic branches would invariably lead to the analyst not taking into account requirements that are in other epic branches and have not yet been merged into dev.
In the end, we settled on this scheme:
The analyst designs a solution in the epic branch, after which he presents it to the development team at a general meeting and generates a stack of pull requests in dev (1 in each repository with changes).
Pull requests are approved by the architect, leading developers of the affected services, testers, and leading analyst.
In dev, all requirements that have been implemented, are already in development, and are only planned to be taken into development are stored.
And the prod branch turned out to be unnecessary, because Gitea does not allow moving individual commits, only all accumulated ones at once. At the same time, some ready-made functionality may not be released immediately. All that was left was to edit the prod branch documentation manually, but I wasn't dedicated enough for that.
Developers see complete and up-to-date documentation in Hugo.
The scheme has been reduced to:
It would seem that everything worked. What could go wrong?
Rake
The bump is the first, expected. Analysts had to be trained work with git and IDE, write in markup language. We solved the problem with a bunch of guides, wandering through the AsciiDoc documentation and sometimes at random.
The second bump, which everyone is silent about. Eye strain analysts has increased significantly. Few things hurt your eyesight as much as code. When I was a programmer, I lost two diopters despite all the gymnastics. Believe me, filling out a table with WYSIWYG is very different from controlling the number of vertical sticks in line 213. Add to this repeated edits when the sticks were still missing, and the table turned into minced meat. And also builds broken by an extra slash. By the way, imagine the face of an analyst who has been calmly pressing the “Save” button in Wiki all his life, and suddenly drops the build.
The third cone is made of waste paper. Number of documents X2 simply because PlantUML diagrams are drawn in a separate file, and also because each Hugo folder must contain an index file for this folder. Imagine the face of a developer who receives five files to agree on a single method.
The fourth bump is aesthetic. It turned out that neither Hugo nor Gitea do not support AsciiDoc from the box fully. In particular, Hugo does not display large and complex tables correctly – determining the width, merging cells, and aligning in a cell fails. Gite does not show PlantUML schemas.
The fifth bump is communication. In Hugo it is forbidden just highlight the text with the error and write to him a comment, like in Confluence. This means that a stream of screenshots or links to text with instructions on what to fix is flying into analysts’ chats.
The sixth cone, technical. As the documentation grew, its build began to come together more slowly, even more slowly, very slowly. We looked for the reason for some time, but, to be honest, not very diligently, because it was already clear that we had more problems than gains.
The seventh cone, the main one. Document approval has become hell.
Imagine: a stack of X2 documents, referencing each other, arrives for approval, but the links do not work; in this case, you cannot look at the diagrams – they are not drawn, you read the PlantUML files and draw the diagram in your head; perhaps incorrect because you are a developer and have never written in PlantUML; in addition, you can see the delta of changes only in the markup language format, and you are trying to compare the read and counted number of sticks with the not very smooth plate that you see in the preview.
Now imagine ten people going through this at once.
Whatever we did! We read the documentation online with screen sharing by its author; we threw to each other PDFs collected from the entire stack of documents; we downloaded epic branches to read in the IDE; We uploaded the documentation without approval to make it possible to read it in a tolerable form with working links.
But all these crutches do not allow you to see the delta of changes and the documentation as a whole at the same time.
The decision to abandon Docs as Code was made after a batch of fifty documents on one small epic was sent for approval. We plan to continue storing documentation in the repository, but use a Confluence-like system for reading, editing, and approval. Integration with git was one of the selection criteria.
