NPM monorepository (Lerna + autodeploy GitHub Actions)

Hey! We are the GrandCore Foundation. We create the ideal organization for the development of free projects: software, ethical online services and product standards. Read more here… Join the our chat in Telegram. We are always glad to like-minded people!

For our new project – universal documentation generator we have a need to create a monorepository, since the generator’s functionality will be extended by plugins. Read below how we fully automated this process using GitHub Actions and Lerna…

What is a monorepository

Monorepository is a collection of many projects in one repository.

Advantages:

• There is no need to maintain a huge number of repositories separately;

• Ability to track and edit code in the entire project by each individual team;

• Atomic commits.

Problems:

• Increased data volume;

• Possible issue with versioning each subproject;

• Reducing the responsibility of each team that works on subprojects of the repository.

Lerna

As the main tool for working with a mono-repository in NPM must be considered Lerna. It is a fairly convenient tool for working on monorepositories. Lerna allows you to solve many problems arising during the work.

Possibilities:

• End-to-end versioning;

• Individual versioning of each subproject;

• Convenient publication of all subprojects;

• Automatic management of dependencies between subprojects;

• Working with Yarn or NPM.

Creation of a monorepository

To demonstrate the initial Lerna settings, we will create a default project, and then demonstrate the option that we implemented in our project.

Install Lerna globally:

npm i lerna -g 

Initialize the project:

lerna init

After that, a directory will be created at the root of the project packages/, file package.json and the file lerna.json…

Content lerna.json:

{
  "packages": [
    "packages/*"
  ],
  "version": "0.0.0"
}

First, the directory with all nested projects is specified (there may be several directories). Next, the version of the monorepository is indicated (for end-to-end versioning, this value is the same in all subprojects).

Creating a new package (subproject):

lerna create <name-pkg>

After entering this command in the directory packges/ (by default) a new directory is created with a name that is specified as a parameter. Next, you need to answer a few standard questions from Lerna. It is important to note that for all packages it is desirable to use names of the following type @project_name/pkg_name… Thus, it is possible to link all packages that are developed within the repository to one single repository or one organization.

Let’s say we created a new package. Then a file is created in the directory with this package package.json… In order for this package to be available publicly in the future and be published correctly, it is necessary to add to this file:

"publishConfig": {
    "access": "public"
  }

Naturally, subprojects can also be created manually.

Publishing modified projects:

lenra publish

Possible problems:

• Not authorized by NPM;

• Forgot to add public access to package.json package;

• A busy or incorrect name for one of the packages.

Important notes:

• Publishing is possible only after commit when using this command in this form;

• The default is the pass-through version for all packages.

Recommended to honor See the Lerna documentation for additional options for this command.

In our project

If you want to use your own project structure, you can change it. As an example, consider our project universal documentation generator…

We decided to leave in the main module only the functionality of the stream manager, as well as the markdown finalizer, compatible with the main static site generators. In order to process the documentation files themselves, you need to connect the appropriate plugins.

Our file lerna.json:

{
  "packages": ["plugins/*", "main/"],
  "version": "0.0.0",
  "command": {
    "run": {
      "npmClient": "npm"
    }
  }
}

In our project, all plugins are in the directory plugins/, and the main module in the folder main/… Thus, the structure of our project is as follows:

├── lerna.json
├── LICENSE
├── main
│   ├── index.js
│   ├── lib
│   │   ├── finMd.js
│   │   └── manager.js
│   ├── package.json
│   └── README.md
├── package.json
├── plugins
│   ├── fin-html
│   │   ├── index.js
│   │   ├── package.json
│   │   └── README.md
│   └── gen-js-jsdoc
│       ├── index.js
│       ├── package.json
│       └── Readme.md
│       ...
└── README.md

Interaction of modules and dependencies

Let’s talk now about the interactions between different parts of the project. Obviously, in a mono-repository, each subproject can somehow depend on others. There is a problem of dependencies between subprojects. Also, it is not very convenient to set dependencies separately for each subproject. Lerna knows how to solve these problems.

Installing dependencies:

lerna bootstrap

When this command is executed, Lerna analyzes all subprojects and performs npm install in each of them. If in the dependencies of one local module (subproject) B there is another Athen Lerna creates a symbolic link in the folder node_modules/ subproject B per subproject A… Thus, there is no unnecessary copying of files in node_modules/… When working with mono repository, we have to run this command instead of installing dependencies in each subproject.

Recommended to honor See the Lerna documentation for additional options for this command.

Autodeploy with GitHub Actions

We created a monorepository, we were even able to publish it to NPM, but now the question arises: “How can you automatically submit changes to NPM?” To solve this problem, you can use GitHub Actions.

Create a folder in the root of the project .github/workflows/, and in it the file main.yml (name can be any). Consider at our example…

Content of our yml file:

name: autodeploy

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  default:
    runs-on: ubuntu-latest
    steps:
      - name: checkout
        uses: actions/checkout@v2

      - name: Install Packages
        run: npm install

      - name: Authenticate
        run: |
          echo "@grandcore:registry=http://registry.npmjs.org/" > .npmrc
          echo "registry=http://registry.npmjs.org/" >> .npmrc
          echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" >> .npmrc
          npm whoami
        env:
          NPM_TOKEN: ${{ secrets.NPMTOKEN }}

      - name: Publish
        run: lerna publish from-package --yes
        env:
          NPM_TOKEN: ${{ secrets.NPMTOKEN }}

What’s going on here:

• A commit or pull request to the master is set as a launch trigger;

• The operating system is Ubuntu;

• Next, npm packages are installed in the cloned repository;

• After that we add the NPM user. To do this, you need to create secret in a project with npm token… In our project, the token is kept in the NPMTOKEN secret.

• We publish using the appropriate command lerna publish from-package --yes… In our case, we update only those packages with which we manually changed the version in their files. package.json…

• You can track progress in the Actions tab in your repository.

---

We will be glad to hear your comments in the comments.

If the reader is interested in our open source projects, we will be glad to see you in our chat…