Mirroring GitHub projects in 2023

For a number of reasons, I decided to mirror your public GitHub projects to other collaborative development platforms. It turned out to be not so easy to do this. This short article describes the difficulties I had to face and the final working solution.

Illustration by John Tenniel
Illustration by John Tenniel

Previously, I used the mirror function GitLab, which is called pull mirroring. She periodically copied code, tags and discussions in issues and pull requests from my GitHub repositories to GitLab. But some time ago, I noticed that my GitLab mirrors started to lag behind the main repositories. I logged into GitLab and saw this nasty banner:

The pull mirroring feature has disappeared from some of my repositories. It is no longer possible to enable it for free:

Buying GitLab premium support (Premium Tier) is not possible in Russia. GitLab forbids even its free trial. So I started looking for other solutions to mirror my GitHub projects:

  • Deploy your standalone GitLab development server, or Forgejo didn’t work for me: I didn’t want to host my open-source projects on an isolated server. This would significantly limit interaction with the community and be an obstacle for contributors.

  • I checked the Chinese platform Giteeand I didn’t like its limited English localization support.

  • I looked at radical, peer-to-peer network for software development. But its “powerful functionality based on the blockchain” seemed too radical to me.

  • Then I looked at codeberg. This collaborative development platform is supported by a non-profit organization of the same name that promotes free software ideas. I like these guys much more than Microsoft owning GitHub. But in March 2020 they disabled the mirroring feature on Codeberg servers due to lack of resources. According to them, “mirrored repositories are easy to create, but consume resources forever” 🙁

  • I also checked sourcehut (Thank you paulmairo for the link). This platform did not suit me because:

    1. It provides only paid services.

    2. The development process on it looks incompatible with GitHub: on the SourceHut platform, email is used to work with tasks and project code.

  • I looked at salsa (Thank you Mic_92 for the idea). This is a Debian Community Collaboration Server that runs on the GitLab software. First I registered an account there. A few days later, the Salsa admin activated it, and I was able to copy one of my projects from GitHub to Salsa. But it turned out that the pull mirroring function in Salsa is also disabled, as well as on gitlab.com in free mode. I AM asked a question about it in their tracker, but, unfortunately, did not receive any response.

  • Then I created mirrors for their GitHub projects on gitflic. This is a small Russian platform for collective development, which is now actively developing. Their support team allowed me to create public repositories and set up mirroring. In addition, they quickly answered questions and gave comments on the problems found. Unfortunately, there is no CI in GitFlic, and there is even no way to copy information from issues and pull requests from GitHub. Obviously mirrors on GitFlic weren’t the final solution for me.

GitHub Mirroring Scheme -> GitFlic” title=”GitHub Mirroring Scheme -> GitFlic” width=”845″ height=”421″ data-src=”https://habrastorage.org/getpro/habr/upload_files/657/97e/630/65797e6304a3a3d17188aeb7528ddef8.png”/><figcaption>GitHub Mirroring Scheme -> GitFlic</figcaption></figure><p>By the way, I would suggest to the GitFlic team to name the Issues section for projects on the platform not “Issues”, but “Issues”.</p><p>To sum it up, I haven’t found any popular collaborative development platform that can provide full-featured mirrors of my GitHub projects.  Therefore, I decided to look at this task from a different angle: how can I manually back up the discussions in my GitHub projects?</p><div class='code-block code-block-2' style='margin: 8px 0; clear: both;'> <script type=

The first idea was to make this information part of the code. Soon I found a project gh2mdwhich allows it. The gh2md tool dumps the contents of the GitHub issues and pull requests of a given project into a Markdown document.

Under the hood, this tool uses the GraphQL interfaces provided by GitHub, so I had to generate a GitHub personal access token with minimal privileges for it. Details on working with such tokens can be found in GitHub documentation.

GitHub access token
GitHub access token

Now my projects contain the file issues.md with a backup of all tasks and discussions. One could use gh2md in CI to update issues.md automatically. But for now, I refrain from giving GitHub Actions scripts full access to the repository.

At some point, I came up with the idea of ​​a CI script that would generate a pull request with file updates issues.md. But I soon realized that this is not a very smart idea: such an automatically generated request will lead to the next triggering of the CI script for updating issues.mdwhich will create another pull request, and so on 🙂 I decided not to invent problems for myself and added a manual update issues.md to the list of release procedures for their projects.

There is another way: you can upload tasks and discussions from GitHub to the project’s git repository. This can be done using a special tracker git bug (Thank you Sergei Bronnikov per link):

git-bug project
git-bug project

In the end, after backing up the information from GitHub issues and pull requests, I decided to back up my projects on Codeberg and then just perform the operation git push simultaneously for both GitHub and Codeberg. The Codeberg platform successfully performed a one-time migration of my projects from GitHub, even copied all the discussions and tasks. So this would be the perfect solution for me if mirroring was enabled in Codeberg… Alas!

Then the question arose before me: what to do with outdated information in Codeberg issues? As a last resort, you could manually delete and recreate projects on Codeberg to keep up to date, but I did not like this solution at all.

GitHub Mirroring Scheme -> Codeberg” title=”GitHub Mirroring Scheme -> Codeberg” width=”845″ height=”421″ data-src=”https://habrastorage.org/getpro/habr/upload_files/6d3/047/f63/6d3047f6377d695d15a8da2daa5ea74c.png”/><figcaption>GitHub Mirroring Scheme -> Codeberg</figcaption></figure><p>I began to study the settings of the repositories and found a workaround: Codeberg allows you to use an external issue tracker!  I’ve set up a GitHub issue link and numbering format, disabled codeberg pull requests, and now my project mirrors just link to GitHub issues and pull requests.</p><figure class=Setting up the Codeberg repository
Setting up the Codeberg repository

This is more or less a working solution. If something goes wrong with GitHub, I will recreate the repositories in Codeberg, include an internal tracker and pull requests in them, and then announce to the community and GNU/Linux distributions that the development of my open source projects has been transferred to this platform.

Final GitHub mirroring scheme -> Codeberg” title=”Final GitHub mirroring scheme -> Codeberg” width=”845″ height=”421″ data-src=”https://habrastorage.org/getpro/habr/upload_files/7ed/36c/537/7ed36c5372acad154354b79a001f7d04.png”/><figcaption>Final GitHub mirroring scheme -> Codeberg</figcaption></figure><p>Thank you for your attention.  I will be glad to comments and suggestions.</p><p> <script type=

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *