Mirroring GitHub projects in 2023
For a number of reasons, I decided to mirror your public GitHub projects to other collaborative development platforms. It turned out to be not so easy to do this. This short article describes the difficulties I had to face and the final working solution.
Previously, I used the mirror function GitLab, which is called pull mirroring. She periodically copied code, tags and discussions in issues and pull requests from my GitHub repositories to GitLab. But some time ago, I noticed that my GitLab mirrors started to lag behind the main repositories. I logged into GitLab and saw this nasty banner:
The pull mirroring feature has disappeared from some of my repositories. It is no longer possible to enable it for free:
Buying GitLab premium support (Premium Tier) is not possible in Russia. GitLab forbids even its free trial. So I started looking for other solutions to mirror my GitHub projects:
Deploy your standalone GitLab development server, or Forgejo didn’t work for me: I didn’t want to host my open-source projects on an isolated server. This would significantly limit interaction with the community and be an obstacle for contributors.
I checked the Chinese platform Giteeand I didn’t like its limited English localization support.
I looked at radical, peer-to-peer network for software development. But its “powerful functionality based on the blockchain” seemed too radical to me.
Then I looked at codeberg. This collaborative development platform is supported by a non-profit organization of the same name that promotes free software ideas. I like these guys much more than Microsoft owning GitHub. But in March 2020 they disabled the mirroring feature on Codeberg servers due to lack of resources. According to them, “mirrored repositories are easy to create, but consume resources forever” 🙁
I also checked sourcehut (Thank you paulmairo for the link). This platform did not suit me because:
It provides only paid services.
The development process on it looks incompatible with GitHub: on the SourceHut platform, email is used to work with tasks and project code.
I looked at salsa (Thank you Mic_92 for the idea). This is a Debian Community Collaboration Server that runs on the GitLab software. First I registered an account there. A few days later, the Salsa admin activated it, and I was able to copy one of my projects from GitHub to Salsa. But it turned out that the pull mirroring function in Salsa is also disabled, as well as on gitlab.com in free mode. I AM asked a question about it in their tracker, but, unfortunately, did not receive any response.
Then I created mirrors for their GitHub projects on gitflic. This is a small Russian platform for collective development, which is now actively developing. Their support team allowed me to create public repositories and set up mirroring. In addition, they quickly answered questions and gave comments on the problems found. Unfortunately, there is no CI in GitFlic, and there is even no way to copy information from issues and pull requests from GitHub. Obviously mirrors on GitFlic weren’t the final solution for me.
The first idea was to make this information part of the code. Soon I found a project gh2mdwhich allows it. The gh2md tool dumps the contents of the GitHub issues and pull requests of a given project into a Markdown document.
Under the hood, this tool uses the GraphQL interfaces provided by GitHub, so I had to generate a GitHub personal access token with minimal privileges for it. Details on working with such tokens can be found in GitHub documentation.
Now my projects contain the file issues.md
with a backup of all tasks and discussions. One could use gh2md in CI to update issues.md
automatically. But for now, I refrain from giving GitHub Actions scripts full access to the repository.
At some point, I came up with the idea of a CI script that would generate a pull request with file updates issues.md
. But I soon realized that this is not a very smart idea: such an automatically generated request will lead to the next triggering of the CI script for updating issues.md
which will create another pull request, and so on 🙂 I decided not to invent problems for myself and added a manual update issues.md
to the list of release procedures for their projects.
There is another way: you can upload tasks and discussions from GitHub to the project’s git repository. This can be done using a special tracker git bug (Thank you Sergei Bronnikov per link):
In the end, after backing up the information from GitHub issues and pull requests, I decided to back up my projects on Codeberg and then just perform the operation git push
simultaneously for both GitHub and Codeberg. The Codeberg platform successfully performed a one-time migration of my projects from GitHub, even copied all the discussions and tasks. So this would be the perfect solution for me if mirroring was enabled in Codeberg… Alas!
Then the question arose before me: what to do with outdated information in Codeberg issues? As a last resort, you could manually delete and recreate projects on Codeberg to keep up to date, but I did not like this solution at all.
This is more or less a working solution. If something goes wrong with GitHub, I will recreate the repositories in Codeberg, include an internal tracker and pull requests in them, and then announce to the community and GNU/Linux distributions that the development of my open source projects has been transferred to this platform.