How we set up CI in two steps

I work as a junior developer in VK’s internal mobile development department. When I joined the team, we didn’t have CI. At the same time, we had seven applications in one repository, and with each update we had to collect them separately, spending a lot of time and effort on this. I decided to automate the build by writing a human CI. And this is his story.

First run

First, we needed to check commits and pull requests to the dev branch. Secondly, when adding pull requests, we wanted to receive apk files for tests (and upload them to the Firebase App Distribution), and when added to master, we wanted to receive aab files for the app store. In general, the task looked simple, and I sat down to implement it.

It turned out exactly the way I wanted. But since we have a lot of applications (flavors), and Mac Mini of 2014 was allocated for CI, everything ended quite naturally: even on a more recent and powerful Mac, the assembly was completed in 5-8 minutes, and on Mini we received application files … only after 2 hours. Obviously not the result we wanted.

Second entry

The overall CI pipeline had to stay the same, just needed to speed up the build of the required flavors. We also wanted to automate version raising, which for 7 applications was not the most trivial task. I started looking for a solution. More than ever, by the way, on the Android Academy Global channel came out report on the topic of CI, which helped me find the idea and start development.

Development process

To begin with, I fixed the most requested: a commit to the task branch and a pull request with a feature to the dev branch. In general, the assembly of the main flavor was already implemented, but just in case, I also added the assembly of a random flavor so that it would not take a long time to fix errors arising from resources: for example, if someone added the required flavor for all flavors file to one of them.

script:
 - |
   COMMANDS=("./gradlew --no-daemon --stacktrace assembleCollageDebug"
   ...) # тут остальные команды
 - $(shuf -n1 -e "${COMMANDS[@]}")

Version uplift

There are many solutions in Java and Python on the web, but they are all quite cumbersome, so it did not make sense to contribute them to a repository or clone them. All of our versions are numbered according to the template .., and we decided to write a Gradle task that raises the desired version fragment. Versions of all flavors have been added to the versions.properties file. The task takes the version and number of the desired flavor, increases it by one, and raises the required fragment from the version. It looks like this:

– файл version.properties до –
…
app_code=1260
app_name=2.0.60
…

– вызов команды –
./gradlew bumpVersion --flavor app --field minor

– файл после –
…
app_code=1261
app_name=2.1.0
…

We write the version name in a separate file, and then we create a branch from it release/flavor version. Then we make a commit and send it to this branch, the versions are already raised. You can see more about commit and push with GitLab CI here.

Trigger per task

Now it was necessary to set up a task that would work on a trigger and perform the actions described above, because we don’t want to merge and create branches manually. We wanted to add this functionality to the chatbot we had, so I started looking at Gitlab for information about triggers. This issue is well described in the documentation, but you may not see something you need the first time. The result is a query like this:

request = requests.post(f'https://self-hosted-gitlab.com/api/v4/projects/{project_id}/trigger/pipeline/', data={
   "token": token,
   "variables[flavor]": flavor,
   "variables[field]": field,
   "ref": 'dev'
}).json()

And in gitlab-ci.yml the condition looks like this:

only:
 variables:
   - $CI_PIPELINE_SOURCE == "trigger" && $CI_COMMIT_BRANCH == "dev" && $flavor == "app""

This is where all the magic happens.

Pull requests

After we have raised the version and pushed the code to the branch, another task should work out, which creates pull requests in dev and master. To do this, we run the GitLab API methods associated with pull (merge) requests and perform the task:

release_merge_request:
 tags:
   - android
 stage: release
 script:
   - 'curl --request POST
           --form title="Release [`cat ./ci/app_name.txt`]"
           --form id=`echo ${CI_PROJECT_ID}`
           --form ref=`echo $CI_COMMIT_BRANCH`
           --form source_branch=`echo $CI_COMMIT_BRANCH`
           --form target_branch=master
           --form remove_source_branch=true
           --form assignee_id=1809
           --form private_token=token
         "https://gitlab.corp.mail.ru/api/v4/projects/${CI_PROJECT_ID}/merge_requests"'
   - 'curl --request POST
           --form title="Release [`cat ./ci/app_name.txt`]"
           --form id=`echo ${CI_PROJECT_ID}`
           --form ref=`echo $CI_COMMIT_BRANCH`
           --form source_branch=`echo $CI_COMMIT_BRANCH`
           --form target_branch=dev
           --form remove_source_branch=true
           --form assignee_id=1809
           --form private_token=token
         "https://gitlab.corp.mail.ru/api/v4/projects/${CI_PROJECT_ID}/merge_requests"'
 rules:
   - if: '$CI_COMMIT_BRANCH =~ /^release.*/ && $CI_PIPELINE_SOURCE == "push"'

This is an intermediate step to:

  1. do not create manually pull requests later;

  2. submit the required build to the app store.

Let me tell you about the second point.

Publication

We create another task that will be triggered on a pull request from the release branch in dev (or master, because pull requests are the same there) – exactly in the pull request, so that the task that collects and publishes the desired application is launched, in depending on the title given in the title.

Now we need to submit the new version to the app store. For this we use Gradle Play Publisher plugin. Using this plugin, we upload the assembly to the store for internal testing, then we copy the same release to the release (if there are alpha / beta tests, you can first send it there, as you wish). The chain looks like this:

  1. A pull request is created.

  2. In relation to it, a task is launched that collects the app bundle and runs a Gradle task that publishes the bundle.

  3. Just in case, we also save it as an artifact in Gitlab.

app_release_bundle:
 variables:
   GIT_CHECKOUT: "true"
 tags:
   - android
 stage: bundle
 script:
   - echo $APP_KEYSTORE | base64 -d > app.jks
   - export FIREBASE_TOKEN=`echo $FIREBASE_CI_TOKEN`
   - ./gradlew --no-daemon bundleWorldRelease
     -Pandroid.injected.signing.store.file=$(pwd)/app.jks
     -Pandroid.injected.signing.store.password=$APP_PASSWORD
     -Pandroid.injected.signing.key.alias=$APP_ALIAS
     -Pandroid.injected.signing.key.password=$APP_KEY_PASSWORD
   - mkdir release
   - cp app/build/outputs/bundle/worldRelease/app-world-release.aab app/release/appRelease.aab
   - ./gradlew --no-daemon publishWorldReleaseBundle
 rules:
   - if: '$CI_COMMIT_BRANCH =~ /^release.*/ && $CI_PIPELINE_SOURCE == "merge_request_event" && $CI_MERGE_REQUEST_TITLE =~ /\[(app)]/'
 artifacts:
   expire_in: 3 days
   paths:
     - app/release/appRelease.aab

And a bit more

It’s no secret that a lot of things can be automated on CI, not just checks and releases. For example, translations. We already had a program that downloaded translations from the cloud and produced an .xml file (or a file for iOS, it was enough to specify the platform in the launch argument). In the end, I decided to bring the translations out too. The result is a task that takes a script for translations, copies the necessary files to the right place and sends the result to the repository. It remains only if necessary to add this to your branch. The code:

make_new_translate:
 image: python:latest
 cache: []
 variables:
   GIT_CHECKOUT: "true"
 tags:
   - android
 stage: translate
 before_script:
   - 'which ssh-agent || ( apt-get update -y && apt-get install openssh-client -y )'
   - eval $(ssh-agent -s)
   - echo "${SSH_PRIVATE_KEY}" | tr -d '\r' | ssh-add - > /dev/null
   - mkdir -p ~/.ssh
   - ssh-keyscan self-hosted-gitlab.com >> ~/.ssh/known_hosts
 script:
   - cd
   - git clone ...
   - cd (склонированный проект)
   - pip install -r requirements.txt
   - python translate.py # + параметры запуска
   - (cd ./Build/android/ && tar c .) | (cd /builds/$CI_PROJECT_PATH/app/src/main/res && tar xf -) # копируем в проект
   - # аналогичные действия проделываем с модулями, если есть
   - cd /builds/$CI_PROJECT_PATH
   - git add ./app/src/main/res
   - # + модули
   - git config --global user.email "" # что-нибудь, можно и почту автора
   - git config --global user.name "[CI]"
   - git checkout -B translations/all-`echo $CI_JOB_ID`
   - git commit -m 'Translations'
   - git push git@self-hosted-gitlab.com:group/repo.git HEAD:translations/all-`echo $CI_JOB_ID`
 rules:
   - if: '$CI_PIPELINE_SOURCE == "trigger" && $CI_COMMIT_BRANCH == "dev" && $translate == "true"'

Summary

Although the implementation of this CI is not something complicated, however, thanks to it, we save time on routine tasks, such as raising versions, building release artifacts and publishing them to the application store, and so on (which, in general, was a fairly predictable result ).

91

Similar Posts

Leave a Reply