Having tried different Source Controls in conjunction with UE (Gitlab, SVN, Perforce) at the moment, for an indie project I chose the most convenient solution, although not the easiest to install:
Gitlab – To use it on your server, you need to have a Linux machine (I use mint), some time and patience to install the server and that’s it, we are happy owners of a free gitlab server. Next, we set up a connection via SourceTree and use;
Another charm of Gitlab, I would call the built-in tool for CI / CD, which is very easy to configure and allows one click of a button: Build a light, pack a game, upload it to steam, send a message to discord, etc., you just need a server with the UE installed. I’ll tell you about setting up CI under Windows.
Let’s skip the moment when to install the Gitlab / Source tree, create a new project and other items in it. Let’s assume that all this is already there and you need to configure the runner along with the CI.
Create a folder for it, for example: C: Gitlab-Runner, download the binary: 64-bit or 32-bit, drop it into the folder created earlier and change the name to: gitlab-runner.exe (more details). The main thing is to make sure that the folder has write access and you can run files from it. Run cmd to register and install Gitlab-runner.
First of all, we install and start the runner, below is an example, taking into account the cases when a username and password are needed to start a Windows service.
cd C:Gitlab-Runner .gitlab-runner.exe install --user ENTER-YOUR-USERNAME --password ENTER-YOUR-PASSWORD .gitlab-runner.exe start
Registering a new Gitlab-runner.
We go through a web browser to the Gitlab server, go to our new project. Further Settings-> CI / CD-> Runners and expand the tab, which will indicate the settings for the new server.
We open a command line on the server with Gitlab-runner installed and register a new runner for registering and fill in the requirements point by point, specifying the server URL, a unique token (it can be regenerated if necessary), name and tags as server identifiers. An important detail is with which executor we will process the yml file (I use shell) more details…
cd c:Gitlab-runner .Gitlab-runner.exe register
That’s it, the new Runner is registered, by refreshing the page we can see all the available servers. An important point for further work is to go to the server settings and install, without this checkbox, CI hangs in pending mode:
I will also add that in “General pipelines” you can change the timeout (by default 2 hours) to a large value (I have 16h), after this time the CI process is disabled.
In such a very simple way, we prepared our server for work, as a build machine for CI / CD.
Setting up yml and first launch
For CI / CD, you need to create a file (.gitlab-ci.yml) in the root of the project and upload it to the repository, thereby giving instructions for work. Gitlab will check the given script and if something goes wrong it will issue a yaml invalid error in the CI / CD. In order not to push dozens of broken files, right on the site in the CI / CD-> Pipelines tab there is a tool: “CI Lint”, with which you can check the script for correctness.
Since we previously indicated that we are using the Shell executor, the yml file is created according to the powershell principle (for other executors it may differ). Below I gave an example, assembling a client in two forms (dev / spike) and a server, as well as uploading solutions to Steam. As a bonus, sending a message about a successful build to the discord.
Sample script for .gitlab-ci.yml
# last update 10.04.2019 12:58PM # BUILD_CONFIG - Shipping / Development / DebugGame / Debug variables: GIT_STRATEGY: none # we disable fetch, clone or checkout for every job. GIT_CHECKOUT: "false" # as we only want to checkout and fetch in the preperation stage. CI_DEBUG_TRACE: "false" CLIENT_BUILD_CONFIG: Shipping DEV_CLIENT_BUILD_CONFIG: Development CLIENT_PLATFORM: Win64 SERVER_BUILD_CONFIG: Development SERVER_PLATFORM: Win64 BUILD_DIR: D:SoulhaimBuilds CLIENT_DIR: D:SoulhaimBuildsclientez-client CLIENT_STEAM_PATH: $CLIENT_DIRWindowsNoEditor* CLIENT_ARCH_NAME: $CLIENT_BUILD_CONFIG-$CI_COMMIT_SHA DEV_CLIENT_DIR: $BUILD_DIRdev_clientez-client DEV_CLIENT_STEAM_PATH: $BUILD_DIRdev_clientez-clientWindowsNoEditor* DEV_CLIENT_ARCH_NAME: $DEV_CLIENT_BUILD_CONFIG-$CI_COMMIT_SHA SERVER_DIR: $BUILD_DIRserverez-server SERVER_ARCH_NAME: $SERVER_BUILD_CONFIG-$CI_COMMIT_SHA LOCAL_ARCH_PATH: D:BuildsArchives STEAM_PATH: D:steamworks_sdk_148asdktoolsContentBuilder UE4_ROOT: D:UnrealEngine stages: - preperations - build_path - build_reflection - build_light - build_client - load_to_steam - upload_to_steam - pack_client - upload_client - clean_steam_client - dev_build_client - dev_load_to_steam - dev_upload_to_steam - dev_pack_client - dev_upload_client - dev_clean_client - build_server - pack_server - upload_server - clean_server - clean job_preperations: stage: preperations variables: GIT_STRATEGY: fetch GIT_CHECKOUT: "true" script: - echo "$CLIENT_STEAM_PATH" - echo "$STEAM_PATHcontentSoulhaim" - (Remove-Item -Path $BUILD_DIR -Force -Recurse -ErrorAction Ignore) - echo "Hello" - echo "$UE4_ROOT" - echo "$UE4_ROOTEngineBuildBatchFilesRunUAT.bat" - echo "$CLIENT_DIR" - echo "$CLIENT_BUILD_CONFIG" - echo "$CLIENT_ARCH_NAME" - echo "$DEV_CLIENT_DIR" - echo "$DEV_CLIENT_BUILD_CONFIG" - echo "$DEV_CLIENT_ARCH_NAME" - echo "$SERVER_DIR" - echo "$SERVER_ARCH_NAME" only: - web job_build_path: stage: build_path script: - Start-Process -NoNewWindow -Wait "D:Soulhaim_build_path.bat" only: - web job_build_reflection: stage: build_reflection script: - Start-Process -NoNewWindow -Wait "D:Soulhaim_build_Reflection.bat" only: - web job_build_light: stage: build_light script: - Start-Process -NoNewWindow -Wait "D:Soulhaim_build_light.bat" only: - web job_build_client: stage: build_client script: #-ForceDebugInfo -CrashReporter - for debug files - Start-Process "$UE4_ROOTEngineBuildBatchFilesRunUAT.bat" -NoNewWindow -Wait -ArgumentList "BuildCookRun", "-project=$CI_PROJECT_DIRSoulhaim.uproject", "-nop4", "-build", "-cook", "-compressed", "-stage", "-ForceDebugInfo", "-allowcommandletrendering", "-platform=$CLIENT_PLATFORM", "-clientconfig=$CLIENT_BUILD_CONFIG", "-pak", "-archive", "-archivedirectory=$CLIENT_DIR", "-utf8output" only: - web job_Load_To_Steam: stage: load_to_steam script: cp -r "$CLIENT_STEAM_PATH" "$STEAM_PATHcontentSoulhaim" only: - web job_Upload_To_Steam: stage: upload_to_steam script: - Start-Process "D:steamworks_sdk_148asdktoolsContentBuilderrun_build.bat" -NoNewWindow -Wait only: - web job1_pack_client: stage: pack_client script: - Start-Process -NoNewWindow -Wait "C:Program Files7-Zip7z.exe" -ArgumentList "a", "$CLIENT_DIR.zip", "$CLIENT_DIR" only: - web job2_copy_to_local_client: stage: upload_client script: - cp "$CLIENT_DIR.zip" "$LOCAL_ARCH_PATHclient-$CLIENT_ARCH_NAME.zip" only: - web job3_clean_Steam_Folder: stage: clean_steam_client script: (Remove-Item -Path "$STEAM_PATHcontentSoulhaim*" -Force -Recurse -ErrorAction Ignore) only: - web dev_job_build_client: stage: dev_build_client script: #-ForceDebugInfo -CrashReporter - for debug files - Start-Process "$UE4_ROOTEngineBuildBatchFilesRunUAT.bat" -NoNewWindow -Wait -ArgumentList "BuildCookRun", "-project=$CI_PROJECT_DIRSoulhaim.uproject", "-nop4", "-build", "-cook", "-compressed", "-stage", "-ForceDebugInfo", "-CrashReporter", "-debug", "-allowcommandletrendering", "-platform=$CLIENT_PLATFORM", "-clientconfig=$DEV_CLIENT_BUILD_CONFIG", "-pak", "-archive", "-archivedirectory=$DEV_CLIENT_DIR", "-utf8output" only: - web job_dev_Load_To_Steam: stage: dev_load_to_steam script: cp -r "$DEV_CLIENT_STEAM_PATH" "$STEAM_PATHcontentSoulhaim" only: - web job_dev_Upload_To_Steam: stage: dev_upload_to_steam script: - Start-Process "D:steamworks_sdk_148asdktoolsContentBuilderrun_build.bat" -NoNewWindow -Wait only: - web job1_dev_pack_client: stage: dev_pack_client script: - Start-Process -NoNewWindow -Wait "C:Program Files7-Zip7z.exe" -ArgumentList "a", "$DEV_CLIENT_DIR.zip", "$DEV_CLIENT_DIR" only: - web job2_dev_copy_to_local_client: stage: dev_upload_client script: - cp "$DEV_CLIENT_DIR.zip" "$LOCAL_ARCH_PATHclient-$DEV_CLIENT_ARCH_NAME.zip" only: - web job_clean_client: stage: dev_clean_client script: (Remove-Item -Path $BUILD_DIR -Force -Recurse -ErrorAction Ignore) only: - web job_build_server: stage: build_server script: - Start-Process "$UE4_ROOTEngineBuildBatchFilesRunUAT.bat" -NoNewWindow -Wait -ArgumentList "BuildCookRun", "-nocompileeditor", "-project=$CI_PROJECT_DIRSoulhaim.uproject", "-nop4", "-build", "-cook", "-compressed", "-stage", "-noclient", "-server", "-serverplatform=$SERVER_PLATFORM", "-serverconfig=$SERVER_BUILD_CONFIG", "-pak", "-archive", "-archivedirectory=$SERVER_DIR", "-utf8output" only: - web job1_pack_server: stage: pack_server script: - Start-Process -NoNewWindow -Wait "C:Program Files7-Zip7z.exe" -ArgumentList "a", "$SERVER_DIR.zip", "$SERVER_DIR" only: - web job2_copy_to_local_server: stage: upload_server script: - cp "$SERVER_DIR.zip" "$LOCAL_ARCH_PATHserver-$SERVER_ARCH_NAME.zip" only: - web job3_unpack_local_server: stage: upload_server script: - Start-Process -NoNewWindow -Wait "C:Program Files7-Zip7z.exe" -ArgumentList "x", "$LOCAL_ARCH_PATHserver-$SERVER_ARCH_NAME.zip", "-oD:" only: - web job4_send_notify_server_to_slack: stage: upload_server script: - Start-Process "D:Soulhaim_Message_Discord.bat" "New Awesome build is ready" -NoNewWindow -Wait only: - web job_clean_server: stage: clean_server script: (Remove-Item -Path $BUILD_DIR -Force -Recurse -ErrorAction Ignore) only: - web job_clean: stage: clean script: (Remove-Item -Path "$STEAM_PATHcontentSoulhaim*" -Force -Recurse -ErrorAction Ignore) only: - web
At the first stage (preperation), we clear the folder from the old build, and also take all the latest changes from the repository. It is advisable to make only this stage as the first launch, since the next steps are related to the project and we need a path to it.
The next steps are to calculate navmesh light reflection, the scripts of which are located in regular * .bat files. An example for rendering light on two maps: Through the “+” you can specify a list of maps.
Separately, it is necessary to add that if you are working with a binary version, then it is advisable to calculate navmesh light reflection in the same way on the binary version, otherwise, on the version from the source, the engine will require you to rebuild the solution, and it will get stuck in the process (perhaps there is a solution , but unfortunately I didn’t find it).
"C:Program FilesEpic GamesUE_4.26EngineBinariesWin64UE4Editor-Cmd.exe" "D:GitLab-RunnerbuildsComtV2y9 SoulhaimSoulhaimSoulhaim.uproject" -nop4 -run=resavepackages -buildlighting -quality=Preview -allowcommandletrendering -map=LobbyMap+GameMap
Next, step by step, the assembly will take place Shipping client-> Steam, Development client-> Steam, Server, archive all this and place it in a separate folder in the form of client / server-Dev / ship-hash.zip, if there are a lot of builds, then the time from time to time it will be necessary to clean the folder with archives, or simply remove these steps from CI.
An important point, since the process is automatic, under Steam you need to create a separate user with the rights to upload the build, and without any two-factor protection, otherwise the process will hang waiting for a confirmation code that will have nowhere to drive.
Launching is extremely simple, go to Ci / CD-> Pipelines and press the big green RunPipeline button, select which branch we want to build from and start the process. Then, if all is well, it remains to sit back in a comfortable chair with tea and wait until the process reaches the end (or fails). Any stage can be opened and problems can be investigated, if any.
Bonus for sending messages to discord
First, go to the discord channel settings-> integration-> Webhooks, create a new one and copy its URL, and then just paste the message and link into the bat:
curl -d "content=Build is ready: Soulhaim" -X POST https://вебхук вашего канала в дискорде
That’s actually the whole process of setting up CI CD in Gitlab, you can read more on the official website in the documentation, but I tried to describe the process in detail with examples. I hope the article will be useful to you, and I did not forget to indicate anything important in it.
Thank you all for your attention and may the power of knowledge be with you.