Cocoapods pod install without internet on CI using Tuist

In some companies, for security reasons, continuous integration runners do not have Internet access. And for successful execution of the command pod install need access to GitHub repository with subspecs to download them to a local directory. But for iOS development of a project that uses Cocoapods as a dependency manager, you don’t have to use the command pod install on CI, if the entire Pods folder is fully tracked by Git and Pods are integrated into all projects.

However, when used in a project Tuist (a very useful tool for reducing merge conflicts that is quickly gaining popularity today) xcodeproj files will be generated anew each time on CI, naturally, without the Pods integrated into them. In this case, you need to call the command pod install after tuist generation for the purpose of integrating Pods into xcodeproj projects. But even though all the Pod sources are in place, the team pod install will give an error without the Internet, without introducing Pods into our projects.

There is a solution to this problem: the team pod install you won’t need the Internet if there is a valid and unexpired one trunk specs repo with all the subspecs of the libraries used in the project from the GitHub repository in the directory ~/.cocoapods/repos/trunk. The implementation of such a solution will be discussed in this article.

Stages of the pod install command

Team pod install performs several sequential steps to install Pods:

1. Analyzing dependencies – the longest stage, has many substages. Here, dependencies for targets are analyzed by Podfile, and sub-specs of the frameworks used are downloaded from the centralized Specs repository from Github in the sub-stage Resolving dependencies of `Podfile`. It is important that if all the necessary subspecies are already available in the local directory ~/.cocoapods/repos/trunk, then the download does not occur.

2. Downloading dependencies – Pod sources are downloaded if any are missing according to the previous analysis. If everything is enough, then nothing will be downloaded.

3. Generating Pods project – the Pods project Pods.xcodeproj and service files in the folder are generated Pods/Target Support Filessuch as xcconfig files needed for linking pods with our targets.

4. Integrating client projects – integration of Pods into the targets of our xcodeproj projects.

There are cases when we need to execute the last two stages of this command. For example, the 4th stage of integrating Pods into targets is needed when using Tuist in a project. But if there is no Internet, the command fails with a CDN error at the first stage (Analyzing dependencies) on CI. This error symbolizes the impossibility of both finding the necessary files locally and downloading them by URL https://cdn.cocoapods.org/. Depending on whether the trunk folder is missing completely, or whether it is missing some file or subspecies for the current Podfile configuration, the wording of the error will be different, but the reason for its appearance is the same.

There are many cases of need or desire for successful command execution pod install on CI without a network, even in the case of tracking all Pod sources in the Pods directory:

• Executing the 3rd stage of the command can be useful because Cocoapods during it generates a Project and a lot of service files in the folder Pods/Target Support Files, the number of which can reach several thousand depending on the size of the project, the number of targets and configurations. This means you can upload project files to .gitignore Pods.xcodeproj and the whole folder Pods/Target Support Files, so that they are regenerated on CI. This approach will help avoid merge conflicts in these files and numerous changes to them every time the Pods are updated.

• During the 4th stage, the integration of Pods into our projects occurs, and this is necessary when using tuist after their generation. Integrating Pods into projects involves: attaching xcconfig files generated by Cocoapods to each target, adding a Pods framework for targets, adding build scripts to the build phases of tagrets, such as “Embed Pods Frameworks” and “Check Pods Manifest.lock”.

• Some useful plugins for Cocoapods can only be used if pod install is successful. For example, plugins such as cocoapods-binary or cocoapods-xcremotecachecaching-related functions are executed in the pre-/post-install script during the pod install command. The turn of such scripts occurs only after the 2nd or 3rd stage of the command. Therefore, due to a command failure at the very beginning, such plugins will never be executed.

Solution

As mentioned above, if all the necessary files and subspecs are already in the local directory ~/.cocoapods/repos/trunk, then they are not downloaded during the 1st stage of the command pod installand therefore it does not need Internet access.

This way we can simply copy our local folder to the current one trunk from ~/.cocoapods/repos/trunk directly into the project, and on CI before the command pod install move this folder to the runner directory without a network, i.e. V ~/.cocoapods/repos/.

Locally, at home: Being in the root directory of the project, after successful execution pod install copy the folder trunk somewhere in the project, for example, in the root. Archive it so that there is one archive file trunk.zipnot a folder with thousands of files and subspecs.

 cp -r ~/.cocoapods/repos/trunk $PWD
 zip -rm trunk.zip trunk >/dev/null

On CI without internet: Before pod install just in case, delete the existing folder trunk, if such a thing exists. Then unzip our folder trunk.zip and move it to the directory ~/.cocoapods/repos/ on the runner. Next it is called pod install and works successfully without accessing the network, since now everything podspecs available locally in trunk.

build_job:
  script:
    - pod repo remove trunk || echo trunk removed already
    - unzip trunk.zip -d ~/.cocoapods/repos >/dev/null

    - tuist generate -n     # only if you use tuist 
    - pod install
    - xcodebuild ......     # build project

With this approach, the folder archive trunk.zip will become outdated and will need to be updated only when the version of some Pod is updated or when a new Pod is added that is not in this trunk. That is, an update of the old trunk.zip a new one will be needed in cases where new podspecs are added to the trunk that are not in the old trunk.

The advantage of this approach is the decentralization of storage trunk with subspecs for different branches, since trunk.zip is not stored in one place for all branches, but directly in the project for a specific Podfile configuration. Thus, trunk.zip with subspecs of some Pods needed for this branch can be stored on one branch, and trunk.zip with subspecs of completely different Pods suitable for this branch can be stored on another branch. And for both of them it will be successfully completed pod install on CI.

Improving this approach

Anyway update trunk.zip will be needed only after the next pod install or pod update in order to update the Pods. Therefore, you can automate the copying of new trunk.zip with subsequent Pod updates, by placing local commands in the Cocoapods provided pre-/post-hooks in the Podfile, for example in the post_integrate function:

post_integrate do
  if ENV['UPDATE_TRUNK'] == "true"
    puts "Copying and zipping trunk from ~/.cocoapods/repos"
    system("cp -r ~/.cocoapods/repos/trunk $PWD")
    system("zip -rm trunk.zip trunk >/dev/null")
  end
end

If desired, you can supplement the commands with a condition under which the update trunk.zip will only be executed if the variable UPDATE_TRUNK == true. This check is needed so that the update trunk.zip did not occur on CI after pod install, and also locally once again when it is not needed. In this form for updating trunk.zip you just need to set the variable UPDATE_TRUNK before the next one pod install/update:

export UPDATE_TRUNK=true && pod install
                or
export UPDATE_TRUNK=true && pod update
                or
export UPDATE_TRUNK=true; pod install --repo-update

Cocoapods creates a repo folder trunk by default in the considered directory to accumulate downloaded subspecs in it. Moreover, subspecies of libraries used in other projects are downloaded to the same trunk folder. And in principle, for this approach it’s okay that when copying a trunk it will contain extra sub-specs, the main thing is that it contains sub-specs used in the current project. But if you wish, you can delete the current one trunk before the next one pod install and a new clean trunk will be created during the execution of the command. This may reduce the size trunk.zip, since it will not contain unnecessary sub-specs before copying it into the project.

pod repo remove trunk
pod install
cp -r ~/.cocoapods/repos/trunk $PWD
zip -rm trunk.zip trunk >/dev/null

Also, if you don’t want to delete the current local trunk, you can create the same repo folder to download subspecs of libraries only used in the current project into it. First, rename the current trunk. Then we create a CDN trunk using the command pod repo add-cdnpassing the name into it trunk and url. And after downloading the subspecs to the new trunk during pod install and archiving it into the project, delete the new one trunk and rename back your native one trunk:

mv ~/.cocoapods/repos/trunk ~/.cocoapods/repos/trunk_backup
pod repo add-cdn trunk https://cdn.cocoapods.org/
pod install
cp -r ~/.cocoapods/repos/trunk $PWD
zip -rm trunk.zip trunk >/dev/null
pod repo remove trunk
mv ~/.cocoapods/repos/trunk_backup ~/.cocoapods/repos/trunk

Result

As stated earlier, during the command pod install on CI in substage Resolving dependencies of `Podfile` instead of downloading subspecs from a turnip, Cocoapods returns and processes local files already in trunk. This can be observed in more detail by calling the command pod install –verbose.

The screenshot shows that before downloading each subspec .podspec.json, a text file with the prefix is ​​first downloaded all_pods_versions and with three characters at the end. This is due to the CDN engine used by Cocoapods for optimization. Cocoapods first takes the name of the Pods specified in the Podfile, calculates SHA hashes from the names using md5, and takes the first three characters from these hashes. For example, for Pod InputMask the md5 hash is 7c16b23066345d6d2fed8e232ce756. So, Cocoapods will take the 7c1 prefix from the hash and download the file all_pods_versions_7_с_1.txt from turnip CDN. This file contains all versions of all Pods whose first 3 hash characters from the name will match our InputMask, that is, also 7c1. Cocoapods will download subspecs for all versions of our Pod, making https requests like – https://cdn.cocoapods.org/Specs/7/c/1/InputMask/1.0.0/InputMask.podspec.json. As you can see, in the InputMask repository the library is stored along the path Specs/7/с/1/InputMask. And along the same path all subspecs for it will be stored in our local directory in the trunk after downloading. This mechanism has been used by Cocoapods since 2019 to significantly speed up the execution of the pod install command, since it is no longer necessary to do a full clone of a huge Specs repository. Instead, only the Podspecs needed for our entire dependency tree are loaded.

Conclusion

We looked at the method of resuming the successful execution of a command pod install on runners without Internet access, but with existing Pods in the project. A method was proposed in which you need to copy a valid and up-to-date trunk on the developer’s local machine with the Internet directly into the project in the form trunk.zipand then on a runner without the Internet, copy trunk from the project to a folder ~/.cocoapods/repos. We discussed possible additional, remaining in the shadows, cases when calling a command pod install needed for CI, such as the need to integrate Pods into projects or the desire to generate service Pods files anew each time, placing them in gitignore. We analyzed in detail the stages of command execution pod installhighlighting stages that are useful to us.