Xcode logs - read on your own?

In this publication, we will try to understand how and where xcode keeps its logs, what is SLF0 and how to read all this, and maybe even understand, and better without IDEthat’s more interesting.

No extra preamble.

To see the build logs of a project or, say, tests, in normal, everyday development, we go and look at the last tab of the toolbar xcode.

The organization of the log as a whole is readable, it is even possible to export all this to txt format.

But, all this is not suitable for us to automate the collection of metrics, for example, to accumulate the dynamics of the assembly time of modules during the run job‘s on CI.

There is no easy way to export this log in full for subsequent interpretation of specific events.

So all logs xcode collects and stores DerivedData/{ProjectName}/Logs.

What is DerivedData

By default it is located in the following path:

open ~/Library/Developer/Xcode/DerivedData

This path can be changed in the Xcode settings:

Setting paths for DerivedData and archives

Actually, DerivedData stores all intermediate information (including logs), as well as artifacts required to optimize the subsequent build.

Inside a folder Logs you can find many logs of various stages that we encounter during development:

These are, in fact, the logs (you may have more of them)

Here we can find the following kinds of files:

*.xcactivitylog – the actual magazine that we will interest Further;
*.xcresult – bundleA that stores summary information. Can be opened in xcode and view the logs as if we just opened the last tab of the toolbar;
LogStoreManifest.plist – contains summary information in a readable form about all xcactivitylog files, can be used, for example, to monitor the readiness of the next log with logs.

Read xcactivitylog

So the log file is a compressed gzip journal, the name is a unique identifier UUID.

After unpacking, we get a log written in the serialization format SLF0.

An example of an unpacked log in SLF0 format

SLF010#21%IDEActivityLogSection1@0#39"Xcode.IDEActivityLogDomainType.BuildLog25"Build XCActivityLogParser212"eyJ0eXBlIjp7ImJsdWVwcmludFByb3ZpZGVyIjp7fX0sImJsdWVwcmludFByb3ZpZGVyX3Byb3ZpZGVyRmlsZVBhdGhTdHJpbmciOiJcL1VzZXJzXC92b3JvYnlvdlwvRG9jdW1lbnRzXC9YQ0FjdGl2aXR5TG9nUGFyc2VyXC9YQ0FjdGl2aXR5TG9nUGFyc2VyLnhjb2RlcHJvaiJ9cbbac35a7833c541^0000007fc3632d42^1(1@1#27"com.apple.dt.IDE.LogSection16"Prepare packages212"eyJ0eXBlIjp7ImJsdWVwcmludFByb3ZpZGVyIjp7fX0sImJsdWVwcmludFByb3ZpZGVyX3Byb3ZpZGVyRmlsZVBhdGhTdHJpbmciOiJcL1VzZXJzXC92b3JvYnlvdlwvRG9jdW1lbnRzXC9YQ0FjdGl2aXR5TG9nUGFyc2VyXC9YQ0FjdGl2aXR5TG9nUGFyc2VyLnhjb2RlcHJvaiJ9f73bc45a7833c541^4303c95a7833c541^---0#0#0#46"Compile plug-ins and run any prebuild commands--36"52BE500F-D551-461D-975D-BF4B4AA236BF--0"0(0#0#0#---36"2976A337-D8BA-4626-B5F2-41F0F7CB232E--

There is no official documentation on the web that describes this type of serialization. Everything that we managed to find is the fruit of the work of enthusiasts, for which many thanks to them.

Each log starts with a mandatory header SLF0.

The following is a sequence of tokens in the following format:

<side_value><delimiter><side_value>

is an optional value, and depends on the particular delimiter.

Current version SLF0 supports 7 separatorseach of which is a description of a value of a particular type:

# integer

Describes the type integerFor example:

10#

Value on the left: UInt64the value of the token itself.

Right value: absent.

” string

Describes the type stringFor example:

21%IDEActivityLogSection

Value on the left: UInt64describes quantity characters that comes to the right of the delimiter and is the value of the string token. In this case, this means that the string value you are looking for is the string on the right, consisting of 21 characters. Characters are counted in utf-16and does not match the default count for instances of type stringwhich returns the number of graphemes).

Right value: lineconsisting of quantities characters, which is left from separator.

^double

Describes the type doubleFor example:

4fcbedd82b32c541^

Value on the left: is a big endian floating point number encoded in hexadecimal. Used in log to represent type timeInterval.

An example of reduction to the usual double out of tongue swift:

guard let integerValue = UInt64("4fcbedd82b32c541", radix: 16) else { exit(0) }
let double =  Double(bitPattern: integerValue.byteSwapped)
print(double) // 711219121.857767

Right value: absent.

– null

Describes null value, i.e. we simply do not have the value of any field, for example:

Value on the left: absent.

Right value: absent.

(array

Describes arrayindicating the number of elements it contains, for example:

2(

Value on the left: number of elements in the array. Array elements are values with type class_instanceabout them will be a little lower.

Right value: absent.

% class_name

Describes class_nameV SLF0 the decisive success factor is order of all tokens in the log, it is important to keep their order when parsing. Received indexing class_name – searched for when parsing class_instance.

38%IDEActivityLogCommandInvocationSection

Value on the left: here everything is by analogy with the type stringwhich is described above.

Right value: class name.

@class_instance

Describes class_instance, i.e. indicates the name of the class, from which it should become clear how to interpret all the tokens that will follow this one class_instance.

1@

Value on the left: UInt64the value of the token itself, indicating the position of the corresponding class_name in the log (required to understand what type the tokens that follow are).

Right value: absent.

For the convenience of understanding this serialization language, I outlined a description of the syntax in the form Backus-Naura (may come in handy when developing an appropriate parser):

<header> := SLF0

<integer_delimiter> := #
<double_delimiter> := ^
<null_delimiter> := -
<string_delimiter> := "
<array_delimiter> := (
<class_name_delimiter> := %
<class_instance_delimiter> := @

<delimiter> := <integer_delimiter>|<double_delimiter>|<null_delimiter>|<string_delimiter>|<array_delimiter>|<class_name_delimiter>|<class_instance_delimiter>
<side_value> := <character>|<digit>|<side_value><character>| <side_value><digit>
<value> := [<side_value>]<delimiter>[<side_value>]
<log> := <value>|<log><value>

<all_log> := <header><log>

The only thing to paint what is And did not, I think it’s clear.

So, knowing all the above syntactic and semantic rules, let’s try to read some specific log (from example above), after reading we get the following list of tokens, going one by one (I use the written tokenizer):

Example of received tokens xcactivitylog

integer(10)
class_name("IDEActivityLogSection")
class_instance(1)
integer(0)
string("Xcode.IDEActivityLogDomainType.BuildLog")
string("Build XCActivityLogParser")
string("eyJ0eXBlIjp7ImJsdWVwcmludFByb3ZpZGVyIjp7fX0sImJsdWVwcmludFByb3ZpZGVyX3Byb3ZpZGVyRmlsZVBhdGhTdHJpbmciOiJcL1VzZXJzXC92b3JvYnlvdlwvRG9jdW1lbnRzXC9YQ0FjdGl2aXR5TG9nUGFyc2VyXC9YQ0FjdGl2aXR5TG9nUGFyc2VyLnhjb2RlcHJvaiJ9")
double(711389365.529138)
double(63113904000.0)
array(1)
class_instance(1)
integer(1)
string("com.apple.dt.IDE.LogSection")
string("Prepare packages")
string("eyJ0eXBlIjp7ImJsdWVwcmludFByb3ZpZGVyIjp7fX0sImJsdWVwcmludFByb3ZpZGVyX3Byb3ZpZGVyRmlsZVBhdGhTdHJpbmciOiJcL1VzZXJzXC92b3JvYnlvdlwvRG9jdW1lbnRzXC9YQ0FjdGl2aXR5TG9nUGFyc2VyXC9YQ0FjdGl2aXR5TG9nUGFyc2VyLnhjb2RlcHJvaiJ9")
double(711389365.53308)
double(711389365.570412)
null
null
null
integer(0)
integer(0)
integer(0)
string("Compile plug-ins and run any prebuild commands")
null
null
string("52BE500F-D551-461D-975D-BF4B4AA236BF")
null
null
string("")
array(0)
integer(0)
integer(0)
integer(0)
null
null
null
string("2976A337-D8BA-4626-B5F2-41F0F7CB232E")
null
null

At first glance, everything is very scary and no less clear than it was before. However, let’s try to figure it out.

To begin with, please pay attention to the class names, in fact there are a limited number of them, at the time of this writing:

IDEActivityLogSection
IDEActivityLogCommandInvocationSection
IDEActivityLogUnitTestSection
IDEActivityLogMajorGroupSection
IDEActivityLogMessage
DVTDocumentLocation
DVTTextDocumentLocation

When trying to find information about the above types, it becomes clear that they are all described in a private framework IDEFoundation.frameworkwhich is part of xcode and by default located at:

open /Applications/Xcode.app/Contents/Frameworks/IDEFoundation.framework/Versions/A/

Further, through reverse engineering you can find all these classes and reproduce their set of fields, for subsequent comparison with the values of the tokens obtained above (all manipulations are for research purposes).

Out of curiosity, I dug around with ktool and was able to get a description of all classes without much difficulty:

ktool dump --headers /Applications/Xcode.app/Contents/Frameworks/IDEFoundation.framework/Versions/A/IDEFoundation

I will give an example IDEActivityLogSectionfor the sake of convenience – only its initializer, which reflects all the fields we need:

-(id)initWithSectionType:(NSInteger)arg0
              domainType:(id)arg1
                   title:(id)arg2
                subtitle:(id)arg3
                location:(id)arg4
               signature:(id)arg5
    timeStartedRecording:(CGFloat)arg6
    timeStoppedRecording:(CGFloat)arg7
             subsections:(id)arg8
                    text:(id)arg9
                messages:(id)arg10
            wasCancelled:(char)arg11
     wasFetchedFromCache:(char)arg12
commandDetailDescription:(id)arg13
              resultCode:(NSInteger)arg14
        uniqueIdentifier:(id)arg15
   localizedResultString:(id)arg16
        xcbuildSignature:(id)arg17

The only problem is that the order of the tokens does not match the order obtained by researching the framework.

However, the guys from Tulza XCLogParser everything has already been found and compared, many thanks to them, you can see how all the received types look with the correct field order Here.

For example, total fields with correct order for IDEActivityLogSection :

public let sectionType: Int8
public let domainType: String
public let title: String
public let signature: String
public let timeStartedRecording: Double
public var timeStoppedRecording: Double
public var subSections: [IDEActivityLogSection]
public let text: String
public let messages: [IDEActivityLogMessage]
public let wasCancelled: Bool
public let isQuiet: Bool
public var wasFetchedFromCache: Bool
public let subtitle: String
public let location: DVTDocumentLocation
public let commandDetailDesc: String
public let uniqueIdentifier: String
public let localizedResultString: String
public let xcbuildSignature: String

Knowing all this, let’s try to decrypt the received tokens:

The first token is always the version of the serialization format SLF0in our case 10-I:

integer(10) // Версия SLF0

Next comes the first declaration of the type name, which will be referenced by tokens with type class_instance:

class_name("IDEActivityLogSection") // Название класса (нужно для понимания, к какому типу относятся поля у class_instance)

Below begins the description of the first object with the values of its parameters, class_instance(1) indicates that the following tokens should be interpreted as class_namewhich is listed under the corresponding relative index 1in our case – IDEActivityLogSection:

class_instance(1) // Указывает, что объект типа IDEActivityLogSection

integer(0) // sectionType
string("Xcode.IDEActivityLogDomainType.BuildLog") // domainType
string("Build XCActivityLogParser") // title
string("eyJ0eXBlIjp7ImJsdWVwcmludFByb3ZpZGVyIjp7fX0sImJsdWVwcmludFByb3ZpZGVyX3Byb3ZpZGVyRmlsZVBhdGhTdHJpbmciOiJcL1VzZXJzXC92b3JvYnlvdlwvRG9jdW1lbnRzXC9YQ0FjdGl2aXR5TG9nUGFyc2VyXC9YQ0FjdGl2aXR5TG9nUGFyc2VyLnhjb2RlcHJvaiJ9") // signature
double(711389365.529138) // timeStartedRecording (TimeInterval)
double(63113904000.0) // timeStoppedRecording (TimeInterval)

An array declaration follows, followed by a description of its elements:

array(1) // Массив из одного элемента

class_instance(1) // Элемент массива, также типа IDEActivityLogSection

integer(1) // sectionType
string("com.apple.dt.IDE.LogSection") // domainType
string("Prepare packages") // title
string("eyJ0eXBlIjp7ImJsdWVwcmludFByb3ZpZGVyIjp7fX0sImJsdWVwcmludFByb3ZpZGVyX3Byb3ZpZGVyRmlsZVBhdGhTdHJpbmciOiJcL1VzZXJzXC92b3JvYnlvdlwvRG9jdW1lbnRzXC9YQ0FjdGl2aXR5TG9nUGFyc2VyXC9YQ0FjdGl2aXR5TG9nUGFyc2VyLnhjb2RlcHJvaiJ9")
double(711389365.53308) // timeStartedRecording (TimeInterval)
double(711389365.570412) // timeStoppedRecording (TimeInterval)
null // subSections
null // text
null // messages
integer(0) // wasCancelled
integer(0) // isQuiet
integer(0) // wasFetchedFromCache
string("Compile plug-ins and run any prebuild commands") // subtitle
null // location
null // commandDetailDesc
string("52BE500F-D551-461D-975D-BF4B4AA236BF") // uniqueIdentifier
null // localizedResultString
null // xcbuildSignature

And, finally, the remaining fields of the root section:

string("") // text
array(0) // messages
integer(0) // wasCancelled
integer(0) // isQuiet
integer(0) // wasFetchedFromCache
null // subtitle
null // location
null // commandDetailDesc
string("2976A337-D8BA-4626-B5F2-41F0F7CB232E") // uniqueIdentifier
null // localizedResultString
null // xcbuildSignature

For ease of understanding, the final parsing of the log, written in the format json:

{
   "version": 10,
   "mainSection": {
      "sectionType": 0,
      "domainType": "Xcode.IDEActivityLogDomainType.BuildLog",
      "title": "Build XCActivityLogParser",
      "signature": "eyJ0eXBlIjp7ImJsdWVwcmludFByb3ZpZGVyIjp7fX0sImJsdWVwcmludFByb3ZpZGVyX3Byb3ZpZGVyRmlsZVBhdGhTdHJpbmciOiJcL1VzZXJzXC92b3JvYnlvdlwvRG9jdW1lbnRzXC9YQ0FjdGl2aXR5TG9nUGFyc2VyXC9YQ0FjdGl2aXR5TG9nUGFyc2VyLnhjb2RlcHJvaiJ9",
      "timeStartedRecording": 711389365.529138,
      "timeStoppedRecording": 63113904000.0,
      "subSections": [
         {
            "sectionType": 1,
            "domainType": "com.apple.dt.IDE.LogSection",
            "title": "Prepare packages",
            "signature": "eyJ0eXBlIjp7ImJsdWVwcmludFByb3ZpZGVyIjp7fX0sImJsdWVwcmludFByb3ZpZGVyX3Byb3ZpZGVyRmlsZVBhdGhTdHJpbmciOiJcL1VzZXJzXC92b3JvYnlvdlwvRG9jdW1lbnRzXC9YQ0FjdGl2aXR5TG9nUGFyc2VyXC9YQ0FjdGl2aXR5TG9nUGFyc2VyLnhjb2RlcHJvaiJ9",
            "timeStartedRecording": 711389365.53308,
            "timeStoppedRecording": 711389365.570412,
            "subSections": null,
            "text": null,
            "messages": null,
            "wasCancelled": false,
            "isQuiet": false,
            "wasFetchedFromCache": false,
            "subtitle": "Compile plug-ins and run any prebuild commands",
            "location": null,
            "commandDetailDesc": null,
            "uniqueIdentifier": "52BE500F-D551-461D-975D-BF4B4AA236BF",
            "localizedResultString": null,
            "xcbuildSignature": null
         }
      ],
      "text": "",
      "messages": [],
      "wasCancelled": false,
      "isQuiet": false,
      "wasFetchedFromCache": false,
      "subtitle": null,
      "location": null,
      "commandDetailDesc": null,
      "uniqueIdentifier": "2976A337-D8BA-4626-B5F2-41F0F7CB232E",
      "localizedResultString": null,
      "xcbuildSignature": null
   }
}

Actually, that’s all I wanted to tell. In the resulting, readable form, you can explore the log and isolate the information that may be useful to you in your projects.

Thank you for your attention!