When Names Aren't Enough. REST Client in F#. Part 2. Methods

In the last part, we projected external contracts into DTOs using REST as an example. In this part, we will project contract methods into something that will allow us to call them like this:

let! issues = gitflic.project.["kleidemos"].["myFirstProject"].issue.GET(limit = 24)

The imitation of the method path is of the greatest interest, but its implementation will be preceded by a number of self-sufficient stages.

Request objects instead of network access

Call HttpClient is the culmination of the method, so the format of interaction with it can determine the entire structure of our client. I find such a connection depressing, so I avoid it until the last minute and may refuse to include it in the package at all. And recently I got hooked on Godotthe android version of which does not allow working with the network bypassing the engine mechanisms. Because of this, this practice has become mandatory even in relation to my legacy.

Instead of calling the network, each method will create a request object that will be interpreted at the point of use by some external actor (in the broad sense). This actor should come with a generalized binding that does not penetrate into the depths of the actorbut it provides API typification from the outside. For historical reasons, I will call the complex of the actor and its binding a runner, but this naming cannot be borrowed uncritically. Please note that we are interested in the request object from the runner's perspective, and not HttpClient or actor. And in the first approximation it can look like this:

type 'output Request = {
	Url : string
	HttpMethod : string
	Content : string
	Headers : (string * string) list
}

'output in this case, it just points to the return type. It can be reified by adding a deserialization function to the record. But it's better to throw out both the function and Contentsince these two fields make serialization part of the client, and it can entail a ton of additional settings and a potential run to the sea. Since we are counting on the request being launched by an external player, the same player can be assigned responsibility for [де]serialization:

type Request<'input, 'output> = {
	Url : string
	HttpMethod : string
	Headers : (string * string) list
	Input : 'input
}

Thus we have just defined a monad for which we can specify zero, map etc.

This may look nice, but then you have to ask yourself why it is here Input. Because of it, a request cannot be created without an instance of the corresponding type, but at the same time we are not interested in this field either as a data source or as a result of calculations. That is, all operations on it will be performed in accordance with the goals and wishes of the user. If we throw out this field, we will save information about the input and output, and the runner will still be able to interpret the request:

type Request<'input, 'output> = {
	Url : string
	HttpMethod : string
	Headers : (string * string) list
}

For the same reasons, it is necessary to trim Urlbecause if it includes the domain address, port, etc., then all this information will be required at the creation stage Request. This will again increase the context without any tangible benefit to us. This information should belong to the runner, who will apply it at the launch stage, and we can limit ourselves to the relative path:

type Request<'input, 'output> = {
	Path : string
	HttpMethod : string
	Headers : (string * string) list
}

Finally I am free to highlight Query-parameters from Path. Merging them together won't be a problem, but extracting them from there will require a lot of effort:

type Request<'input, 'output> = {
	Path : string
	Query : (string * string) list
	HttpMethod : string
	Headers : (string * string) list
}
	with
	member this.PathAndQuery : string = ...

Some might be interested in why Query And Headers were given as a list, not Mapwhich would provide primitive protection against duplicates, perhaps speed up searches, etc. This is an acceptable step if it is dictated by considerations other than sharing validation. It should be understood that our Request although it is positioned as an object at the input of the system, in reality it is an object at the output of the system, of our system. The runner is an external entity that is outside the responsibility of our library, and therefore the validation of its input data will lie with it. We simply cannot replace it. We still do not have the right to generate garbage, but our task now boils down to producing a well-readable object. Therefore, a hypothetical split Path on tokens will give a greater effect than validated types.

The resulting Request does not touch on authorization/authentication issues. Firstly, this process is always too individual to be described here blindly. Secondly, it is much easier to solve on the runner side. We are required to implement commonly used authorization elements in some parallel corner of the client. Based on this implementation and on a fairly transparent request structure, the runner will be able to supplement the request as needed when loading into HttpClient (and its analogues).

Methods as objects

Next, we are interested in the process of creating requests. Potentially, it can be a client method, accept several parameters (including optional ones) and return Request<'input, 'output>:

val GetProjectIssues: ownerAlias: string * projectAlias: string * ?page: int * ?size: int -> Request<unit, _>

This is a normal move, and in itself it has no defects. But if between the client and Request-om will be another type that displays a specific method of our API, then we can hook onto it and throw in more complex functionality:

val GetProjectIssues: ownerAlias: string * projectAlias: string * ?page: int * ?size: int -> GetProjectIssues

This is just an additional node that we will get to in the same way as before, but the transformation step is Request<'input, 'output> will have to be done explicitly. Since we already have all the necessary parameters by this point, this transformation can be unified from the user's perspective. According to the classics, this should be another factory:

type IRequestFactory<'input, 'output> =
	abstract member Create : unit -> Request<'input, 'output>

However, our query is an immutable record with structural equivalence, i.e. its instance cannot be changed, but can be compared with others. This raises the question of how normal the situation will be when multiple calls Create will return Notidentical queries. This question does not have a correct answer due to lack of context. But I know the final point of our research, and it assumes idempotent operations, i.e. the queries will be identical. The semantics of factories does not reflect this property in any way, so I prefer cruder names and closer to the essence:

type IAsRequest<'input, 'output> =
	// Делать `AsRequest` свойством или методом -- дело внутренних установок.
	abstract member AsRequest : unit -> Request<'input, 'output>

Looking at the example from the beginning of the article, it may seem rational to build a hierarchy of types-methods through inheritance. With inheritance, the calculation Path is written more simply and is edited by cascade, but essentially this is where the advantages of inheritance end. With codegen, we don't really need this safety net, and we'll benefit more from records that don't have inheritance, but do have {with}-syntax, templates and equivalence.

In fact, this is not a mutually exclusive choice, since if you have a codegen, you can achieve record like behavior and from the usual type. The only question is what is cheaper to do.

Before this article I had no idea that uri, url, path etc., which is, and I plan to forget this information. But regardless of the recording format, a string as a basic assembly element is a source of problems. A string is as convenient for interaction with the external environment as a string is convenient for interaction with the external environment. json under certain conditions. Not counting infrastructure domains, json almost always transforms into full-fledged objects before work. In rare cases, JObject or ExpandoObject. Adding or replacing properties in json-line through regex-s and other operations with string — acceptable for cartoons and, perhaps, for the top league of byte-lovers (I don’t know, I just admit the possibility), but not for most real applications.

WITH Uri we must act in a similar manner. Uri — this is not just minced meat that cannot be turned back. This is minced meat that has been breaded and heat-treated, that is, these are ready-made cutlets. You do not need to put forgotten components in them in the hope of grinding and frying everything again. So wherever an addition or correction of the path is supposed, all internal elements must be presented as full-fledged objects.

In concrete terms, this means that the object of the method that produces Requestmust contain all required and optional parameters:

type GetProjectIssues = {
	ownerAlias : string
	projectAlias : string
	page : int option
	size : int option
}

A Path And Query will be generated on the fly.

with
interface IAsRequest<unit, response> with
	override this.AsRequest () = {
		Path = $"/project/{this.userAlias}/{this.projectAlias}/issue"
		Query = [
			let inline (!) key value = [
				match value with
				| Some value -> key, string value
				| None -> ()
			]
			yield! !"page" this.page
			yield! !"size" this.size
		]
		HttpMethod = "GET"
		HttpHeaders = []
	}

Other potential parameters, such as credentials or website addresses, were discarded in the previous paragraph. So now we have a complete description of the method that is available for reading, comparing, and with-modifications. The first is relevant for the UI, the second is for caching, and the last is for recursive page loading.

There may be only one question left here. Since we have a separate object for each method, why Request remains a separate record, and not an interface attached directly to the method-object? The behavior of the record (as well as DU) cannot be changed, due to which the record can forcibly break domains where it is necessary to clearly designate the boundaries of responsibility of two different systems. The interface simply forms a connection without additional restrictions. I can’t say that this coercion was strictly necessary here, rather, it simply proved itself well and became the default strategy.

Simulation of hierarchy

Those who have read Part 4 of “Big Code” have already seen how graphs can be mapped to types using type extensions. Our graph is reduced to a tree, which is much simpler, since it assumes movement only in one direction. The reverse movement is possible, but it is ironcladly predetermined for each specific node.

We will need to create a type for each non-terminal node in the tree, including its root, regardless of whether it contains individual data or not:

type GitFlic = ...
type project = ...
type ``project {userAlias}`` = ...

Records without fields are syntactically impossible, so nodes without data must be simulated either via DU, or via structures, or via singleton:

[<RequireQualifiedAccess>]
type project = Instance

// Для справки:
// Такая структура имеет свойство эквивалентности.
// project() = project() // = true
type project = struct end

type ``project`` private () =
    static member val instance = ``project`` ()

All these types will be stored in a separate namespace, which we are unlikely to open separately. In this case, only external readability is important to us, it should be clear what is being discussed only by the mention in the signature (including variables). The naming of DU cases is a little stricter than the naming of types, which, together with their nesting, will rather confuse us, so I prefer to use an empty structure or singleton. Both cases involve an atypical “constructor”, but the client user will not need it.

Data nodes are elementary:

type ``project {userAlias}`` = {
	userAlias : string
}

There is no need to skimp on types, even if their data matches:

type ``project {userAlias} {projectAlias}`` = {
	userAlias : string
	projectAlias : string
}

type ``project {userAlias} {projectAlias} issue`` = {
	userAlias : string
	projectAlias : string
}

If we are talking about a tree API, we cannot cut off a node from its descendants unless it has a separate type. The absence of separate types can lead to both infinite loops and swallowing of path segments. Both options also create prerequisites for misinterpretation when names coincide in different parts of the tree:

gitflic.project.["kleidemos"].["myFirstProject"].issue.issue.issue.issue

gitflic.project.["kleidemos"].["myFirstProject"].["someId"].GET

Obviously, with this approach, methods (terminal) and non-terminal nodes differ in their structure only in that the former implement the interface IAsRequest. Whether to store them together or separately is a matter of ease of navigation:

type ``project {userAlias} {projectAlias} issue {localId} GET`` = {
	ownerAlias : string
	projectAlias : string
	page : int option
	size : int option
}
	with
	interface IAsRequest<...

Next, each non-terminal node needs to add properties leading to its child nodes:

type GitFlic with
	member this.project = project.instance

type ``project {userAlias} {projectAlias}`` with
	// Типов с таким набором полей несколько,
	// но явное указание типа результата позволяет компилятору понять, 
	// о чём идёт речь.
	member this.issue : ``project {userAlias} {projectAlias} issue`` = {
		userAlias = this.userAlias
		projectAlias : this.projectAlias
	}

If the transition requires parameterization, then an indexer is needed. For those who didn't know or forgot, the definition of an (unnamed) indexer in F# looks like this:

type project with
	member this.Item
		with get userAlias : ``project {userAlias}`` = {
			userAlias = userAlias
		}

// project().["userAlias"]

In one of the latest versions, F# learned to work with indexers without a dot before the square bracket, but this innovation creates the preconditions for different interpretations. From now on seeing the record f[x]I can't be sure that this is a function call on a list of one element. I really don't like this, the arguments of the authors of the feature seem dubious to me, and the whole action looks like a sudden C#-like aggravation. That's why I avoid simplified notation in my projects and articles, since we work with lists many times more often than with indexers.

Some of the final transitions to API methods (terminal nodes) require optional parameters, which can only be expressed through class methods. For unification purposes, it is preferable to use the same approach for non-parameterized cases:

type ``project {userAlias} {projectAlias} issue`` with
	member this.GET (?page : int, ?size : int)
		: ``project {userAlias} {projectAlias} issue GET`` = {
		userAlias = this.userAlias
		projectAlias = this.projectAlias
		page = page
		size = size
	}

Optionally, a property can be added to all nodes to navigate to the top of the tree:

type ``project {userAlias} {projectAlias} issue GET`` with
	member this.Parent =
		: ``project {userAlias} {projectAlias} issue`` = {
		userAlias = this.userAlias
		projectAlias = this.projectAlias
	}

But since all parameters from the path are available for reading directly, it is often easier to go through the entire chain from gitflic to another method by transferring values ​​from the existing record to the path.

At the end of all this you need to pull out GitFlic.instance as a global “variable”:

[<AutoOpen>]
module Auto =
	let gitflic = Routes.GitFlic.instance

gitflic.project.["kleidemos"].["myfirstproject"].GET()

Runner

A runner is a purely individual thing, but a minimal example can be given. This is how the basic procedure for processing a request on the base might look like HttpClient And Hopac:

module Runner =
	open Thoth.Json.Net

	type Config = ...

	let run (client : HttpClient) config (input : 'input) (request : Request<'input, 'output>) : 'output Job = job {
		use content = new StringContent(
			if typeof<'input> <> typeof<unit>
			then Encode.Auto.toString input
			else ""
		)
		content.Headers.ContentType <- Headers.MediaTypeHeaderValue.Parse "application/json"

		use msg = new HttpRequestMessage(
			HttpMethod.Parse request.HttpMethod
			, config.ApiAddress + request.PathAndQuery
			, Content = content
		)
		for key, value in request.Headers do
			msg.Headers.Add(key, value)
		do  let auth = config.AuthorizationHeader
			msg.Headers.Add(auth.Name, auth.Value)
        
		let! response = client.SendAsync msg
		if typeof<'output> = typeof<unit> then
			return unbox<'output> ()
		else
			let! respContent = response.Content.ReadAsStringAsync()
			return
				respContent
				|> Decode.Auto.unsafeFromString
	}

Error codes etc. are not displayed here, but in F# such things are best decided taking into account the specifics of the project and the wishes of the team. Most likely, the code processing will be placed here, but it is not a fact.

It is worth paying attention to how it behaves unit V 'input And 'output. In both directions, an instance of this type should be treated as an empty body. In the case of a request, we place an empty string in Contentalthough ideologically it is better (but more verbose) not to ask at all msg.Content. And in case of a response, instead of reading and deserializing, we immediately return () (copy) unit).

Based on the function run you can define a runner object and its main methods:

module Runner =
	...

	type Main = {
        HttpClient : HttpClient
        Config : Config
    }

	let create client config : Main = ...

	type Main with
		// `request` и `input` в зависимости от предпочтительного варианта использования можно менять местами
		member this.Run (input : 'input) (request : Request<'input, 'output>) : 'output Job = ...
		member this.RunAsRequest (input : 'input) (preRequest : IAsRequest<'input, 'output>) : 'output Job = ...

Then it makes sense to expand Request And IAsRequest:

// После модуля Runner (а не "в")
[<AutoOpen>]
module RunnerAuto =
    type Request<'input, 'output> with
        member this.Run (input : 'input) (runner : Runner.Main) : 'output Job = ...

    type IAsRequest<'input, 'output> with
        member this.Run (input : 'input) (runner : Runner.Main) : 'output Job = ...

I tend to forget the names of loose objects and functions, so I can duplicate the root gitflic and attach it to the runner:

module Runner =
	type Main with
		member this.api = Routes.GitFlic.instance

runner.api.project.["kleidemos"].["myfirstproject"].GET()

Builder

If you need a deep understanding of builders (aka Computation Expressions or CE), That Here There is a translation of a large cycle dedicated specifically to this topic. I am talking about everyday use, so I will only explain what influences the decisions made.

In fact, we often get enough of explicitly calling runners through pipes or type extensionsbut if you want, you can combine the runner and builder. We usually don't bother too much and create our own builder based on an existing asynchronous one. For example, this is what it looks like Start runner-builder based Hopac.JobBuilder:

module Runner =
	type JobBuilder (httpClient, serverConfig) =
		inherit Hopac.JobBuilder()

		// Применяется для разруливаня return! к Request<unit, 'output>
		member this.ReturnFrom (request : Request<unit, 'output>) =
			run httpClient serverConfig () request

		// Для let! к Request<unit, 'output>
		member this.Bind (request : Request<unit, 'output>, f : 'output -> 'y Job) = job {
			let! output = this.ReturnFrom request
			return! f output
		}

This builder has some peculiarities. Firstly, it cannot be created without parameters:

JobBuilder(clinet, config) {
	...
}

But this is not a serious problem, since he usually goes in the convoy. Runner.Main:

module Runner =
	type Main with
		member this.job = JobBuilder(this.HttpClient, this.Config)

And the user creates it like this:

runner.job {
	...
}

Secondly, he is the heir. Hopac.JobBuilderso it can work with Job, Task, Async And ObservableMoreover, it can work as a pure ancestor and not interfere with ours in any way. Request<unit, 'output>:

runner.job {
	do! timeOutMillis 1000
	return 42
}

Some people might be put off by this, but I consider this backward compatibility an advantage, since in practice the code can evolve in roundabout ways, and it would be inconvenient to force a change to the builder just because a REST call left the scope. Semantics will suffer, of course, but it is more convenient to postpone fixing them until the commit, when key decisions have already been made.

While processing is defined in this builder let!, do! And return! For Request<unit, 'output>. But in F# interfaces are implemented explicitly, because of this the method AsRequest will be closed to calling until we cast the type to the interface:

runner.job {
    do! timeOutMillis 100
    let! project = (runner.api.project.["kleidemos"].["myfirstproject"].GET() :> IAsRequest<_,_>).AsRequest()
    return project.language
}

This is a pretty verbose entry, so it makes sense to extend the builder with two more methods:

// Аналогичная пара для IAsRequest<unit, 'output>
member this.ReturnFrom (preRequest : IAsRequest<unit, 'output>) =
    run httpClient serverConfig () (preRequest.AsRequest())

member this.Bind (preRequest : IAsRequest<unit, 'output>, f : 'output -> 'y Job) = job {
    let! output = this.ReturnFrom preRequest
    return! f output
}

As a result, it will be possible to write like this:

runner.job {
    do! timeOutMillis 100
    let! project = runner.api.project.["kleidemos"].["myfirstproject"].GET()
    return project.language
}

Now it remains to deal with the requests that have 'input different ()We would probably be satisfied with an address like this:

runner.job {
    let! project = gitflic.project.POST() {
        title = "created-via-api"
        isPrivate = true
        alias = "created-via-api"
        ownerAlias = "kleidemos"
        ownerAliasType = "USER"
        language = "F#"
        description = "Created via api."
    }

    return project.id
}

But the builder can influence the interpretation of the code strictly in certain nodes. Get to the segment .POST() { he can't. We need to pass it on. request And input to enter the builder, which can be done through tuple:

let! project = 
	gitflic.project.POST()
	, {
		title = "created-via-api"
		...
	}

The problem with this option is the weak coupling of the tuple parameters. The compiler will first look at what we created, compare it with the existing overloads, and only then point out our error. With such a sequence, the type of the second element will have to be specified manually. Instead, it is better for the first parameter to explicitly define the second via a method. Even better, if this method returns a specially designed structure:

module Runner =
	...

	type Send<'input, 'output> = {
        Request : Request<'input, 'output>
        Input : 'input
    }

[<AutoOpen>]
module RunnerAuto =
    type IAsRequest<'input, 'output> with
        member this.Send input : Runner.Send<'input, 'output> = {
            Request = this.AsRequest()
            Input = input
        }

Although interfaces in F# are explicit, extensions to them are implicit. Therefore, the appeal gitflic.project.POST().Send correct and suggested by the IDE.

It remains to add a couple more methods to the builder:

// Аналогичная пара для Send<unit, 'output>
member this.ReturnFrom (send : Send<'input, 'output>) =
    run httpClient serverConfig send.Input send.Request

member this.Bind (send : Send<'input, 'output>, f : 'output -> 'y Job) = job {
    let! output = this.ReturnFrom send
    return! f output
}

And we can write like this:

runner.job {
    let! project = gitflic.project.POST().Send {
        title = "created-via-api"
		...
    }

    return project.id
}

Original Hopac.JobBuilder written in a slightly outdated manner that knows nothing about the mechanism Sourcewhich is why each source type required 2 additional methods from us. But if the builder was written as it should be, then we would only need to add conversion functions to Job for each type from the list (Request, IAsRequest, Send), after which the compiler completed the remaining transformations automatically. For those interested, I suggest going to the source code FsToolkit.ErrorHandling (as well as in related threads), where there are several builders built exactly according to this scheme. For those who intend to expand frequently JobBuilderI recommend thinking about replacing it with your own version, the language allows it.

Interim conclusion

This time we projected the REST API method tree, and then made the projection friends with the builder. In reality, the projection would require a code generator, but the truncated result of the generation can be seen Here. The generator itself is not there, I left it for the next part. In it I will give the generator, analyze the most interesting moments in detail, and also show what other projections of the method tree may be useful to us.

Author of the article @kleidemos

UFO arrived and left a promo code here for our blog readers:
— 15% on any VDS order (except for the Progrev tariff) — HABRFIRSTVDS