Site API generation based on user-specified functions

Main idea

The idea is quite simple: a `.php` file is created in a package in a certain directory, which should return an anonymous processing function of the following form:

return function (ApiPut $api, string|int $id, array $value = []) {/**/};
return function (ApiGet $api, string|int $id) : array {/**/};
return function (ApiLifeTime $api) : array{/**/};
return function (ApiDirect $api, string|int $id) : array {/**/};

The function name is the filename + subdirectory. Those. for a file located in `auth/user/get.php` the name `auth_user_get` will be generated.

Function types

Depending on the first parameter, all functions are divided into four types:

  • ApiPut – function for changes values.

  • APIGet – function for reading values. The result of the call is cached until the dependencies change. For example, the function returns the text of an article by ID. At the first call, a query is made to the database and the result is cached. Subsequent calls with the same ID return the cached value. When the function for changing an article is called by this identifier, the cached value is reset.

  • ApiLifeTime – function for reading values. The result of the call is cached until the specified time has elapsed.

  • ApiDirect – function direct call. For read functions (ApiGet and ApiLifeTime), ignores cached values ​​and always calls the given function.

Each type of object contains the functions available to call on it.

  • ApiPut – can call all functions, except functions of the form ApiPut. Those. a function that modifies a value cannot call another modifier function. In this case, all calls will be direct, i.e. values ​​in the cache will be ignored. This is because when the data changes, the actual data values ​​are needed.

  • APIGet – allows you to call functions like ApiGet. Functions of the ApiDirect type are not available, since these functions always return different values, which means they cannot be cached.

    Since the result of ApiLifeTime caching changes only from time to time, its change will not lead to the recalculation of the cached value in the ApiGet function, so calls to functions of the ApiLifeTime type are also prohibited.

    Since this is a read function, it cannot change anything, so functions of the ApiPut type are available in it, but only in the mode *dependencies*. Those. you can call a function of the form ApiPut with key fields, which will connect the ApiGet function with the ApiPut function through the specified parameters (but there will be no data change!). This is an indication to the system that when calling a function of the form ApiPut with such key parameters, it is necessary to reset the value cached in ApiGet. For example, we have a function for getting an article `article_get`:

 return function (ApiGet $api, string|int $id) : array {
    // Указать зависимость от функции article_put
    $api->article_put($id);
    // Выбрать статью из БД
    return db()->select('...')->get();
 };

Now if the `article_put` function is called to change the article, the cached value will be reset.

Final call pattern:

To generate, we look for all files with certain API functions and generate a class file to call these functions.

This file is not the final version, but simply illustrates what should happen approximately.

class Api
{
    // Фукнции
    protected ApiPut $apiPut = new ApiPut;
    protected ApiGet $apiGet = new ApiGet;
    protected ApiLifeTime $apiLifeTime = new ApiLifeTime;
    protected ApiDirect $apiDirect = new ApiDirect;
    // Функция article_put
    protected $article_put_fn = null;
    public function article_put(string|int $id, array $data) {
        // Сгенерировать ключ КЕШ-а по ключу
        $key = 'article_get:id'.serialize([$id]);
        // Функция загружена?
        if( is_null($this->article_put_fn) ) {
            // Загрузить функцию
            $this->article_put_fn = require "<путь до файла функции>";
        }
        // Вызвать функцию
        $ret = $this->article_put_fn($this->generatorPut, $id,$data);
        // Сбросить закешированные зависимые значения
        cache()->remove($key);
        // Вернуть результат работы функции
        return $ret;
    }
    // Функция article_get
    protected $article_get_fn = null;
    public function article_get(string|int $id) {
        // Сгенерировать ключ КЕШ-а по ключу
        $key = 'article_get:id'.serialize([$id]);
        // Проверить наличие в КЕШ-е
        if( cache()->has($key) ) {
            // Если есть в КЕШ-е, то читать значение
            $ret = cache()->get($key);
        } 
        else 
        {
            // Функция загружена?
            if( is_null($this->article_get_fn) ) {
                // Загрузить функцию
                $this->article_get_fn = require "<путь до файла функции>";
            }
            // Вызвать функцию
            $ret = $this->article_get_fn($this->generatorGet, $id);
            // Записать значение в КЕШ навсегда
            cache()->put($key, $ret, 0);
        }
        // Вернуть результат работы функции
        return $ret;
    }
    // Функция article_lifetime
    protected $article_lifetime_fn = null;
    public function article_lifetime(string|int $id) {
        // Сгенерировать ключ КЕШ-а по ключу
        $key = 'article_get:id'.serialize([$id]);
        // Проверить наличие в КЕШ-е
        if( cache()->has($key) ) {
            // Если есть в КЕШ-е, то читать значение
            $ret = cache()->get($key);
        } 
        else 
        {
            // Функция загружена?
            if( is_null($this->article_get_fn) ) {
                // Загрузить функцию
                $this->article_get_fn = require "<путь до файла функции>";
            }
            // Вызвать функцию
            $ret = $this->article_get_fn($this->generatorLifeTime, $id);
            // Записать значение в КЕШ на заданное время
            cache()->put($key, $ret, $this->generatorLifeTime->getTTL());
        }
        // Вернуть результат работы функции
        return $ret;
    }
    // Функция article_direct
    protected $article_direct_fn = null;
    public function article_direct(string|int $id) {
        // Функция загружена?
        if( is_null($this->article_direct_fn) ) {
            // Загрузить функцию
            $this->article_direct_fn = require "<путь до файла функции>";
        }
        // Вызвать функцию
        return $this->article_direct_fn($this->generatorDirect , $id);
    }
}

ApiPut, ApiGet, ApiLifeTime, ApiDirect class files are also generated. Since we have a list of all functions and their parameters, generating such files is a technical matter.

When called from a function fn_a kind of ApiGet function fn_b of the ApiGet type, it is necessary to take into account that the function fn_a depends not only on its connections, but on the connections of the function fn_b. Those. For example, we have the following dependencies:

In this case, the function fn_b depends on fn_z. A function fn_a depends on fn_z and fn_y (fn_b not taken into account, since it cannot change the data). Those. when calling a function fn_z reset the cached value for functions fn_b and fn_a. And when calling the function fn_y reset the cached value of the function only fn_a.

The idea of ​​caching is based on a report Leaving the cache in highly loaded systems / Pavel Parshikov (Avito)

APIGet

Each key has a set time. Each element of the cache contains

When reading data from the cache, all dependent keys are checked against the current installation time of these keys. If the time of some key does not match,

so the value in the cache needs to be recalculated.

In this case, the CACHE for storing data and the CACHE for storing timestamps of keys can be different. Because timestamps will be accessed more frequently, it makes sense to store them in memory. Moreover, the key + timestamp will not take up much memory even for a larger number of keys.

Let’s consider in more detail. For example, we have the following chain of API function calls:

Call chain
Call chain

On the first call, the data cache is set to the following values

Data cache state
Data cache state

The set time of the value and the set time of all child calls are stored with the data. In all cases it = **t1** (although in practice the values ​​may differ, but in our example, we will assume that all timestamps have the same value).

There is also a cache of timestamps.

Timestamp Cache Status
Timestamp Cache Status

State 1 shows timestamp cache values ​​after the first call. When calling the set value function fn_z the timestamp for the function is reset fn_c.

State 2 shows timestamp cache values ​​after function cache flush fn_c.

When a value is requested, the following occurs:

1. Requesting data from the data cache

2. Checking that the timestamps of all child keys match what is stored in the CACHE data in the field rel_keys.

If the timestamp is different (or missing), then the value is regenerated. If all labels match, then the data is up-to-date.

ApiLeftTime

We store the value and the time until which this value is valid. When reading, the time is checked. If the time is exceeded, then

  1. Change the time to +30 seconds (the value is not important, the main thing is that it be more than new data is generated)

  2. We start the function of generating new data

  3. After generating new data, we change the value in the CASH

This will reduce the load on the server in case of data recalculation. Those. only the first request will cause them to be recalculated, the rest will read either already “extended” or new data.

An example of calling API functions

All API functions are called from the corresponding service

    // Запуск
    public function run(IApi $api)
    {
        // Вызов функции типа ApiPut
        $api->test_dbg_put(1, ['aa' => 11]);
        // Вызов функции типа ApiGet
        $ret1 = $api->test_dbg_get(1);
        // Вызов функции типа ApiDirect
        $ret2 = $api->test_dbg_direct();
    }

Results

The developer does not need to think about caching, just write a function and indicate its type. Everything else will be generated automatically.

The only service needed to get the data is the generated API service.

As a bonus, you can generate code to call the API on the frontend.

In this case, the same function, for example, to get an article, can be used both on the backend and on the frontend. But here you still need to add a condition that all data that the API function returns must be converted to JSON format.

Similar Posts

Leave a Reply