Using APM with Laravel and Lumen

APM stands for Application Performance Monitoring (application performance monitoring). If you come across this abbreviation on your way, then it is most likely about measuring the performance of your application and your servers. How do they cope, how much memory do they consume, where are the bottlenecks? And that’s not all. With APM, you can set up special notifications that will notify you, for example, when memory consumption has reached very high levels or a remote call is taking too long. Triggers for such notifications can be based on a fairly wide range of indicators and events. But let’s not get ahead of ourselves.

A bit of history

When I was still working in Pathao, we used New Relic. New Relic is very handy. To get started with it in PHP you just need to install its package and agent. And that is all. You will immediately see all the necessary information on the New Relic dashboard as soon as the server starts serving requests. But my current companyDigital Health Care Solutions, formerly known as Telenor Health), offered Elastic APM. So I had to figure out how it works. Up to this point, I have already tried at least 4 times to force myself to master Elastic Search, but all to no avail. Therefore, I looked for available ready-made packages. And I found something. It was a very good package, but under the hood it sent HTTP Requests to the APM server. Which is actually quite costly. And it didn’t support APM Server 7.x.

github.com

That’s why I had to create my own package almost from scratch. In this article, I’m going to show you how you can use my package with absolutely any PHP code. The package is already easy enough to use with Laravel or Lumen. But you don’t have to use any of these. The beauty is that you can use it however you want.

What is ElasticAPM?

Did you pay attention to the image at the beginning of the article? Let’s go back to it for a moment. This image shows the basic structure of how it all works. You need to install an APM agent for your specific language (APM agents still don’t support all languages, so keep that in mind). This agent will collect data about the execution of your code. It will then send them to the APM server. The APM servers then pass this data to the Elasticsearch server and you can view this data in Kibana. Actually, nothing supernatural. But I found that APM Dashboard UI comes with XPack which means you have to shell out a bit.

installation

Elastic provides an APM agent for PHP. This agent will collect data from our server and transfer it to the APM server. I hosted my PHP application in a docker container. The docker file looks like this:

FROM php:7.4-fpm

RUN apt-get update

RUN apt-get install -y libpq-dev libpng-dev curl nano unzip zip git jq supervisor

RUN docker-php-ext-install pdo_pgsql

RUN pecl install -o -f redis

RUN docker-php-ext-enable redis

RUN curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer

RUN apt-get -y update && apt-get -y install build-essential autoconf libcurl4-openssl-dev --no-install-recommends
RUN mkdir -p /home/apm-agent && \
    cd /home/apm-agent && \
    git clone https://github.com/elastic/apm-agent-php.git apm && \
    cd apm/src/ext && \
    /usr/local/bin/phpize && ./configure --enable-elastic_apm && \
    make clean && make && make install
COPY ./elastic_apm.ini /usr/local/etc/php/conf.d/elastic_apm.ini
RUN mkdir /app
WORKDIR /app


CMD ["php", "-S", "0.0.0.0:80", "index.php"]

Docker file for PHP container. Ignore the CMD section.

In the snippet above, you should pay attention to the commands in line 16. We clone the APM git repository and then set it up and install it in our container.

IN line 22 we copy .ini file. This .ini file:

extension=elastic_apm.so
elastic_apm.bootstrap_php_part_file=/home/apm-agent/apm/src/bootstrap_php_part.php
elastic_apm.enabled=true
elastic_apm.server_url="http://docker.for.mac.localhost:8200"
; elastic_apm.secret_token=
elastic_apm.service_name="PHP APM Test Service"
; service_version= ${git rev-parse HEAD}
elastic_apm.log_level=DEBUG
; Available Log levels
; OFF | CRITICAL | ERROR | WARNING | NOTICE | INFO | DEBUG | TRACE

elastic_apm.ini file

In file elastic_apm.ini line 2 specifies the path to our cloned git repository file − src/bootstrap_php_part.php. Line 4 specifies the URL APM servers. If you are not using docker you can do git clone repository and then just install it. Next, we integrate with the php extension. That’s all for installing the APM agent.

Before we dive into the package itself, you can get some background on the basic concepts from agent documentation.

Basically it comes down to just two things:

• Transaction: When your application is running, it generates transactions. Each request is considered a transaction. Each transaction has a name and a type.

• Span: When you execute some code, the information you are dealing with can be sent to the server as a span. A span is a record of an operation performed by a single piece of code. A database request can be a span, just like HTTP request information.

The package itself and working with it

github.com

So first let’s figure out how to integrate the package with PHP. Then we’ll look at how it integrates with Laravel/Lumen. The agent itself requires PHP version ≥ 7.2, which is why the minimum requirement for the package is PHP 7.2.

installation

composer require anik/elastic-apm-php

PHP integration

• Class Anik\ElasticApm\Agent is the public access point for all interactions. The Agent class cannot be instantiated – it is a singleton. That is, whenever you need to interact with it in any way, you will call Agent::instance(). You will receive the same object, no matter where you call it from, throughout the lifecycle of the request.

• To set transactions Name And type you will need to instantiate the object Anik\ElasticApm\Transaction with parameters name And type. After successfully instantiating the object, you will need to pass it to the class Agent through his method setTransaction.

Agent::instance()->setTransaction(new Transaction('name', 'type'));

• If you want to send the data of this transaction to the server, you will have to use span. To be able to create a new span, the interface must be implemented Anik\ElasticApm\Contracts\SpanContractwhich contains methods such as getSpanData(), getName(), getType(), getSubType(). But if you use the trait Anik\ElasticApm\Spans\SpanEmptyFieldsTraitthen you can omit the methods getAction() And getLabels(). If you want to send data to your APM server, then you would do well to implement these methods. I won’t go into method return values ​​here, you can find that information in the agent documentation above.

Agent::instance()->addSpan(" class="formula inline">implementedSpanObject);

• Finally, when you’re done adding spans, you’ll need to send all of these before returning the result. transactions And spans to the APM agent. To do this, use

Agent::instance()->capture();

The above method processes all transactions and spans and then passes them to the agent, after which the agent takes care of sending them to the server.

Note: If you want to do everything yourself, you can use Agent::getElasticApmTransaction()to get the agent’s current transaction, or Agent::newApmTransaction($name, $type)to create a new transaction. Be sure to call the method end()if you created a new object Transaction. Or, if you want to put the spans you add into a new transaction, you can use the method Agent::captureOnNew()to send them with a new transaction. You don’t have to call endwhen you use captureOnNew. If you suddenly need to get a fresh instance Agentyou can first call Agent::reset()and then Agent::instance()But Agent::reinstance() will do the same. Finally, keep in mind that if you call any of the methods capture*()then an object must be provided with it Transaction. Without the Transaction object, you will get an exception Anik\ElasticApm\Exceptions\RequirementMissingException.

This is where we are done with PHP integration.

Integration with Laravel/Lumen

For Laravel:

For Lumen:

// в ваш файл bootstrap/app.php.
use Anik\ElasticApm\Providers\ElasticApmServiceProvider;
$app->register(ElasticApmServiceProvider::class);
$app->configure('elastic-apm');

You can also feel free to modify the configuration file according to your requirements.

Application Bug Tracking

If you want to send error data to your APM server, then

// ЭТОТ РАЗДЕЛ СЛЕДУЕТ ЗАКОММЕНТИРОВАТЬ
/**
 *  $app->singleton(
 *      Illuminate\Contracts\Debug\ExceptionHandler::class,
 *      App\Exceptions\Handler::class
 *  );
 */
// ИСПОЛЬЗУЙТЕ ЭТОТ РАЗДЕЛ
use Illuminate\Contracts\Debug\ExceptionHandler;
use Anik\ElasticApm\Exceptions\Handler;
use App\Exceptions\Handler as AppExceptionHandler;
use Symfony\Component\HttpKernel\Exception\NotFoundHttpException;
use GuzzleHttp\Exception\ConnectException;
$app->singleton(ExceptionHandler::class, function ($app) {
    return new Handler(new AppExceptionHandler($app), [
        // NotFoundHttpException::class, //(1)
        // ConnectException::class, //(2)
    ]);
});

Anik\ElasticApm\Exceptions\Handler takes an array of exception classes as the second parameter (which will not be sent to the APM server). Default errors NotFoundHttpException are not sent to the APM server. The lines marked (1) and (2) have been commented out to point this out to you.

If your application encounters mistakewhich was successfully intercepted exception handler (Exception Handler), and transactions are already configured, then the APM server is guaranteed to receive the error stack trace. Because the PHP agent does not provide an API to send a stack trace, your trace may be truncated by the exception handler.

Keep track of your application’s requests and responses

We have a special built-in middleware to keep track of the number of requests your application is serving, status codes and the time it took to process them.

<?php
use Anik\ElasticApm\Middleware\RecordForegroundTransaction;
class Kernel extends HttpKernel {
    protected $middleware = [
        // ...        
        RecordForegroundTransaction::class,
        // ..
    ];
}

use Anik\ElasticApm\Middleware\RecordForegroundTransaction;
$app->middleware([
    // ...
    RecordForegroundTransaction::class,
    // ...
]);

The transaction name for processed requests follows the following logic:

• If the route handler uses the parameter usesi.e.; HomeController@index (controller method).

• If the route handler uses the parameter asi.e.;['as' => 'home.index'] (named route).

• If this is not the option above, then – HTTP_VERB ROUTE_PATHi.e.; GET /user/api.

• If nothing matches, 404, then index.php or a user-provided name from the config is used.

Tracking Remote HTTP Calls

If you are using Guzzle, you can use Guzzle’s built-in middleware.

use GuzzleHttp\HandlerStack;
use GuzzleHttp\Client;
use Anik\ElasticApm\Middleware\RecordHttpTransaction;

$stack = HandlerStack::create();
$stack->push(new RecordHttpTransaction(), 'whatever-you-wish');
$client = new Client([
    'base_uri' => 'https://httpbin.org',
    'timeout'  => 10.0,
    'handler'  => $stack,
]);
$client->request('GET', '/');

Queue Worker Tracking

To keep track of jobs, you need to use the built-in (Job) middleware whenever you dispatch a new job. You can use any of the below:

use Anik\ElasticApm\Middleware\RecordBackgroundTransaction;
use Illuminate\Contracts\Queue\ShouldQueue;
class TestSimpleJob implements ShouldQueue 
{
    public function middleware () {
        return [ new RecordBackgroundTransaction()];
    }
    
    public function handle () {
        app('log')->info('job is handled');
    }
}

use Anik\ElasticApm\Middleware\RecordBackgroundTransaction as JM;
use App\Jobs\ExampleJob;
dispatch((new ExampleJob())->through([new JM()]);

Note: If you are using php artisan queue:work, this means that the task is running for a long time. That is why only one transaction will be sent. If no process is created, then you won’t get a transaction or a span. On the other hand, if you use queue:listeni.e.: php artisan queue:listen – a new process will be used for each task, so you will get a new transaction and spans for that transaction for each task.

Query progress tracking

The execution of the request is processed automatically and passed to the APM server.

That’s all. I hope you enjoyed it. Don’t forget to give this project a star!

Tracking Redis Requests

The execution of a Redis request is not handled automatically. If you are using Redis as your Cache Driver, you need to explicitly indicate that you want to enable Redis Query Logging by adding ELASTIC_APM_SEND_REDIS=true to your .env file.

And also for the sake of self-development, here is a docker-compose.yml file for ES, Kibana and APM (Do not use in production!)

version: "2"

services:
    php:
        build:
            dockerfile: php.dockerfile
            context: .
        volumes:
            - .:/app
        ports:
            - 8008:80
        links:
            - apm
    
    elasticsearch:
        image: bitnami/elasticsearch:7.8.0
        volumes:
            - ~/.backup/elasticsearch/elastic-apm:/bitnami/elasticsearch/data
            - ./bitnami_es_config.yml:/opt/bitnami/elasticsearch/config/elasticsearch.yml
        ports:
            - 60200:9200
        environment:
            - BITNAMI_DEBUG=true
    
    kibana:
        image: bitnami/kibana:7.8.0
        ports:
            - 5601:5601
        volumes:
            - ~/.backup/kibana/elastic-apm:/bitnami
        links:
            - elasticsearch
        environment:
            - KIBANA_ELASTICSEARCH_URL=elasticsearch
    
    apm:
        image: docker.elastic.co/apm/apm-server-oss:7.8.0
        ports:
            - 8200:8200
        user: apm-server
        links:
            - elasticsearch
            - kibana
        command: --strict.perms=false
        environment:
            - apm-server.host=0.0.0.0
            - apm-server.kibana.enabled=true
            - apm-server.kibana.host="http://kibana:5601"
            - output.elasticsearch.hosts=["elasticsearch:9200"]
            - output.elasticsearch.max_retries=1
    
    apm2:
        image: docker.elastic.co/apm/apm-server-oss:6.8.9
        ports:
            - 8201:8200
        user: apm-server
        links:
            - elasticsearch
            - kibana
        command: --strict.perms=false
        environment:
            - apm-server.host=0.0.0.0
            - apm-server.kibana.enabled=true
            - apm-server.kibana.host="http://kibana:5601"
            - output.elasticsearch.hosts=["elasticsearch:9200"]
            - output.elasticsearch.max_retries=1

docker-compose.yml

Material prepared in anticipation of the start online course “PHP Developer. Professional”.