DeepPavlov for developers: # 2 setup and deployment
We agree that all the library startup scripts are executed in the Python environment with the DeepPavlov library installed (for installation, see the first article, about virtualenv can be read here). The examples in this article do not require knowledge of Python syntax.
Modes of interaction with NLP models DeepPavlov
DeepPavlov currently supports 4 ways to interact with NLP models (both pre-trained and user-created):
- REST server (mode riseapi) Is the main tool for integrating models, which was described in detail in the previous article. Documentation.
- TCP or UNIX socket server (mode risesocket) – if you need low-level integration. Documentation.
- Bot in Telegram (mode telegram) – demo mode, allows you to interact with the model via Telegram. Documentation.
- Command line (mode interact) – demo and debug mode, allows you to interact with the model through the command line. The model in the interaction mode via the command line is initialized with the following command:
python -m deeppavlov interact
Configs of all models supplied with DeepPavlov are registered in the library. The list of supplied models can be found in the MODELS section of the DeepPavlov documentation, their configs can be found here.
In any of the above modes, NLP models are initialized, which are based on neural networks. This makes them quite demanding on computing resources. You can improve the performance of models by using the GPU. To do this, you will need an nVidia graphics card with sufficient video memory (depending on the model you are running) and a supported version of the CUDA framework. You can find all the necessary information on launching DeepPavlov models on the GPU here.
Library settings files
All library settings are contained in three files:
- server_config.json – REST and socket server settings, as well as a Telegram connector
- dialog_logger_config.json – settings for logging queries to models
- log_config.json – library logging settings
By default, settings files are located in
python -m deeppavlov.settings
You can find out the exact path to the directory with the settings files. You can also set the path to a directory convenient for you by specifying it in the environment variable DP_SETTINGS_PATH. After the first run of the above command (a server with any trained DeepPavlov model), files from the default directory will be copied to the directory from DP_SETTINGS_PATH. Team
python -m deeppavlov.settings -d
resets settings by copying settings files from the default directory on top of files to DP_SETTINGS_PATH.
From the settings of DeepPavlov you should pay attention to:
server_config.json, parameter model_args_names:
From the last article we remember:
– arguments to the REST API DeepPavlov named;
– any model in DeepPavlov is identified by the name of its config.
So, the default argument names for each model are taken from its config.
We will analyze the structure of model configs in detail in the following articles, now we only note that the argument names in the REST API can be redefined as follows:
model_args_names: [“arg_1_name”, ..., “arg_n_name”]
The sequence of argument names corresponds to the sequence in which the arguments are defined in the model config, an empty string as the parameter value model_args_names matches the default names.
Please note that for logging uvicorn a logger is used, which is configured separately. You can read about the configuration structure of the logging Python module here.
Running pre-trained models in Docker
Any pre-trained DeepPavlov model can be launched in the Docker container in the REST service mode. Detailed instructions are in our repositories on DockerHub: here for the CPU, here for the GPU. API models in containers are fully consistent with the description from the previous article.
To make it easier to work with pre-trained NLP models from DeepPavlov, we started providing them in SaaS mode. To use the models, you need to register in our service and get a token in the Tokens section of your personal account. The API documentation is in the Info section. Under one token, you can send up to 1000 requests to the model.
Currently, the service is launched in the Alpha version and its use is provided free of charge. Further, the set and format for the provision of models will be expanded in accordance with user requests. The request form can be found at the bottom of the Demo page.
The following models are now available in DeepPavlov Cloud:
- Named Entity Recognition (multilingual) – recognition of named entities;
- Sentiment (RU) – classification of tonality of the text;
- SQuAD (multilingual) – the answer to a question to the text as a fragment of this text.
In this article, we got acquainted with the features of configuration and deployment of DeepPavlov models, learned about Docker DP images and the possibility of free access to DP models as SaaS.
In the next article, we will train a simple DeepPavlov model on our dataset. And do not forget that DeepPavlov has a forum – ask your questions regarding the library and models. Thanks for attention!