what is it, why is it needed, how does it work
Imagine you have N editors or IDEs and M programming languages. It turns out that for them to work correctly, you need to support N*M plugins. But what if there are many such editors and languages?.. The solution can be LSP — a single interface for interaction between the language server and editors, which helps narrow the problem down to N+M.
My name is Denis Mamatin, I work in the R&D department of SberTech. Our team develops and tests new technologies. In this article, I will tell you what the LSP protocol is, how it can help simplify development, and I will consider a small example of an LSP server.
What is LSP
LSP (Language Server Protocol) is a protocol between an editor or IDE and a language server that extends work with a text document. The protocol supports a number of functions for convenient text editing. For example, auto completion (auto complete), transition to the declaration (go to), find all occurrences (find all) and others.
The main idea of LSP is to standardize the interaction between the language server and the development tool (IDE). One LSP server is used for different editors (VScode, IntelliJ, vim).
N*M problem
Before LSP, an editor had to support each programming language. Let's say if you have N editors and M programming languages, you need N*M plugins. The image below shows an example of what this looks like.
This problem concerns not only the developers of these plugins, but also ordinary programmers. For example, a new version of Go was released, which added generics (they already exist, but let's assume). As a developer, I want a basic function from the IDE – a hint of compilation errors. In addition to the fact that I do not have this, the IDE indicates that my code with generics is an error. And all because the IDE does not support the new version of Go. Although in the same Java, generics have existed for many years, and the IDE can work with them.
Now all I can do is wait and hope that the IDE developers will add support. And that would be half the trouble, if not for one “but”: we have N editors. Petya waits for a new version for editor-1, and gets it a day later, and Vasya waits a month, because editor-2 is supported by one contributor. Let me remind you once again that we have M programming languages. As an IDE developer, I would go away crying.
And here LSP comes to the rescue: with it the problem narrows to N+M. We get a situation as in the figure below.
You can read more about how LSP is useful here.
Specification
The protocol consists of two parts: the HTTP header and the content. Only one header is required – Content-Length. The content is a message in JsonRPC format. Request example:
Content-Length: ...\r\n
\r\n
{
"jsonrpc": "2.0",
"id": 1,
"method": "textDocument/completion",
"params": {
...
}
}
Пример ответа:
Content-Length: ...\r\n
\r\n
{
"jsonrpc": "2.0",
"id": 1,
"result": [
...
]
}In addition to the usual “request-response”, the client and server can send notifications. A notification is a message that does not require a response. For example, opening a document.
IN documentation messages are defined as TypeScript interfaces.
interface Message {
jsonrpc: string;
}
interface RequestMessage extends Message {
id: integer | string;
method: string;
params?: array | object;
}
interface NotificationMessage extends Message {
method: string;
params?: array | object;
}
interface ResponseMessage extends Message {
id: integer | string | null;
result?: string | number | boolean | array | object | null;
error?: ResponseError;
}
interface ResponseError {
code: integer;
message: string;
data?: string | number | boolean | array | object | null;
}Let's say we want to find a structure declaration. Then the message from the client would look like this:
{
"jsonrpc": "2.0",
"id": 1,
"method": "textDocument/definition",
"params": {
"textDocument": {
"uri": "file:///someFolder/main.go"
},
"position": {. // позиция, где находился курсор
"line": 1,
"character": 13
}
}
}Server response:
{
"jsonrpc": "2.0",
"id": 1,
"result": {
"uri": "file:///someFolder/exampleStructure.go",
"range": {
"start": {
"line": 0,
"character": 6
},
"end": {
"line": 0,
"character": 12
}
}
}
}Some examples of other functions:
textDocument/hover— hovering the cursor over a position in the text.JSON request:
{ "jsonrpc": "2.0", "id": 1, "method": "textDocument/hover", "params": { "textDocument": { "uri": "file:///path/to/document.txt" }, "position": { "line": 10, "character": 3 } } }JSON response:
{ "contents": [ { "value": "Some value", "kind": "markdown" } ], "range": { "start": { "line": 0, "character": 0 }, "end": { "line": 0, "character": 6 } } }textDocument/completion— autocompletionJSON request:
{ "jsonrpc": "2.0", "id": 2, "method": "textDocument/completion", "params": { "textDocument": { "uri": "file:///path/to/document.txt" }, "position": { "line": 5, "character": 10 } } }JSON response:
"isIncomplete": false, "items": [ { "label": "Some label", "kind": 1, "documentation": "Some doc", "detail": "console", "textEdit": { "newText": "Some label", "range": { "start": { "line": 3, "character": 1 }, "end": { "line": 3, "character": 1 } } } } ] }textDocument/signatureHelp— getting help/help on the current cursor positionJSON request:
{ "jsonrpc": "2.0", "id": 3, "method": "textDocument/signatureHelp", "params": { "textDocument": { "uri": "file:///path/to/document.txt" }, "position": { "line": 8, "character": 5 } } }JSON response:
{ "signatures": [ { "label": "print()", "documentation": "Prints a message to the console.", "parameters": [] } ] }references – find references
JSON request:
{ "jsonrpc": "2.0", "id": 4, "method": "textDocument/references", "params": { "textDocument": { "uri": "file:///path/to/document.txt" }, "position": { "line": 12, "character": 8 }, "context": { "includeDeclaration": true } } }JSON response:
[ { "uri": "file:///path/to/document.txt", "range": { "start": { "line": 10, "character": 5 }, "end": { "line": 10, "character": 10 } } } ]documentHighlight— highlighting all occurrences of a symbol in the scope. For example, all places in a file where the function name is “mentioned” are highlighted.JSON request:
{ "jsonrpc": "2.0", "id": 5, "method": "textDocument/documentHighlight", "params": { "textDocument": { "uri": "file:///path/to/document.txt" }, "position": { "line": 3, "character": 7 } } }JSON response:
[ { "range": { "start": { "line": 10, "character": 5 }, "end": { "line": 10, "character": 10 } }, "kind": "write" } ]textDocument/formatting– format the entire documentJSON request:
{ "jsonrpc": "2.0", "id": 7, "method": "textDocument/formatting", "params": { "textDocument": { "uri": "file:///path/to/document.txt" }, "options": { "tabSize": 4, "insertSpaces": true } } }JSON response:
[ { "range": { "start": { "line": 0, "character": 0 }, "end": { "line": 10, "character": 0 } }, "newText": "some text" } ]textDocument/declaration— go to the adJSON request:
{ "jsonrpc": "2.0", "id": 8, "method": "textDocument/declaration", "params": { "textDocument": { "uri": "file:///path/to/document.txt" }, "position": { "line": 10, "character": 5 } } }JSON response:
{ "uri": "file:///path/to/document.txt", "range": { "start": { "line": 10, "character": 5 }, "end": { "line": 10, "character": 10 } } }textDocument/definition— go to definitionJSON request:
{ "jsonrpc": "2.0", "id": 9, "method": "textDocument/definition", "params": { "textDocument": { "uri": "file:///path/to/document.txt" }, "position": { "line": 8, "character": 3 } } }JSON response:
{ "uri": "file:///path/to/document.txt", "range": { "start": { "line": 10, "character": 5 }, "end": { "line": 10, "character": 10 } } }textDocument/rename— renameJSON request:
{ "jsonrpc": "2.0", "id": 11, "method": "textDocument/rename", "params": { "textDocument": { "uri": "file:///path/to/document.txt" }, "position": { "line": 3, "character": 7 }, "newName": "newIdentifier" } }JSON response:
{ "changes": { "file:///path/to/document.txt": [ { "range": { "start": { "line": 0, "character": 10 }, "end": { "line": 0, "character": 15 } }, "newText": "newName" } ] "documentChanges": [ { "textDocument": { "uri": "file:///path/to/document.txt", "version": 1 }, "edits": [ { "range": { "start": { "line": 0, "character": 10 }, "end": { "line": 0, "character": 15 } }, "newText": "newName" } ] } ] }
Initialization
The specification describes a large set of functions (in the documentation – capabilities) that the client and server can support. However, it would be strange to require the client or server to support all available functions. Therefore, the client and server first exchange initialize messages, in which they indicate what they “can do”.
The message itself would look something like this:
{
"jsonrpc": "2.0",
"id": 1,
"method": "initialize",
"params": {
"processId": null,
"rootUri": "file:///path/to/root",
"capabilities": {
"textDocument": {
"synchronization": {
"willSave": true,
"willSaveWaitUntil": true,
"didSave": true
},
"hover": {
"dynamicRegistration": true
},
"completion": {
"dynamicRegistration": true,
"completionItem": {
"snippetSupport": true
}
},
"signatureHelp": {
"dynamicRegistration": true
}
},
"workspace": {
"symbol": {
"dynamicRegistration": true
}
}
}
}
}Another scenario
When a user opens a file in an editor, the client notifies the language server that the document has been opened by sending the message “
textDocument/didOpen” The contents of the document are no longer stored in the file system and are transferred to the tool's RAM.When changes are made to a document, the client notifies the server of the modifications using the message “
textDocument/didChange” In response to this, the language server updates the program's semantic data, analyzes it, and sends information about errors and warnings to the tool via the message “textDocument/publishDiagnostics“.When the user selects the Go To Definition command for a symbol in the editor, the client sends a ” Go To Definition ” request to the server.
textDocument/definition” The request contains two parameters: the document URI and the position in the text from which the request was initiated. In response, the server transmits the document URI and the position of the symbol definition in this document.When closing a document (file), the user initiates the client sending a notification “
textDocument/didClose” This notification informs the language server that the document is no longer loaded into RAM and the file contents in the file system have been updated.
To sum up
Messages consist of a jsonRPC header and body.
Messages can be requests/responses or notifications that do not require a response.
The text editor does not need to wait for a response to a request to send additional requests.
The client can send notifications such as
textDocument/didOpenortextDocument/publishDiagnostics.The client and server exchange initialize messages declaring the supported features.
LSP server
Let's write our own small LSP server. We'll use golang as the main tool, but you can use any other programming language.
Client
But first, you need a client to run the server. VSCode and JetBrains IDE have a client extension template that you need to make some minor edits to and it will work. Let's look at the VScode example from Language Server Extension Guide — for a detailed immersion, I recommend studying it. Let's clone it repository and we will assemble the project.
git clone https://github.com/microsoft/vscode-extension-samples.git
cd vscode-extension-samples/lsp-sample
npm install
npm run compile
code .We will get the following project structure:
├── client
│ ├── src
│ │ ├── test
│ │ └── extension.ts // тут будет лежать основной код для расширения
├── package.json
└── server // Сервер у нас свой, это мы удаляем
└── src
└── server.tsVSCode supports communication with the server via IPC, socket, and output stream. In our example, we will consider working via a socket. We will also add the port to listen to in the extension settings and remove the link to the package with the server from tsconfig.json.
In package.json we add the port and remove everything related to the server:
{
"name": "lsp-sample",
"description": "A language server example",
"author": "Microsoft Corporation",
"license": "MIT",
"version": "1.0.0",
"repository": {
"type": "git",
"url": "https://github.com/Microsoft/vscode-extension-samples"
},
"publisher": "vscode-samples",
"categories": [],
"keywords": [
"multi-root ready"
],
"engines": {
"vscode": "^1.75.0"
},
"activationEvents": [
"onLanguage:plaintext"
],
"main": "./client/out/extension",
"contributes": {
"configuration": {
"type": "object",
"title": "Configuration",
"properties": {
"serverPort": {
"type": "number",
"default": 9091,
"description": "Port for lsp server"
}
}
}
},
"scripts": {
"vscode:prepublish": "npm run compile",
"compile": "tsc -b",
"watch": "tsc -b -w",
"lint": "eslint ./client/src --ext .ts,.tsx",
"postinstall": "cd client && npm install && cd ..",
"test": "sh ./scripts/e2e.sh"
},
"devDependencies": {
"@types/mocha": "^10.0.6",
"@types/node": "^18.14.6",
"@typescript-eslint/eslint-plugin": "^7.1.0",
"@typescript-eslint/parser": "^7.1.0",
"eslint": "^8.57.0",
"mocha": "^10.3.0",
"typescript": "^5.3.3"
}
}Extension code:
import * as vscode from "vscode";
import { workspace, ExtensionContext } from 'vscode';
import * as net from 'net';
import {
LanguageClient,
LanguageClientOptions,
StreamInfo
} from 'vscode-languageclient/node';
let client: LanguageClient;
export function activate(context: ExtensionContext) {
const config = vscode.workspace.getConfiguration();
const serverPort: string = config.get("serverPort"); // Получаем порт из настроек окружения vscode
vscode.window.showInformationMessage(`Starting LSP client on port: ` + serverPort); // Отправим пользователю информацию о запуске расширения
const connectionInfo = {
port: Number(serverPort),
host: "localhost"
};
const serverOptions = () => {
// Подключение по сокету
const socket = net.connect(connectionInfo);
const result: StreamInfo = {
writer: socket,
reader: socket
};
return Promise.resolve(result);
};
const clientOptions: LanguageClientOptions = {
documentSelector: [{ scheme: 'file', language: 'yaml' }], // Указываем расширение файлов, с которыми поддерживаем работу
synchronize: {
fileEvents: workspace.createFileSystemWatcher('**/.clientrc')
}
};
client = new LanguageClient(
'languageServerExample',
'Language Server Example',
serverOptions,
clientOptions
);
client.start();
}
export function deactivate(): Thenable<void> | undefined {
if (!client) {
return undefined;
}
return client.stop();
}Server
Let's look at an example of a server for YAML files. The server will support the functions hover, completion, initialize, didChange, didOpen, diagnosticThe main focus will be on supporting the LSP protocol, and the logic for processing these functions will be on stubs.
To exchange messages, we will need the models described in the protocol, but transferring them ourselves is an expensive and uninteresting task. Therefore, we will take those already generated from gopls — LSP server for Go (license allows). Need files tsprotocol.go, uril.go, tsdocument_changes.go, pathutil/utils.go.
Let's create a router that will process the called method, read from the socket and write a response to it.
type Mux struct {
concurrencyLimit int64
requestHandlers map[string]handlers.Request
notificationHandlers map[string]handlers.Notification
writeLock *sync.Mutex
conn *net.Conn
}There are two types of handlers: requests (initialize, hover, completion) and notifications (initialized, didOpen, didChange). The difference is that the notification does not wait for a response.
type Request interface {
Call(params json.RawMessage) (any, error)
}
type Notification interface {
Call(params json.RawMessage) error
}Let's create a server that creates a connection, adds handlers, and runs code diagnostics. For diagnostics, we'll create a channel that will receive changes in the code — in our case, all the content. The diagnostic process itself is a separate goroutine that reads the channel with changes in the code and sends any notifications to the client.
func (s *Server) runDiagnostic(documentUpdates chan protocol.TextDocumentItem) {
for doc := range documentUpdates {
diagnostics := s.createDiagnostics(doc)
err := s.mux.Notify(
handlers.PublishDiagnosticsMethod,
protocol.PublishDiagnosticsParams{
URI: doc.URI,
Version: doc.Version,
Diagnostics: diagnostics,
})
if err != nil {
slog.Error("error to send diagnostic notify", slog.Any("err", err))
return
}
}
}Server functions
The first thing you need to do is tell the editor what functions the server supports. To do this, you need to send a response to the client for the event initialize. We indicate support hover And completion and the type of synchronization. There are several types of synchronization: none, full (the entire file is sent to the server when changed), sequential (“symbol-by-symbol”). We will consider full synchronization.
const (
TextDocumentSyncKindNone protocol.TextDocumentSyncKind = iota
TextDocumentSyncKindFull
TextDocumentSyncKindIncremental
)
func (i Initialize) Call(params json.RawMessage) (any, error) {
slog.Info("received initialize method")
iParams, err := i.parseParams(params)
if err != nil {
slog.Error("Error to parse initialized params")
return nil, err
}
slog.Debug("initialized params", slog.Any("params", *iParams))
result := protocol.InitializeResult{
Capabilities: protocol.ServerCapabilities{
TextDocumentSync: TextDocumentSyncKindFull,
HoverProvider: &protocol.Or_ServerCapabilities_hoverProvider{},
CompletionProvider: &protocol.CompletionOptions{},
},
ServerInfo: &protocol.ServerInfo{
Name: meta.Name,
Version: meta.Version,
},
}
return result, nil
}
func (i Initialize) parseParams(params json.RawMessage) (*protocol.InitializedParams, error) {
var initializeParams protocol.InitializedParams
if err := json.Unmarshal(params, &initializeParams); err != nil {
return nil, err
}
return &initializeParams, nil
}Now we need to define two notifications: didOpen And didChangeto receive code changes from the client. In these requests, we pass the received code to the change channel. This channel is read by the diagnostics goroutine.
type DidChange struct {
documentUpdates chan protocol.TextDocumentItem
}
func NewDidChange(documentUpdates chan protocol.TextDocumentItem) *DidChange {
return &DidChange{documentUpdates: documentUpdates}
}
func (d DidChange) Call(params json.RawMessage) error {
slog.Info("received didChange notification")
changeParams, err := d.parseParams(params)
if err != nil {
slog.Error("Error to parse didChange params")
return err
}
slog.Debug("didChange params", slog.Any("params", *changeParams))
d.documentUpdates <- protocol.TextDocumentItem{
URI: changeParams.TextDocument.URI,
Version: changeParams.TextDocument.Version,
Text: changeParams.ContentChanges[0].Text,
}
return nil
}Messages in this channel look like this:
type TextDocumentItem struct {
// путь до файла
URI DocumentURI `json:"uri"`
// Номер версии документа, увеличивается после change, including undo/redo
Version int32 `json:"version"`
// Содержимое документа
Text string `json:"text"`
}For inquiries hover And completion you will need a map that will store the current state of files by their path. This state is updated when reading from the change channel in the document. Hover accepts the following as input arguments:
type HoverParams struct {
TextDocumentPositionParams // позиция курсора в документе (строка и номер символа)
WorkDoneProgressParams // необязательный параметр о состоянии прогресса
}The response is expected to be:
type Hover struct {
// Отображаемый ответ. Может быть обычным текстом или markdown
Contents MarkupContent `json:"contents"`
// Можно выделить часть текста, для который выполняется hover
Range Range `json:"range,omitempty"`
}Processing request:
func NewHover(fileURIToContents *map[string][]string) *Hover {
return &Hover{fileURIToContents: fileURIToContents}
}
func (h Hover) Call(params json.RawMessage) (any, error) {
slog.Info("received hover method")
hParams, err := h.parseParams(params)
if err != nil {
slog.Error("Error to parse hover params")
return nil, err
}
slog.Debug("hover params", slog.Any("params", *hParams))
hoverItem := h.createHoverItem(hParams)
return hoverItem, nil
}
func (h Hover) createHoverItem(hParams *protocol.HoverParams) *protocol.Hover {
hoverItem := &protocol.Hover{
Contents: protocol.MarkupContent{
Kind: protocol.Markdown,
Value: "some hover",
},
Range: protocol.Range{
Start: protocol.Position{
Line: hParams.Position.Line,
Character: hParams.Position.Character,
},
End: protocol.Position{
Line: hParams.Position.Line,
Character: hParams.Position.Character,
},
},
}
return hoverItem
}Request parameters for completion:
type CompletionParams struct {
// Опциональный параметр, контекст вызова. Например вызов по символу, как пример "." у инстанса класса
Context CompletionContext `json:"context,omitempty"`
TextDocumentPositionParams // позиция в документе, аналогично hover
WorkDoneProgressParams // аналогично hover
PartialResultParams // опционально, передача ответа в виде стрима
}A list is expected in response CompletionItemwith the required parameter “value to insert”.
func (c Completion) Call(params json.RawMessage) (any, error) {
slog.Info("received completion method")
completionParams, err := c.parseParams(params)
if err != nil {
slog.Error("Error to parse completion params")
return nil, err
}
slog.Debug("completion params", slog.Any("params", *completionParams))
suggestions := []protocol.CompletionItem{
{
Label: "some completion",
Kind: protocol.ValueCompletion,
},
}
return suggestions, err
}As you can see above, all requests and notifications are processed in a similar way. The biggest difficulty is the logic of document processing — building AST and working with it. We will consider this issue in another article. There are few difficulties in the implementation of the protocol itself. The optimal approach would be to look at the documentation and, for example, look at existing LSP servers (for example, gopls).
To sum it up
The LSP protocol has had a great impact on IDE development. It has become easier to support a large zoo of languages for different editors. In addition, it is not necessary to limit yourself to a programming language. You can use the LSP protocol for other documents, such as OpenAPI, k8s files, etc. In this article, I would like to draw the community's attention to this technology so that more solutions for ease of development appear. Especially since such approaches as IaC and AaC are becoming increasingly popular these days.
Server source code: https://gitverse.ru/Asako/LSP-server-example
Client source code: https://gitverse.ru/Asako/LSP-client-example
