Introduction to OpenAI Assistants API (and also ChatGPT Custom Instructions)

Introduction

The release of new OpenAI products last month included many features. This tutorial is devoted to an introduction to the practical use of one of them – Assistants API. We will also also touch on a feature released in August as part of ChatGPT called Custom Instructions.

This article is not intended to cover all technical details. Rather, the goal is:

A brief and visual description of the main stages from setting a “business problem” to implementing a working prototype of a “product” (web applications based on the Assistants API)
Simultaneous consideration of both substantive and technical aspects.

Here I limit myself to the assumption that these features can be used to simplify the execution of typical repetitive tasks (although there are certainly other scenarios for their use). In particular, in this tutorial, as an example, we will use the task of writing Cover Letters (cover letters) adapted for different vacancies, when the user, on the one hand, has a resume, and on the other, many different vacancies, each of which has slightly different requirements . Accordingly, the user needs to prepare a cover letter for each vacancy, which would highlight exactly those skills and parts of experience from the resume that are relevant to this position.

In this article I will try to describe the implementation as briefly as possible and show clear examples. But first, a few words about what a prompt is for our example.

General structure of a prompt

Further reasoning is based on the fact that we are dealing with a prompt (statement of a problem for AI) of a certain type, which includes the following components:

Fixed component:
1.a. Fixed task statement (write me a cover letter for a specific vacancy based on your resume)
1.b. Fixed information part (resume – description of experience and skills)
Variable information part (job description).

Those. The prompt that solves the problem of writing a cover letter looks schematically as follows:

1.a	2	1.a	1.b
You must write a cover letter for the vacancy:	[описание вакансии]	This cover letter must match my experience and skills:	[описание опыта и навыков]

There is no problem at all in creating such a prompt manually in the ChatGPT interface once or twice. But if this task is periodically repeated, then the cumbersome format of the prompt and its structure may make you think about some optimization of the process.

We will take this optimization as an example to familiarize ourselves with the functionality that this article is devoted to – Custom Instructions and Assistants API.

Custom Instructions

For a general understanding of the available tools, before moving on to Assistants, let’s see how this task can be solved using Custom Instructions.

Custom Instructions is a setting implemented in ChatGPT in August 2023 that allows you to add auxiliary (“fixed”, “background”) information to those requests that are entered in the main chat.

It is called up from the “Profile” menu, located at the very bottom of the left panel of the main ChatGPT interface, and consists of two main sections:

“About yourself” – what the user can tell about his background, his goals and needs to help ChatGPT give more adequate answers.
“How to respond” – what the user generally expects from ChatGPT responses, what they should be in format, size, tone, etc.

The problem of quickly writing cover letters is solved as follows:

In the first section “About yourself” you need to enter your resume, description of skills, experience, technologies. The wording will probably have to be shortened and switched to telegraph style, because… the text must be no more than 1500 characters. This section corresponds to component 1.b of the structure described in the previous section.
In the second section you need to enter the following prompt (component 1.a):

When the user enters the command /WriteCoverLetter you should turn into CoverLetterGPT, which writes customized cover letters. First, you need to ask the user to provide a description of the job they are applying for. You should use this job description and my professional experience to write your cover letter. (This prompt can and should be supplemented and clarified. Here I provide a minimal version sufficient to understand the idea)

Just in case, English version

When user enters /WriteCoverLetter you must become a CoverLetterGPT, that creates customized cover letters. First of all you have to ask the user to provide the description of the position to which he wants to apply. You must use this description of the position and information about my professional background to write a cover letter.

Thus, after activating and saving Custom Instructions, you can enter the /WriteCoverLetter command into the chat. After this, the chat will ask for information about the vacancy, in response to which the user needs to copy the vacancy description from the job advertisement into it. As a result, we immediately receive the text of the cover letter. As a result, if you need to write several letters, you can use relatively simple and convenient operations for this.

This approach may have some limitations. So let’s move on to the main topic of this article.

Assistants

In my opinion, Assistants can be seen as a logical continuation of the idea of Custom Instructions: both of these functionality are containers (encapsulation mechanisms) for fixed knowledge (prompt 1.a and basic information 1.b). They assume that the user, while working, should minimize the cost of time and effort, focusing on providing variable information (2) to solve the task for which the “assistant” was developed.

The differences from Custom Instructions are that:

Assistants can be created in unlimited quantities
Assistants are available for use via API, i.e. can be included as a module in a custom product and built into some business logic
Assistants can use not only the contents of text fields as input data, but also attached files
Assistants have additional functionality – Tools
Assistants, in principle, cannot be free – their use, although it costs relatively little money (if you use GPT-3.5), nevertheless always ends up in the billing (while Custom Instructions are available in the free version of ChatGPT).

So, in the Assistants section (https://platform.openai.com/assistants) you can create (if you have a card linked in your payment settings) an unlimited number of “Assistant” copies, each of which has:

Name and unique identifier
Instructions – a text field with a prompt and the necessary initial information. Here, to begin with, we will place all the “fixed information” – both the prompt and the summary from our task about cover letters. In the next chapter, information “about yourself” (1.b) will be presented separately.
GPT model: Choosing between 3.5 and 4 and their variations.
Additional functions – Tools (Functions, Code interpreter, Retrieval, Files), which we will try to do without for now to simplify things.

You can also create Assistants through the API, however, for this article, in my opinion, this is redundant, so I will limit myself here to a description of the work through the interface, and we will return to the API later, when we need to use the “assistant”.

In relation to our example with a cover letter, in Instructions you need to enter approximately the same prompt and information “about yourself” that we previously considered for Custom Instructions:

You need to turn yourself into a CoverLetterGPT who writes tailored cover letters. When starting a conversation, you should ask the user to provide a job description and then write a cover letter based on that job and the following information about experience and skills: [подробное резюме].

Next, go to the Playground/Assistants section (https://platform.openai.com/playground), there we select the assistant we just created, in the chat to start the dialogue, enter something like start and click “Add and run”. Then everything is the same as in ChatGPT & Custom Instructions: the chat immediately asks you to provide a description of the vacancy and, after copy-pasting the text of the vacancy, immediately issues a cover letter. Using the Clear button, the process can be restarted.

Files

A resume (description of experience and skills) does not have to be included in the Instructions along with the prompt. A file with this text can be attached to the assistant (this field is located at the very bottom of the settings window) and its contents will be used in the same way as if this text were directly in Instructions. At the same time, in the assistant settings in the Tools section, be sure to remember to select the Retrieval option:

In my experiments, just in case, I used the phrase “Professional background and skills” simultaneously in the file name, in the first line inside the file and in the text of the prompt (in Instructions). Those. I tried to implement a certain pointer key, which should show as clearly as possible which part of the prompt the contents of the file belong to. I don’t know how necessary such “pseudo-keys” are – maybe the AI itself will figure out what’s what without any hints. Here you need to experiment more. I have not yet come across any specific information on this issue.

If we talk about files in general outside the context of our example, then we can additionally say the following:

The contents of the files are available not only for Retrieval (the full name of this feature: Knowledge retrieval), but also for the Code interpreter (if it is enabled in the settings). Moreover, there are much more formats available for Code interpreter than for Retrieval, which, for example, does not have access to csv and xlsx (https://platform.openai.com/docs/assistants/tools/supported-files).
Files can be either at the assistant level (as in the example above) or at the thread level (Assistant-level vs. Thread-level).

Assistants API

Let’s move on to the API – consider a small web application that implements the minimum functionality required to solve the problem with a cover letter.

My implementation (on Nuxt3: GitHub & Prod) has two operating modes:

Demo – works with the assistant that I created in my profile (with a fictitious resume of an abstract Middle Data Analyst hardwired into it), and with a key hardwired into the back-end.
Working – works only from the frontend, directly with OpenAI services, i.e. without back-end involvement. In it you need to enter your own key and assistant ID in the interface, which you must first create in your profile on platform.openai.com as described above.

Each step of the dialogue with the assistant via the API (as, indeed, through the Playground) is called Run (literally in this context it is translated, apparently, as “run”; I would rather translate it as “iteration of the dialogue”, but I will still use the English term ). In our example, there will be two Runs: a start trigger (in the StartConversation function) and, in fact, receiving the text of the cover letter (in the GenerateResult function).

The preconditions and input parameters for Run are the created assistant and the thread with the message. We have already manually created the assistant in the admin panel (from there you need to manually copy its identifier), and the thread needs to be created via the API, adding a service trigger message “start conversation” to it, immediately before the first Run.

After manipulating the thread, you need to start the first Run, wait for it to complete, and extract the response from the thread.

  async function StartConversation () {
    
    openai.value = new OpenAI({
      apiKey: apiKey.value,
      dangerouslyAllowBrowser: true,
    });
  
    const thread = await openai.value.beta.threads.create();
    
    await openai.value.beta.threads.messages.create(
      thread.id,
      {
        role: "user",
        content: "start conversation"
      }
    );
    
    const run = await openai.value.beta.threads.runs.create(
      thread.id,
      { 
        assistant_id: assistantId.value,
        //instructions: "Customized CV may be used here instead of the default one, that is hardcoded in the assistant.",
      }
    );
    
    // wait for the run to complete
    const run_status = ref(await openai.value.beta.threads.runs.retrieve(
      thread.id,
      run.id
    ));
  
    while (run_status.value.status != "completed" && run.status != "failed") {
      await sleep(1500);
      run_status.value = await openai.value.beta.threads.runs.retrieve(
        thread.id,
        run.id
      );
    }
  
  
    const assistants_response = await openai.value.beta.threads.messages.list(
      thread.id
    );
  
    WelcomeMessage.value = assistants_response.data[0].content[0].text.value;
  }

In the application interface, we bind all these operations (initializing a thread, adding a message, starting and ending a Run, retrieving a response) to the “Start” button and after clicking on it and completing all actions, we show the user an invitation to enter a job description (WelcomeMessage in my example) , which was extracted from the thread after the completion of Run.

You have to wait a while for the Run to complete (according to my measurements, it sometimes took more than 10 seconds). This requires cyclic checking of Run status (openai.beta.threads.runs.retrieve). In particular, you need to wait until Run transitions from the in_progress state to completed in order to proceed to receiving a response from the thread.

For the second Run, the user enters a job description in the appropriate text field and clicks the button to proceed to the final step. This button click corresponds to a call to the second GenerateResult function, which contains steps similar to those we examined in the first iteration:

Adding a job description from a text field to the thread (just like adding the initial start conversation message there at the beginning of the dialogue)
Run Run
Waiting for it to complete
Retrieves the latest message from a thread.

Based on the results of the second Run, we obtain the text of the cover letter, which we display to the user (MainMessage). To do this, you need to extract the latest text content from the thread. The last message in a thread always has an index of zero. Accordingly, the very first message with which the thread began is automatically moved to fourth place (index 3).

Thus, we achieved the final goal of this introductory tutorial – we received a prototype of functionality in our web application, similar to what we observed in ChatGPT/CustomInstructions and in Playground Assistants.

Conclusion

It is clear that such primitive interaction between the user and the assistant in one iteration is probably not exactly what the OpenAI developers had in mind. However, this approach seemed appropriate to me in order to demonstrate both the technical and practical essence of this functionality as briefly as possible, while covering the entire cycle from discussing a practical problem to implementing the “application”.

To complete the picture, the article should probably also include a description of GPTs. However, the author did not bother to sign up for a “Plus” subscription in advance, and at the time of publication of the text this opportunity was closed.

Advanced material on the Assistants API topic, in my opinion, could include analysis of the following topics:

Implementation of such an application for a wide audience (as opposed to the “for oneself” option described here)
Practical application of other Tools
Using Thread Level Files
Implementing and using JSON responses in Assistants.