How to Reduce ChatGPT Hallucinations: 6 Ways
One of the main problems with working with neural networks is their tendency to invent unreliable answers. This is called “hallucinations”. To avoid them, you need to understand how to make promts. In this article, I will share six ways to reduce hallucinations. It’s all about how you create queries.
Hallucinations is a phenomenon in which ChatGPT generates content that does not correspond to reality. This can happen due to insufficient training data, incorrect interpretation of the input data, or due to the architecture of the model. This leads to incorrect or misleading results.
Machine learning experts offer different solutions to the problem. Among them are the creation of specialized data sets, the use of reinforcement learning (RLHF), the use of a search neural network that will parse external information for a generative neural network, etc. OpenAI CEO Sam Altman believesthat it will be possible to eliminate hallucinations in 15-2 years. Then a balance will be found between creativity and perfect accuracy of answers.
I investigated the problem from a user experience point of view – how to improve the model’s responses using prompts. For examples I used chat MashaGPTwhich gives remote access to ChatGPT.
1. Avoid ambiguity
When communicating with a chatbot, it is better to write clearly and precisely. Prompts that are vague, ambiguous, or lacking in detail cause the neural network to reason in an attempt to fill in the details you missed.
For example, here’s a very general query:
Tell us about the development of computer technology.
There are two ways to reduce uncertainty: either provide expanded context or ask for clarifying questions. I asked ChatGPT to improve my original request:
You are a prom generator. Help me compose a detailed PM with context and detail so that you can answer me as accurately as possible. Initial prompt: “Tell us about the development of computer technology.”
Now let’s ask the neural network to clarify the context of my request:
My initial prompt: “Tell me about the development of computer technology.” Ask me clarifying questions so that you understand the context of my request, and based on my answers, create a final detailed answer to my request.
2. Avoid combining unrelated concepts
Prompts that combine unrelated concepts may well cause the language model to generate a response that implies that the unrelated concepts are in fact related.
For example, a neural network can justify even an absurd connection between completely different processes:
Explain the relationship between the fermentation process in winemaking and the development of batteries for electric vehicles.
AI doesn’t really know anything about our world. It will try to fit what it is asked to do into its model. If he cannot fill in the variables using real facts, he will try to interpolate, providing fictions or hallucinations where he needs to fill in the gaps. Simply put, in unsuccessfully formulated requests, we ourselves set the generation of an incorrect response.
You can tell the bot that he is making a mistake and give him a rule:
If you don’t know the answer, don’t draw any conclusions or make up answers. Just tell me you don’t know the answer.
This is similar to the RLHF learning model, but to me it seems more like the neural network is simply following the user’s instructions. I am ready to agree with any comment.
3. Indicate sources of information
Unlike search engines, neural networks do not link to sources, but generate answers from the entire array of data on which they were trained. Therefore, they often make up news, scientific articles or books. For example, I asked the bot to make a selection of sources on the topic “Factors for increasing the economic efficiency of using fixed assets of an enterprise.”
All Russian-language sources are fake. With English speakers it is slightly better, since the dataset for ChatGPT was used mainly in English, but there are also inaccuracies there. Search bots like Perplexity AI and You.com, which are connected to the Internet, have the same problem. They may produce a news digest on a topic with non-existent links.
Therefore, do not write in the prom for the neural network so that it refers to research in the generated text – they can be fabricated. What the chatbot reports:
Instead, send ChatGPT data directly – the text of the article or a brief description of the book, if the volume is large. Then the neural network will be able to use them in its responses within the history of one chat.
4. Don’t forget about feedback
Like people, neural networks learn from experience. Generative neural networks learn from interactions with users. For example, if ChatGPT provided incorrect information about the date of a historical event, you can ask again, providing context or verifying the information through reliable sources. This will help reduce hallucinations and ensure that the answer you receive is accurate.
5. Tell the neural network what you want and don’t want from it
To narrow the context, tell ChatGPT directly what result you want. Conversely, preventing the neural network from generating responses that you don’t need is a great way to also prevent hallucinations. Simply ask the bot to exclude certain facts or data. For example:
I want you to help me create a Powerpoint presentation. You will need to research your given topic, formulate precise, concise and clear content for each page, and create a compelling piece of work that is both informative and engaging. Audience – students, optimal length – 6 slides. Topic: Pandas/NumPy libraries, modules and functions.
As you can see, the neural network even swears a little when it is assigned functionality beyond its capabilities. However, she understood the task and completed it quite clearly.
Additionally, you need to specify the avatar role for the chatbot. This way he can better understand in which direction to generate the response:
Act like a project manager. I have a project that I need to present to my client now. Write the text that I will use in my presentation, and then make a list of points that I should include in my presentation. My project: SaaS platform for marketing automation, solving Performance and CRM problems.
6. Monitor the temperature
Temperature is a parameter in language models that determines the level of randomness when generating text. Its value is from 0 to 1.0. The higher it is, the higher the level of “creativity” of the neural network in its responses. Temperature also plays a role in ChatGPT hallucinations as it controls the randomness of the results. While a lower temperature will produce relatively predictable results, a higher temperature will increase the randomness of the responses. This may increase the chances of “hallucinogenic” responses. For example:
Write a letter to send to clients. Subject: Message about the imminent end of the free trial subscription period for the video editor. An invitation to sign up for a paid subscription with a promotional code giving a 10% discount. The style is friendly and informative. Temperature=0
Now let’s go through the generation again. Temperature=1
There is more “creativity”, but along with it there are also errors in the text. Be sure to indicate the volume – this will reduce the amount of “water” in the text and will not allow the neural network to invent additional details.
Check, check, check
Whether you’re using a neural network to write code, create articles, or create homemade recipes, refine your promts using the methods described above. This may help ChatGPT do its job better, but you will still need to check its responses. I hope the article was not boring and useful. Write in the comments what other ways you know to reduce hallucinations.
