from idea to implementation
In general, this statement is not without foundation: ChatGPT is developing exponentially, and each new version of the neural network works an order of magnitude better than the previous one. However, I think we can still breathe a sigh of relief for the next six months – until the release of the 5th or even 6th iteration. Although GPT-4 can simplify the life of a developer, it does not yet replace him. Creating a Java project with ChatGPT is a long iterative process of trial and error, refining queries, close checks and correcting the “hallucinations” of the neural network. However, judge for yourself.
First phase of the experiment: domain analysis and UML generation
As part of the experiment, we decided to create an educational platform from scratch, like Udemy or Coursera. The advantage of ChatGPT is that it is a universal neural network, which means that with its help we can create not only a code, but also describe the entire structure of the project. For this reason, ChatGPT was used, and not specialized Copilot or Codex. I will also immediately clarify that the experiment was conducted in early spring – before the release of the new GPT-4o model.
So, I write a ChatGPT query:
“I want to create a system Educational platform: creating a system for conducting online courses, testing and tracking students' progress.
Specify use cases for this system.”
And I get this answer:
“To create an educational platform with online courses, testing and tracking of students' progress, you can consider the following use cases:
1. Registration and authentication of users:
Registration of new users (students and teachers).
Login using your credentials (username and password).
Password recovery.
2. Managing user profiles:
Editing personal information.
Changing account settings.
View and edit the list of courses the student is subscribed to.
3. Creating and editing courses:
Creating a new course indicating the name, description and other parameters.
Editing an existing course.
Adding and editing modules and lessons within a course.
Assigning teachers to the course.
4. Managing course materials:
Uploading and organizing educational materials (texts, videos, audio, images).
Creating and editing tests and assignments for students.
Setting up the conditions for completing a course and receiving certificates.
5. View courses and educational materials:
Search and select courses.
View lessons and study materials.
Taking tests and completing assignments.
6. Tracking students' progress:
View your course progress (for students).
Monitoring students' academic performance and activity (for teachers).
Generation of reports and statistics on rates.
7. Interaction between users:
Exchange of messages and comments between students and teachers.
Notifications about new materials, tasks, events, etc.
8. Managing access and user rights:
Assigning roles and access rights to users (administrators, teachers, students).
Sounds logical, let's move on to visualizing the requirements.
I write the following query:
“Develop a UML diagram in PlantUML format for these use cases.
Clarification: Can I not use the aliases UC1, UC2, etc? And place this diagram vertically so that teacher is under student?”
I get this response:
@startuml
actor Student as student
actor Teacher as teacher
actor Administrator as admin
left to right direction
student --> (User Registration)
student --> (User Authentication)
student --> (Password Recovery)
student --> (Profile Management)
student --> (Search & Enroll in Courses)
student --> (View Lessons & Study Materials)
student --> (Take Tests & Complete Assignments)
student --> (Monitor Student Progress)
student --> (Messaging & Comments)
student --> (Notifications)
teacher --> (User Authentication)
teacher --> (Profile Management)
teacher --> (Create/Edit Course)
teacher --> (Manage Course Materials)
teacher --> (Create/Edit Tests & Assignments)
teacher --> (Assign Teachers)
teacher --> (Monitor Student Progress)
teacher --> (Generate Reports & Statistics)
teacher --> (Messaging & Comments)
teacher --> (Notifications)
admin --> (Profile Management)
admin --> (Create/Edit Course)
admin --> (Manage Course Materials)
admin --> (Assign Teachers)
admin --> (Assign Roles & Access Rights)
@endumlThe generated UML script was then visualized using an open source tool PlantUML:
The next step is the domain model. We set the following prompt:
“Develop a domain model for the system described in the use case diagram:
We obtain a domain model with key entities associated with the educational platform, such as users, courses, lessons, materials, tests, assignments, and others:
Let's visualize the domain model using the same tool:
The model is still a bit raw, so we are refining the request:
«Please specify fields for the Teacher, Student, Administrator classes.”
Here is the updated domain model:
The Teacher, Student, and Administrator classes now inherit from the User base class and have the following additional fields and relationships specific to each role:
1. Student:
enrollments: List of course enrollments (Enrollment) for this student.
testResults: List of test results (TestResult) for this student.
assignmentSubmissions: List of completed assignments (AssignmentSubmission) for this student.
2. Teacher:
3. Administrator:
In this case, the administrator inherits all fields from the User base class and there is no need to add additional fields. If specific properties or functionality for the administrator are required, they can be added later.
Let's continue to clarify the request:
“Rename Lesson to Module and add duration information for Material, Test and Assignment.”
We get:
Now the Lesson class has been renamed to Module, and a new field “duration” has been added to the Material, Test and Assignment classes to reflect the duration of the material, test or task. This duration can be measured in minutes, hours or days depending on the context and system requirements.
The second phase of the experiment: connecting and using the code generator
ChatGPT is not an applied neural network for development, it cannot – yet – create the actual files and directory structure we need, but can only output code in text form, which is of little use without the appropriate environment. Therefore, at this stage we connect the JAIG code generator – a separate tool designed to generate code using the ChatGPT API. It is supplied as a .jar file and a settings file. JAIG was created specifically for course IBS for development with the help of machine intelligence, but you can use any other available “converter” for the same purposes or even write your own simple script that will manage sending and receiving requests in ChatGPT.
JAIG is connected as an external tool. In the ExternalTools settings you only need to set the correct configuration:
Program: java
Arguments: -jar jCodeGen/jCodeGen.jar $FilePathRelativeToProjectRoot$
Working directory: $ProjectFileDir$
To use JAIG, you need to create a folder, place a request in it in a file with the .txt extension — say, prompt.txt — and start generating. The request can include links to existing files and folders. Please note: the link must start with “/” and contain the full path from the project root. The contents of the files will be automatically substituted into the full request prompt-request.txt. Based on this file, the system will generate the final request and then receive a response from the AI, saving it in the prompt-response.txt file. If the neural network response contains code and the packages are specified correctly in this code, then automatic parsing of the response is enabled: .java files are distributed among folders according to the package. The parsing result is written to the prompt-response-parsed folder. Next, the parsing result must be copied to the src folder, the generated code must be analyzed and errors must be fixed. Based on the adjustments made, a patch will be generated for automatic correction during the next code generation.
So the application development cycle using ChatGPT and JAIG looks like this:
Phase 3 of the experiment: Implementing the domain model in Java
Generating a Java Domain Model Based on a UML Diagram
In the root of the project, we create a folder requests and in it a subfolder 01_PlantUML_to_domain_model. In the subfolder, we put a plain-ASCII file 01_PlantUML_to_domain_model.txt with the following request:
/task/requirements.txt
/task/plant-uml.txt
Generate domain model classes in Java. Specify all required fields.
The system generated a domain model, but without specifying packages. We correct the request:
/task/requirements.txt
Below are all the classes of the domain model:
/requests/01_PlantUML_to_domain_model-response.txt
Split all classes into packages if the base package is ru.ibs.eduplatform.model.
Each package must contain no more than 5 classes.
AI offers a breakdown of packages based on functionality:
We ask the neural network to split the classes into packages and add Lombok annotations:
/requests/01_PlantUML_to_domain_model-response.txt
/requests/02_packages-response.txt
Add Lombok annotations @Data, @NoArgsConstructor to all domain model classes.
Add @AllArgsConstructor to all domain model classes except inherited ones.
Use ru.ibs.eduplatform.model as a base package.
Make sure all imports are correct and match the generated packages.
Print the code for all classes, including the package name of each class and imports.
Each class must have a package specified.
Make sure all classes are included in the results.
As a result, we have generated all the domain model classes. Now we need to parse the response, that is, perform a semantic analysis of the generated code. Preliminary parsing is performed automatically. For all classes, their packages were correctly specified, so after 5 seconds, the code generator automatically arranged the classes in folders according to their packages and wrote the result to the prompt-response-parsed subfolder.
Testing the domain model by filling it with test data
We write another request:
/src/main/java/ru/ibs/eduplatform/model/**
Generate a class that populates the domain model with test data.
Create 2-3 students, add 2 courses and some materials,
2 tests with 2-3 questions, enroll students in the course.
Display information about all courses.
For each course, display a list of students (name, last name, email), a list of materials, and a list of tests.
The answer must contain only the code.
Use ru.ibs.eduplatform as a package.
The neural network forgot to add constructors for inherited classes, so I had to modify the query:
/src/main/java/ru/ibs/eduplatform/model/**
For domain model classes that inherit from User, add constructors using super() and calling the superclass constructor.
Display only the code of classes that have constructors added to them.
Generating user interface
Now that we've sorted out the constructors, let's move on to the application's appearance:
/src/main/java/ru/ibs/eduplatform/model/**
The data model is presented above.
Generate code for a Java program with a command-line user interface that will display a menu with the following items:
Add a student (must allow student data to be entered)
Find a student (when selecting, you need to request the student's name, find him by name and show full information)
Enroll a student in a course (should allow you to find a course by the student's first and last name, add that student to the course, and create an Enrollment by adding it to the student's enrollments)
Add course (should allow entering course details)
Find Course (should find a course by name and output all course details and a list of enrolled students)
The interface must be in Russian.
The answer must contain only the code.
The output is the UI.java file with the first version of the code.
Fragment of the received code
The third phase of the experiment: ensuring persistence and working on errors
It is almost impossible to get the perfect code the first time. ChatGPT 4.0 does not “digest” complex queries consisting of many requirements well.
How to fix it:
correct the code manually; for example, add a missing import from a particular class, structure, or package;
change the order of requirements in the request, listing unfulfilled requirements closer to the end, where they will have higher priority;
add a requirement clarification to the request;
change the temperature to get a different answer;
split the request into several requests to simplify the neural network's task.
Generating user interface
Clarification for ChatGPT:
The AI generates a new user interface in UI.java.
Database support
We write another request:
Add JPA support, package and import declarations for each generated class.
Use jakarta.persistence package in imports instead of javax.persistence.
Generate constructors without id (id should be generated automatically).
/src/main/java/ru/ibs/eduplatform/model/**
Add fully qualified imports for all imports from all classes.
Great, now all domain objects have JPA support.
To generate Spring Data repositories, we write an almost identical query:
Add JPA support, package and import declarations for each generated class.
Use jakarta.persistence package in imports instead of javax.persistence.
/src/main/java/ru/ibs/eduplatform/model/**
Add fully qualified imports for all imports from all classes.
Request for database support for interface:
/src/main/java/ru/ibs/eduplatform/model/**
/src/main/java/ru/ibs/eduplatform/repositories/**
Use the data model and repositories to update the UI class.
Data should be loaded from the database when requested by the user and placed into the database after any changes.
Modify the UI class to use repositories.
/src/main/java/ru/ibs/eduplatform/UI.java
Result: all data when working through the command line interface is loaded from the database and saved to the database.
Supplement: Display all students and courses
Request:
/src/main/java/ru/ibs/eduplatform/repositories/**
/src/main/java/ru/ibs/eduplatform/EduplatformUIApplication.java
Add to the menu items show all students and all courses
The system generated a new class EduplatformUIApplication, which includes items about displaying students and courses:
case 6:
showAllStudents();
break;
case 7:
showAllCourses();
break;Working on bugs: LazyInitializationException
Of course, there were some errors. After adding a course, when trying to search for a course, the program threw a LazyInitializationException. This error occurs when Hibernate tries to load data from related entities after the session is closed. In this case, when calling the `findCourse()` method, the session is already closed, and Hibernate cannot load data from related entities.
In response to the complaint, GPT suggested adding the `@Transactional` annotation to the `findCourse()` method:
@Transactional
public void findCourse(Scanner scanner) {
// ...
}At the same time, the machine intelligence also explained that such a solution should allow Hibernate to open a new session when executing the `findCourse()` method and load data from related entities.
But that's in theory. In practice, the program threw an exception again. It turned out that the annotation doesn't work because we call this method inside the same class. In Spring, the `@Transactional` annotation works through a proxy, and when we try to call a method inside the same class, the proxy is not used and the transaction is not started.
To solve this problem, GhatGPT suggested to move the `findCourse` method to a separate service class and add the `@Transactional` annotation to it. Then inject this service class into `EduplatformUIApplication` and call the `findCourse` method through this service class:
@Service
public class CourseService {
@Transactional
public void findCourse(Scanner scanner) {
// ваш код метода findCourse
}
}Problem solved.
Refactoring: moving business logic into services
Request:
Update the UI class using the Command design pattern.
/src/main/java/ru/ibs/eduplatform/UI.java
By using the Command design pattern, adding and working with commands becomes easier to maintain and modify.
Database persistence
Another problem with the AI-generated application: the data was not written to disk, meaning it was deleted from memory after the program was restarted.
To make the database persistent, ChatGPT suggested changing the H2 settings in the `application.properties` file:
“`
spring.datasource.url=jdbc:h2:file:./data/db;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE
spring.datasource.driverClassName=org.h2.Driver
spring.datasource.username=sa
spring.datasource.password=
spring.jpa.database-platform=org.hibernate.dialect.H2Dialect
spring.jpa.hibernate.ddl-auto=update
“`
The `DB_CLOSE_DELAY=-1` and `DB_CLOSE_ON_EXIT=FALSE` parameters specify that the database should remain open and not be closed when the application terminates.
Fix: Server shutdown
The next error is that when I selected “Exit” in the menu, the server continued to work.
To terminate the application and server, ChatGPT suggested adding a call to `System.exit(0)` after `scanner.close()` in the `run` method. Here is the updated code:
@Override
public void run(String... args) throws Exception {
...
scanner.close();
System.exit(0); // Завершаем работу сервера
}Fix: Don't add course if it already exists
The patches didn't end there. Another problem was discovered – the program allowed adding duplicate courses with the same names.
To avoid adding a course if a course with the same name already exists, ChatGPT recommended making the following changes to the `addCourse` method in the `CourseServiceImpl` class:
@Override
public void addCourse(String name, String description) {
Course existingCourse = courseRepository.findByName(name);
if (existingCourse != null) {
System.out.println("Курс с таким названием уже существует.");
return;
}
Course course = new Course(name, description);
courseRepository.save(course);
System.out.println("Курс успешно добавлен.");
}Phase 4 of the experiment: REST API implementation and natural language interface development
We will not dwell on this phase in detail. I think the detailed analysis of the errors above speaks for itself: it is not possible to take the ready-made code from ChatGPT and immediately launch it into work – both for the 3.5 model and on the “four”.
Next, we generated REST requests, controllers and services, tested the code using auto-generated unit tests and finally generated Swagger documentation.
The result was as follows:
It is impossible to cover all of our attempts to “attach” machine intelligence to code development in one short article, so let’s just briefly list where we actually managed to achieve acceptable results:
drafting requests with links to source codes and other documents;
sending requests via ChatGPT API (automatically or manually);
convenient integration with the development environment (IDEA);
viewing queries during generation, code highlighting;
parsing the generation results, decomposing the sources into files and packages;
writing generated files to the required folder (for example, to the canonical Maven folders src/main/java and src/test/java);
flexible generation settings, including selection of model (gpt3/gpt4, etc.) and temperature, globally or for each individual request;
creation and “auto-rolling” of patches for changes in the code;
generating a series of requests from templates (based on the Apache Velocity framework);
execution of requests in a single package (batch files);
a library of prompts, for example, for refactoring a selected fragment, generating javadoc comments, explaining a particular snippet, identifying vulnerabilities and anti-patterns, etc.;
rollbacks of changes in code.
Experiment: Conclusions
The fourth iteration of ChatGPT works noticeably better than version 3.5. In our previous experiment, a colleague spent a total of 10 days “dancing” with the neural network. At the same time, manually writing the code would have taken one working day. ChatGPT 4.0, on the contrary, shortens, rather than delays, the development process. Thanks to the transition to the “four”, in our experiments there was a stable tendency to save development time by at least 20-25%. However, the generated code still requires serious revisions, and the model periodically makes rather ridiculous mistakes. Let's see what ChatGPT 5.0 will be like.
We also hope that API access will appear to large language models from other vendors that show interesting results specifically for code generation, such as Llama or Vicuna.
P.S. JAIG, which turned out to be the key to communicating with machine intelligence, is an open source tool and is available at link.
