Is it time to start programming with LLM or is it too early?
In my previous article, a big discussion broke out among respected readers about whether the model is capable of adequately programming and freeing the programmer from significant coding labor costs.
Its title, which is written with a bit of humor and self-irony, was perceived by some as very straightforward. And yet, the question is still relevant. In this article, I will use LLM to refactor two samples of dirty code and analyze the results.
First, let us summarize the opinions of the majority of those who spoke on the topic.
Some believe that the model is not capable of replacing a programmer at any stage of his work, even at the most basic, when it is necessary to write the simplest code without errors, and they support their opinion with a variety of logical reasoning. The most common argument is that the training sample, although large, is finite, and therefore the model cannot go beyond the knowledge obtained from it, period.
Others, more curious, tried to get, as a rule, from well-known proprietary models some code according to their requests and in many cases were disappointed with the result. Either it was just text similar to code, but in general, strictly speaking, not code, or code, but with broken logic and errors. After several such attempts, a stable opinion was formed that it simply does not work, and if someone succeeds and works, then it is either some kind of magic or deception.
There are still a few voices of those who allow the possibility of using LLM as a programmer, but under certain conditions. And there are very few programmers who have learned to use models in practice so that they have become real assistants in writing code. And this, it turns out, is possible.
But why do some succeed and others don’t?
There are several reasons at once. The first is an attempt to use the most famous and popular, promoted models with general knowledge for writing code, such as GPT-4, Claude, Gemini, the Llama family of models and other remarkable models in many respects. But these are not models specialized for writing code. It's the same as if we asked an average and very well-read and educated person to write a program for us, and he simply studied everything little by little and unsystematically – literature, medicine, economics and much more, and additionally programming.
Actually, it would probably be a bit strange to expect a highly qualified and impeccable answer from him.
Therefore, to work on writing program code, you need to use specialized models trained specifically for this. They exist, for example, Codestral, Deepseek and others.
The second reason for failures is the request itself, the prompt (or rather their sequence), which must be composed competently and professionally. Yes, yes, this is also a kind of science (or art, if you like), which you need to learn and master the skills, as in any other business, if you expect a good result.
The third reason is methodological. Let's say we formulate a task, trying to cover everything we can at once, generalizing somewhere, leaving something out somewhere, putting forward contradictory requirements somewhere, and want to get the whole code in one fell swoop. Well, the result will be disastrous. Or there is a large volume of dirty, obscure code and it needs to be optimized and refactored. We just shove it all into a prompt with instructions, expecting a miracle. Unfortunately, it won't happen.
It is important to understand that tasks are solved in stages, in parts, in successive steps, from simple to complex, and also under constant control.
The fourth reason is purely technical. Not a single small model of 8B-13B size, for example, Code-Llama-13B or Nxcode-CQ-7B-orpo (it is also an improved CodeQwen1.5-7B) or deepseek-coder-6.7b has ever given me a satisfactory result in more or less serious coding tasks. These sizes are clearly not enough, it is better not to even waste your time and effort. Codestral at least has 22B weights and this is the minimum to have practical value. Therefore, specially trained for programming models of a larger size will probably be even more advanced and useful, with weights of 34B or 70B or more and here everything depends only on your hardware, if you work locally.
If these reasons don't stop us from working, will LLM be able to write acceptable code?
Yes, it can, and in most cases it will be high-quality code at a good level. And how can we now consider the model: is it a convenient new tool for writing code and nothing more? But it takes on a significant part of the intellectual and creative work that we used to do ourselves.
I think in programming there is a gradual transition from the concept of a person as a tool to a new concept of a manager as an executor, in which we come up with ideas, give her tasks, she carries them out, and we control the process. In principle, there is nothing new, just a different, more productive paradigm in work. Previously, a person would take a piece of iron, clamp it in a vice and grind it long and hard, wasting his physical and time resources. Now there are 5-axis CNC machines and, in fact, the same person, having slightly increased his level, now enjoys controlling this machine. At the same time, oddly enough, the quality of the product is several times, and maybe even orders of magnitude higher.
In the previous article, it was shown that LLM can help to create quite high-quality and working program code on demand, if certain rules are followed and the process is controlled. As one of the commentators in the discussion put it, finally the programmer began to program, setting tasks and working with the results, and not to engage in coding, having transferred this function to the model.
What else can LLM help with?
For example, in improving the existing code. Perhaps many have faced the problem of a significant amount of low-quality code, which was inherited from previous developers or this is your own code, written earlier without following the rules or in a hurry, the main thing is that it works. Understanding the logic of its work, understanding the meaning of intricate algorithms, simply tidying up and removing all the garbage, adding readability and the possibility of support in the future – these are tedious and thankless tasks, but necessary.
I have also encountered this problem more than once, and so now we will try to apply LLM to solve the problem of refactoring and optimizing sloppy and dirty code, without even delving into the meaning of its work. We will proceed from a general understanding of the work of experimental code samples and will rely on formal criteria for assessing the quality of the work done by the model.
As a model, I will again use the quantized Codestral-22B-v0.1-Q4_K_M, specially trained for coding in 80 languages (You can find out about the original model on the developers website).
Attempt number 1.
The first piece of code we will try to make look decent is a carelessly written function, apparently written in a hurry. It is intended, apparently, to read time-ordered data from two text files and form several return arrays from them. The processing algorithm is very confusing, with numerous unclear conditions and checks. Apparently the input data is not fully correlated and therefore efforts were made to harmonize them. In addition, unknown global variables are used in the processing.
The model has to understand the internal logic, as far as possible in a situation of great uncertainty, and without violating it, change the code according to our conditions. What are these conditions? In a nutshell, refactoring and optimization, but since the model needs to specify everything in the most detailed and unambiguous way, the set of requirements will be as follows:
– variable names should be in accordance with the logic of their use for better understanding;
– individual functional blocks need to be separated into separate functions;
– individual operations should be optimized;
– exception handling should be improved;
– we could ask for more comments on the code, but we won’t do that here yet.
Before I start refactoring, I should say that trying to get a good result in one go is likely to fail for many reasons at once. This is probably the most inflated expectation that leads to disappointment for many. My experience is that it is better to do it in several steps, successively achieving local goals, and in this case I will do the refactoring in three steps.
No, sometimes, with a lucky combination of circumstances, you can get an unexpectedly good result right away, but, as a rule, with one or more errors that are not very easy to detect. And yet, sometimes even this option is much more effective than doing everything manually.
So, in the first step we put our function code in the prompt and give the model basic instructions:
Source
def load_data(src, tb, lrn, nr0):
g.j_n = 0
mg = (5 in g.ohlcv)
if mg:
src2 = f"{g.tkr}/{g.TFrame}/bsl/manual_{g.bsl}{g.bs_N}.txt"
rows2 = read_data(src2)
ind2 = 0
rows = read_data(src)
jnd = -1
tb0 = datetime.strptime(tb.replace('24:00','00:00'), '%d/%m/%y %H:%M')
_s1 = tb0
data_date, data_close, data_all = [], [], [];
for i in np.arange(len(rows)):
data_list = rows[i].split(",")
if len(data_list) != 7:
print(f"The format of the input array element from {src} is incorrect: {data_list}")
exit(0)
s = f"{data_list[0]} {data_list[1]}"
try:
s1 = datetime.strptime(s.replace('24:00','00:00'), '%d/%m/%y %H:%M')
if mg:
s2 = datetime.strptime(rows2[ind2], '%d/%m/%y %H:%M')
if ind2 < len(rows2)-1 and s1 > s2:
s = f"Date={s} of {src} exceeded the date={rows2[ind2]} из {src2}"
show_text(s,1)
exit(0)
except Exception as err:
s = f'Date format error {s}'
show_text(s)
print(s + " in " + src)
exit(0)
if i - nr0 + 1 == 1 or s == tb or _s1 < tb0 < s1:
g.TimeBegin = s
if lrn > len(rows) - i:
print(f"The length of the array g. lrn={g. lrn} is specified greater than the length of the input array from {src} starting with {tb} ({len(rows) - i})")
exit(0)
_s1 = s1
data_date.append(s)
koef = 1.0
if g.tkr == 'GMK' and int(s[6:8]+s[3:5]+s[0:2]) <= 240321:
koef = 0.01
try:
data_close.append(float(data_list[5])*koef)
except Exception as err:
s = f'Format Error close {data_list[5]}'
show_text(s)
print(s + " in " + src)
exit(0)
if nr0 > 0:
try:
if mg:
if s == rows2[ind2]:
vl = 1
if ind2 < len(rows2)-1:
ind2 += 1
else:
vl = 0
data_all.append([float(data_list[2])*koef,float(data_list[3])*koef,float(data_list[4])*koef,float(data_list[5])*koef,float(data_list[6]),vl])
else:
data_all.append([float(data_list[i])*(1 if i==6 else koef) for i in[2,3,4,5,6]])
except Exception as err:
s = f'Format Error {data_list}'
show_text(s)
print(s + " in " + src)
exit(0)
return data_date, np.array(data_close), np.array(data_all)
Refactor this code. Carefully analyze all if-else conditional statements, preserving their accuracy and correctness. Change only those parts of the code that you are sure are free of possible errors, otherwise leave these parts of the code unchanged. Pay special attention to the algorithms for creating returned lists, optimize the process of their formation with maximum accuracy. Change the names of variables in accordance with the logic of their use for better understanding. When creating code, consider the scope of variables that can change within functions and should always be returned if they are used as input parameters in other functions. Carefully check that all such variables are used and passed between functions logically and correctly without losses. The code obtained by refactoring should exactly match the source code in functionality and internal data conversion algorithms at each step.
Here is the result obtained.
def load_data(source_file, start_time, learning_period, normalization_factor):
global j_n
j_n = 0
is_market_data = (5 in g.ohlcv)
if is_market_data:
source_file2 = f"{g.tkr}/{g.TFrame}/bsl/manual_{g.bsl}{g.bs_N}.txt"
rows2 = read_data(source_file2)
index2 = 0
rows = read_data(source_file)
start_time_obj = datetime.strptime(start_time.replace('24:00','00:00'), '%d/%m/%y %H:%M')
previous_time = start_time_obj
dates, closes, all_data = [], [], [];
for i, row in enumerate(rows):
data_list = row.split(",")
if len(data_list) != 7:
print(f"The format of the input array element from {source_file} is incorrect: {data_list}")
exit(0)
current_time_str = f"{data_list[0]} {data_list[1]}"
try:
current_time = datetime.strptime(current_time_str.replace('24:00','00:00'), '%d/%m/%y %H:%M')
if is_market_data:
next_time = datetime.strptime(rows2[index2], '%d/%m/%y %H:%M')
if index2 < len(rows2)-1 and current_time > next_time:
error_message = f"Date={current_time_str} of {source_file} exceeded the date={rows2[index2]} из {source_file2}"
show_text(error_message, 1)
exit(0)
except Exception as err:
error_message = f'Date format error {current_time_str}'
show_text(error_message)
print(error_message + " in " + source_file)
exit(0)
if i - normalization_factor + 1 == 1 or current_time_str == start_time or previous_time < start_time_obj < current_time:
g.TimeBegin = current_time_str
if learning_period > len(rows) - i:
print(f"The length of the array g.learning_period={g.learning_period} is specified greater than the length of the input array from {source_file} starting with {start_time} ({len(rows) - i})")
exit(0)
previous_time = current_time
dates.append(current_time_str)
coefficient = 1.0
if g.tkr == 'GMK' and int(current_time_str[6:8]+current_time_str[3:5]+current_time_str[0:2]) <= 240321:
coefficient = 0.01
try:
closes.append(float(data_list[5])*coefficient)
except Exception as err:
error_message = f'Format Error close {data_list[5]}'
show_text(error_message)
print(error_message + " in " + source_file)
exit(0)
if normalization_factor > 0:
try:
if is_market_data:
value = 1 if current_time_str == rows2[index2] else 0
if index2 < len(rows2)-1 and current_time_str == rows2[index2]:
index2 += 1
all_data.append([float(data_list[j])*(1 if j==6 else coefficient) for j in range(2, 7)] + [value])
else:
all_data.append([float(data_list[j])*(1 if j==6 else coefficient) for j in range(2, 7)])
except Exception as err:
error_message = f'Format Error {data_list}'
show_text(error_message)
print(error_message + " in " + source_file)
exit(0)
return dates, np.array(closes), np.array(all_data)And her comments.
This refactored version of the code improves variable names and optimizes list creation for better readability and performance. It also ensures that all variables used in other functions are returned correctly and logically. The functionality and internal data transformation algorithms remain the same as the original code.
The model successfully completed the task by improving the quality of the code, which became more readable and logical, and it does not contain errors related to any defects in the code transformation process. The names of variables were changed, optimization was performed, etc., but the function still remains monolithic and unstructured.
Therefore, in the second step we ask for a deeper refactoring in the prompt:
Now, separate the individual functional blocks into functions. Pay close attention to the function input parameters and return parameters.
Here is the following result obtained
def parse_time(time_str):
try:
time_obj = datetime.strptime(time_str.replace('24:00','00:00'), '%d/%m/%y %H:%M')
return time_obj
except Exception as err:
error_message = f'Date format error {time_str}'
show_text(error_message)
print(error_message)
exit(0)
def parse_data_row(data_list, coefficient):
try:
close = float(data_list[5]) * coefficient
row_data = [float(data_list[j]) * (1 if j==6 else coefficient) for j in range(2, 7)]
return close, row_data
except Exception as err:
error_message = f'Format Error {data_list}'
show_text(error_message)
print(error_message)
exit(0)
def load_data(source_file, start_time, learning_period, normalization_factor):
global j_n
j_n = 0
is_market_data = (5 in g.ohlcv)
if is_market_data:
source_file2 = f"{g.tkr}/{g.TFrame}/bsl/manual_{g.bsl}{g.bs_N}.txt"
rows2 = read_data(source_file2)
index2 = 0
rows = read_data(source_file)
start_time_obj = parse_time(start_time)
previous_time = start_time_obj
dates, closes, all_data = [], [], [];
for i, row in enumerate(rows):
data_list = row.split(",")
if len(data_list) != 7:
print(f"The format of the input array element from {source_file} is incorrect: {data_list}")
exit(0)
current_time_str = f"{data_list[0]} {data_list[1]}"
current_time = parse_time(current_time_str)
if is_market_data:
next_time = datetime.strptime(rows2[index2], '%d/%m/%y %H:%M')
if index2 < len(rows2)-1 and current_time > next_time:
error_message = f"Date={current_time_str} of {source_file} exceeded the date={rows2[index2]} из {source_file2}"
show_text(error_message, 1)
exit(0)
if i - normalization_factor + 1 == 1 or current_time_str == start_time or previous_time < start_time_obj < current_time:
g.TimeBegin = current_time_str
if learning_period > len(rows) - i:
print(f"The length of the array g.learning_period={g.learning_period} is specified greater than the length of the input array from {source_file} starting with {start_time} ({len(rows) - i})")
exit(0)
previous_time = current_time
dates.append(current_time_str)
coefficient = 1.0
if g.tkr == 'GMK' and int(current_time_str[6:8]+current_time_str[3:5]+current_time_str[0:2]) <= 240321:
coefficient = 0.01
close, row_data = parse_data_row(data_list, coefficient)
closes.append(close)
if normalization_factor > 0:
if is_market_data:
value = 1 if current_time_str == rows2[index2] else 0
if index2 < len(rows2)-1 and current_time_str == rows2[index2]:
index2 += 1
row_data.append(value)
all_data.append(row_data)
return dates, np.array(closes), np.array(all_data)And her comments.
In this refactored version, I've created separate functions for parsing time strings (`parse_time`) and parsing data rows (`parse_data_row`). This improves code readability and makes it easier to maintain and debug. The `load_data` function now calls these helper functions as needed, and it returns the expected output.
The model highlighted two semantic blocks in the parse_time and parse_data_row functions, and the code acquired an even more decent appearance.
Again, I did not see any transformation errors. In principle, this is already quite enough, although this process could be continued and probably achieve a greater effect. My task is to show an effective way to obtain the desired result and this is not a dogma, but simply a well-thought-out algorithm. I would like to immediately emphasize an important remark: this method of using the model for programming tasks does not always give a 100% result, sometimes the model makes mistakes, for example, in conditional operators or when optimizing individual operations.
Therefore, in the third step we give her a self-test and in the prompt we ask her to work on her mistakes:
Excellent. At this final stage, compare the source code and the final code obtained during the refactoring process to search for possible errors that arose accidentally or through carelessness during the refactoring process. Find these problem areas in the resulting code and fix them.
Here is the final version.
def parse_time(time_str):
try:
time_obj = datetime.strptime(time_str.replace('24:00','00:00'), '%d/%m/%y %H:%M')
return time_obj
except Exception as err:
error_message = f'Date format error {time_str}'
show_text(error_message)
print(error_message)
exit(0)
def parse_data_row(data_list, coefficient):
try:
close = float(data_list[5]) * coefficient
row_data = [float(data_list[j]) * (1 if j==6 else coefficient) for j in range(2, 7)]
return close, row_data
except Exception as err:
error_message = f'Format Error {data_list}'
show_text(error_message)
print(error_message)
exit(0)
def load_data(source_file, start_time, learning_period, normalization_factor):
global j_n
j_n = 0
is_market_data = (5 in g.ohlcv)
if is_market_data:
source_file2 = f"{g.tkr}/{g.TFrame}/bsl/manual_{g.bsl}{g.bs_N}.txt"
rows2 = read_data(source_file2)
index2 = 0
rows = read_data(source_file)
start_time_obj = parse_time(start_time)
previous_time = start_time_obj
dates, closes, all_data = [], [], [];
for i, row in enumerate(rows):
data_list = row.split(",")
if len(data_list) != 7:
print(f"The format of the input array element from {source_file} is incorrect: {data_list}")
exit(0)
current_time_str = f"{data_list[0]} {data_list[1]}"
current_time = parse_time(current_time_str)
if is_market_data:
next_time = datetime.strptime(rows2[index2], '%d/%m/%y %H:%M')
if index2 < len(rows2)-1 and current_time > next_time:
error_message = f"Date={current_time_str} of {source_file} exceeded the date={rows2[index2]} из {source_file2}"
show_text(error_message, 1)
exit(0)
if i - normalization_factor + 1 == 1 or current_time_str == start_time or previous_time < start_time_obj < current_time:
g.TimeBegin = current_time_str
if learning_period > len(rows) - i:
print(f"The length of the array g.learning_period={g.learning_period} is specified greater than the length of the input array from {source_file} starting with {start_time} ({len(rows) - i})")
exit(0)
previous_time = current_time
dates.append(current_time_str)
coefficient = 1.0
if g.tkr == 'GMK' and int(current_time_str[6:8]+current_time_str[3:5]+current_time_str[0:2]) <= 240321:
coefficient = 0.01
close, row_data = parse_data_row(data_list, coefficient)
closes.append(close)
if normalization_factor > 0:
if is_market_data:
value = 1 if current_time_str == rows2[index2] else 0
row_data.append(value)
if index2 < len(rows2)-1 and current_time_str == rows2[index2]:
index2 += 1
all_data.append(row_data)
return dates, np.array(closes), np.array(all_data)And her comments.
Now the code should be free of any errors introduced during the refactoring process.
The model found some places that were questionable from her point of view, although I did not see them, and decided to record them.
Okay, let it be as she decided.
It is clear that the source code has now transformed from a solid mess into a completely readable, reasonably factored code that does not cause rejection and can now be normally perceived and further supported.
Attempt number 2.
The second code fragment we will try to refactor is a generally understandable algorithm for creating a neural network based on a certain template from a string variable that is unknown to us. The algorithm looks very sophisticated and branched, with numerous conditions and checks. Apparently, there are many options for building a neural network and all of them must be provided with various tricks. Also, unknown global variables initiated earlier outside this code are used in the processing process.
In the first step, we put our code in the prompt and give the model basic instructions:
Source.
t1 = ['simple_rnn','gru','lstm']
t2 = [SimpleRNN, GRU, LSTM]
act_model = act_list_model
init_model = init_list_model
if len(g.ohlcv) == 1:
if g.type_model == 0:
in_dots = Input((g.nr0,))
elif g.type_model == 1:
padding = 'same'
in_dots = Input((g.nr0,1,))
elif g.type_model in [2,3,4]:
in_dots = Input((1, g.nr0,))
x = in_dots
if len(g.nr_leyers) == 0:
nn = g.nr0
for i in np.arange(100):
nn = int(nn/2)
if nn < 10:
break
x = Dense(nn, activation=act_model, kernel_initializer=init_model)(x)
x = Dropout(g.slider2.val if g.slider2.val != g.slider2.valinit else g.dropout)(x)
else:
flat = 0
for i in np.arange(len(g.nr_leyers)):
ley = g.nr_leyers[i]
if ley[0] == 'dense':
if g.type_model == 1 and flat == 0:
x = Flatten()(x)
flat = 1
x = Dense(ley[1], activation=act_model, kernel_initializer=init_model, bias_initializer=init_model)(x)
if ley[0] == 'dense_l2':
x = Dense(ley[1], activation=act_model, kernel_initializer=init_model, bias_initializer=init_model, kernel_regularizer=l2(g.nr_leyers[i][2]), bias_regularizer=l2(g.nr_leyers[i][2]))(x)
if ley[0] == 'dropout':
x = Dropout(g.slider2.val if g.slider2.val != g.slider2.valinit else ley[1])(x)
if ley[0] == 'conv1d':
x = Conv1D(ley[1], ley[2], strides=1, padding=padding, activation=act_model, kernel_initializer=init_model)(x)
if ley[0] == 'max_pooling1d':
x = MaxPooling1D(ley[1])(x)
if ley[0] in t1:
sequences=(True if flat < len([1 for c in g.nr_leyers if ley[0] in c])-1 else False)
flat += 1
j =t1.index(ley[0])
x = t2[j](ley[1], activation=act_model, recurrent_dropout=g.dropout, kernel_initializer=init_model, recurrent_initializer=init_model, bias_initializer=init_model, return_sequences=sequences)(x)
out_dots = Dense(2 if g.bsl == "bsl" else 1, activation=act_model, kernel_initializer=init_model)(x)
g.ae = Model(in_dots, out_dots)
else:
x_list, in_dots_list = [], []
for j in np.arange(len(g.ohlcv)):
d_cut = 0
if g.type_model == 0:
in_dots = Input((g.nr0,))
elif g.type_model == 1:
padding = 'same'
in_dots = Input((g.nr0,1,))
elif g.type_model in [2,3,4]:
in_dots = Input((1, g.nr0,))
x = in_dots
_i = 0
if len(g.nr_leyers) == 0:
nn = g.nr0
_i = 100
for i in np.arange(_i):
nn = int(nn/2)
if nn < 10:
break
x = Dense(nn, activation=act_model, kernel_initializer=init_model)(x)
x = Dropout(g.slider2.val if g.slider2.val != g.slider2.valinit else g.dropout)(x)
d_cut += 1
if d_cut == dropout_cut:
_i = i + 1
break
else:
flat = 0
_i = len(g.nr_leyers)
for i in np.arange(len(g.nr_leyers)):
ley = g.nr_leyers[i]
if ley[0] == 'dense':
if g.type_model == 1 and flat == 0:
x = Flatten()(x)
flat = 1
x = Dense(ley[1], activation=act_model, kernel_initializer=init_model, bias_initializer=init_model)(x)
if ley[0] == 'dense_l2':
x = Dense(ley[1], activation=act_model, kernel_initializer=init_model, bias_initializer=init_model, kernel_regularizer=l2(g.nr_leyers[i][2]), bias_regularizer=l2(g.nr_leyers[i][2]))(x)
if ley[0] == 'dropout':
d_cut += 1
x = Dropout(g.slider2.val if g.slider2.val != g.slider2.valinit else ley[1])(x)
if ley[0] == 'conv1d':
x = Conv1D(ley[1], ley[2], strides=1, padding=padding, activation=act_model, kernel_initializer=init_model)(x)
if ley[0] == 'max_pooling1d':
x = MaxPooling1D(ley[1])(x)
if ley[0] in t1:
sequences=(True if flat < len([1 for c in g.nr_leyers if ley[0] in c])-1 else False)
flat += 1
j =t1.index(ley[0])
x = t2[j](ley[1], activation=act_model, recurrent_dropout=g.dropout, kernel_initializer=init_model, recurrent_initializer=init_model, bias_initializer=init_model, return_sequences=sequences)(x)
if d_cut == dropout_cut:
_i = i + 1
break
x = Model(in_dots, x)
in_dots_list.append(in_dots)
x_list.append(x)
x = concatenate([x.output for x in x_list])
if len(g.nr_leyers) == 0:
nn = g.nr0
for i in np.arange(_i,100):
nn = int(nn/2)
if nn < 10:
break
x = Dense(nn, activation=act_model, kernel_initializer=init_model)(x)
x = Dropout(g.slider2.val if g.slider2.val != g.slider2.valinit else g.dropout)(x)
else:
for i in np.arange(_i,len(g.nr_leyers)):
ley = g.nr_leyers[i]
if ley[0] == 'dense':
if g.type_model == 1 and flat == 0:
x = Flatten()(x)
flat = 1
x = Dense(ley[1], activation=act_model, kernel_initializer=init_model, bias_initializer=init_model)(x)
if ley[0] == 'dense_l2':
x = Dense(ley[1], activation=act_model, kernel_initializer=init_model, bias_initializer=init_model, kernel_regularizer=l2(g.nr_leyers[i][2]), bias_regularizer=l2(g.nr_leyers[i][2]))(x)
if ley[0] == 'dropout':
d_cut += 1
x = Dropout(g.slider2.val if g.slider2.val != g.slider2.valinit else ley[1])(x)
if ley[0] == 'conv1d':
x = Conv1D(ley[1], ley[2], strides=1, padding=padding, activation=act_model, kernel_initializer=init_model)(x)
if ley[0] == 'max_pooling1d':
x = MaxPooling1D(ley[1])(x)
if ley[0] in t1:
j =t1.index(ley[0])
x = t2[j](ley[1], activation=act_model, recurrent_dropout=g.dropout, kernel_initializer=init_model, recurrent_initializer=init_model, bias_initializer=init_model, return_sequences=False)(x)
out_dots = Dense(2 if g.bsl == "bsl" else 1, activation=act_model, kernel_initializer=init_model)(x)
g.ae = Model(in_dots_list, out_dots)Transform this code into a function whose input parameters are global variables and which must return the created Model. Change the names of local variables according to the logic of their use for a better understanding of the functional context of this code. Perform reasonable optimization of individual logical operations.
Here is the result obtained.
def create_model(g):
# Define the types of layers and their corresponding Keras classes
layer_types = ['simple_rnn', 'gru', 'lstm']
layer_classes = [SimpleRNN, GRU, LSTM]
activation_model = g.act_list_model
initializer_model = g.init_list_model
if len(g.ohlcv) == 1:
return _create_single_model(g, layer_types, layer_classes, activation_model, initializer_model)
else:
return _create_multi_model(g, layer_types, layer_classes, activation_model, initializer_model)
def _create_single_model(g, layer_types, layer_classes, activation_model, initializer_model):
input_layer = _get_input_layer(g)
x = input_layer
if not g.nr_leyers:
x = _add_default_layers(x, g, activation_model, initializer_model)
else:
x = _add_custom_layers(x, g, layer_types, layer_classes, activation_model, initializer_model)
output_layer = Dense(2 if g.bsl == "bsl" else 1, activation=activation_model, kernel_initializer=initializer_model)(x)
model = Model(input_layer, output_layer)
return model
def _create_multi_model(g, layer_types, layer_classes, activation_model, initializer_model):
input_layers, models = [], []
for _ in np.arange(len(g.ohlcv)):
input_layer = _get_input_layer(g)
x = input_layer
if not g.nr_leyers:
x, dropout_cut = _add_default_layers(x, g, activation_model, initializer_model, return_dropout_cut=True)
else:
x, dropout_cut = _add_custom_layers(x, g, layer_types, layer_classes, activation_model, initializer_model, return_dropout_cut=True)
model = Model(input_layer, x)
input_layers.append(input_layer)
models.append(model)
x = concatenate([model.output for model in models])
if not g.nr_leyers:
x = _add_remaining_default_layers(x, g, dropout_cut, activation_model, initializer_model)
else:
x = _add_remaining_custom_layers(x, g, dropout_cut, layer_types, layer_classes, activation_model, initializer_model)
output_layer = Dense(2 if g.bsl == "bsl" else 1, activation=activation_model, kernel_initializer=initializer_model)(x)
model = Model(input_layers, output_layer)
return model
def _get_input_layer(g):
if g.type_model == 0:
return Input((g.nr0,))
elif g.type_model == 1:
return Input((g.nr0, 1,))
elif g.type_model in [2, 3, 4]:
return Input((1, g.nr0,))
def _add_default_layers(x, g, activation_model, initializer_model, return_dropout_cut=False):
nn = g.nr0
dropout_cut = None
for i in np.arange(100):
nn = int(nn / 2)
if nn < 10:
break
x = Dense(nn, activation=activation_model, kernel_initializer=initializer_model)(x)
x = Dropout(g.slider2.val if g.slider2.val != g.slider2.valinit else g.dropout)(x)
if return_dropout_cut and i == g.dropout_cut - 1:
dropout_cut = i + 1
break
if return_dropout_cut:
return x, dropout_cut
return xAnd her comments.
Similar functions for _add_custom_layers, _add_remaining_default_layers, and _add_remaining_custom_layers can be defined here.
This code creates a function that takes the global variable `g` as input and returns a Keras model. The function first checks if there's only one element in `g.ohlcv`. If so, it calls `_create_single_model` to create a single model. Otherwise, it calls `_create_multi_model` to create multiple models and concatenate them.
The helper functions are named more descriptively to reflect their purpose, and some common operations have been extracted into separate functions for better readability and maintainability.
Since the functions _add_custom_layers, _add_remaining_default_layers, and _add_remaining_custom_layers are not defined, we will force the model to write them in separate steps with simple prompts.
Write the function _add_custom_layers.
Write a function _add_remaining_default_layers.
Write a function _add_remaining_custom_layers.
As a result, we got a well-structured code of the main function as a result of refactoring and a number of functions corresponding to the logical blocks of this code.
Variable names have become more meaningful, and some reasonable code optimization has been performed.
The code for these functions.
def _add_custom_layers(x, g, layer_types, layer_classes, activation_model, initializer_model, return_dropout_cut=False):
flat = 0
dropout_cut = None
for i, ley in enumerate(g.nr_leyers):
if ley[0] == 'dense':
if g.type_model == 1 and flat == 0:
x = Flatten()(x)
flat += 1
x = Dense(ley[1], activation=activation_model, kernel_initializer=initializer_model, bias_initializer=initializer_model)(x)
elif ley[0] == 'dense_l2':
x = Dense(ley[1], activation=activation_model, kernel_initializer=initializer_model, bias_initializer=initializer_model, kernel_regularizer=l2(ley[2]), bias_regularizer=l2(ley[2]))(x)
elif ley[0] == 'dropout':
x = Dropout(g.slider2.val if g.slider2.val != g.slider2.valinit else ley[1])(x)
if return_dropout_cut and i == g.dropout_cut - 1:
dropout_cut = i + 1
break
elif ley[0] == 'conv1d':
padding = 'same' if g.type_model == 1 else 'valid'
x = Conv1D(ley[1], ley[2], strides=1, padding=padding, activation=activation_model, kernel_initializer=initializer_model)(x)
elif ley[0] == 'max_pooling1d':
x = MaxPooling1D(ley[1])(x)
elif ley[0] in layer_types:
sequences = (True if flat < len([1 for c in g.nr_leyers if ley[0] in c])-1 else False)
flat += 1
j = layer_types.index(ley[0])
x = layer_classes[j](ley[1], activation=activation_model, recurrent_dropout=g.dropout, kernel_initializer=initializer_model, recurrent_initializer=initializer_model, bias_initializer=initializer_model, return_sequences=sequences)(x)
if return_dropout_cut:
return x, dropout_cut
return x
def _add_remaining_default_layers(x, g, dropout_cut, activation_model, initializer_model):
nn = g.nr0
for i in np.arange(dropout_cut, 100):
nn = int(nn / 2)
if nn < 10:
break
x = Dense(nn, activation=activation_model, kernel_initializer=initializer_model)(x)
x = Dropout(g.slider2.val if g.slider2.val != g.slider2.valinit else g.dropout)(x)
return x
def _add_remaining_custom_layers(x, g, dropout_cut, layer_types, layer_classes, activation_model, initializer_model):
for ley in g.nr_leyers[dropout_cut:]:
if ley[0] == 'dense':
x = Dense(ley[1], activation=activation_model, kernel_initializer=initializer_model, bias_initializer=initializer_model)(x)
elif ley[0] == 'dense_l2':
x = Dense(ley[1], activation=activation_model, kernel_initializer=initializer_model, bias_initializer=initializer_model, kernel_regularizer=l2(ley[2]), bias_regularizer=l2(ley[2]))(x)
elif ley[0] == 'dropout':
x = Dropout(g.slider2.val if g.slider2.val != g.slider2.valinit else ley[1])(x)
elif ley[0] == 'conv1d':
padding = 'same' if g.type_model == 1 else 'valid'
x = Conv1D(ley[1], ley[2], strides=1, padding=padding, activation=activation_model, kernel_initializer=initializer_model)(x)
elif ley[0] == 'max_pooling1d':
x = MaxPooling1D(ley[1])(x)
elif ley[0] in layer_types:
j = layer_types.index(ley[0])
x = layer_classes[j](ley[1], activation=activation_model, recurrent_dropout=g.dropout, kernel_initializer=initializer_model, recurrent_initializer=initializer_model, bias_initializer=initializer_model, return_sequences=False)(x)
return xBut there is one interesting point that requires close consideration. The fact is that in the _add_custom_layers function it a logical error was made. Here is this place in the original code:
if ley[0] == 'dropout':
d_cut += 1
…
if d_cut == dropout_cut:
_i = i + 1
break
Which was converted to the following code:
Elif Lee[0] == 'dropout':
if return_dropout_cut and i == g.dropout_cut – 1:
dropout_cut = i + 1
break
Yes, it would be easy to correct this oversight manually, but we are exploring the capabilities of the model itself.
So I decided to point this mistake out to her directly.
There is an error in the _add_custom_layers function, since its code logically differs from the corresponding fragment of the original code, which uses the d_cut += 1 operation. Find the errors and fix them.
The result of an attempt to fix the code.
def _add_custom_layers(x, g, layer_types, layer_classes, activation_model, initializer_model, return_dropout_cut=False):
flat = 0
dropout_cut = 0
for i, ley in enumerate(g.nr_leyers):
if ley[0] == 'dense':
if g.type_model == 1 and flat == 0:
x = Flatten()(x)
flat += 1
x = Dense(ley[1], activation=activation_model, kernel_initializer=initializer_model, bias_initializer=initializer_model)(x)
elif ley[0] == 'dense_l2':
x = Dense(ley[1], activation=activation_model, kernel_initializer=initializer_model, bias_initializer=initializer_model, kernel_regularizer=l2(ley[2]), bias_regularizer=l2(ley[2]))(x)
elif ley[0] == 'dropout':
x = Dropout(g.slider2.val if g.slider2.val != g.slider2.valinit else ley[1])(x)
dropout_cut += 1
if return_dropout_cut and dropout_cut == g.dropout_cut:
break
elif ley[0] == 'conv1d':
padding = 'same' if g.type_model == 1 else 'valid'
x = Conv1D(ley[1], ley[2], strides=1, padding=padding, activation=activation_model, kernel_initializer=initializer_model)(x)
elif ley[0] == 'max_pooling1d':
x = MaxPooling1D(ley[1])(x)
elif ley[0] in layer_types:
sequences = (True if flat < len([1 for c in g.nr_leyers if ley[0] in c])-1 else False)
flat += 1
j = layer_types.index(ley[0])
x = layer_classes[j](ley[1], activation=activation_model, recurrent_dropout=g.dropout, kernel_initializer=initializer_model, recurrent_initializer=initializer_model, bias_initializer=initializer_model, return_sequences=sequences)(x)
if return_dropout_cut:
return x, dropout_cut
return xAnd her comments.
You're correct. I apologize for the oversight. In the `_add_custom_layers` function, I mistakenly used `i` instead of `dropout_cut` to keep track of the number of dropout layers added. Here's the corrected version:
Now the function correctly keeps track of the number of dropout layers added using `dropout_cut`, and breaks the loop when it reaches `g.dropout_cut`.
The model now believes that the new version of this feature “correctly keeps track of dropout layers.”
Unfortunately, it is not. The model actually made a good attempt to fix it, but forgot to simply move one line of code.
New code snippet:
dropout_cut = 0
…
Elif Lee[0] == 'dropout':
dropout_cut += 1
if return_dropout_cut and dropout_cut == g.dropout_cut:
break
Now everything seems to be correct, but the line that was there before has disappeared:
dropout_cut = i + 1
Let's sum it up.
Overall, I am quite satisfied with the ability of this model to produce high-quality code refactoring and optimization.
If it weren't for the mistake that was made and only a partial, rather than a complete correction, I would have given the model a 5-point rating.
I'm deducting half a point for the unsolved problem and it turns out to be a solid 4+ for the sum of the two experiments performed.
Could the same result have been achieved without LLM? Of course it could and even better, but why, please tell me, to toil, waste energy, time and overcome a persistent reluctance to do such uncreative work. I prefer to delegate the dirty work to the model. And in places even part of the usual programming functions under vigilant control, of course, since errors are not uncommon. To minimize them, it is advisable to factor the code in not very large parts, applying a reasonable compromise between the number of actions and the quality of the resulting code.
So what are the best prompts to write?
Specific, detailed, with precise wording, for which similar in meaning but expressed in different ways are acceptable. A sequence of limited but interconnected by a common approach prompts works best.
There is no need to try to cram everything into one huge prompt, we do not write programs like that, but structure them at different levels of abstraction. Act on the principle from simple to complex, with consistent improvements and you will be happy.
And in general, let's ask the model herself how to write her prompts correctly, let her speak out on this matter, and we will get valuable information first-hand.
Let's write a prompt like this:
How to write a prompt correctly so that the quality of the program code you create is maximum. Formulate the answer point by point.
Model response.
1. Describe the task or code requirements in as much detail as possible. This will help me better understand your expectations and ensure a more accurate result.
2. Specify the programming language you prefer to use (in this case Python).
3. If you have specific requirements for the code structure or writing style, please let us know. For example, you may prefer a functional style, object-oriented, or procedural.
4. Please specify the version of Python you are using, as some features may only be available in newer versions.
5. Provide examples of input data and expected output, if possible. This will help me test the code and make sure it meets your requirements.
6. Please indicate which performance optimizations or improvements are important for your task.
7. Please advise if there are any memory or execution time limitations.
8. Specify whether you want to comment the code and what comment style you prefer.
9. Indicate whether the code will be integrated with other systems or libraries, and provide any additional information that might be useful.
10. Finally, please let me know if you have any preferences for refactoring or code reuse.
Time is probably the most valuable resource in our lives, and wasting it on something that is boring, routine work like coding (let alone digging into someone else's code) is at least unwise when you have such a capable assistant (you can call it assistant, as you like) as an LLM.
That's all for now, thank you for your attention.
Here is the previous publication on the topic.
