Multi-criteria optimization for ranking and selection of trading systems
Selection of trading systems: how to choose the best from an arbitrary number of available ones
The article will discuss the selection mechanism or calculation of the coefficient for ranking a pool of trading systems. We will look at one of the methods that can be used to determine the best options.
It is important to understand that the results of analyzing the efficiency and effectiveness of trading systems can be complex and ambiguous. For example, a high profit trading system may have a large drawdown, but it can also generate consistent income over a long period of time. Therefore, when choosing a trading system, it is important to take into account the entire set of characteristics and indicators. This is the problem addressed in the article.
Formulation of the problem
So, let's say we have a certain set of trading systems, and some of them are better, and some are worse. Based on a set of metrics, it is difficult to determine whether a particular system is better or worse. One system can have a high profitability, and at the same time, a high drawdown and a lower recovery factor. The other has a minimal drawdown, but the profit seems very mediocre. And these are just three metrics, and, of course, many more can be calculated. For example:
the ratio of annualized profit on the test and on the inspection
proportion of total trading time used, for example, proportion of days on which trades were made in relation to the total number of trading days
time share of a trading system in the market – the time during which capital was occupied by a position in relation to the total trading market time
maximum, average and minimum transaction lengths – in bars, minutes, hours or days
average number of transactions per unit of time, for example, per hour or per day or per week
average number of trades per trading day and average number of trades per calendar day
maximum length of a series of profitable trades and the same for losing trades
annualized return
share of commissions in a transaction, share of commissions in the final profit, amount of commission per point of income
maximum realized profit achieved
average profit per hour, day, month
recovery factor
all kinds of MAE, MAPE, SMAPE, RMSE, MSPE, etc.
the square of deviations and the square of negative deviations, and by analogy –
Sharpe ratio and Sortino ratio
expected value
maximum realized drawdown, relative realized drawdown, maximum unrealized drawdown
maximum unrealized potential profit
number of profitable/losing trades, total profit and loss on profitable and unprofitable trades
financial results for long and short transactions
correlation between the yield curve and the market price of an asset
benchmark profit/drawdown for the “buy and hold” case for a given asset for the same period in relative units to the corresponding indicators of the system itself
profit benchmark along the optimal path – the maximum theoretical possible profit taking into account commissions for a given asset
Let's say you have a matrix where each trading system has a row, and each of the indicators above has a column. I would like to rank trading systems in some way in order to select the best, but for ranking it is desirable to have a continuous range of values, which can already be divided into quantiles, quartiles, or use other methods. But here we have a set of motley metrics, and sorting by several of them at the same time will not give anything.
Search history
At first I did this manually – literally applying a filter to something like the matrix described above, and the technique used was “it seems to me.”
Then I automated it, but the essence did not change, it was a decision “probably this should be like this,” but there was no certainty.
Next, I used a regression neural network whose input was this normalized matrix, and the results were pretty good.
Now I want to approach this process from an even more formal point of view: in fact, this is what the neural network I created seemed to do, and now there will be a mathematical method.
Formulation of the problem
We have to use many criteria—metrics for testing trading systems—to find one or a subset of the best ones. Such problems are problems of the multicriteria optimization class. This will be discussed below. Essentially, the task is as follows: we need to calculate (or construct) the metrics of an ideal trading system and find the distance to such an ideal system from all others. This distance will be the value for selecting systems – a certain value by which you can sort systems or rank a list of them.
Found solutions
So, wondering what tools there were to do this, I dug up the following.
First, the method classes:
methods based on artificial intelligence, such as neural networks, genetic algorithms, fuzzy logic, etc. These methods allow you to create complex and flexible models that can learn from data, identify patterns and predict market behavior;
methods based on optimization, such as simplex method, gradient descent, Newton's method, etc. These methods allow you to find optimal values of strategy parameters that maximize or minimize a given objective function, such as profit, risk, Sharpe ratio, etc.
Methods based on statistics, such as regression analysis, hypothesis testing, time series analysis, etc. These methods allow you to test the statistical significance and reliability of strategies, as well as evaluate their risks and returns.
Here are some methods, in particular, that I came across during my searches (and seemed applicable to solving the contextual problem):
One of the popular methods for multi-criteria selection of trading strategies is the ELECTRE (Elimination Et Choix Traduisant la REalité) method, which is based on pairwise comparison of strategies and the exclusion of those that are dominated by others according to one or more criteria. This method allows you to take into account not only quantitative, but also qualitative criteria, as well as criteria weights and decision-making thresholds;
Another common method for multi-criteria selection of trading strategies is the TOPSIS (Technique for Order of Preference by Similarity to Ideal Solution) method, which is based on calculating the distances between strategies and ideal points corresponding to the maximum and minimum values of each criterion. This method allows you to determine the best and worst strategy, as well as calculate the relative proximity of each strategy to the ideal solution;
Another interesting method for multi-criteria selection of trading strategies is the ML-LDM (Multi-Lingual Linguistic Decision Making) method, which is based on the use of linguistic variables to describe criteria and evaluate strategies. This method allows us to take into account the uncertainty and fuzziness of data, as well as various areas of expertise and preferences of participants in the decision-making process;
the next one will be the AHP (Analytic Hierarchy Process) method, which is based on building a hierarchy of criteria and alternatives, as well as determining priorities using pairwise comparison of hierarchy elements;
the PROMETHEE method (Preference Ranking Organization METHod for Enrichment Evaluations), which is based on calculating the degree of preference of one alternative over another for each criterion, as well as the aggregation of these degrees taking into account the weights of the criteria;
The VIKOR method (VIseKriterijumska Optimizacija I Kompromisno Resenje), which is based on determining the best and worst alternative for each criterion, as well as calculating the distances between the alternatives and these reference points.
And that's not all.
Selected solution
I liked the TOPSIS method, which was mentioned above. This is what I will use. Graphically, the solution to the problem can be represented, to some approximation, in the form of a “radar” type diagram, where the metrics of trading systems are plotted on the circle, and the values of these metrics are plotted along the axes from them to the center. But in fact, we will first look for an ideal system (which does not exist), we can call it an ideal point or even a utopian trading system. And then the distance to her from everyone else.
The relative closeness of an algorithmic or any other trading strategy to an ideal solution is a measure that shows how close a given strategy is to the best possible strategy across all criteria. One way to calculate relative proximity is to use the TOPSIS method.
Here is the sequence of steps to solve the problem of the article's problem:
generate a decision matrix in which the rows correspond to alternative strategies, and the columns correspond to evaluation criteria (for example, profit, risk, stability – as shown above);
normalize the decision matrix to bring all values to the same scale and eliminate the influence of units of measurement (remember that we have fractions somewhere, percentages somewhere, quantities and sums somewhere);
multiply the normalized matrix by a vector of criterion weights to account for their relative importance. This is the weakest point – you need to determine the weight of each criterion, but here you can go iteratively, adjusting them during the research process;
define the ideal solution as a vector consisting of the best values for each criterion (metric), and the anti-ideal solution as a vector consisting of the worst values for each metric. This will be our space of possibilities;
calculate the Euclidean distance between each alternative and the ideal solution, and between each alternative and the anti-ideal solution. By the way, you can use not necessarily Euclidean space, but, for example, Chebyshev, Manhattan or another, but this will complicate the task. Although they would be better suited for uncorrelated metrics;
The distance is easy to calculate: as the sum of the squares of the differences between the ideal value of each metric and one of the existing ones. Thus, we obtain a certain number that characterizes the distance of a given vector in relation to the ideal solution (vector). Once again: we calculate the difference between the ideal value of the metric and the value of the same metric for a given trading system, square it, sum all the squares;
calculate the relative proximity of each alternative to the ideal solution as the ratio of the distance to the anti-ideal solution to the sum of the distances to the ideal and anti-ideal solutions. This value will range from 0 to 1, where 0 means complete non-compliance with the ideal solution (anti-ideal solution), and 1 means complete compliance (ideal solution); You can do the opposite – expand this continuum as you like;
rank the alternatives in descending order of relative proximity and select the one that matters most as the best strategy (or top best).
It sounds complicated, but in reality the formulas are quite simple – sums of squares and nothing more. What do you think of the idea?
Conclusion
Finally, a tub of cold water: the ideal solution in algorithmic trading is a strategy that maximizes profit and minimizes risk and other undesirable factors. However, in reality, such a solution may be unattainable or non-existent due to many uncertainties and restrictions in the market. Therefore, instead of searching for an ideal solution, it is more advisable to search for the most acceptable or optimal solution that satisfies the given criteria and restrictions. To do this, you can use various optimization methods, such as the simplex method, gradient descent, Newton's method, etc. These methods allow you to find optimal values of strategy parameters that maximize or minimize a given objective function, such as profit, risk, Sharpe ratio, etc.
It’s good when a person has a goal, it is formalized, “we have a plan,” we are moving in this direction and are passionate about our idea. This is one of the main factors that make us happy 🙂
