my way to smart choice
I would like to share my experience of searching for securities on the American market that are traded on the NYSE, NASDAQ and AMEX.
It is difficult to buy shares of foreign companies from Russia in 2024, but options still remain: a foreign broker, an insurance company, or some Russian brokers that have not been subject to sanctions.
I usually buy index funds, but sometimes I want to buy specific stocks. Which specific company's stock should I choose, since there are 10,522 of them traded on the American market as of August 2024? The answer to this question is complex and depends on many factors. True, I often don't want to spend a lot of time on analysis, but I also don't want to buy a completely random stock.
There is a popular resource Yahoo Finance, which provides various data on stocks, including fundamental data, as well as consolidated recommendations from analysts of various investment companies: the predicted price of a security and a recommendation: buy / sell / hold. All this data is presented on Yahoo in a structured form. Many forecasts can be given for one company, For example, for Apple Inc. (AAPL) in August 2024, 38 such forecasts were given from various investment companies.
I had an idea – why not collect this data for each paper, filter it by growth potential – the percentage between the current and the forecast price according to these analysts, and also take into account how many companies-analysts conducted the analysis over the last two months. It is necessary to filter and take into account the current dividend yield. In practical research, it turned out that not all shares have such data on the forecast price, but only 4,250 out of 10,522 papers. The remaining 6,272 shares do not have data on the forecast price.
Stocks with forecast prices – you can go through each of the 4,250 papers and if it meets the requirements – include it in the selection. Well, and then work with the selection yourself, when the mechanical selection is done.
Why the American market?
The dominance of the American capital market is not just a recent phenomenon; it is the result of a century of economic, technological, and geopolitical development.
This is illustrated by one chart that shows the history of global stock market capitalization. Before World War I, the world was truly multipolar in terms of economic power. The United Kingdom, the largest economy at the time, accounted for only 24% of global market capitalization, followed by the United States with 15% and Germany with 13%. No single country dominated the world's capital markets with an overwhelming majority.
But the situation changed dramatically in the 1920s, when the United States began to distance itself from its peers. By the end of the 20th and early 21st centuries, the United States had solidified its position as the world’s leading capital market. Today, it accounts for a staggering 61% of global stock market capitalization, far surpassing any other country. The scale of this dominance is such that no other nation reaches even double digits. Even China, the world’s second-largest economy, remains a distant competitor in the capital markets arena.
Over the past century, no country has successfully challenged America’s leadership in global finance. Japan came closest in the 1980s, riding a wave of economic prosperity and rapid growth. But by the early 1990s, the Japanese bubble had burst, and it retreated from its challenge to American dominance.
Several factors explain the rise of the American capital market. The sheer size, resilience, and growth of the U.S. economy have played a central role. Wall Street has become synonymous with financial power, supported by strong institutions, deep pools of capital, and a regulatory environment that, despite its complexities, has fostered a stable and predictable market.
Other countries have seen their relative influence in global capital markets decline. A century ago, for example, Russia accounted for 6% of global market capitalization.
Where can I get a list of tickers traded on the NYSE, NASDAQ and AMEX?
If you are looking for a complete list of tickers traded on the NYSE, NASDAQ and AMEX, you can find it on the official Nasdaq Trader website. Page by to this address provides access to a complete catalog of all shares listed on these exchanges.
Nasdaqtrader.com provides downloadable files containing ticker symbols, company names, and other information. These files are available in a variety of formats, including .txt and .zip, making it easy to access and work with the data.
To convert to json format I use a script nasdaqtraded.sh:
#!/bin/bash
# $ sh nasdaqtraded.sh
echo -n '["' > nasdaqtraded.json
curl -o nasdaq.txt ftp://ftp.nasdaqtrader.com/symboldirectory/nasdaqtraded.txt
cat nasdaq.txt | grep -Eo '^\w\|\w*' | sed 's/^\w|//g' | sed 'H;1h;$!d;x;y/\n/,/' | sed 's/,/\",\"/g' >> nasdaqtraded.json
echo '"]' >> nasdaqtraded.json
sed -i ':a;N;$!ba;s/\n//' nasdaqtraded.json
rm nasdaq.txt
Where can I get price data, financial information and forecasts for companies?
Although the official Yahoo Finance API was closed back in 2017, an undocumented working version of it appeared almost immediately, which is still alive.
About a year ago, in the past 2023, the API, which is still alive and working, began to require cookies for each request. In the usual way, only an error was returned Invalid Crumb. However, on stackoverflow.com almost immediately a solution to this problem has appeared.
Now in 2024, the working code for a request to the Yahoo Finance API looks like this:
async function getCredentials() { // на основе https://stackoverflow.com/a/76555529
// Inline the API and User-Agent values
const API = 'https://query2.finance.yahoo.com';
const USER_AGENT = 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36';
// get the A3 cookie
const response = await fetch('https://fc.yahoo.com', {
headers: {
'User-Agent': USER_AGENT
},
timeout: 10000 // Увеличиваем таймаут до 10 секунд
})
const cookie = response.headers.get('set-cookie')
// now get the crumb
const url = new URL('/v1/test/getcrumb', API)
const request = new Request(url, {
headers: {
'User-Agent': USER_AGENT,
'cookie': cookie
}
})
const crumbResponse = await fetch(request)
const crumb = await crumbResponse.text()
return {
cookie,
crumb
}
}
async function USAStockGetName(ID, cookie, crumb) { //получаем имя бумаги
// https://query1.finance.yahoo.com/v10/finance/quoteSummary/GRMN?modules=price&crumb=rxBh.H4z62E
const url = `https://query1.finance.yahoo.com/v10/finance/quoteSummary/${ID}?modules=price&crumb=${crumb}`;
console.log("%s. URL for %s: %s", getFunctionName(), ID, url);
try {
const response = await fetch(url, {
headers: {
'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36',
'cookie': cookie
},
timeout: 10000 // Увеличиваем таймаут до 10 секунд
});
const json = await response.json();
const value = json.quoteSummary.result[0].price.longName;
console.log("%s. Name for %s: %s", getFunctionName(), ID, value);
if (value == 0) return 'нет';
return value.replace(/\'/g, '');
} catch (e) {
console.log(`Ошибка: ${e}.`);
}
}That is, Yahoo Finance can be chosen as a data source because the API provides structured data, including forecast prices and buy/sell/hold recommendations from different analysts.
The Problem with Stock Picking: Too Many Options
When there is a pool of more than 10,000 companies, it is difficult to select something manually. The Yahoo Finance website has good screeners – filters, but they do not work according to analysts' recommendations.
That's why I wanted to write my own implementation of the screener.
Stock Filtering System – How the Script Works
This script is designed to automate the process of finding undervalued stocks traded on US stock exchanges (NYSE, NASDAQ and AMEX) by retrieving data from Yahoo Finance. Here's how it works:
Initialization and configuration. The script starts by recording the current time and initializing the necessary modules, such as
fs,path,momentAndfetch. The script allows you to set threshold values for stock selection, including:Growth potential: minimum percentage increase between the current and projected stock price (e.g. 40% or more).
Analysts' recommendations: minimum number of analyst firms that have provided recommendations in the last two months (e.g. at least 20).
Dividend yield: minimum current dividend yield (e.g. 1% or more).
Loading ticker database. The script downloads a JSON file (
nasdaqtraded.json), which contains tickers of companies whose shares are traded on US stock exchanges. It ensures the uniqueness of tickers by filtering out duplicates.
Analysis process. The main functionality involves iterating through tickers and getting financial data for each stock from Yahoo Finance. The script checks:
Growth potential: the difference between the current stock price and the projected price.
Analytical trend: number of analyst recommendations for a stock.
Dividend yield of stocks.
Only shares that meet all specified conditions (growth potential, analyst recommendations and dividend yield) are included in the final selection.
Error handling. The script monitors for errors, especially when there is no predicted price data for certain stocks.
Sorting and registering data. The selected stocks are sorted by growth potential, and the results are logged periodically: every 100 stocks processed, the script saves the current progress to disk, creating an HTML file summarizing the results.
Once the script has processed all the tickers, the script generates a final report that includes all the selected stocks that match the criteria and outputs the total time taken to execute.
Step by step guide on how to use YahooFinance_Analyst Price Targets script
Before you get started, make sure you have the following installed on your computer:
Node.js and npm (Node package manager): Node.js allows you to run JavaScript on your computer, and npm is the package manager that comes with Node.js. You can download both from official site of Node.js.
Code editor (optional). It is recommended to use a code editor such as Visual Studio Code to edit and view project files. Download it from Visual Studio Code website.
Download the project repository
The first step is – download (clone) the project from GitHub.
Installing dependencies
The next step is to install the necessary packages needed to run the project.
For Windows, use the file first start.bat.
For Mac and Linux, in Terminal, enter:
npm installThis command will install all the packages listed in the file package.json.
Launch of the project
Once the dependencies are installed, you can run the project.
For Windows:
Use the file YahooFinance_Analyst Price Targets_start_windows.bat
The launch takes about 10 hours:
For Mac:
Open a terminal and run the command, changing the path to your own:
node /Users/mike/Desktop/YahooFinance_Analyst Price Targets/index.js 2>&1 | tee /Users/mike/Desktop/YahooFinance_Analyst Price Targets/log/log_$(date +%F_%T).txtThe process also takes about 10 hours, all logs can be viewed in the folder log.
For Linux:
Open a terminal and run the command, changing the path to your own:
node /home/mike/YahooFinance_Analyst Price Targets 2>&1 | tee /home/mike/YahooFinance_Analyst Price Targets/log/log_$(date +%F_%T).txtView results
After successful execution of the script in the folder searching_results another file with the current date will appear.
You can click on the column names – the sorting will change.
Past search results for several years you can see here.
Instead of conclusions
Script always available on GitHub.
Author: Mikhail Shardin
August 19, 2024
