Is dependency management in Javascript going to a new level? Working with ES modules without assemblers
Initially, this article was intended as a story about the differences and purpose of fields dependencies, devDependencies And peerDependencies V package.json. This topic was chosen by the guys in my telegram channelby the way, subscribe if you haven't already. However, when I looked at the amount of content on this topic, I realized that there is enough of it even in the Russian segment. At the same time, I read one article that seemed very good to me, and there were also thoughts on the topic of future dependency management.
As a result, I decided to briefly retell the above-mentioned article in order to better understand the topic myself, as well as sketch out a project for managing dependencies directly on the client, through ES Modules. So you can read either the original and full article from the author, or a shortened version at first half this article. And the analysis of the ESM work will be second half.
History of Dependency Management Development
In the distant past, which I believe many have already forgotten, there was no NodeJS, so libraries or scripts were included directly in HTML using the tag script:
<script src="https://habr.com/ru/articles/825424/<URL>"></script>In place <URL> you need to put a link to js file. Typically this was a link to CDN:
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.6.1/jquery.min.js"></script>In this case, we completely rely on the CDN provider, since we have no control over what we download. But earlier with this method there was a bonus in the form of a cross-domain cache, but this is more not relevant for safety reasons.
To have full control, you had to download the library code and store it in the same repository as the application. However, as the number of libraries grew over time, it became more and more difficult to manage.
Here came to light Bower – a package manager that automates the loading of libraries. Moreover, everything that with its appearance needed to be stored in the repository with the application –bower.jsonwhich looks like this:
{
"name": "my-app",
"dependencies": {
"react": "^16.1.0"
}
}Here you can already see the similarity with what is used now.
It was enough to run the command bower install and Bower will install all the dependencies that are in dependencies. At the same time, it was possible to use them in the project, both with the help of various task managers of the type Grunt or Gulpand the old-fashioned way, through the tag script:
<script src="https://habr.com/ru/articles/825424/bower_components/jquery/dist/jquery.min.js"></script>It is also worth noting that with the advent of Bower, versioning entered the scene in the form that we are all accustomed to. Since Bower has its own package registry, there was a need to somehow navigate this. Now it was enough to specify the range of versions according to SemVer to load a particular library.
This formalization and automation in dependency management and the emergence of various modular systems allowed library developers to use third libraries, thereby creating the term transitive dependencies.
The key to understanding how any package manager works is understanding how dependency resolution works. At installation time, Bower picks up the appropriate dependencies according to the dependencies field, but since transitive dependencies also appear, the resolution process becomes recursive and is a tree traversal.
It is also worth noting that in addition to dependencies with the advent of Bower the field appeared devDependencies. Based on the dev prefix, it is clear that all dependencies that help us in development, but are not needed in the application code itself, are indicated here, that is, various libraries for testing, formatting, etc. The package manager will only download direct ones during installation. devDependenciesand will ignore transitive ones:
In this case, in Bower, all downloaded dependencies will be located in one directory in a flat form:
This structure increases the risk of conflicts, which can arise when a project's dependencies depend on different versions of the same library:
Since the installation goes to one directory, Bower cannot install multiple versions of the same package. In this case, the developer must manually select which version to use, which entails additional risks if we are talking about major versions.
For this purpose, a field has appeared resolutions:
{
"resolutions": {
"library-d": "2.0.0"
}
}
This problem was a strong limiter, so it is not surprising that a solution was soon found. And it came from NodeJSwhen a package manager was developed for this platform – NPM.
NPM originally had nested dependency resolution model, that is, for each dependency its own directory is created node_moduleswhere its own dependencies are stored, thereby avoiding conflicts.
However, this approach also has its drawbacks, namely problems with duplicate packages and hierarchy depth. Therefore, with an irresponsible approach, the weight of the directory node_modules could become colossal, and it was also possible to run into limiting maximum path length on Windows.
As a result, NPM 3 switched to hoisted a resolution model in which the package manager tries to place all packages at the top level. And only when a version conflict occurs, a separate nested directory is created node_modules for a specific package, where the conflicting dependency is located.
The principle of this model is that NodeJS goes through the entire directory when searching for a dependency. node_modules from bottom to top, that is, “floats up”.
When talking about Bower, dependencies and devDependencies were introduced, which are also present in NPM. However, NPM has a number of other fields with dependencies, which are also described in detail in the original article, which I mentioned at the beginning, and also here. So I'll go over it briefly. TO dependencies And devDependencies added:
peer dependencies – this type of dependencies is most often used for library development. A striking example is
react-dom– is a library for working with DOM, which means that you will use it in your projectreact. That isreact-domdoes not clearly indicate what she needsreactbut why? React can be used in different environments. For front-end development, the usual combination isreactAndreact-domhowever for mobile developmentreact-domnot needed, but wantedreact-native. Thus, peerDependencies indicates a relationship, but not rigidly, shifting responsibility for the presence of the required dependency to the developer.bundledDependencies – are intended for those cases when you pack your project into one file. This is done using the command
npm pack, which turns your folder into a tarball file. Thus, all dependencies come immediately with the package and the package manager will no longer resolve them.optionalDependencies – this directory is usually used to install dependencies that are context-specific. For example, in various CI/CD or operating system scenarios.
However, about peerDependencies I would like to add a couple more points. NPM 7 and higher will automatically install the missing ones peerDependencies. However, if the versions conflict, the installation will fail with an error. For example, the following error will occur when trying to install Storybook version 6 in an application with React 18:
If you run the installation with the flag --force or --legacy-peer-depsas the error text itself suggests, NPM will work as before NPM 7, but this may lead to problems with duplicates.
To solve such problems, NPM, similar to Bower, has a field overrideswhere you can solve this problem:
{
"dependencies": {
"react": "18.2.0"
},
"devDependencies": {
"@storybook/react": "6.3.13"
},
"overrides": {
"@storybook/react": {
"react": "18.2.0"
}
}
}As I wrote earlier peerDependenciesare typically used to develop libraries that require a host library (in the example with react-dom react acts as a host library). However, some libraries can work without a host library, that is, they can work without it in one key, and with it in another. In this case, the dependency on the host library is optional and this can also be specified in package.json across the field peerDependenciesMeta:
{
"peerDependencies": {
"react": ">= 16"
},
"peerDependenciesMeta": {
"react": {
"optional": true
}
}
}However, NPM is not the only one that plays a significant role in the field of dependency management. Over time, analogues appeared: Yarn And PNPM. They also contributed and pushed the same NPM to evolve. For example, Yarn solved the problem of being able to download different versions of dependencies at different times.
Usually, dependencies are not specified strictly, limiting only the major version, thereby shifting responsibility to the package manager. In this case, there is a risk that during development one dependency will be loaded, and at the time of delivery of the code to production, a more recent version will be released and it will be installed, which can affect the behavior of the application. Of course, such a situation is unlikely, but still possible.
To solve this problem, Yarn added lock filewhich saves the result of the dependency resolution process, that is, saves the versions of the packages that were installed. Subsequently, using yarn.lockYarn will simply install the dependencies from the list, skipping the resolution step. This makes the installation step predictable and faster.
NPM also used this method and added its own lock fileTo take it into account as the main source, you need to install it through the command npm ci.
Yarn also speeds up the installation of packages due to a local cache, which allows you to create your own package registry on your machine in order to replace the network request to copy folders in the file system during the installation process.
In turn, PNPM also solved another problem of NPM, namely the problem of phantom dependencies. NPM uses hoisted a dependency resolution model where all packages “float” to the top, and only duplicate packages with different versions remain in place. This behavior has an effect that is not obvious at first glance. All packages that “float” become available for import in the application, although they may not be listed as dependencies V package.json.
Now imagine a situation where a patch was released. library-awhich is no longer used library-b. In this case, the application will crash because the import library-b will end in error.
In NPM you can monitor this with ESLint pluginand PNPM, unlike NPM and Yarn, does not try to make a structure node_modules as flat as possible, instead it rather normalizes the dependency graph.
So far, everything we've seen has looked more like a tree than a graph. And indeed, the “nested” model is closest to the tree structure, but in fact it simply duplicates the dependencies that can be placed in directed acyclic graph.
There was also a similar dilemma in file structures, which was solved symlinks. They allow you to create a link to a file or directory instead of duplicating the content. This is exactly the idea that PNPM uses.
Once installed, PNPM creates a .pnpm directory in node_modules, which is conceptually a key-value store. In this file, the key is the package name and its version, and the value is the contents of that package version.
This data structure eliminates the possibility of duplicates. The structure of the node_modules directory itself will be similar to the “nested” model from NPM, but instead of physical files there will be symlinks that lead to the same package storage.
IN node_modules each package will contain only symlinks to those packages that are specified in its package.jsonwhich completely eliminates the problem of phantom dependencies and eliminates the need for an ESLint plugin.
NPM 9 introduced a flag install-strategythe “linked” value in it includes a PNPM-like installation model with symbols, but at the moment NPM 10 has already been released, and this feature remains experimental.
The Future of Dependency Management Development
Nowadays, more and more libraries are switching from CommonJS modules to EcmaScript modules. In particular, my previous article about the transition from Webstorm to Cursor appeared due to the fact that msw the second version has one problem with Jest. Because of this, I began the transition to Vitest, which in turn caused the transition from Webstorm to Cursor, since the old Webstorm did not support Vitest.
As you hopefully remember, the first part of this article is a more or less brief excerpt from another article in which the author also shared that ESM is, in his opinion, the future of dependency management. More than a year has passed since that article was written, so I became interested in trying to implement an application without using NodeJS and builders. And now I will share what I came up with.
Previously, we added interactivity to the application by connecting scripts directly to the HTML. Therefore, in my opinion, it is ironic that everything is gradually returning to what we left long ago. But I won’t get ahead of myself and will start the story in order.
First of all, I started looking for information about full-fledged SPA on ESM and found nothing except for a number of articles in the foreign segment:
The second article was the one that interested me the most, as it already has an implementation of an application on ESM – the classic ToDo App, which I used as an example. You will also find there report 2019 from Fred Schott, where he raised the question of why assemblers are needed and whether they are needed at all.
The main message of this report, one might even say the answer to the question: “Why would it be great to use ESM directly on the client?” is the rejection of a huge toolkit. Just remember how many hours in your life you have spent on setting up this or that assembler or task manager. In addition, you will not have to spend time on the assembly itself, which can sometimes be tedious. And the identity of the environment for development and sales is also achieved.
5 years passed from the publication of this report to the publication of the article. Has anything changed globally during this time?
No.
NPM was created for Node, Web found a way out in the form of collectors and it is very difficult to deploy this machine. All libraries are written in CJS, and not all add ESM support. In addition, a number of critical points remain open:
The ability to use tools like Typescript is lost;
There is no way to minify the code and use Tree-shaking;
There is no possibility to use aliases for imports.
And in my opinion, the cons outweigh the pros so far.
Anyway, let's look at how to use ESM using the good old example Lodash. We can import the function we need or the entire library directly in any js file:
import get from "https://esm.sh/lodash-es@4.17.21/get.js";But inserting such a path every time is expensive, so it is recommended to use the tag script with type importmap:
<script type="importmap">
{
"imports": {
"get": "https://esm.sh/lodash-es@4.17.21/get.js"
}
}
</script>After which in any js The following notation can be used in the file:
import get from "get";Having looked at how to work with ESM, all that remains is to select a number of tools that will be required for development. I chose the following:
These two libraries can work in tandem with each other, bringing development closer to what I'm used to when working with React.
import { h, render } from 'https://esm.sh/preact';
import htm from 'https://esm.sh/htm';
// Initialize htm with Preact
const html = htm.bind(h);
const MyComponent = (props, state) => html`<div ...${props} class=bar>${foo}</div>`;
render(htm`<${MyComponent} />`, container);Also Here you can see even more tools that can be used directly on the client without a builder.
That's all about the tools, now let's sort it out projectwhich I sketched specifically for this article.
To run it locally install serve and run the command npx servelocated in the project directory.
Let's start with index.html :
<!DOCTYPE html>
<html lang="en">
<head>
<title>Agify</title>
<link
href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/css/bootstrap.min.css"
rel="stylesheet"
integrity="sha384-QWTKZyjpPEjISv5WaRU9OFeRpok6YctnYmDr5pNlyT2bRjXh0JMhjY6hW+ALEwIH"
crossorigin="anonymous"
/>
<link
rel="stylesheet"
href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.11.3/font/bootstrap-icons.min.css"
/>
<link href="https://habr.com/ru/articles/825424/styles/styles.css" rel="stylesheet" />
<link rel="manifest" href="https://habr.com/ru/articles/825424/./manifest.json" />
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<script type="importmap">
{
"imports": {
"preact": "https://esm.sh/preact",
"preact/": "https://esm.sh/preact/",
"htm": "https://esm.sh/htm",
"bootstrap": "https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/js/bootstrap.esm.min.js"
}
}
</script>
</head>
<body>
<script type="module" src="https://habr.com/ru/articles/825424/js/App.mjs"></script>
</body>
</html>Here we connect ready-made bootstrap styles, the necessary libraries and a module App.mjsabout which the further story will go.
import { render } from "preact";
import html from "./render.mjs";
import { Agify } from "./Agify/index.mjs";
import { Footer } from "./Footer/index.mjs";
import { Header } from "./Header/index.mjs";
import { Layout } from "./Layout/index.mjs";
const App = () => {
return html`
<${Layout} Header=${Header} Content=${Agify} Footer=${Footer} />
`;
};
render(html`<${App} />`, document.body);Here is the call renderfrom preact, which will render the application. Pay attention to the component App. I sketched it out to show the syntax and way of passing properties to child components. Also worth mentioning is the function htmlif you go to the file render.mjsthen you will see that connection preact And htmwhich I wrote about above:
import { h } from 'preact';
import htm from 'htm';
export default htm.bind(h);Everything is really very close to React. True, since jsx is passed to the html jsx function as a string, there is no highlighting in the IDE. For VS Code I got around this via a plugin lit-html. Lit – is another library that can be used to implement a similar task.
In the module App.mjs the component is rendered Layoutwhere a number of other components are passed through properties: Header, Agify And Footer. All except Agify are responsible for displaying the corresponding sections, and with Agify everything is a little more interesting.
For those who don't know, I copied the Agify application from my other article about Typescript Generics. So, if anyone is interested in implementing the same application in React, then welcome to sandbox. And here let's look at the component code Agify:
import { useState } from "preact/compat";
import get from "get";
import html from "../render.mjs";
export const Agify = () => {
const [value, setValue] = useState("");
const [age, setAge] = useState("");
const [loading, setLoading] = useState(false);
const handleSubmit = (e) => {
e.preventDefault();
setLoading(true);
fetch(`https://api.agify.io?name=${value}`)
.then((res) => {
return res.json();
})
.then((data) => {
setAge(get(data, "age"));
})
.finally(() => {
setLoading(false);
});
};
const handleReset = () => {
setValue("");
setAge("");
};
return html`
<div class="h-100 d-flex justify-content-center align-items-center agify">
${loading
? html`
<div
class="h-100 w-100 d-flex justify-content-center align-items-center agify-spinner-container"
>
<div class="spinner-border" role="status">
<span class="visually-hidden">Loading...</span>
</div>
</div>
`
: ""}
<div class="w-50 d-flex flex-column gap-3 agify-content">
<h2 class="text-center agify-title">
Estimate your age based on your first name
</h2>
<form class="agify-form" onSubmit=${handleSubmit}>
<div class="input-group mb-3">
<input
aria-describedby="button-addon2"
aria-label="Enter your first name"
class="form-control"
onChange=${(e) => setValue(e.target.value)}
placeholder="Enter your first name"
type="text"
value=${value}
/>
<button
class="btn btn-outline-secondary"
disabled=${!value}
type="submit"
id="button-addon2"
>
<i class="bi bi-search"></i>
</button>
</div>
<div
class="d-flex flex-column justify-content-center align-items-center gap-3 agify-result"
>
<h3 class="agify-result-title">
Your age is:
${age ? age : html`<i class="bi bi-question-circle"></i>`}
</h3>
<button
class="btn btn-secondary"
disabled=${!age}
type="button"
onClick=${handleReset}
>
Reset
</button>
</div>
</form>
</div>
</div>
`;
};Agify is a regular field where we have to enter our name, a button to search for the estimated age by name, a text with the answer and a reset button. Nothing complicated.
But what's interesting is that preact gives us the opportunity to use the React API, already familiar to many, in this case useStateand also interesting are examples of nested html rendering in conditional operators:
${age ? age : html`<i class="bi bi-question-circle"></i>`}To summarize, I will say that, as in 2019, as in 2022, and in 2024, the community has not accumulated a sufficient number of convenient tools and approaches to implement serious projects without assemblers. You have to sacrifice too much for little. The choice of tools is a trade-off between the complexity of the build process and performance + optimization. Using third party libraries in a non-build application can be difficult if there is no version of ESM available. Plus, I love Typescript too much to give it up.
But overall, I'm very interested in the idea that everything goes in a spiral. And this concept leads us to where it all began. It's very interesting to know if it does.
Thank you all for your time. If you liked this article, please subscribe to my telegram channelwhere you can influence the choice of topics for future articles, as well as mine YouTube channel.
