Will IPFS be able to completely replace HTTP?
My name is Vitaly Kireev, I am the head of R&D at SpaceWeb. At the beginning of last year, we introduced IPFS technology into our hosting, and all our clients were able to post content on the IPFS network. We didn’t decide to take such a step right away: IPFS is still an experimental technology, and the R&D team has a lot of questions about it.
In this article we will talk about the basic principles of the IPFS device and discuss the advantages and disadvantages. We’ll also share why a classic hosting provider needed to implement IPFS technology and what difficulties we encountered in the process. The article will be useful for those who are just starting to get acquainted with IPFS or plan to use the technology for their projects.
Briefly: what is IPFS
IPFS is a peer-to-peer protocol with content-addressable data storage. It allows you to create a distributed file system in which there is no centralized server, but there are many nodes that make the content available. The principle of operation is similar to BitTorrent, where several computers (nodes) freely exchange files and distribute them throughout the network.
Simply put, if, when sending an HTTP request, we tell the server “find me a file at such and such an address,” then the IPFS request will sound like “find me a file with such and such content.” If such a file exists and is accessible (determined by comparing hashes), the peer-to-peer network will return the file to you.
How does file exchange between nodes work in IPFS?
When a file is published to an IPFS network, it is divided into blocks of a certain size. For each block, a unique cryptographic hash Content Identifier (CID) is created, which allows the file to be placed on multiple nodes at once, regardless of where it was originally launched.
To assemble a single file from different blocks with hashes, a directed acyclic Merkle tree graph (Merkle DAG) is used. By connecting blocks by hashes, we get the same blockchain, that is, a chain of blocks that allows us to restore the contents of a file, regardless of how we received these blocks.
To understand where to get the necessary blocks, IPFS has protocols for exchanging data – both the content itself and metadata associated with the file. And here is the most interesting thing. For example, a centralized storage system knows for sure that a specific server has the file you need, which means you can access it by IP address and gain access. But a decentralized system consists of an unknown number of nodes. That is, in an IPFS network we will never know the number of nodes and determine where exactly our site is located.
But this problem is solved in IPFS using the Lip P2P library. It supports two modes of message transmission on the network:
Standard content transfer between two nodes. Works if the file is already hosted on one node and it knows that the second node needs a file with a certain hash
A situation when a node and its environment do not have the necessary content. Then the second part of the protocol, related to the exchange of metadata, works. This is where gossips come into play. Network nodes begin to poll each other about a specific file. After the request, the node can either give the content or say that it does not exist, but assume that the file may be on other nodes. And so on until you find the file you need.
Advantages and disadvantages of IPFS technology
Advantages of IPFS:
+ Free. Since the storage is the local machines of other users, the technology does not require paying for the services of the hosting provider’s data centers.
+ Security. Access to sites hosted on a peer-to-peer network cannot be blocked and is not subject to sanctions or DDoS attacks. IPFS also has built-in encryption for content exchange.
+ “Eternal storage.” Under ideal conditions, content uploaded to such a p2p network will live forever. But or almost – as long as at least one node remains on the network (we all remember “give it speed”, whatever).
+ Data access speed. The file will be transferred to the user from the node that is closest to him.
+ Data authentication. If in the classic client-server model a file can be replaced, while preserving its address for the user, then the p2p network excludes this – the search is not based on the address, but on the hash.
+ File versioning at the network level. You can both track changes and roll back to earlier versions.
+ Crypto domains. They need to be registered once and do not need to be renewed like regular ones. These are domains in the .crypto, .blockchain and similar zones; they operate on the Ethereum blockchain.
Disadvantages of IPFS:
– No one guarantees the availability of data, as is the case with hosting. There were no peers – the files were no longer accessible. But if among the IPFS nodes there are several reliable ones, for example, hosting providers, there should not be problems with availability.
– The canonical URL in the IPFS network is ipfs://QmQJXYB8LnxWBkx5XR2WRCJYKLZB2Xxy9LoPngw98eNk4r/ – such a link is not supported by most browsers out of the box; you need to install official plugins. Opera and Brave have native support for the technology. Gateways save the situation; we open the link via https://ipfs.sweb.ru/ipfs/(hash address)/(file name and extension). I'll tell you more about this below.
– Public availability of data. If content is stored on unprotected user machines, the level of security will be low. When choosing IPFS, you need to focus on your goals – if you need to distribute content as widely as possible, this will be a good option. But an IPFS network is not suitable for storing sensitive data.
– Low motivation of other users to become network participants. We partially solve this problem with the help of Filecoin, a cryptocurrency that is awarded to users for storing other people’s data.
Why does a classic hosting provider need to create a public IPFS gateway: how we developed the project
As we mentioned at the beginning, the development of IPFS was an experiment for us. This is why we finally decided to take this step:
IPFS has clear value for the hosting audience. We are talking primarily about confirming the authenticity of data, storage reliability, speed of data access, and versioning.
The technology is based on principles that have already proven their worth – for example, BitTorrent, whose analogue is IPFS.
There is potential for commercial use. Dozens of successful commercial services that use IPFS technologies have already been implemented around the world. They are mainly from the Web3 world: decentralized exchanges and NFT projects.
It took 4 months to develop the idea, the active implementation phase took 1 month.
During the implementation process, interesting nuances also arose.
The idea was that we, as a reliable hosting provider, provide a stable node within IPFS that will always contain content.
At the beginning of our work, we made a test bench, raised IPFS and tried to open an Opera file to check the availability of content. But it didn't open. The reason was simple – other nodes did not know about its existence.
Then we started studying in more detail and found the built-in publishing function. It allows IPFS to notify popular nodes about a file right out of the box, and the configs of these nodes are already registered. After completing the publishing procedure, your content will be distributed through the LIP P2P library and appear on popular IPFS nodes.
After the content was distributed, the path to the content through the SpaceWeb IPFS gateway or another public IPFS gateway began to look like this:
https://{gateway URL}/ipfs/{content ID}/{optional path to resource}.
For example:
https://ipfs.sweb.ru/ipfs/QmQJXYB8LnxWBkx5XR2WRCJYKLZB2Xxy9LoPngw98eNk4r/
Client side: who is IPFS suitable for and how it is being used now
IPFS is chosen when you need to distribute content as much as possible and make it available 24/7 even when one of the nodes is turned off. For example, if a company needs to publish public information – about products, team, etc. IPFS technology will help maintain it as stable as possible.
But if you need to store confidential information, such as customer data, it is better to choose a classic hosting provider. It will be much easier to provide secure storage on dedicated servers.
Nothing gives as much useful knowledge about the demand for your technology as testing with real users. We tracked user behavior for a year and this is the data we received:
839 clients connected IPFS, of which 171 people used the network. We uploaded 1.7 TB of data to the network. We have allocated 30GB to each user for free to upload data to IPFS, so anyone can test the technology.
42.5% uploaded various files to IPFS. Most likely, just to evaluate the network's performance.
14% store backups in IPFS.
30% store media content.
11% use the network to host static sites and their files.
2.5% store scripts (JavaScript, Python, etc.).
The first two cases are an example of the targeted use of technology, the last one is not. Although the solution is completely okay.
Here are a couple of cool projects that our clients have posted on IPFS: first And second.
What conclusions did we draw?
The technology is promising, and there is real demand for it among hosting users. It is clear that IPFS will not replace HTTP. At least for now. The use of IPFS is mostly targeted, which is also good news. We understand that to launch IPFS and ensure stable operation of the network, the presence of a large number of its active users is necessary – so for now the system lives in hybrid mode. Using our own server capacity, we guarantee the operability of the network even if there are not enough nodes. In the future, we plan to supply clients with unstopable domains and link them to the IPFS hash.
What do you think about IPFS? Have you tried it yet? Do you believe in prospects?
