Meta Information, File System Capabilities, and the Decentralized Networks of the Future

A file must have a unique name within its folder, but several files in a folder may well have the same tags or even a completely identical set of tags. The folder is also a kind of tag. You can think of a folder name as the “master” tag that defines the location of a file in a hierarchical system.

Putting it all together

Further presentation is no longer a description of existing technologies, but thoughts about how and what can be improved. These are mainly the following ideas:

  • Unified storage of all kinds of metadata in extended attributes

  • Full support for extended attributes by file managers

  • DBMS integrated into the file system for indexing metadata

  • Unified metadata presentation file format

Storing meta information in extended attributes

So, we have extended attributes, and we have a lot of meta information about files. In all solutions, meta information is stored either in the name or in a separate database. But what if we store meta information in extended attributes? Advantages:

  • Access unification: information is tied to a file at the file system level, not at the level of a specific program. Other programs may have no idea about the file format, but they will work correctly with its meta information. This is exactly what happens with the existing minimum of metadata in modern operating systems: the file size and creation time do not depend on the format.

  • The ability to add meta information not provided by the file format.

  • The ability to change meta information without changing the file itself. This is important, since when the file changes, the hashes inevitably change, and the hash is the key for accessing the file in the decentralized network.

What is needed for this? Support for attributes from operating systems (already available) and file managers. Convenient means of entering and editing attributes. Convenient search tools.

Storing meta information in special files (“metafiles”)

This is a new abstraction: a special file containing meta information about another file or directory (“metafile”). This is both a torrent, and a library card, and an album or disk cover. There is everything you need: various hashes of the file, and a preview picture, and a table of contents, and a brief annotation, and a list of tags / keywords, and information about authors / publishers, and the date of creation, and a link to the source on the Internet, where the file is from. has been downloaded.

  • This is a ready-made object for publication in file-sharing networks of any kind; now you do not need to fill out dull forms on the torrent tracker, all the information

  • It is obtained by simply clicking on a file and selecting an item in the context menu of any file manager.

  • Such a file, unlike a torrent, contains human-oriented information, and therefore is understandable: you can always open it and see what it refers to; it can be googled simply by selecting the appropriate command from the context menu of the file.

  • Such a file is implementation-independent and can be used by any client of any decentralized network that supports this format. Those. it is not a new decentralized network, but a convenient tool for other networks, both existing and those that will appear in the future. The file has a specification that takes into account most networks, and can be expanded in the future without breaking backward compatibility

Have you ever shared a torrent tracker? There are many fields to fill. For example, for a book – title, author, year of publication, publisher, number of pages, format, cover (upload a picture to the server and insert a link), table of contents, annotation … All this information is present in the meta-information file ready-made. Thus, the design of the distribution is reduced to simply uploading the metadata file to the tracker. And in the decentralized networks of the future, trackers will not be needed: information about the content file can be generated from the metadata file automatically.

Storing meta information along with content files (“meta wrappers”)

If the format of the content file provides for storing part of the meta information inside the file itself (such as jpg or mp3), it can be stored there as well. This is useful when transferring files using tools that do not support transferring meta information. In addition, there is another interesting option that requires support for the file system, or at least file managers: a special universal wrapper file format. Such a file contains the content file and all the meta information about it not in the form of extended attributes, but in the form of a special format such as an archive. Such a file can be distributed by means that do not support extended attributes – for example, download via http. And once downloaded, it is transparently interpreted as a content file; meta information is also transparently transferred to extended attributes and indexed in the file database.

Outwardly, it may look like something that looks like an archive file with a special extension. The compression itself is optional (although it is possible – in this case it will be rather an archiver with support for compressing and unpacking meta-information). But it is possible without compression – then it will be a simple Unix-way solution, something similar to tar from Unix.

Decentralized social networks

Within the framework of my concept of decentralized file-sharing social networks, setting a like, reposting or adding an object to “favorites” means downloading this object and joining the distribution of this object in the decentralized network. However, in addition to the “direct interest” in the content, there can also be an indirect one – for example, a subscription to a particular person or community. Newsletters by subscription are downloaded from the network, but instead of the content itself, “metafiles” are downloaded – small files containing the minimum necessary information about the content – both machine-oriented and human-oriented.

Total

Convenience of working with metadata is one of the most important components for the further development of the decentralized Internet. Search by title, author, keywords, decentralized classification and structuring, in particular for finding “similar” information (if you downloaded something from the decentralized network, then you can rely on the Community and try to download what other users have classified as “similar “) – all this is based on meta information, and therefore simple (corresponding to the Unix-way philosophy) and universal (independent of a specific network) means of working with meta information are needed. Several of these proposals I also bring up for general discussion within the framework of this article. I hope it will be interesting.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *