MagmaFS

Magma network filesystem

Developer(s)	Tx0 <tx0@strumentiresistenti.net>
Operating system	Linux and BSD kernels
Type	Network file system
License	[[1]]
Website	http://www.magmafs.net/

Magma is a network file system based on Distributed hash table, written in C, compatible with Linux and BSD kernels using FUSE.

Terminology and basic principles

Network topology is described by a structure of nodes called a lava ring. Each node is called a vulcano. Each vulcano hosts a portion of a common key space, delimited by two SHA1 keys. Each vulcano also is in charge of mirroring the key space of the previous node, to ensure data redundancy.

Magma can store many kink of objects: files, directories, symbolic links, block and characted devices, FIFO pipes. All this kink of objects are called flares. A flare of any cast has some basic properties, like a path and a hash key. But directory flares will have some specific informations that don't apply to symbolic links for example.

A flare can be moved between vulcano nodes with a set of operations which are part of magma network protocol. To be easily movable, each type of flare, including directories, has been reimplemented as a two files set, the first containing flare informations (metadata) and the second containing flare contents.

Routing

Since each vulcano node has complete network topology available, routing is just a matter of matching flare key with nodes key-space and find the node holding the flare. Network topology is also saved in the distributed directory /.dht/ inside magma filesystem. Vulcano nodes can periodically check their informations against contents of /.dht/ directory to know if something has changed.

Load balancing

Each vulcano node has some parameters declared at boot, like bandwidth and storage available. A separate thread called balancer is devoted to distribute keys to avoid nodes overloading or underloading. Each node has a dynamic load value associated, which is computed by the formula:

$l=kl*{\frac {b_{n}}{b_{a}}}*{\frac {s_{n}}{s_{a}}}$

where $kl$ is the node key load calculated on logarithmic scale; $b_{n}$ is node bandwidth and $b_{a}$ is average bandwidth; $s_{n}$ is node storage and $s_{a}$ is average storage

Magma server

Magma server magmad manages intercommunication between DHT nodes and magma clients. Flare system provides a network event loop that accept incoming connections. Three kind of connection are accepted.

• A flare protocol connection is used to operate on flares: opening files and directory, reading and writing, getting information and exchange flares during balancing operations. Flare protocol is a binary protocol.

• A internode protocol connection is used to exchange DHT information and to join new nodes. Internode protocol is a binary protocol.

• A console protocol connection is used to let administrators query lava network, perform simple operations like listing directory contents and show file content and finally issue some administrative commands to nodes like shutting down. Console protocol is a text based protocol accessible via Telnet.

Magma client

Magma client magma.mount is based on FUSE, being compatible with Linux and BSD kernels. Magma client uses a flare protocol connection to contact and operate with a near Magma server. Network topology and flare location is totally transparent to clients which simply query one server as if all information are located on that host only.

A cryptographic layer is planned for Magma client, allowing file contents only encryption. Implementing cryptography on client side is due to scalability (computational power will increase at same rate of computational request) and cryptographic key privacy (keys or passphrases will never reach the server).

Alternative NFS interface

As an alternative to Magma client, which is supported only by Linux and BSD kernels, Magma server plans to offer a NFS interface for other Unices. Since NFS is an established standard, no new features can be added. For example, cryptographic layer will be unavailable to client mounting Magma shares through NFS.