IPFS as a support system to handle large files in Blockchain Ecosystem
A major challenge in blockchain space is storing large files efficiently on the blockchain. Blockchain technology provides an immutable data storage that allows transaction to only be appended and never to be modified or removed. However, the data storage on blockchain has a cost model that differs from conventional data storage in terms of size and cost. For instance in terms of size, the Bitcoin blockchain [3] provides the OP_RETURN opcode to store arbitrary data in transactions. It was limited to80 bytes which has been reduced to 40 in February 2014. In term of cost, storing 80 bytes on the Bitcoin blockchain costs roughly US$0.03617.
There are several Off-chain data storage solutions designed to be friendly to blockchain, such as Storj1, FileCoin2, Sia3,and IPFS4. These solutions share the idea of peer-to-peer distributed file system, where the data is shredded, encrypted, and distributed to multiple nodes in the network to ensure their safety and availability.
InterPlanetary Filesystem (IPFS) has been actively used by the blockchain developer community to address the storage problem. IPFS is based on cryptographic hashes is a file sharing system that can be leveraged to more efficiently store and share large files. It relies on a distributed hash table (DHT) to retrieve file locations and node connectivity information.
IPFS can be leveraged to support applications while keeping blockchain small. The cryptographic hashes that serve to securely identify a file’s content can be sent to blockchain, thus proving that such a file was available to someone at a certain time. It has a block storage model with hyperlinks to address the contents forming a Merkle Directed Acyclic Graph (DAG). Since IPFS is distributed, it has no single point of failure.
IPFS Storage method:
Large data files are stored on IPFS using the DAG cryptographic mechanism. The ProductTree object is used to store the provenance data, one can also use the links attribute to store links to traceability data, where hash represents the hash address of the data on IPFS.
Example; Say you would like to store a image and a sensorData of an agriculture setup.
{ “data”: {“type”: “ProductTree”, “productID”: “Rice_3067”},
“links”: [ {“hash”: “QmQZzTMN2X54SxC2jMuAug6Qcz1K-YS5ZB12i3gGusvBnkn”, “name”: “Image_20170910_package”, “size”:”7987"}, {“hash”:”QmUgeQaCjhZ8V42DKBLtTCvzxojUt-LpaX6QmXX9rTLNnxF”, “name”: “SensorDataList_20170911”, “size”:”5765"} ] }
Data attribute stores type. links attribute stores sensor DataPackage links, and hash is the IPFS hash address of the sensor DataPackage.
To summarise the above action in simple language:
Input: data, old ProductTree IPFS hash (OPT_hash)
Get MoonX’s stories in your inbox
Join Medium for free to get updates from this writer.
Out: data IPFS hash, new ProductTree IPFS hash (NPT_hash)
- OPT_Data Get oldProductTree object according to OPT_hash;
2：if Data is video or image then
3：Data_hash Store the data directly to IPFS, and obtain the data
IPFS hash address;
4：NPT_Data_hash Link the data_hash into OPT_Data, generate new
ProductTree object, and obtain NPT_Data_hash;
5：else
6：Data_hash Store the data to IPFS by creating a custom object, and
obtain the data IPFS hash address;
7：NPT_hash Link the data into the sensorDataList object and obtain
new ProductTree IFPS hash;
8：end if
9：return Data_hash, NPT_ hash;
Each time the data is stored in the IPFS, the updated ProductTree hash address and the hash address of the corresponding data block are encapsulated into a transaction and written into blockchain. The data is sent to the blockchain node in json format. The blockchcian node initiates the transaction and writes to the block after receiving the transaction request.
Dapps on EOS blockchain have actively been utilising the multitude benefits of IPFS to enhance the possibilities of storing large size data in a decentalized fashion to fully utilize the value of blockchain ecosystem. Some of the Dapps that are actively utilizing IPFS are ThePhemi.com, Everipedia and Steemit.
