Time to research: Hyperbolic

12 min readOct 10, 2024

Hyperbolic provides scalable GPU access, AI inference services, and solutions for monetizing idle computers. It serves businesses, researchers, data centers, and people, striving to democratize AI through competitive pricing and flexible usage plans.

Hyperbolic offers a solution to slash costly API or machine rental expenses by up to 80% without compromising quality. They offer an enormous range of GPU options to meet your needs with only a fraction of the cost from large cloud providers like AWS.

Hyperbolic also acts as a conduit to unlocking the earning potential of idle GPUs. When it comes to generating solutions for an equal and revolutionary future using AI, Hyperbolic prioritizes research and expertise.

Their network is highly available, with fault-tolerant features and real-time backups to provide ongoing operation. Designed with growth in mind, the Hyperbolic platform adapts seamlessly with your application and growing user base.

Their platform allows any GPU to host AI inference endpoints. It is designed for inclusivity and performance across several machines. By combining cutting-edge cryptography and secret computation, Hyperbolic assures the integrity of your requests as well as unprecedented data security.

Their novel compensation mechanism is designed to reward efforts, ensuring that everyone in their ecosystem benefits proportionally as the platform grows.

Architecture

Hyperbolic’s ecosystem works similarly to the solar system; everyone benefits, from the tiniest node to the largest data center, including those who only engage with the system temporarily. Each player, whether a GPU supplier or an AI service user, contributes to a dynamic environment in which all pieces work together seamlessly.

Hyperbolic Decentralized Orchestration Layer

Hyperbolic’s system is built around a decentralized orchestration layer powered by the Hyperbolic Decentralized Operation System (Hyper-dOS).

This core layer oversees and optimizes the utilization of global GPU infrastructure, combining computing capacity from data centers, mining farms, personal machines, and on-premises machines all around the world. This combined power enables AI services by providing the required computational resources.

Hyper-dOS organizes these resources into various independent clusters. Key features of this system include:

Auto-scaling: Clusters can automatically adjust their size based on the demand for computing power.
Self-healing: Clusters can detect and recover from any failures by themselves, ensuring continuous operation.
Customizable: Clusters can be tailored to meet specific needs and preferences, making the system flexible and adaptable.

AI Services Layer

Hyperbolic’s system is built around a decentralized orchestration layer powered by the Hyperbolic Decentralized Operation System (Hyper-dOS).

Services include:

Inference Services: Provides powerful and efficient inference capabilities, allowing AI models to make accurate predictions and decisions in real-time.
Model Training and Fine-tuning: Offers tools for developers to train and fine-tune their AI models, ensuring they can adapt to specific tasks and datasets effectively.
AI Model Evaluation: Tools and benchmarks to evaluate the performance and accuracy of AI models, helping developers to continuously improve their models.
Vector Database: A high-performance, scalable database designed to store and manage vector embeddings, enabling efficient retrieval and processing of large-scale AI data.
Retrieval-Augmented Generation (RAG): Combines retrieval-based methods with generative models to enhance the quality and relevance of AI outputs by leveraging external knowledge bases and databases.
Vector Embedding: Tools to create, manage, and utilize vector embeddings, which represent data in high-dimensional spaces for various AI applications like recommendation systems and natural language processing.

Verification and Privacy Layer

These layers ensure the security and integrity of AI outputs within the Hyperbolic ecosystem.

Verification Layer (Proof of Sampling): Implements a sampling-based verification protocol to ensure the honesty and correctness of AI inferences and other AI services. This mechanism, known as Proof of Sampling (PoSP), is the gold standard for verification in AI. Developed by the Hyperbolic team in collaboration with researchers from UC Berkeley and Columbia University, PoSP leverages game-theoretic models to secure decentralized systems. It also supports bringing verifiable AI results onto various L1 and L2 blockchains such as Ethereum, Avalanche, NEAR, Polygon, Scroll, and Manta.
Privacy Layer (Confidential Computing): Safeguards data and computations through advanced cryptographic techniques, ensuring confidentiality and privacy. It allows AI services on Hyperbolic to be adopted by Web2 enterprises, providing them with the necessary security and privacy protections.

Hyperbolic Blockchain Layer

The Hyperbolic blockchain will serve as the foundation of the ecosystem, facilitating settlement and governance. It also facilitates arbitration and verification of their verification layer, ensuring that everything is visible and reliable.

Application Layer

Hyperbolic’s application layer allows users to interact with AI-powered apps created by the community. They invite builders, startup owners, developers, and academics to engage with their team and learn how Hyperbolic may help your application reach its full potential.

With their decentralized orchestration layer, AI services layer, verification layer, privacy layer, and blockchain, they make it possible to build many AI applications that cannot be implemented on regular Web2 platforms.

Here are some potential applications they are developing with industry partners:

GPU Exchange
Revenue-sharing for AI Agents
Reputation and Incentive System
AI-powered DAO
GPU Restaking
Fractionalized GPU Ownership

Proof of Sampling (PoSP)

Proof of Sampling (PoSP) employs sampling techniques and game theory to ensure security and efficiency by designing a system in which all participants are rewarded to behave honestly. At the heart of this strategy is an incentive system that rewards actors who act honestly while penalizing those who do not.

By randomly picking and confirming transactions, PoSP minimizes the network’s processing cost, making it a scalable alternative for decentralized applications.

How it works:

A node (called an Asserter) computes a function and submits a cryptographic commitment to an orchestrator.
The Orchestrator, a trustless random beacon, then decides whether to accept the Asserter’s claim or initiate a challenge by randomly selecting validators to independently compute and submit their own commitments.
If all commitments match, the result is accepted; otherwise, an arbitration process is initiated to identify and penalize dishonest nodes.

The Hyperbolic research team performed thorough mathematical analysis and demonstrated that as long as the chance of challenge exceeds a particular threshold based on the setup settings, this PoSP achieves what is known in game theory as a pure strategy Nash Equilibrium.

It is a state in which all network participants are motivated to operate honestly since any other strategy would be less advantageous.

In other words, when the system is properly configured, with rewards and punishments balanced against the likelihood of being challenged, the most sensible and profitable strategy for all nodes is to compute the outcome precisely.

Read here for the detailed breakdown of PoSP.

SpML

Building on PoSP concepts, Hyperbolic created SpML, a PoSP implementation tailored exclusively for decentralized AI inference. SpML addresses the shortcomings of previous verification systems, like opML (which uses fraud proofs) and zkML (which employs zero-knowledge proofs).

SpML employs an effective way to provide speedy processing and good security while avoiding the computational overhead associated with zkML and the fraud risk shown in opML.

The SpML protocol improves security by randomly contesting the outcome of another validator. The asserter is uninformed of whether their results will be questioned or which validator will be chosen to carry out the challenge, ensuring an unbiased verification process.

If their outputs are identical, the transaction is deemed genuine, and the servers are compensated. If a disparity arises, the network votes to settle it, ensuring system integrity and confidence.

SpML outperforms opML and zkML by combining their strengths and effectively addressing their limitations. It provides greater security than opML by lowering the risk of undiscovered fraud while preserving scalability and efficiency.

SpML, in contrast to zkML, provides a more scalable and efficient solution by lowering the computational cost associated with zero-knowledge proofs while maintaining a high level of security via its sampling-based method.

Read here for the detailed breakdown of SpML.

Stable Diffusion

Stable Diffusion is a deep learning model used to turn text into images. It can generate high-quality, variant-style graphics from just a few lines. This is an example of the most recent model, Stable Diffusion XL, with the request “a photo of an astronaut riding a horse on Mars”.

Hyperbolic is an open-source platform that generates graphics in seconds utilizing Stable Diffusion models and numerous backends.

ControlNet

ControlNet improves image-to-image generation by conditioning input images, allowing for flexible control through techniques such as pose and edge recognition.

Consider having a gorgeous photograph with excellent composition but wanting to fill it with new content. This is where ControlNet shines. Let’s explore its capabilities with an example:

In this example, the image on the left is the original from the internet. In the center lies the image’s depth structure, which was generated with the Hyperbolic AI preprocessor. The image on the right was created using SDXL + ControlNet, which smoothly integrates depth information and the text prompt “an astronaut on Mars”.

LoRA

LoRA is a revolutionary AI picture generating technique that uses little adjustments to existing models to achieve large improvements. Pixel-Art-XL, for example, is a LoRA adaption for Stable Diffusion XL that was created expressly to improve the pixel art generating capabilities. Using the same prompt, SDXL and SDXL+Pixl-Art-LoRA produce very distinct images.

They support a variety of LoRAs for Stable Diffusion v1.5 and Stable Diffusion XL, each of which adds unique functionality to the models.

Supported LoRA list:

TVM Supported Resolutions

GPU Marketplace Pricing

Supply side

Supply Rewards: Earn points for providing GPU resources on their platform.

Rental Income: Gain rental payments for the GPUs you supply, detailed in the payment structure below.

Machines shared are used by a client:

Maximum Rental Payment Income (Suppliers set their own price that should not exceed the following):

80GB VRAM

H100 SXM: $3.20/hour
H100 PCIe: $3.00/hour
A100 SXM: $1.80/hour
A100 PCIe: $1.60/hour

48GB VRAM

L40: $1.00/hour
L40S: $1.00/hour
RTX 6000 Ada: $0.90/hour
RTX A6000: $0.75/hour
A40: $0.5/hour

24GB VRAM AND UNDER

RTX 4090: $0.50/hour
RTX 3090 Ti: $0.30/hour
RTX 3090: $0.30/hour
RTX A5000: $0.30/hour
RTX A4000 Ada: $0.30/hour
RTX A4500: $0.30/hour
RTX A4000: $0.30/hour
RTX 3080: $0.20/hour
RTX 3070: $0.20/hour
A30: $0.20/hour
Tesla T4: $0.20/hour

Platform Fee:

Get ginoongbakulaw’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

Hyperbolic will take a 10% platform fee from the rental income.

Example: Let’s say a supplier decides to set up an H100 SXM at $2.5/hour, when the machine is being used by a client, they will be receiving 90% of the price they decide to share on the network, aka $2.25/hour.

Machines shared on the network are sitting idle:

Hourly Point Reward Examples:

H100 SXM: 320 points/hour
H100 PCIe: 300 points/hour
A100 SXM: 180 points/hour
A100 PCIe: 160 points/hour
3090: 30 points/hour
4090: 50 points/hour

Example: Whenever a supplier shares their machine on our network, if their machine is sitting idle, they will receive the amounts of point / hour as shared above.

Demand Side

Initial Credits:

Every renter will get $10 credits upon sign-up.

Renting Machines:

Renters purchase machines to rent using their credits.
Renters’ credits will decrease as they use the rented machines.
Renters’ instances will be terminated after the credit drop under $0.

Prices start from:

H100 SXM: Coming Soon
H100 PCIe: $2.20 +
A100 SXM: Coming Soon
A100 PCIe: $2.00 +
3090: $0.30 +
4090: Coming Soon

AI Inference Pricing

Basic Tier

Usage: Up to 60 requests per minute for free users, 600 requests per minute for users who deposit a minimum of $10 into their accounts. If users require higher rate limit, they can contact support@hyperbolic.xyz.

Important disclaimer: Each source IP address will be limited to 100/min to prevent DDoS attacks.

Features: Access to text-to-text, text-to-speech, and text-to-image models, text-to-video models and fine-tuning services.

Cost:

$10 credit for free trial

Text-to-text:

Instruct models:

Llama 3.1 8B (BF16): $0.1 per 1M tokens
Llama 3.1 70B (BF16): $0.4 per 1M tokens
Llama 3 70B (BF16): $0.4 per 1M tokens
Hermes 3 70B (BF16): $0.4 per 1M tokens
Llama 3.1 405B (BF16): $4 per 1M tokens
DeepSeek V2.5(BF16): $2 per 1M tokens
Qwen 2.5 72B (BF16): $0.4 per 1M tokens
Llama 3.2 3B (BF16): $0.1 per 1M tokens

2. Base models:

Llama 3.1 405B parameters BASE (FP8): $2 per 1M tokens
Llama 3.1 405B parameters BASE (BF16): $4 per 1M tokens
Llama 3.2 90B Vision BASE (BF16): $2 per 1M tokens

Text-to-image:

Pricing Formula: $Base Rate * (width/1024) * (height/1024) * (steps/25) per image

How it works?

Base Rate for SD family and Flux Dev: $0.01 is the cost for generating a standard image at 1024x1024 pixels with 25 steps
Base Rate for Flux Pro: $0.05 is the cost for generating a standard image at 1024x1024 pixels with 50 steps
Image Size Adjustment: The price scales with the width and height of the image. For example:

512x512 pixels: The price would be $Base Rate * (512/1024) * (512/1024) * (steps/25)
2048x2048 pixels: The price would be $Base Rate * (2048/1024) * (2048/1024) * (steps/25)

4. Step Adjustment: The price also scales based on the number of steps used in generating the image: