Introduction

In the previous article, The Intersection of Web3.0 x AI, we dived into core design themes that we believe are critical to the success of the integration of AI into Web3 technology. As a refresher, those themes are composability, interoperability, and verifiability.

• Composability: Developers can piece together models like AI legos easily in smart contracts to create powerful use-cases and build whatever they want.
• Interoperability: Developers can make interchain queries to models from any chain and gain access to whatever data they need for inference.
• Verifiability: Developers can customize the security of their inference in all applications ranging from high-leverage use-cases to whimsical for-fun use-cases.

In this article, we’re going to dive deeper into what these design themes entail and how we’re designing around these themes to build the future of Web3.0 x AI.

Design Principles for On-Chain AI

Theme #1: Composability

One of the most important features that has enabled open-source software to accelerate at such a rapid pace is its high degree of composability: the ability to take software modules, plug them into an application and immediately benefit from its utility. On-chain applications are the same, the composability of smart contracts empowers developers to combine existing contracts like building blocks, allowing for the rapid creation and deployment of innovative solutions without the need to reinvent the wheel. Through composability, web3.0 ecosystems flourish, fostering collaboration, experimentation, and the evolution of novel primitives on the blockchain.

We foresee the development of on-chain AI to follow a similar theme, we must build modular and composable infrastructure that allows developers to be able to stitch models together seamlessly to enable a renaissance of AI development on the blockchain. Engineers should be able to easily pick and choose models and piece them together like lego blocks together in smart contracts to build powerful use-cases easily. Especially for developing use-cases like autonomous on-chain trading agents, these are likely not going to be powered by monolithic “super models” and will likely be powered by a combination of highly-specialized models aimed to perform specific tasks instead. We can draw parallels to algorithmic trading in quantitative finance, where researchers piece together individual building blocks like forecasting models, convex optimizers, risk models, and more to create sophisticated trading systems instead of using unreliable and unexplainable monolithic models to do end-to-end trading.

Stacking multiple models together to improve results is hardly a new use-case in the traditional world of machine learning, there is a wealth of existing literature and research on model ensembles and these ideas have existed for a long time. Composability has always been important in the modeling world, as model ensembles are powerful and can lead to more robust results for a couple reasons, the most important being:

1. Reducing generalization error — Aggregating model outputs, also known as model stacking, can reduce generalization error of individual models that may have reasons such as overfitting or underfitting to specific datasets.
2. Robustness — Model ensembles are typically more robust to noise and variability in the data. If one model makes a mistake due to noise or outliers, other models in the ensemble can compensate for it, leading to more reliable predictions.
3. Decorrelation — To make model stacking even more powerful, a meta-model can weight the outputs of models differently (e.g. with a correlation matrix) to appropriately decrease weights for models that have highly-correlated outputs and increase weights for vice versa to maximize the stability of a model’s outputs.

To take full advantage of the composability offered by the blockchain application layer, we’ve also filed an ERC to propose an extensible framework for designing LLM-backed agents via smart contracts on EVM-based blockchains. The framework will be designed in a way that takes advantage of both the composability and interoperability of smart contracts so agents can communicate and interop seamlessly with each other.

Ultimately, creating an environment with easy access to these hosted models through simple interfaces like a straightforward smart contract function call will allow for maximum composability that would allow for the creation of powerful on-chain use-cases from individual models as building blocks. Your average developer should be abstracted away from mechanisms like model hosting, inference execution, hardware acceleration, and proof generation/validation to be able to rapidly iterate and create new applications.

Theme #2: Interoperability

For on-chain AI to thrive, interoperability is crucial. We divide interoperability into two main types: (1) AI models should be universally accessible from anywhere (e.g. smart contracts on other chains, REST APIs…etc) and (2) AI models should have seamless, programmatic access to data, tools, and smart contracts on other chains for maximum effectiveness.

The importance of the former is quite self-explanatory, the model as well as the results of the model should be made available to developers wherever they may be. Primarily, we foresee two types of cardinalities of access to on-chain AI for maximum interoperability: one-to-many which allows the source chain to run on-chain inference and expose the result to multiple chains via a view function that can be accessed via interchain read queries, and many-to-many which allows parties on any chain to invoke cross-chain smart contract calls to run inference on the source chain in an ad-hoc fashion.

As for the latter point, models must have access to a wealth of data spanning various chains, decentralized applications, and oracles made available through interoperability protocols for it to reach its full potential. Access to a diverse set of cross-chain data sources empowers AI models to engineer features from disparate sources, enhancing their capabilities and impact. For example, reputation AI models should be able to leverage historical data from multiple chains to improve their classification of whether a certain address is an authentic user or an airdrop-farming bot. Similarly, DeFi risk models should be able to leverage a wide variety of cross-chain data sources for more accurate risk assessments of the market. Interoperability unlocks a vast reservoir of data that facilitates the operation of on-chain AI. The utility of on-chain AI would also be limited if it did not have access to programming interfaces that allowed it to operate seamlessly in a multi-chain fashion. Providing access to a large number of chains and decentralized applications allows these models to perform more actions and simply have more impact.

The above reasons go to show why it’s important for on-chain AI to be, well, on-chain. By allowing models to live directly on a chain with built-in features of interoperability, we are able to empower models (or agents) to be accessible from anywhere, while granting them a greater ability to gather data it needs for inference to perform actions on-chain in a variety of modalities that can be tailor-made to different use-cases.

Theme #3: Verifiability

Computational verifiability, as a generic concept, is a recurring theme in all peer-to-peer networks. As an example, validators on proof-of-stake networks must attest to the validity of a block for consensus to be reached, which effectively ensures the integrity and verifies the transactions of the proposed block through cryptoeconomic security. This age-old idea is no different for inference of on-chain AI models. Varying degrees of security are needed in order to completely prevent or deter adversarial attack vectors on consumers of inference. This is especially important in the context of AI, where models are often a black box and the explainability of the results are low.

It’s important to note that different use-cases will require different levels of verifiability. For high-leverage use-cases in DeFi where an inference result will impact trades it will be vital for the computation to be verified cryptographically, while optimistic security mechanisms like ZK fraud proofs or opML or may be better suited for use-cases like generating dialogue with LLMs for NPCs in on-chain games where immediate security isn’t top-of-mind.

It’s important for the future of Web3.0 x AI infrastructure to provide a multitude of inference security options that offer a variety of tradeoffs in cost and security to be able to encompass all sorts of on-chain use-cases. Developers should be able to easily pick and choose whatever mode of security they would like to fit their use-case from a latency, cost, and practicality standpoint.

Enter: OpenGradient

OpenGradient is building an EVM-compatible blockchain network that aims to be the scalable and secure execution layer for AI. The network features a revolutionary approach to reinventing blockchain architecture, allowing developers to inference AI models directly from their solidity smart contracts with a simple function call.

Since the OpenGradient Network directly offers access to inference AI models via precompiles in smart contracts, it is able to take maximum advantage of the composability of smart contracts. Simply string inference calls to different models in your smart contract and you’ll be able to easily create powerful use-cases.

As for interoperability, since OpenGradient is an EVM-compatible network, smart contracts on OpenGradient are able to interact with contracts on other chains via interchain queries and cross-chain calls facilitated by major cross-chain solutions. Having models live on-chain on a network like OpenGradient is powerful, as that allows models to interface with other protocols and models directly; such technical designs maximizes the impact models can have and sets up the Web3.0 world for a renaissance of on-chain AI agents. The OpenGradient team is also cooking up an ERC that architects the future for how on-chain agents and models will interact with each other on EVM-compatible networks.

Lastly, OpenGradient offers a wide selection of a variety of different security mechanisms for inference ranging from cryptographic to cryptoeconomic. The full range of options are listed in our docs here, so regardless of whether you need immediate high-cost cryptographic security or optimistic cryptoeconomic security, we’re sure you’ll be able to find a solution that fits your use-case. Block data and inference artifacts are all batch-posted and persisted to the data availability layer as well, allowing anyone to verify the proofs themselves.

Conclusion

Conclusively, the successful decentralization of AI inference within the Web3.0 framework hinges on three critical themes: composability, interoperability, and verifiability. Composability, inspired by the success of open-source software, enables developers to seamlessly combine modular AI components to create innovative applications easily. Interoperability ensures that AI models are universally accessible and can leverage diverse data sources across multiple chains, enhancing their capabilities. Verifiability, essential for maintaining trust and security, is essential to all blockchain compute and offering a variety of choices for developers to choose from to tailor to specific use-cases is vital. We hope this article gave you a good run-down of the technological themes we believe are critical to the success of Web3.0 x AI.

Follow our journey here!

Twitter: https://twitter.com/OpenGradient

Abstract

Web3.0 x AI has been a very hot topic as of late, and many builders are rushing to build all sorts of applications, protocols, and infrastructure in this intersection of technologies. Ranging from on-chain ChatGPT to AI clones to “autonomous AI agents” to ML models in DeFi, there is a disarray of ideas blooming from the hype in this space, some of which make more sense than others.

But why does AI need to be decentralized? Does this intersection even make sense? More importantly, how do we differentiate noise from signal? In this article, we will provide a generic but robust framework for thinking about building infrastructure in the intersection between decentralized networks and inference of artificial intelligence models. Keeping in mind that there are many gross exaggerations of what Web3.0 x AI can achieve today, we will take on a pragmatic approach to explaining the impact of the intersection of these technologies.

Introduction

Web3.0 and AI are both loaded terms that encompass a wide range of different technologies and themes. The intersection of both fields can be broadly broken down into two core subspaces:

• Integrating Web3.0 into AI: Building generic AI infrastructure to inherit the properties of modern peer-to-peer blockchain networks including decentralization, censorship-resistance, utilization of token incentives…etc.
• Integrating AI into Web3.0: Building infrastructure that allows Web3.0 to leverage sophisticated models to empower both new and existing on-chain use-cases.

While there is significant overlap between the two problem spaces as they are not mutually exclusive (e.g. quite plausible to think that on-chain trading agents will be accessible and operate in a decentralized fashion), we’ve created this subcategorization intentionally since the generic problems technologists are trying to tackle in these two verticals, as we explain later in this article, fundamentally have different properties and time horizons.

Let’s dive in.

Decentralizing AI (Web3.0 -> AI)

Question: So what does it mean to integrate Web3.0 into AI?

Answer: At a high level, as we have previously described, it’s about building infrastructure that decentralizes the use of AI models so that users will always be able to have access to open-sourced and credibly neutral AI. As closed-source AI gradually takes over the world, the introduction of biases into these models for gain could pose a risk to the neutrality of models, having an open-platform for researchers to share, iterate, and develop AI could pose as a critical form a resistance to these risks. This is not dissimilar to cryptocurrencies, where we want to build a system robust to centralized points of failure: we want to help people become financially resistant to bank defaults or faults of governments. Similarly, relying on closed-source AI companies like OpenAI means they have significant control over the information you consume.

Question: Okay, but why is that important?

Answer: Especially when it comes to a powerful technology like AI, there are risks associated with having a single centralized entity with complete control. As people become increasingly reliant on powerful tools like ChatGPT, if a centralized entity were to have control over AI, they could effectively curate consumable information made accessible to the public, control the narratives exposed to people, and effectively redefine the way people think about certain topics. This becomes an increasingly scary idea when you think about the fact that this bias doesn’t stop at human consumption, other forms of technologies and models are beginning to programmatically depend more and more on AI: biased AI can churn out biased information, which is used as training data to create more biased models.

Question: So what does decentralization of AI inference actually look like?

Answer: Before we think about inference, let’s scrutinize what decentralization means broadly. The core value propositions to decentralization anything via a distributed ledger primarily include transparency, verifiability, and censorship-resistance. Using Ethereum as an example: the network offers complete transparency to the transactions that occur on the network, ensures verifiability of blocks of transactions via consensus and fork choice, and offers censorship-resistance by allowing anyone to gain access to the network by permissionlessly running a node. Similarly, these properties can all be inherited and applied to infrastructure aiming to decentralize inference as well: transparency of execution of inference, verifiability to ensure the integrity of the inference, and censorship-resistance by allowing anyone to permissionlessly upload or inference any models.

Question: If this is so important, why don’t AI folks care about Web3.0 x AI?

Answer: There isn’t active censorship of AI technology just yet, everyone still has access to AI and to our knowledge there aren’t centralized entities attempting to consciously introduce biases to AI in a way that has proven harmful yet (ahem Gemini). As a result, the primary problem that people in the AI camp are trying to solve is how to make models better: how to improve quality of model output, how to reduce error rate, how to increase model explainability…etc. It’s understandable that censorship-resistance and neutrality aren’t top of mind when the models themselves still have plenty of room for improvement, but one can also already observe entities like the US government preemptively taking steps in the direction of increasing their control.

This is also why the integration of Web3.0 into AI is a longer-term vision; censorship of AI is very much a function of the capabilities of those models and what one can do with those models. Currently many people might not believe it makes sense to put AI on-chain (aas s decentralization introduces computational overhead), but as AI gets better and becomes more powerful, it’s quite plausible to see centralized entities stepping in to restrict access to, regulate, or monetize the technology.

As a result, OpenGradient is actively building a network for AI models that directly front-runs this: it will be critical for AI in the long-run to be accessible through decentralized infrastructure that helps democratize access to this intelligence to prevent centralization of influence, introduction of biases, or outright censorship.

Question: That was a lot of wishful thinking associated with a dystopian future where AI is censored and regulated, what can Web3.0 do for AI do today?

Answer: One thing Web3.0 has been empirically useful for is creating economic incentive structures through distribution and consumption of cryptocurrency tokens. Akin to how tokens are used as computational gas on Ethereum, tokens can also be used to align incentives and stimulate supply-side contribution to the mission of the token issuer. In the status quo there is little to no incentive to create open-sourced models as closed-source models are the only way to monetize R&D, however cryptoeconomic incentive structures introduce multiple avenues that can play a vital part in accelerating open-source AI development by allowing researchers to monetize licensed open-source models. A couple modalities of token monetization models include:

• Bounty-hunting systems where researchers get token compensation for building open-sourced models that achieve specific purposes
• Pay-per-inference consumption (similar to OpenAI)
• Token-equity model where the model ownership itself is decentralized.

These are all exciting catalysts that Web3.0 can bring for open-source AI to blossom and rapidly accelerate, and are top-of-mind development efforts at OpenGradient as well.

On-Chain AI (AI -> Web3.0)

Question: We covered what Web3.0 can do for AI, now what can AI do for Web3.0?

Answer: Integration of AI into existing Web3.0 applications and technologies is a very real use-case that could be developed today, which is why at OpenGradient we think of the integration of AI into Web3.0 as a medium-term vision that we are working on realizing. Examples include autonomous algo-trading agents, risk models in lending protocols, quoting dynamic spreads in AMMs, NFTs with dynamic art, GameFi that leverages stateful LLMs…etc. It’s important to note that due to the prominence of generative AI tools, people often over-index on that when they hear the term “AI”. It’s critical to consider the fact that classical machine learning can also be very powerful in Web3.0 especially in the context of DeFi with the aforementioned use-cases as examples. Usage of sophisticated modeling techniques in trading/finance is not unprecedented either, the quant finance industry revolves around leveraging modeling to tackle asset management and trading. One of these impactful use-cases the team is actively tackling is conducting research on leveraging OpenGradient’s infrastructure to apply ML models for computing dynamic fees in Uniswap V4, which could ultimately reduce impermanent loss for AMM liquidity providers.

Question: If AI -> Web3.0 is really that powerful, why aren’t there more AI-empowered dApps?

Answer: We’re already beginning to see a lot of dApps slowly leverage AI and ML in their decentralized applications, like AI-driven yield farming or AI-driven crypto price prediction. However, developing AI applications in Web3.0 is extremely challenging. Architecting production-ready AI/ML systems to service inference requests in a scalable fashion is already a non-trivial effort, on top of that developers must also worry about securing the inference. Trustlessness and security is a must for everything on-chain to prevent adversarial attacks (i.e. need to ensure the inference is executed correctly or else people could fake results as an attack vector), and trustless compute for AI/ML via cryptographic mechanisms is both very new and very difficult to develop. To build a Web3.0 application leveraging AI, developers must secure GPU compute, host models on an inference server, build a proof-generation system, utilize hardware acceleration for inference and proof-generation, and also have on-chain contracts to validate the proofs.

At OpenGradient we’ve recognized this as one of the major obstacles to rapid acceleration in this space: as a result we are developing infrastructure that makes it super easy for Web3.0 developers to seamlessly leverage access to AI inference directly on-chain while allowing them to choose from a variety of cryptographic security mechanisms to tailor to their use-cases. Choose a model from the decentralized filestore, pass it into a simple function call and you’re able to run secured inference!

Question: What do we need to do to accelerate and realize on-chain AI?

Answer: In addition to providing an end-to-end solution available via a simple function call, to ultimately reduce these barriers and realize the full potential of the Web3.0 x AI vision, we believe there are three principles to creating infrastructure that will facilitate the blossoming of this technological intersection.

• Composability: Developers can piece together models like AI legos easily in smart contracts to create powerful use-cases and build whatever they want.
• Interoperability: Developers can make interchain queries to models from any chain and gain access to whatever data they need for inference.
• Verifiability: Developers can customize the security of their inference in all applications ranging from high-leverage use-cases to whimsical for-fun use-cases.

These are the design themes the OpenGradient Network is incorporating into its infrastructure to tailor use of AI to Web3.0 use.

OpenGradient: Building the Future of Web3.0 x AI

At OpenGradient, we’re building a blockchain network that is a composable execution layer for on-chain AI inference. The network features access to scalable and secure model inference, allowing developers to seamlessly leverage AI models in composable smart contracts to create powerful applications.

We have a multi-pronged vision when it comes to the Web3.0 x AI integration. As we have discussed, there are multiple verticals of integration when it comes to Web3.0 x AI, and this is how we are comprehensively presenting an approach to this integration.

(1) AI → Web3.0 (Short-Term): Revolutionize Web3.0 protocols and dApps with smarter features through seamless and secure access to AI and ML.

(2) Accelerating Open-Source AI (Medium-Term): Accelerate open-source AI by creating a platform for Web3.0 incentive structures for AI researchers like avenues for tokenizing models, charging gas for inference, decentralizing model ownership…etc.

(3) Web3.0 → AI (Long-Term): Integrate with Web2.0 AI to become the universal inference provider for verifiable AI that is scalable, secure, and censorship-resistant.

Conclusively, we hope this article gave you a good framework of thinking about the impact of the intersection of Web3.0 and AI, and how there are two primary subclasses of problems being addressed specifically in the realm of inference.

Follow our journey here!

Twitter: https://x.com/OpenGradient

Today, the OpenGradient team is excited to announce that we will be working with UAGP (Uniswap-Arbitrum Grant Program) as research grant recipients to conduct research on applied machine learning in AMM fee optimization. See the Twitter official announcement here.

The research will be focused on developing models to intelligently compute dynamic fees for the Uniswap V4 AMM to reduce net impermanent loss for liquidity providers.

The model inference, proof generation, and validation that secures the inference will all be powered by OpenGradient, an EVM blockchain network that is a composable execution layer for on-chain AI inference. The network features access to scalable and secure model inference, allowing developers to seamlessly leverage AI models in composable smart contracts to create powerful decentralized applications and use-cases.

You may remember AMM dynamic fees as one of the most impactful use-cases of on-chain inference in our previous article Applications of AI/ML on the Blockchain, the team cannot be more excited to be tackling one of these use-cases head on in this research initiative.

The Uniswap AMM

AMM pools on decentralized exchanges have skyrocketed in popularity over the past couple years, with AMMs like Uniswap crossing $10 billion in total value locked (TVL) at its peak with billions of dollars in trading volume every day. They’ve also continuously iterated on the design of their protocol, with the recent announcement of the imminent release of their V4 design.

As much as we love decentralized market making, unfortunately most liquidity providers in AMMs not only cannot make money, but actually end up with net losses as a direct result of impermanent loss which is loss of liquidity position value as a result of diverging price ratio between the two assets in the AMM pool.

In the above, one can see that impermanent loss can often outweigh fees made from an AMM pool. If market-making is not profitable in Web3, how are centralized market-makers like Citadel Securities consistently generating >$1 billion in revenue for 15 straight quarters?

The answer lies in the fee mechanism of market-making: centralized market-makers have sophisticated models of varying the spreads they quote depending on the market while AMMs are stuck charging the same fee agnostic of the market environment. This means when the market is extremely volatile arbitrageurs trade cheaply against the LPs which incurs impermanent loss, and when the market is very calm traders may opt to trade on centralized venues with cheaper fees which reduces revenue for LPs.

Enter: OpenGradient

Like aforementioned, OpenGradient is a blockchain network that can support native AI/ML inference directly computed, secured, and verified on-chain; all designed in a way that is so seamless that leveraging AI/ML is as easy as a simple function call.

We’re designing inference to suit all sorts of use-cases depending on demands for speed, cost, verifiability, and security. In additional to vanilla inference, other flavors include secure inference through protocols like opML (optimistic machine learning) and zkML (zero-knowledge machine learning), inference designed for privacy in trusted execution environments (TEE)…etc. The verifiable and secure nature of the inferences will unlock high-stakes use-cases of AI and ML like DeFi, which we are ecstatic to explore.

The research the OpenGradient team is conducting is centered around creating a dynamic fee computation model that can vary the fees quoted in AMM trading depending on the market condition, this can both reduce net impermanent loss for LPs during volatile markets and bring lower fees to retail traders during non-volatile markets.

As can be seen in the explanation above Uniswap V4 will become an increasingly sustainable DeFi protocol in the adversarial world of on-chain trading with OpenGradient’s dynamic fees.

• Liquidity Providers win because they will suffer from less net impermanent loss in the long run.
• Regular traders win because when users trade normally most of the time in non-volatile markets they can actually get lower fees.
• Opportunistic arbitrageurs lose because they make less net profit (due to higher fees) when they arb against a pool.

If you’ve been following the OpenGradient publication on Medium, you would already have context through a previous experiment we’ve done with ML-driven fees. Check out the write-up here where we ran some AMM simulations with a proof-of-concept regression model quoting dynamic fees: Protecting AMM Liquidity with On-chain ML Models.

In addition to simply just AMM dynamic fee calculation, OpenGradient’s infrastructure is widely applicable to all sorts of different use-cases in Web3. By taking advantage of interchain queries , we aim to bring trustless inference to different blockchain ecosystems to empower their dApps. Smart contracts on EVM blockchains like Arbitrum can make a simple interchain query and be able to enable all sorts of intelligent dApps and smart features.

Lastly, we’re excited about the fact that OpenGradient’s infrastructure and integration with Uniswap opens doors to new verticals of research in DeFi. Iterating on models to help improve the initial fee calculation model, or building out sophisticated models to create new use-cases on Uniswap like:

• On-chain systematic trading strategies
• AI-driven smart execution
• Models to predict trade toxicity for varying trading fees depending on trader profile
• Using HMM market regime models to vary fees in different market environments
• Incorporate AI-generated credit scores to vary fees

Conclusion

The OpenGradient team is so excited to be not just developing the network, but also researching core AI/ML use-cases in Web3.0 DeFi. For the Web3.0 space to move forward, it’s important we equip the ecosystem with the tools necessary that will make foundational protocols like AMMs (and more!) increasingly intelligent, smooth, and mature. We plan to be front and center in the movement to empower the crypto ecosystem with more compute that will allow for the development of smarter features.

If you liked this, stay tuned to our socials! We are building infrastructure that will enable more use-cases than just intelligent AMMs. I hope you’re just as excited as we are to see trustless, scalable, and native AI/ML inference introduced to Web3.

Twitter: https://x.com/OpenGradient

Discord: https://discord.gg/y8RD2VEa

Thank you to Jason Morton from EZKL for the feedback and review

We’ve all heard of machine learning, but what is zero-knowledge machine learning (zkML)? In this article, we proceed to break down what it is from a high level and explain its significance in various use-cases.

Introduction

Zero-knowledge machine learning (zkML) is a cryptographic protocol where the party that computes the output on an AI model given an input also generates a cryptographic proof that effectively proves something about the computation. (Note: computing an output from an input on an AI or ML model is also known as “inference”)

With zkML, the party that computes the inference can prove things like:

“I ran X input on Y model to generate Z output”

Okay, it’s quite apparent what the “ML” in zkML is about. What about zero-knowledge (ZK)? The magic of zkML comes from the fact that the prover can prove these statements while keeping certain elements (either X, Y or Z in the example above) hidden from verifiers. In other words, the prover can prove something is true while still ensuring others retain no knowledge of certain elements of the computation. More practically, an example of the zero-knowledge property may look like:

“I ran X input on Y model to generate Z output, and I can prove this without revealing the specific weights of Y model.

It’s almost like creating a cryptographic “receipt” for the inference, but instead of proof of purchase, it’s a proof that a specific computation has occurred.

But why is zkML important? And how can it affect you in your everyday life? Keep reading and we promise we will clear things up!

The Significance of zkML

The increasingly prevalence of machine intelligence is undeniably pervasive, infiltrating nearly every facet of modern life and rapidly expanding its footprint across multiple industries. From recommendation systems that drive your online shopping behavior to the quantitative trading systems that power the financial markets, AI has become an integral part of our daily experiences.

But usually, regular folks don’t think about that, at all.

When you type in your query on ChatGPT, you implicitly trust that OpenAI executes on the right query with the right model. When you ask Siri a question about Trump’s political stance, you trust that Apple’s servers are conducting the inference correctly.

At first, it seems like there’s nothing wrong with this, but when you put the ballooning compute costs for AI companies in perspective, there’s actually a strong financial incentive for these companies to inference on cheaper models or manipulate queries to significant reduce their computational costs.

Seems relatively harmless when ChatGPT responses suddenly drop in quality, or if your generated-art NFT sucks. But what if a fintech company uses cheaper models which results in inaccurate risk assessments? Or if a healthcare AI company uses a cheaper model which results in a false negative diagnosis?

We’re not saying AI companies do this, but we’re saying if they decided to do it, nobody would know and there is absolutely no recourse for people to verify or do anything about it.

Use-Cases

So what are the core use-cases of zkML? Here we break it down into a couple core categories and a variety of examples. Note that the reason why zkML is highly applicable for blockchains is because most blockchain virtual machines (VMs) don’t support computation-heavy operations like inference, so in order for the computation to be done outside of the VM, zkML is needed within the VM to validate the operations. This is why zkML is often seen as the key piece of infrastructure that effectively enables blockchains to harness AI and ML.

1. DeFi
   – Dynamic spread calculation in AMMs for liquidity protection
   – Sophisticated risk engines for DeFi protocols
   – On-chain derivatives pricing models
2. NFTs/GameFi
   – NFT art generation with diffusion models
   – On-Chain GameFi dialogue generation with LLMs
3. Web3.0 Security
   – On-chain smart contract vulnerability detection
4. AI-driven on-chain reputation Systems
   – On-chain reputation systems to measure flow toxicity in AMM trading
   – On-chain reputation systems to flag sybil accounts in airdrops
5. Validity ML
   – Ensuring traditional AI/ML companies are running inference on the correct model with the correct inputs. This could be particularly important in fields like HealthCare AI, or default risk assessment models in finance.

For more on use-cases specific to Web3.0, feel free to check out our previous article: Applications of AI/ML on the Blockchain.

OpenGradient

The problems described above are precisely ones that we are tackling at OpenGradient. The OpenGradient team is building a blockchain network that features native AI and ML inference secured by zkML technology from EZKL.

In other words, we’re building open-source, decentralized infrastructure that allows anyone to permissionlessly upload models, inference other models, and validate zkML proofs for the network all on-chain. This allows anyone to plug-and-play any models with the guarantee that the inference is cryptographically secured and allows consumers of the inference to validate and trust the model output. OpenGradient will also feature cross-chain communication and traditional Web2.0 API endpoints so anyone can harness the power of verifiable inference from any context. We aim to bring zkML-secured inference to everyone, in a way that’s so easy that it’s a simple function call.

Goal: Censorship-resistant, verifiable, and permissionless AI everywhere.

The Magic of zkML

So… how does this magic work? We’d highly recommend looking through EZKL’s blog as well as their docs to get a stronger technical understanding of zkML, but here’s a high-level overview.

1. Compilation: The AI model is converted into a mathematical circuit using tools like Circom. In the context of ZK proofs, tools like Circom are commonly used to describe computations as circuits in a high-level domain-specific language.
2. Setup Phase: There are two steps in the setup phase, firstly a trusted party generates public parameters that will be used in the proof system, e.g. using a multi-party compute (MPC) like the Perpetual Powers of Tau. Secondly, the prover (or party who sets up the compiled circuit) generates the circuit-specific proving keys and verifying keys used for proof generation and validation.
3. Witness Generation: The actual data (witness) related to the statement is mapped into the circuit.
4. Proof Generation: Using the proving key and the mapped witness, a prover generates a succinct proof.
5. Proof Validation: The generated proof is made available along with the statement to verifiers. The verifiers can use the public verification key to validate a proof cheaply in constant time.

An important key factor to keep in mind is that proof generation is quite computationally expensive, but proof validation is very cheap. Because of this, it makes sense for projects like OpenG to adopt an architecture where a proof is generated once, but validated numerous times by many nodes across the network.

Conclusion

Conclusively, zkML has quite tremendous implications on AI and its applications. It prevents model providers from engaging in fraudulent inference for cost reduction purposes, it provides guarantees in the integrity of your inference, and ultimately it could do all this without revealing information that needs to remain private.

We hope you’re as bullish on zkML as we are, feel free to follow us on our socials.

Website: https://www.opengradient.ai/

X (Twitter): https://x.com/OpenGradient

Website: https://ezkl.xyz/

Github: https://github.com/zkonduit/ezkl

Twitter: https://twitter.com/ezklxyz

Introduction

Blockchain technology and artificial intelligence (AI) are some of the hottest fields there are in the technology world right now, and for good reason. AI is rapidly automating tasks and making decisions that were once thought to be the exclusive domain of humans, while blockchain technology serves as a new paradigm of decentralized compute that can potentially revolutionize the modern financial system.

In this article, we plan to dive into the current state of the integration between these two fields. We’ll go into the use-cases of AI on the blockchain, why AI is significantly more prevalent in Web2.0 compared to Web3.0, and both the existing solutions to the problem as well as the major challenges faced by the integration.

While AI has been around for decades, the recent rise in its popularity driven by the commercialization of large language models has really shed a lot more light on the impact AI can have on our lives. The use-case for AI, however, extends far beyond just ChatGPT; AI is being used in healthcare for diagnostics and treatment. In consumer applications, AI is being used in feed-ranking algorithms to determine what users see on the main page. The use-case for AI is endless and research put the CAGR (compound annual growth rate) of the AI market cap to be around 20% for the next decade.

Blockchain technology has also skyrocketed in popularity over the past half-decade, with tens of billions of dollars of capital now locked on-chain. Blockchain technology brings value in the form of increased efficiency, greater transparency, and censorship-resistance to the financial system, and has incited a Cambrian Explosion of dApps (decentralized applications) being developed on the blockchain: from financial applications like decentralized exchanges or lending markets, to newer use-cases like NFT (non-fungible token) collecting or tokenization of virtual assets in games.

The contrast between the state of AI in Web2.0 and Web3.0, however, remains quite stark. AI technology has seemed to penetrate every corner of our lives in Web2.0, from Siri to your Instagram feed to the front page of recommendations when you do your online shopping on Amazon. Unfortunately, the same cannot be said about most modern blockchain use-cases: AMMs (automated market maekers) are still quoting static and arbitrarily-determined spreads, systematic trading algorithms or simple model-driven analysis and most compute-heavily workloads cannot be performed on-chain. These are impactful use-cases as well, as DeFi applications lock up billions of dollars in value. So…why is that? Although the convergence of AI and blockchain has many implications, as AI can further augment existing use-cases and applications in Web3.0 like it did to Web2.0, the integration of the two fields continues to be a challenge.

Difficulties in the AI x Blockchain Convergence

One of the difficulties of AI and blockchain integration comes from the ostensible clash in design ethos when it comes to decentralization of the two technologies. On one hand, AI training and inferences are both extremely expensive, both in dollar terms and in terms of computational requirements: as an example it costs approximately $12 million USD just to train GPT-3. Moreover, companies develop extremely specialized hardware to train models and service inferences as well: AI inference as an example can be up to 60–70% more efficient if run on GPUs. The high barriers of computation and cost results in high levels of centralization when it comes to both training and inference: these models are often both trained and served on powerfully equipped HPC clusters or servers where massive amounts of user inference requests are computed.

On the other hand, one of the core pillars of blockchain technology often documented in the classic blockchain trilemma is decentralization. In case one isn’t familiar, modern blockchains often have to design their architecture in a way that strikes a balance between security, decentralization, and scalability. In this instance, increasing the compute requirement for running a full node could have positive impacts on performance and thus, scalability of the blockchain, but it would have negative impacts on the level of decentralization since the requirements to run a full node are higher now. This is a major obstacle when it comes to being able to incorporate processes with heavy computational requirements into the blockchain like training or performing inference on AI models, as blockchains want to remain as decentralized as possible.

In the following sections, we take a look at limitations and challenges of both the native implementation of AI on the blockchain as well as the off-chain approach in the context of model inference.

Challenges of Native Inferences

On-chain native implementation of AI is difficult primarily for three different reasons. Firstly, as discussed above, there are compute limitations introduced due to hardware requirements for validator nodes. Secondly, there is a scaling bottleneck due to gas limitations per block in modern proof-of-stake networks. Last but hardly the least, it’s difficult to scale when every full node runs every single inference in order to verify state.

Before moving on, to lend clarification to what is meant by the “native” implementation, we’re defining it as direct on-chain inference or training of a model by the validator nodes in a blockchain network.

As we’ve discussed in the previous section, there are immediate hardware limitations that makes CPU validators not suitable for servicing AI inferences. GPUs have many more cores than CPUs, which allows them to perform more calculations at the same time. For example, a typical GPU might have 1000 cores, while a typical CPU might have only 4 cores. GPUs are also more optimized for floating-point operations, which are the most common type of operation used in AI tasks. Without running a network consisting of highly-performant GPU-based full nodes, which would come at the cost of decentralization, it could be difficult to achieve scalable on-chain inference.

Inference or training on CPU validators is already inefficient as it is, and on top of that one must consider the fact that the amount of gas per block is highly limited. For those who are unfamiliar, “gas” is essentially a measure of computation on the blockchain, more computationally heavy processes will cost you more gas on the blockchain to run. Blockchains have a gas limit per block, a maximum limit as to how much cumulative computation a block’s worth of transactions can afford. This is important because it sets a limitation on the size of the block, and ensures that full nodes can keep up with network validation. In other words, if too much computation was put into a block, validator nodes would gradually stop being able to keep up with the network due to space and speed requirements. When you consider that network congestion is already a major concern for blockchains, the incorporation of expensive AI processes could seriously exacerbate this problem.

Lastly, security is also an important aspect of the blockchain network, so modern proof-of-stake blockchains full nodes re-execute the transactions in the proposed block and attest to post-block state as a way to reach consensus. This means that if on-chain AI existed, for every inference run by a user, every full node would have to run the inference once. Now, imagine thousands of users around the world running thousands of inferences on the blockchain, and your home CPU validator executing ALL those inferences. It’s nothing to bat an eye at in the context of regular transactions, but when it comes to inferences from AI models that are already computationally expensive to begin with, which are moreover run on inefficient hardware, it further serves as an obstacle for native AI on the blockchain.

Challenges of Off-Chain Inferences

The challenges delineated above with the native implementation has prompted projects to consider the off-chain approach. There are a couple methods this can be done but it typically resembles a workflow like this: some request is made on-chain and that prompts a listening node to make an API request off-chain to a model provider which computes the request, and the responsible party writes the results of the request into smart contract memory on-chain where it can be retrieved.

While this is more scalable and efficient than the native approach discussed above, this introduces an entirely new problem: integrity.

AI models are often called a “black box”, and for good reason. It’s because people often have very little transparency and insight as to why and how an inference result is generated from a model. As a result, one can imagine that running off-chain inferences exposes blockchain networks to completely novel vectors of attack. Here’s a practical example to paint the picture: imagine a protocol that relies on off-chain ML to predict the leading returns of tokens before entering long/short positions to reflect those forecasts. Now, if I were the model provider that serviced these inferences, I could deliberately not run the model and return some bogus result, and proceed to simply front-run or sandwich their transactions for an easy profit. I could erroneously return a result that signals them to buy $DOGE, and proceed to buy $DOGE before them, and sell it right after they buy to make a profit on their market impact. That’s bad, and extremely dangerous because unlike many security vulnerabilities it can go completely undetected.

In order for blockchains to truly develop use-cases that depend on AI, security protocols that ensure the integrity of the inferences must be put in place.

The Solution

The solution to the AI x Blockchain integration are mainly two-pronged and involve zero-knowledge machine learning (ZKML) and optimistic machine learning (OPML). As we will detail in future blog posts, these are both solutions the OpenGradient network will integrate natively on-chain.

Zero-Knowledge Machine Learning

ZKML is a bleeding field of research that involves generating cryptographic proofs that aim to prove that some computation, in this case the inference done by some model, was computed correctly and with integrity. As described succinctly by EZKL, we can create ZK proofs that cryptographically prove statements like:

“I ran this publicly available neural network on some private data and it produced this output”

or

“I ran my private neural network on some public data and it produced this output”

The term “zero-knowledge”, as one might infer, is derived from the fact that certain things like the model input, or the model weights, can actually be kept private from the verifier without any compromise to the cryptographic security of the proof.

The advent of ZK is massive for decentralized AI/ML, because the execution of the inference as well as the proof generation can now be done on one beefy machine, and the proof can be cheaply verified across a network of light nodes to ensure that the inference was done correctly instead of having to replicate the computation across the entire network!

However, whereas validation of the ZK proof can be done cheaply in constant time, the biggest challenges faced by ZKML currently involve the sky-high computational and time costs that comes with generating the ZK proof. As a result, companies like Cysic are creating hardware tailored to proof generation and developers at EZKL are constantly improving their software to speed up proof generation.

Optimistic Machine Learning

Optimistic machine learning is another solution to decentralizing AI inference, with “optimistic” referring to the idea of optimistically trusting the results of an inference unless (or until) someone challenges the result.

In a challenge, the challenger first puts up a financial stake and a verification game is played, where a bisection protocol is used to locate the disputed step that caused the divergence in computation. The arbitrator smart contract resolves the challenge, heavily slashing the inference computer or the challenger depending on the success or failure of the challenge respectively.

This “innocent until proven guilty” optimistic model comes with both pros and cons compared to ZKML. The biggest advantage is the lower time/computational cost that comes with adopting a model that doesn’t generate a proof on every inference, which is significantly lower especially with larger models where both the inference and proof generation can be extremely expensive. However, the biggest disadvantage is the lack of the immediate cryptographic security that ZKML offers, and in the event of a successful challenge, the potential halting of a blockchain or henceforth the replaying of transactions…etc.

Conclusion

While the intersection of AI and Blockchain is an interesting space as AI could revolutionize Web3.0 use-cases like it did in Web2.0, inherent limitations in the software architecture and hardware requirements do make this a difficult challenge.

And that’s one problem that OpenGradient is tackling head-on, through bleeding-edge solutions like OPML and ZKML, the OpenGradient network will be able to execute on-chain inference in a scalable and secure fashion. Stay tuned to our medium page, our website, and our twitter for further updates that are coming very very soon.

Twitter: https://twitter.com/OpenGradient

Website: http://opengradient.ai/