Startup demos upcoming decentralized GPU infrastructure network to OpenAI, Uber

A project that started out as an institutional-grade quantitative trading system for cryptocurrencies and stocks has transitioned to become a decentralized network sourcing GPU computing power to serve increasing demand for AI and machine learning services.

Io.net has developed a test network that sources GPU computing power from a variety of data centers, cryptocurrency miners and decentralized storage providers. Aggregating GPU computational power is touted to drastically reduce the cost of renting these sources that are becoming increasingly expensive as AI and machine learning advances.

Speaking exclusively to Cointelegraph, CEO and co-founder Ahmad Shadid unpacks details of the network that aims to provide a decentralized platform for renting computing power at a fraction of the cost of centralized alternatives that currently exist.

Related: Future of payments: Visa to invest $100M in generative AI

Shadid explains how the project was conceived in late 2022 during a Solana hackathon. Io.net was developing a quantitative trading platform that relied on GPU computing power for its high-frequency operations, but was hamstrung by the exorbitant costs of renting GPU computing capacity.

The team unpacks the challenge of renting high-performance GPU hardware in its core documentation, with the price of renting a single NVIDIA A100 averaging around $80 per day per card. Needing more than 50 of these cards to operate 25 days a month would cost more than $100,000.

A solution was found in the discovery of Ray.io, an open-source library which OpenAI used to distribute ChatGPT training across over 300,000 CPUs and GPUs. The library streamlined the project’s infrastructure, with its backend developed in the short space of two months.

Shadid demoed io.net’s working testnet at the AI-focused Ray Summit in Sept. 2023, highlighting how the project aggregates computing power which is served to GPU consumers as clusters to meet specific AI or machine learning use cases.

“Not only does this model allow io.net to provision GPU compute up to 90% cheaper than incumbent suppliers, but it allows for virtually unlimited computing power.”

The decentralized network is set to leverage Solana’s blockchain to deliver SOL and USD Coin (USDC) payments to machine learning engineers and miners that are renting or providing computing power.

“When ML engineers pay for their clusters, these funds are directed straight to the miners that served in the cluster with their GPUs, with a small network fee being allocated to the io.net protocol.”

The project’s roadmap is set to include the launch of a dual native token system that will feature $IO and $IOSD. The token model will reward miners for executing machine learning workloads and maintaining network uptime while considering the dollar cost of electricity consumption.

“The IO coin will be freely traded in the crypto market and is the gate to access the compute power, while the IOSD token will serve as a stable credit token algorithmically pegged to 1 USD.”

Shadid tells Cointelegraph that io.net fundamentally differs from centralized cloud services like Amazon Web Services (AWS):

“To use an analogy, they’re United Airlines and we’re Kayak; they own planes whereas we help people book flights.”

The founder adds that any businesses that require AI computation typically use third-party providers, since they lack the GPUs to handle it all in-house. With demand for GPU’s estimated to increase by ten times every 18 months, Hadid says that these is often insufficient capacity to meet demand, leading to long wait times and high prices.

This is compounded by what he describes as inefficient utilization of data centers that are not optimized for the type of AI and machine learning work that is rapidly increasing:

“There are thousands of independent datacenters in the US alone, with an average utilization rate of 12 – 18%. As a result, bottlenecks are being created, which is having the knock-on effect of driving up prices for GPU compute.”

The upside is that the average cryptocurrency miner stands to gain by renting out their hardware to compete with the likes of AWS. Hadid says that the average miner using a 40GB A100 makes $0.52 a day, while AWS is selling the same card for AI computing for $59.78 a day.

“Part of the value proposition of io.net is first we allow participants to be exposed to the AI compute market and resell their GPUs and for the ML engineers we are significantly cheaper than AWS.”

Figures shared with Cointelegraph estimate that miners with GPU resources at their disposal could make 1500% more than they would from mining a variety of cryptocurrencies.

Magazine: Blockchain detectives: Mt. Gox collapse saw birth of Chainalysis