OmnyNet

Decentralized Compute Mesh

A global network of distributed compute nodes enabling AI inference, edge computing, and peer-to-peer workload distribution.

Distributed AI Inference

Run LLMs across multiple nodes

Split large language models across heterogeneous hardware for efficient inference

Model parallelism across nodes
RPC-based tensor communication
Dynamic load balancing
Support for CPU, GPU, and NPU

Secure WireGuard Mesh

End-to-end encrypted networking

All node communication encrypted via WireGuard tunnels with zero-trust architecture

Headscale coordination server
NAT traversal & hole punching
Private overlay network
Node authentication & ACLs

Edge Computing

Compute at the edge of the network

Deploy workloads close to users for low-latency processing

Global node distribution
Automatic workload routing
Local data processing
Reduced bandwidth costs

Hardware Agnostic

Any device can contribute

From Raspberry Pi to datacenter GPUs, every node adds capacity

CPU inference (x86, ARM)
NVIDIA CUDA acceleration
AMD ROCm / Vulkan support
Apple Metal for macOS

Smart Orchestration

Intelligent workload distribution

Automatically route tasks to optimal nodes based on capabilities

Real-time node discovery
Capability-based scheduling
Fault tolerance & failover
Resource monitoring

Open Architecture

Built on open standards

Leveraging battle-tested open source technologies

llama.cpp RPC backend
Tailscale/Headscale mesh
gRPC communication
Kubernetes native

How It Works

Join the Network

Install the OmnyNet agent on your device. It automatically connects to the mesh via WireGuard and registers its compute capabilities.

Receive Workloads

The orchestrator assigns compute tasks based on your node's capabilities, availability, and network proximity to the requester.

Earn Rewards

Contribute your idle compute resources and earn rewards. The more you contribute, the more you earn from the network.

omnynet-export

Export Your Models for Distributed Inference

Convert any PyTorch or ONNX model to the .omny format

The omnynet-export tool converts your AI models into the .omny format - an ONNX-based format with embedded metadata for reliable distributed inference across the OmnyNet mesh.

What You Can Do:

Export PyTorch models to .omny
Convert existing ONNX models
Auto-detect optimal shard points
Set memory constraints per shard
Inspect exported model metadata

.omny Format Includes:

Pre-defined safe cut points
Exact tensor shapes per shard
Memory estimates for scheduling
Min/max shard constraints
VRAM requirements per node

# Quick Install

curl -sSL https://raw.githubusercontent.com/wcares/omnynet-export/main/install.sh | bash