One edge for APIs, agents and LLMs.

NexusGate authenticates, rate-limits, scans, caches and routes every request, including agent-to-agent tool calls, before it reaches your models. Watch requests move through it live.

Build your config Ask the assistant

Live analytics (simulated)

Requests/sec

p99 overhead (ms)

Cache hit ratio

Threats blocked

Rate limited

Tokens saved

Features

Zero-alloc proxy

Pooled buffers, weighted least-connections balancing and a background health probe that revives failed nodes.

MCP and A2A guard

Stops agent loops and enforces token budgets from X-Agent-ID and X-Max-Loops.

Semantic cache

Prompts at cosine 0.96 or higher skip the LLM.

CNN threat filter

Payload scan with an ONNX seam; signature scanner works today.

Distributed limits

Redis Lua token bucket, fail-open or fail-closed, X-RateLimit headers.

Zero-trust auth

RS256 JWT with cached JWKS and rotation. Set JWKS_URL.

Reliability

Circuit breaker on real status codes and latency, graceful shutdown, request IDs, timeouts.

Security headers

HSTS, nosniff, strict CORS allow-list and a 1 MB payload cap.

Observability

Prometheus metrics, Jaeger and an OTel collector in Compose.

Tools

Config builder

Generates the environment the gateway reads.

Requests/sec per keyBurst Token budget per agentIf Redis is down Upstreams (comma separated)Allowed CORS origin

Savings calculator

What the semantic cache could save on LLM spend.

Requests per dayTokens per requestPrice per 1M tokens ($)Cache hit rate (%)

Labs

Threat filter

Semantic cache

Agent loop guard

Rate limiter

Gateway assistant (Groq)

Paste a free key from console.groq.com/keys. The app finds a model your key can use, so it keeps working when Groq retires one.

Model

Tech stack

Go 1.22Redis + LuaONNX RuntimeJWT / JWKSOpenTelemetryPrometheusJaegerDocker Composek6GitHub ActionsGroq LLM

Roadmap

Item	Status
Proxy, limiter, breaker, A2A guard, JWKS auth	Built
Redis client and HNSW semantic cache	Next
ONNX CNN model and mTLS	Next
OTel exporters	Next

Run and deploy

docker compose up --build
curl localhost:8080/healthz
k6 run loadtest/load_test.js

Is the gateway hosted on GitHub Pages?

No. Pages hosts this static showcase. The gateway runs in Docker on your machine or cloud.

Where does the Groq key go?

Only into your browser tab and to Groq. Never commit a key to the repo.

About the builder

Nikhil Chary Sriramoju

B.Tech Computer Science and Engineering, Vaagdevi College of Engineering (2023 to 2027). Machine Learning Engineer Intern at Yuva Intern (NSDC) and Data Analytics Intern at Thiranex.

NASSCOM / Skill India Digital HubIBM SkillsBuild: AI FundamentalsReliance Foundation: AI/ML Engineer