Portkey

Unpopular opinion: Most of us aren't spending enough tokens. We just hosted a hackathon this weekend at Portkey. Incredible teams, thoughtful solutions. Participants had unlimited access to the world’s best LLMs. As a judge with access to the backend logs, I found myself obsessively refreshing one metric: Total Tokens Consumed. Many have started to use Gamma, NotebookLM, Claude, and ChatGPT - BUT, when it comes to coding, usage is still abysmally low. So here's a recommendation from someone who's gotten a little comfortable with AI doing most of his work: BURN MORE TOKENS! We'll continue to host more hackathons, so follow Portkey to get notified for the next one! HackCulture

The slowest part of any startup isn't building anymore. It's the back-and-forth after customer calls. "Can you add X?" "We need Y to integrate." "Our compliance team requires Z." What if that call ended with a merged PR - not another doc to write? That’s what we’re building at AltorLab. We turn customer meetings into shipped code. Today, we’re announcing our first design partnership with Portkey - a team that lives and breathes production AI infrastructure. We’ve been embedded with their team for the past few weeks now watching how fast-moving AI companies handle enterprise requests. As Jen Abel says - the unsung startup heroes are enterprise early-adopters. Thanks to Rohit Agarwal and Ayush Garg for taking a bet on us. Congratulations to Portkey for tracking $93M in LLM spend across production workloads. We’re opening a small private beta for teams that: → ship daily → talk to customers constantly → want customer requests → code faster If that’s you, comment “Altor” or DM me - I’ll share early access. Built by engineers from Microsoft and Rubrik Special Thanks to Sanskar Jain Siddharth Goyal Pritam Roy Punit Dhoot Akash Agarwal (AK) for all the help along the way.

There’s a very specific kind of excitement engineers don’t talk about enough. It’s when you realise the thing you built is being used at real scale, and that people genuinely depend on it. Last year, Portkey ended up tracking $93M in LLM spend across production workloads. That number took a second to sink in. It was a feeling mix of disbelief and excitement, enough that I went and sanity-checked it myself, and then just sat with the moment. Because that’s the payoff. Seeing infra you’ve obsessed over quietly doing its job, day after day, for teams shipping real AI products. We decided to publish the data and methodology openly, because this kind of visibility into how LLMs are actually used is rare. Sharing the blog in case it’s useful 👇 https://lnkd.in/g7M82P_g

[ALERT]These models will be deactivated tomorrow.

AI Gateway - Your single gateway to 200+ LLMs Portkey is a fast AI gateway library with integrated Guardrails, designed to simplify AI integration. Why AI Gateway? Universal access: Connect to 200+ LLMs through one unified interface Built-in protection: 50+ AI Guardrails integrated seamlessly Lightweight & open: Minimal footprint (122kb), fully open-source Performance: Sub-millisecond latency (<1ms) for real-time applications Proven at scale: Processing 10B+ tokens daily Enterprise-ready: Enhanced security, scalability, and custom deployment options Simplify your AI infrastructure. One library, endless possibilities. Github: https://lnkd.in/ddeePaqR Book 1:1 on topmate.io: https://lnkd.in/dvWW9c76 #llms #generativeai #nlproc #ai #deeplearning

One of the hardest parts of working with LLM costs is knowing whether the numbers you’re using are still correct. Pricing changes, new modalities are introduced, and different providers expose costs in different ways. Without a reliable system behind it, pricing data quietly becomes an assumption. To handle this, we built an always-updated pricing database that continuously tracks provider changes and keeps our internal pricing data accurate, powering roughly $250,000 in LLM spend every day. We’re now open-sourcing the exact pricing database that runs behind the scenes so others can use the same data and inspect how costs are calculated. It covers 2,000+ models across 40+ providers and is accessible via a free public API with no authentication required. https://portkey.ai/models Github: https://lnkd.in/gFPnDcgi ⭐️⭐️⭐️ Portkey

AI spend is becoming a board-level concern. As AI moves from pilots to org-wide usage, companies are onboarding entire teams onto AI platforms. CFOs now care about accuracy, attribution, and predictability of AI costs and not just estimates. The problem: LLM pricing is no longer just token-based. Different models, providers, deployments, cache behavior, reasoning tokens, context tiers, and non-token fees break naive pricing systems. At scale, this leads to 20-40% gaps between dashboards and invoices. We’re open-sourcing how we solve this at Portkey. We solve two things: 1️⃣ A clean, structured database of LLM pricing across 2,000+ models and 50+ providers 2️⃣ Automated, near-realtime updates as providers change rates, models, and billing dimensions Huge credit to Siddharth Sambharia and Narendranath Gogineni for building this. If you’re managing AI spend at scale, this should save you significant time (and money).

🆓 Calculating costs in real-time for trillions of tokens within the 5% error range is HARD. (This is tens of millions of 💲💲💲 every month!) We're open-sourcing our database of 2330 models, their capabilities and pricing so you don't have to build it from scratch. We keep this updated, accurate and simple (no duplicates, no bloat, no marketing bs). Our aim -> within the next 2 months make this database near-realtime using autonomous agents looking at changes across all providers and providing accuracy to reduce the error range to well below 0.5% Built with a lot of ❤️ from Siddharth Sambharia Narendranath Gogineni at Portkey!

We've built a never-goes-old system that powers 400B+ tokens and roughly $250,000 in LLM spend every day. We’re now open-sourcing the database produced by that system so you can use it and see exactly how pricing is handled under the hood. → portkey.ai/models — searchable pricing directory → https://lnkd.in/gKQXHxXq — raw JSON data Free API, no auth needed Supporting many AI models is not an integration problem. It’s a correctness problem. Pricing changes frequently. Token semantics differ per provider. Capabilities evolve independently (tools, reasoning, thinking, multimodal, embeddings, real-time). New models launch, others get deprecated, often without clear signals. For an AI Gateway, this directly impacts cost accounting, prompt studio UX, analytics, and governance. Another obvious challenge is logistics: how do you propagate model and pricing updates across SaaS, OSS, hybrid, and air-gapped deployments without forcing redeployments? But logistics is the easier part. What we built at Portkey: - We decoupled model intelligence from deployments. - We open-sourced a single repository that defines pricing and token calculation logic, normalized model capabilities per provider, and compatibility metadata required by the prompt engineering studio UI. This repo is the canonical source of truth. A GitHub workflow syncs it to an S3-backed registry: - SaaS deployments consume cached S3 data - Hybrid deployments fetch via a lightweight API (cached locally) - Air-gapped deployments bake configs into images and support S3→S3 sync to avoid rebuilds One commit updates everything, solving distribution. Why this is only the foundation: Correctness requires feedback loops. Keeping this data accurate as the ecosystem moves needs more than config management. The next step is making model intelligence continuously correct and self-updating. Stay tuned, something more exciting coming soon. 💯 #AIThatNeverBreaks #EnterpriseAIGateway

We’ve been working on something that came out of a very real, recurring problem: LLM pricing is hard to keep up with. Models change, pricing pages lag, and new billing dimensions keep getting added. A static spreadsheet goes stale in a day. To handle this, we built an always-updated pricing database that continuously tracks provider changes and keeps our internal pricing data accurate, powering roughly $250,000 in LLM spend every day. We’re now open-sourcing the exact pricing database that runs behind the scenes so others can use the same data and inspect how costs are calculated. The database covers 2,000+ models across 40+ providers and is available through a free public API with no authentication required -> https://portkey.ai/models