Codestin Search App

Modular Documentation

The Modular Platform accelerates AI inference and abstracts hardware complexity. Using our Docker container, you can deploy a GenAI model from Hugging Face with an OpenAI-compatible endpoint on a wide range of hardware.

And if you need to customize the model or tune a GPU kernel, Modular provides a depth of model extensibility and GPU programmability that you won't find anywhere else.

Get started

python
from openai import OpenAI

client = OpenAI(base_url="http://0.0.0.0:8000/v1", api_key="EMPTY")

completion = client.chat.completions.create(
  model="google/gemma-3-27b-it",
  messages=[
    {"role": "user", "content": "Who won the world series in 2020?"}
  ],
)

print(completion.choices[0].message.content)

Get started

Serving

Modular’s serving library is compatible with OpenAI APIs, so you can own your endpoint with minimal client-side code changes.

Deploying

Deploy your GenAI models to the cloud and scale your deployments across heterogenous GPU clusters.

Developing

The Modular platform provides full extensibility, so you can write custom ops, hardware-agnostic GPU kernels, and more.

Programming with Mojo🔥

Mojo is a Python-style programming language that allows you to write code for both CPUs and GPUs.

Learning tools

Agentic Cookbook

Turn-key applications that use GenAI models with the Modular platform.

GPU Puzzles

A hands-on guide to mastering GPU programming with Mojo.

Build an LLM with MAX

Learn to build an LLM from scratch with MAX.

500+ models supported

We're on a mission to make open source AI models as fast and easy to use as possible. Every model in our repo has been optimized using MAX Graph to ensure performance and portability across any architecture.

View Model Repo