Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 7e8be20

Browse files
bryantgillespiebryantgillespie
authored
Add Directus Docs AI assistant (#713)
* update all dependencies * add packageManager back * remove hardcoded url * wip * cleanup from review * cleanup again * switch to json for storing redirects * convert to ts * cleanup * ai cleanup * Update README.md * backfill all ids * feat(nav): rename Connect→APIs, Automate→Flows, Manage→Hosting * feat(llms): rename guide section titles to APIs and Flows * docs: replace branded "Directus Connect"/"Directus Automate" with APIs/Flows * docs(homepage): drop Data Engine/Data Studio framing * fix: fold 11.integrations into 12.integrations to clear numeric collision * fix sidebar * docs(nav): rename Guides→Guide, Hosting→Deploy * kill editor and explorer * initial moves * wippity * cleanup * fix nav * extract findNavNode * vanilla to vanilla js * fix claude bs * oops technologies is needed for the imaages * add chips to tutorials with framworks * more cleanup * All Frameworks * feat(layout): redesign docs experience * feat(layout): add prompt and shader debug components * fix(layout): restore reference content and skip API prerender * fix(build): stop masking prerender errors * feat(layout): use Directus docs typography * fix(build): inline unhead for preview * chore(layout): remove unused legacy helpers * Clean up docs navigation and icons * fix layous * Update Railway tutorial image * Skip base-prefixed API prerender * fix(build): lazy-load OAS spec to reduce prerender memory Move @directus/openapi import out of app.vue and into the api layout as a dynamic import. The 590KB spec no longer enters heap during content prerender; it's split into its own chunk loaded lazily at runtime on /api/* routes (which have prerender: false anyway). Delete the prerender module — it was already gated off by default and adds no value when API routes aren't prerendered. Also includes earlier nuxt.config tweaks: sourcemap off, llms-full.txt prerender skip, concurrency 1, icon serverBundle config. * increase concurrency * fix icon * feat(hero): painterly Kuwahara shader with fade-up painting layer * feat(search): add Typesense docs search * Add Typesense search refinements * Address search review feedback * docs(mcp): clarify MCP overview title * feat(mcp): add docs MCP server and tools * Address MCP review feedback * feat(assistant): add AI docs assistant * fix(layout): remove invalid navigation field * feat(assistant): add search palette handoff * refactor(assistant): swap @vercel/kv for @upstash/redis * Add docs assistant hardening * Document assistant contribution setup * Allow assistant on docs preview aliases * Add marketing-page fetch tool, update MSCL licensing in assistant - New get-directus-page MCP tool fetches directus.com pages as Markdown - Wire tool into assistant chat endpoint - Replace stale BSL info with MSCL (source-available, not open source) - Direct pricing questions to live /pricing page - Embed Directus website map in system prompt * Add core-tier, data-safety, and v12 grace facts to assistant prompt Carry high-frequency licensing facts into the prompt and point to /licensing/overview for setup mechanics. * Show readable label for get-directus-page in sources list Was falling through to the raw tool name; now renders e.g. "Read directus.com/pricing". * Address assistant review notes * Remove assistant ADR * Resolve assistant review feedback * Fix UTF-8 tool result truncation * Fix assistant runtime helper import * Avoid assistant module entry in tests
1 parent 55e4ea9 commit 7e8be20

65 files changed

Lines changed: 5917 additions & 342 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.env.example‎

Lines changed: 17 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,27 @@ TYPESENSE_PRIVATE_API_KEY=typesense_private_api_key_required_for_indexing_only
66
DIRECTUS_URL=https://marketing-directus-url.com
77
GOOGLE_TAG_MANAGER_ID=GTM-PTLT3GH
88
POSTHOG_API_HOST=https://directus.com/ingest
9+
POSTHOG_AI_HOST=https://us.i.posthog.com
910
POSTHOG_API_KEY=phc_secret_key_here
1011
NUXT_PUBLIC_SITE_URL=https://directus.com
1112
NUXT_PUBLIC_OG_BASE_URL=https://og.directus.com
1213
# Required outside local OG worker dev.
13-
# NUXT_OG_SIGNING_SECRET=shared_secret_from_website_og_worker
1414
# OG_SIGNING_SECRET=shared_secret_from_website_og_worker
1515
# Optional. Fine-grained PAT, public repos read-only. Required for code search and raises GitHub raw rate limits.
1616
# GITHUB_TOKEN=github_pat_...
17+
# Optional. Enables the in-site assistant chat. Get a key at https://openrouter.ai/keys.
18+
# OPENROUTER_API_KEY=sk-or-v1-...
19+
# Optional. Override the default model.
20+
# AI_MODEL=google/gemini-3.1-flash-lite
21+
# ASSISTANT_ENABLED=true
22+
# Required when assistant is enabled. Long random string for daily fingerprint salting.
23+
# ASSISTANT_FP_SECRET=change-me
24+
# Dev only. Allows local reset endpoint to clear assistant limits.
25+
# ASSISTANT_RESET_TOKEN=change-me
26+
# Dev only. Shorten burst window for curl testing.
27+
# RATE_LIMIT_WINDOW_MS=60000
28+
# Optional. PostHog survey id used for thumbs up/down feedback on assistant responses.
29+
# ASSISTANT_FEEDBACK_SURVEY_ID=019e081e-2c3b-0000-04c3-564ad5dff4ed
30+
# Optional. Upstash Redis for cross-instance assistant daily rate limits. Falls back to per-process memory if unset.
31+
# UPSTASH_REDIS_REST_URL=https://your-db.upstash.io
32+
# UPSTASH_REDIS_REST_TOKEN=your_upstash_token

‎CONTEXT.md‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Directus Docs — Domain Language
2+
3+
Shared vocabulary for this codebase. Seeded with rate-limiting terms; extend as other areas are sharpened.
4+
5+
## Language
6+
7+
### Rate limiting
8+
9+
**Burst limit**:
10+
A short-window cap (seconds) on requests per identity, the cheap first gate against rapid-fire abuse.
11+
_Avoid_: throttle, flood limit
12+
13+
**Daily limit**:
14+
A per-UTC-day cap with multiple keyed dimensions (exact IP, IP prefix, fingerprint, IP+fingerprint combo) and degraded modes. The deep backstop; distinct from a burst limit.
15+
_Avoid_: quota, cap
16+
17+
**Policy**:
18+
What a caller declares to a limiter: `max`, `windowSeconds`, and `onStoreError`. The caller picks a policy; it never reimplements the algorithm.
19+
_Avoid_: config, rule, options
20+
21+
**Store**:
22+
The counting backend behind a limiter — `incr(key, ttlSeconds) -> count`. `MemoryStore` for local dev, `UpstashStore` for production (atomic, holds across serverless instances).
23+
_Avoid_: backend, cache, KV (KV is one specific store)
24+
25+
**Verdict**:
26+
The result of a limit check: `{ ok, retryAfter? }`. What the caller acts on.
27+
_Avoid_: result, decision, response
28+
29+
**Fail closed / fail open**:
30+
When the store errors, a `deny` policy fails closed (refuse the request); an `allow` policy fails open (let it through). Expensive/abusable endpoints deny; cheap public reads allow.
31+
32+
### Assistant
33+
34+
**Conversation history**:
35+
The deep, framework-agnostic owner of persisted conversations (`useAssistantHistory`): localStorage sync, sorting, title derivation, compaction, and the `activeId`. Enforces the invariant that `activeId` points to an existing conversation — mutate it only through `setActive`/`clearActive`/`startNew`/`remove`, never by assigning the returned ref.
36+
_Avoid_: conversation store, chat state
37+
38+
**Message transition**:
39+
A change to the chat's message list — reset, open a saved conversation, delete the active one, or the easter egg. All transitions go through the single `setMessages(next)` writer in `useAssistant`, so the persistence watcher reasons about one mutation idiom. `useAssistant` is the Vue-reactive surface; it is not a separate session module.
40+
_Avoid_: chat singleton, assistant session, set chat messages
41+
42+
## Relationships
43+
44+
- A caller passes a **policy** to a limiter and acts on the returned **verdict**.
45+
- A limiter counts against a **store**; the store knows nothing about limits.
46+
- **Burst limit** and **daily limit** are separate limiters. Burst is the cheap first gate; daily is the deep backstop.
47+
48+
## Example dialogue
49+
50+
> **Dev:** "If Upstash is down, does the burst limit block the assistant?"
51+
> **Maintainer:** "Yes — the assistant burst **policy** is `onStoreError: 'deny'`, so it fails closed. The docs API **policy** is `'allow'`, so a store blip never takes down public docs search."
52+
53+
## Flagged ambiguities
54+
55+
- "rate limit" was used for both the burst and daily limiters. Resolved: they are distinct concepts — **burst limit** (short window, single key, process- or store-counted) vs **daily limit** (per-day, multi-key, degraded modes). Only the burst-style limiters share the `checkRateLimit` core.

‎CONTRIBUTING.md‎

Lines changed: 228 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,228 @@
1+
# Contributing to Directus Docs
2+
3+
This guide covers local development, docs authoring, search indexing, deployment, and the optional AI Assistant.
4+
5+
## Running the Docs
6+
7+
### Requirements
8+
9+
- Node.js 22.18 or later
10+
- pnpm
11+
12+
### Install Dependencies
13+
14+
```bash
15+
pnpm install
16+
```
17+
18+
### Setup Environment
19+
20+
Copy the example environment file:
21+
22+
```bash
23+
cp .env.example .env
24+
```
25+
26+
Update `.env` with the service keys you need for the feature you are working on. Most docs-only changes do not require every optional secret.
27+
28+
### Run Development Server
29+
30+
Start the development server on `http://localhost:3000`:
31+
32+
```bash
33+
pnpm dev
34+
```
35+
36+
### Building Locally
37+
38+
```bash
39+
pnpm build
40+
```
41+
42+
## Authoring Content
43+
44+
Pages live as Markdown files under `content/`. Frontmatter fields are validated by the schema in `content.config.ts`.
45+
46+
### Framework Guides
47+
48+
Framework guides live under `content/frameworks/<framework>/`. The numeric prefix on filenames (`01.`, `02.`, ...) controls sidebar sort order only. It has no semantic meaning; renumber freely.
49+
50+
The `section` frontmatter field controls grouping on the `/frameworks/<framework>` hub page:
51+
52+
- `section: start-here` appears in the "Start Here" block at the top.
53+
- `section: guides` or unset appears in the "Guides" block below.
54+
55+
Minimal frontmatter for a new framework guide:
56+
57+
```yaml
58+
---
59+
title: Fetch Data from Directus with Foo
60+
description: Learn how to integrate Directus in your Foo app.
61+
section: start-here
62+
technologies:
63+
- foo
64+
navigation:
65+
title: Data Fetching
66+
---
67+
```
68+
69+
## Repository Tooling
70+
71+
The repository includes scripts that keep docs routes stable when files move and that index the docs into Typesense.
72+
73+
```bash
74+
pnpm stable-ids:ensure # Add missing stableId frontmatter
75+
pnpm stable-ids:check # Validate stableId frontmatter
76+
pnpm redirects:sync # Update redirects.json for moved pages
77+
pnpm redirects:check # Check redirect coverage without writing files
78+
pnpm index:docs # Build the search index in Typesense
79+
pnpm typesense:cleanup-preview # Delete stale Typesense preview indexes
80+
pnpm typecheck:scripts # Type check repository scripts
81+
```
82+
83+
Stable IDs give each public docs page a permanent identity. Nuxt Content derives its unique page IDs from file paths, so moving a page changes its built-in ID. Redirect sync compares the current branch to `origin/main`, so moved pages keep their old URLs working.
84+
85+
CI runs `pnpm stable-ids:check` and `pnpm redirects:check` for docs changes.
86+
87+
- New docs page: run `pnpm stable-ids:ensure`, then commit the new `stableId`.
88+
- Moved docs page: keep the existing `stableId`, run `pnpm redirects:sync`, then commit `redirects.json`.
89+
- Deleted, split, or merged docs page: run `pnpm redirects:sync`, review `.docs/redirect-decisions-needed.md`, choose target redirects, then re-run `pnpm redirects:check`.
90+
- Before opening a PR: run `pnpm stable-ids:check` and `pnpm redirects:check`.
91+
92+
Redirect scripts compare against `origin/main` by default. To check a release branch or another target, fetch it first, then pass `--base` directly to the script:
93+
94+
```bash
95+
git fetch origin release/v13
96+
node scripts/redirects-sync.ts --base origin/release/v13 --no-write --fail-on-unresolved
97+
node scripts/redirects-sync.ts --base origin/release/v13 --write-deterministic --fail-on-unresolved
98+
```
99+
100+
## AI Assistant
101+
102+
The in-site AI Assistant is optional. It is disabled unless `OPENROUTER_API_KEY` is present and `ASSISTANT_ENABLED` is not set to `false`.
103+
104+
### Required for Vercel
105+
106+
```bash
107+
OPENROUTER_API_KEY=sk-or-v1-...
108+
ASSISTANT_FP_SECRET=long-random-string
109+
```
110+
111+
`ASSISTANT_FP_SECRET` salts daily fingerprint identifiers. Use a long random value and keep it secret.
112+
113+
### Recommended for Vercel
114+
115+
Use one supported Redis env pair for cross-instance burst and daily limits:
116+
117+
```bash
118+
UPSTASH_REDIS_REST_URL=https://your-db.upstash.io
119+
UPSTASH_REDIS_REST_TOKEN=your_upstash_token
120+
```
121+
122+
or:
123+
124+
```bash
125+
KV_REST_API_URL=https://your-db.upstash.io
126+
KV_REST_API_TOKEN=your_upstash_token
127+
```
128+
129+
Without Redis env vars, limits fall back to per-process memory. That is fine for local development, but not enough for production or preview deployments.
130+
131+
### Optional
132+
133+
```bash
134+
AI_MODEL=google/gemini-3.1-flash-lite
135+
ASSISTANT_ENABLED=true
136+
ASSISTANT_FEEDBACK_SURVEY_ID=019e081e-2c3b-0000-04c3-564ad5dff4ed
137+
GITHUB_TOKEN=github_pat_...
138+
POSTHOG_AI_HOST=https://us.i.posthog.com
139+
```
140+
141+
- `ASSISTANT_ENABLED=false` is the kill switch. Set it before build/deploy to hide the assistant and skip the server route; the server also rejects requests when the flag is false.
142+
- `GITHUB_TOKEN` is required for the assistant's source-code search tool and raises GitHub raw-file rate limits.
143+
- `ASSISTANT_FEEDBACK_SURVEY_ID` enables thumbs up/down feedback on assistant responses.
144+
- `POSTHOG_AI_HOST` only needs to be set when AI telemetry should use a different PostHog host than `POSTHOG_API_HOST`.
145+
146+
### Local Testing
147+
148+
Useful assistant tests:
149+
150+
```bash
151+
pnpm exec vitest run modules/assistant/index.test.ts modules/assistant/runtime/composables/useAssistant.test.ts modules/assistant/runtime/server/utils/admit.test.ts modules/assistant/runtime/server/utils/abuse-gate.test.ts modules/assistant/runtime/server/utils/bind-tools.test.ts modules/assistant/runtime/server/utils/rate-limit.test.ts modules/assistant/runtime/server/utils/request-context.test.ts server/utils/rate-limit.test.ts server/utils/docs-api-limit.test.ts
152+
```
153+
154+
Dev-only helpers:
155+
156+
```bash
157+
ASSISTANT_RESET_TOKEN=change-me
158+
RATE_LIMIT_WINDOW_MS=60000
159+
POSTHOG_AI_DEBUG=true
160+
POSTHOG_AI_SELF_TEST=true
161+
```
162+
163+
The reset/status endpoints are only registered in dev and only respond from a local development context.
164+
165+
## Search
166+
167+
Search is powered by [Typesense](https://typesense.org). The browser palette (`UCommandPalette`-based) lives at `app/components/DocsSearchPalette.vue` and queries Typesense directly via `app/services/typesenseService.ts`. The official `typesense` npm client is used by the indexer only.
168+
169+
### Indexing
170+
171+
The indexer at `scripts/index-docs.ts` walks `/content`, chunks each Markdown page, attaches synonyms, and pushes everything to Typesense. OpenAPI indexing is deferred to a later branch. Run it locally with:
172+
173+
```bash
174+
pnpm index:docs
175+
```
176+
177+
CI runs the same command on every push to `main` (production index) and on every PR commit (per-branch preview index). See `.github/workflows/search-index.yml`.
178+
179+
### Collection Naming
180+
181+
Indexes use a blue/green slot pattern with a stable alias:
182+
183+
- `main` -> alias `directus-docs`, slots `directus-docs-a` / `directus-docs-b`
184+
- Branch `bry/foo` -> alias `directus-docs-preview-bry-foo`, slots `...-a` / `...-b`
185+
- Local branch runs use the same branch-derived alias as CI
186+
187+
Each indexer run writes to whichever slot the alias is not currently pointing at, swaps the alias, then deletes the previous slot.
188+
189+
For one-off writes, override the index target with `TYPESENSE_INDEX_TARGET=...`.
190+
191+
The browser reads from `TYPESENSE_COLLECTION` when set. Otherwise it derives the same branch alias as the indexer. The app reads the alias, never the `-a` / `-b` slot name.
192+
193+
### Preview Cleanup
194+
195+
PR preview indexes are deleted when same-repo PRs close. The cleanup job deletes the branch alias and both fixed slots:
196+
197+
```bash
198+
pnpm typesense:cleanup-preview --branch bry/foo
199+
```
200+
201+
For one-time cleanup of accumulated preview indexes, run a dry run first:
202+
203+
```bash
204+
pnpm typesense:cleanup-preview --stale --dry-run
205+
pnpm typesense:cleanup-preview --stale
206+
```
207+
208+
Stale cleanup keeps preview aliases for currently open PR branches and deletes the rest. It requires `TYPESENSE_URL`, `TYPESENSE_PRIVATE_API_KEY`, and authenticated `gh`.
209+
210+
### Ranking
211+
212+
Section boosts and personalization live in `buildPersonalizedSortBy` in `app/composables/useDocsSearch.ts`. The same `sectionPriority` array drives both the Typesense `_eval` boost order and the chip-bar render order in the palette.
213+
214+
### Synonyms
215+
216+
Search synonyms live in `server/data/synonyms.ts` and are pushed to Typesense on every indexer run. Two formats: `multiway` (equivalent terms) and `oneway` (directional shorthand -> canonical, e.g. `db -> database`). Header comment in the file explains both.
217+
218+
### Search-Friendly Content
219+
220+
Write H2s and first paragraphs so they work as standalone search results.
221+
222+
## Deploying the Docs
223+
224+
The documentation automatically deploys to Vercel when changes are merged into the main branch.
225+
226+
1. Open a pull request.
227+
2. Review the deploy preview.
228+
3. Once the PR is approved and merged to `main`, Vercel builds and deploys the updated documentation.

0 commit comments

Comments
 (0)