feat(search): on-device Ask AI grounded on docs search#3284
Open
bartlomieju wants to merge 1 commit into
Open
Conversation
Add an 'Ask AI' action to the search results that synthesizes an answer from the current site-wide search hits using the browser's built-in, on-device model (Chrome Prompt API) when available. It is grounded only on the retrieved docs, lists its sources, and is entirely client-side and free (no server endpoint or API key). When the on-device model is unavailable it falls back to a Claude hand-off with the question and the most relevant pages prefilled. Everything is feature-detected and wrapped in try/catch so existing search is never affected.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A first cut at the highest-leverage idea from the docs review: an in-docs
"Ask AI" that's grounded on our own content, free, and on-brand with the
existing AI-native affordances.
How it works. The search box already does site-wide retrieval via Orama,
so Ask AI reuses those hits as context rather than adding a second
retrieval path. When you run a search, an "Ask AI about " button
appears above the results. Clicking it builds a compact context from the
top hits and answers using the browser's built-in, on-device model (the
Chrome Prompt API) when available. The answer is grounded only on the
retrieved pages and lists them as sources. It is entirely client-side, so
there is no server endpoint, no API key, and no per-query cost.
When the on-device model isn't available (most browsers today), it falls
back to a one-click Claude hand-off with the question and the most
relevant doc links prefilled, so the button is always useful. Everything
is feature-detected and wrapped in try/catch, and the answer panel is
purely additive, so existing search behaviour is unchanged for everyone.
Caveats. The Chrome Prompt API is experimental and gated, so the
on-device path reaches a small set of users today; the rest get the
hand-off. I could not runtime-test the on-device model in this
environment, so the Prompt API call wants a real-browser check before
merge. The on-device context window is small, so retrieved context is
deliberately capped (top 5 hits, truncated). Future work: response
streaming, and widening retrieval beyond the current 8 search hits.