Allow use embeddings with local models #6501
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
Caution
Changes requested ❌
Reviewed everything up to 054faef in 3 minutes and 27 seconds. Click for details.
- Reviewed
307lines of code in9files - Skipped
0files when reviewing. - Skipped posting
10draft comments. View those below. - Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. web-app/src/services/providers/tauri.ts:29
- Draft comment:
When mapping over builtInModels (lines 29–31), the code uses the variable 'models' (initialized as provider.models) inside the map callback. Since 'models' is reassigned during the map, this may cause unexpected behavior. Consider storing the original provider.models in a separate variable (e.g. originalModels) for lookup. - Reason this comment was not posted:
Comment was not on a location in the diff, so it can't be submitted as a review comment.
2. web-app/src/services/providers/tauri.ts:64
- Draft comment:
The logic to derive the base_url by removing '/chat/completions' (line 65) assumes that this exact substring exists. For robustness, consider using URL parsing or a more flexible pattern to handle variations. - Reason this comment was not posted:
Comment was not on a location in the diff, so it can't be submitted as a review comment.
3. web-app/src/services/providers/tauri.ts:87
- Draft comment:
The try/catch blocks for checking tool and embeddings support (lines 85–103) are good, but consider logging more structured information or using a dedicated logger for consistency with the rest of the app. - Reason this comment was not posted:
Comment was not on a location in the diff, so it can't be submitted as a review comment.
4. web-app/src/services/providers/tauri.ts:151
- Draft comment:
In fetchModelsFromProvider, headers are conditionally added based on base_url. Ensure that using a hard-coded origin 'tauri://localhost' is safe and works in all intended environments. - Reason this comment was not posted:
Comment was not on a location in the diff, so it can't be submitted as a review comment.
5. web-app/src/services/providers/tauri.ts:172
- Draft comment:
Error handling for the models fetch (lines 173–191) is thorough. The use of structured error messages based on status codes is a good practice – just ensure that all expected status scenarios are covered. - Reason this comment was not posted:
Comment was not on a location in the diff, so it can't be submitted as a review comment.
6. web-app/src/services/providers/tauri.ts:250
- Draft comment:
In updateSettings (lines 250–264), the mapping from provider settings to engine settings is performed. Confirm that the conversion between 'controller_props' and 'controllerProps' remains consistent across implementations. - Reason this comment was not posted:
Comment was not on a location in the diff, so it can't be submitted as a review comment.
7. web-app/src/services/providers/tauri.ts:245
- Draft comment:
Consider defining a shared helper for building error messages and headers to reduce duplication between fetchModelsFromProvider and other provider methods. - Reason this comment was not posted:
Comment was not on a location in the diff, so it can't be submitted as a review comment.
8. core/src/browser/extensions/engines/AIEngine.ts:287
- Draft comment:
Typographical note: The comment currently says "Check if a embedding is supported by the model". Consider replacing "a embedding" with "an embedding" (or possibly rephrase to "Check if embeddings are supported by the model") for grammatical consistency. - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% While the grammar correction is technically accurate, it's a very minor issue that doesn't affect code functionality or clarity. The meaning is perfectly clear either way. The rules specifically state not to make purely informative comments or comments that are obvious/unimportant. Grammar nitpicks fall into this category. The comment is factually correct and improves documentation quality. Poor grammar in documentation could be seen as unprofessional. While correct, this type of minor grammatical fix doesn't warrant a PR comment. It's too trivial and could be fixed in a future documentation pass if needed. Delete this comment as it violates the rule about not making purely informative comments or comments about unimportant issues.
9. extensions-web/src/jan-provider-web/provider.ts:315
- Draft comment:
Typographical error: The comment "Jan models need to check from wed for embeddings support" contains "wed" which likely should be "web". - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% While the comment is technically correct about the typo, comments about typos in comments are generally not important enough to warrant a PR comment. The typo doesn't affect functionality, and fixing it would be a trivial change that doesn't improve code quality. There's actually a more serious typo in the console.log statement that would be more worth pointing out. The typo could indicate a lack of attention to detail that might manifest in more serious ways elsewhere in the code. Also, clear documentation is important for maintainability. While clear documentation is important, this particular typo is extremely minor and obvious - any reader would understand "wed" means "web" in this context. PR comments should focus on more substantive issues. Delete this comment as it points out a trivial typo in a comment that doesn't impact code functionality or significantly affect readability.
10. extensions/llamacpp-extension/src/index.ts:1994
- Draft comment:
Typographical error: The JSDoc comment reads "Check if a embeddings is supported by the model". Consider revising it to "Check if an embedding is supported by the model" or "Check if embeddings are supported by the model". - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 20% vs. threshold = 50% While the comment correctly identifies a grammatical error, comments about documentation typos are generally not critical enough to warrant a PR comment. The meaning is still clear despite the minor grammatical error. The comment doesn't point out any functional issues or suggest important code improvements. The comment is technically correct - there is a grammatical error. And clear documentation is important for code maintainability. While good documentation is important, minor grammatical fixes like this are too trivial for PR comments and can be fixed during regular code maintenance. The current wording, while imperfect, doesn't impede understanding. Delete the comment. While it points out a real grammatical error, such minor documentation fixes are too trivial for PR comments.
Workflow ID: wflow_44sjInCOu5SDSrKI
You can customize by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.
|
|
||
| async isEmbeddingsSupported(modelId: string): Promise<boolean> { | ||
| // Jan models need to check from wed for embeddings support | ||
| console.log(`Checking embeddings support for Jan model ${modelId}: тще supported`); |
There was a problem hiding this comment.
Typographical error: The log message "тще supported" appears to contain incorrect characters. Did you mean "not supported" or similar?
Suggested change
| console.log(`Checking embeddings support for Jan model ${modelId}: тще supported`); | |
| console.log(`Checking embeddings support for Jan model ${modelId}: not supported`); |
|
@urmauur @louis-menlo could you check this PR? If you agree to mrge it in general I will add to get correct pool_mode from model and pass to llama.cpp. Or we can remove --pooling parameter. In this case llama-server should get correct parameter from model. |
054faef to
6162676
Compare
6162676 to
1aec6d5
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Describe Your Changes
Users can import embeddings models but these moders doesn't work as embedded models.
Also a lot of models support /embeddings and /chat at the same time but can't deteted as embeddings models. Allow users to emable embedding feture manually.
Self Checklist