Adds support for simple global/general model configuration #3197

RobotSail · 2025-02-27T15:51:16Z

Checklist:

Commit Message Formatting: Commit titles and messages follow guidelines in the
conventional commits.
Changelog updated with breaking and/or notable changes for the next minor release.
Documentation has been updated, if necessary.
Unit tests have been added, if necessary.
Functional tests have been added, if necessary.
E2E Workflow tests have been added, if necessary.

cdoern

few preliminary comments

cdoern · 2025-02-27T15:55:58Z

src/instructlab/cli/data/generate.py

    "-dt", "--detached", is_flag=True, help="Run ilab data generate in the background"
 )
+@click.option(
+    "--student-model-id",


so with the new:

cls=clickext.ConfigOption, config_class="command_name", config_sections="sub_class_in_other_command_class", )

structure, you can point to defaults from elsewhere in the cfg. I'd set the default for all of these to the model_path from the train cfg for example

yeah i think thats a good idea
after speaking with @RobotSail we decided to keep the first PR fairly barebones so that this logic at least gets in
I have plans to work on a follow up PR where among other things, all the model references in the config are updated to be ids instead of paths and then it would make sense to wire the click defaults up the way you suggested

cdoern · 2025-02-27T16:06:57Z

src/instructlab/model/chat.py

    logs_dir,
    vi_mode,
    visible_overflow,
+    models: List[cfg._model_config],


hmmmm, so it would be preferable to keep config class types out of the model pkg since its a "cli" only related structure.

but if its not possible that's fine. Could we resolve the model before getting down into this file and just pass it in?

@cdoern This is just so we can do type-checking. How can we structure it so we can reuse the types later throughout the library?

we could move things around so that we have a dedicated place for datatypes that can be used everywhere so it doesnt feel like a front end construct is "bleeding into" the backend in a follow up PR maybe
would that be better @cdoern?

reidliu41 · 2025-03-06T09:01:08Z

src/instructlab/configuration.py

+    models: List[_model_config] = Field(
+        default_factory=lambda: [],
+        description="User-defined custom set of models. This allows people to use their own models for the InstructLab model customization process.",
+    )


sorry, I not sure about the purpose and may not understand correctly, based on the code, user defined models, and then need to specify the model id in the models?
why not just specify the model id?

@reidliu41 Not sure if I follow your question

@reidliu41 it would look something like

chat: ... generate: ... models: - id: 'ibm/granite-7b-lab' path: <local/hf path> ctx_size: 4096 - id: ...

and then the long term goal would be to have model references in other sections (like in chat.model or generate.model) start pointing to these model ids instead of having the path directly so we can centralize all the model info in one place for a given model

Signed-off-by: Jaideep Rao <[email protected]>

RobotSail

@jaideepr97 This looks good! I just have 2 very short comments, but otherwise it should be good to go.

src/instructlab/cli/model/evaluate.py

src/instructlab/configuration.py

Signed-off-by: Oleg <[email protected]>

cdoern

comments on defaults specifically, looks good so far

cdoern · 2025-03-13T17:21:09Z

src/instructlab/cli/data/generate.py

+@click.option(
+    "--student-model-id",
+    # help="ID of the custom model you want to use as the student model.",
+    default=None,


do we want this to default to the model in the train part of the cfg or are we getting rid of those defaults?

We want to leave this blank for now.

cdoern · 2025-03-13T17:22:09Z

src/instructlab/cli/data/generate.py

+    show_default=True,
+)
+@click.option(
+    "--teacher-model-id",


same here for default

@cdoern We should leave it blank for now and figure out the best way for this later.

cdoern · 2025-03-13T17:24:38Z

src/instructlab/cli/data/generate.py

-    system_prompt = get_sysprompt(student_model_arch)
+    system_prompt, legacy_pretraining_format = None, None
+    if not student_model_id:
+        student_model_path = pathlib.Path(ctx.obj.config.train.model_path)


yeah this can be the default in the flag using config_class

Can we please avoid this for now? I would like to leave it as something where it's merely a lookup for models in the config, rather than something with a known default that will be referenced in the existing system. This way we can keep the logic simple and change it later as needed.

not quite - this default is a path, not a model id yet
the way im imagining it, if we dont have to care about breaking changes as such we can rename all the model_path fields to either model_id or just model and have them reference model ids instead of paths. Until we do that, setting defaults in the click flags wont achieve much

cdoern · 2025-03-13T17:25:06Z

src/instructlab/cli/model/evaluate.py

    required=True,  # default from config
 )
+@click.option(
+    "--base-model-id",


default from cfg?

Let's leave it blank to keep logic simple

cdoern · 2025-03-13T17:25:14Z

src/instructlab/cli/model/evaluate.py

    type=click.STRING,
 )
+@click.option(
+    "--judge-model-id",


We should be leaving it blank to keep it simple

cdoern · 2025-03-13T17:27:05Z

src/instructlab/configuration.py

+        system_prompt (str | None): The initial message used to prompt the conversation with this model.
+    """
+
+    id: str = Field(


default_factory for these?

and default for id is missing even if None

This will be a user-specified field for now, we shouldn't have a default here.

src/instructlab/configuration.py

Signed-off-by: Jaideep Rao <[email protected]>

mergify bot added the ci-failure PR has at least one CI failure label Feb 27, 2025

cdoern reviewed Feb 27, 2025

View reviewed changes

booxter self-requested a review February 27, 2025 21:53

ktam3 assigned jaideepr97 Mar 4, 2025

reidliu41 reviewed Mar 6, 2025

View reviewed changes

courtneypacheco mentioned this pull request Mar 6, 2025

add sys prompt to config #2271

Closed

6 tasks

jaideepr97 force-pushed the os/third-party-models branch from 67d4fb2 to 22af5e7 Compare March 11, 2025 20:50

RobotSail added 2 commits March 11, 2025 16:52

wip: add 3rd party model support-s

cc6e556

Signed-off-by: Jaideep Rao <[email protected]>

wip: add model selection by id in more places

f7726ab

Signed-off-by: Jaideep Rao <[email protected]>

jaideepr97 force-pushed the os/third-party-models branch from 22af5e7 to b3146a9 Compare March 12, 2025 14:25

mergify bot added ci-failure PR has at least one CI failure and removed ci-failure PR has at least one CI failure labels Mar 12, 2025

jaideepr97 force-pushed the os/third-party-models branch from b3146a9 to 61478cc Compare March 12, 2025 17:26

mergify bot added testing Relates to testing ci-failure PR has at least one CI failure and removed ci-failure PR has at least one CI failure labels Mar 12, 2025

jaideepr97 force-pushed the os/third-party-models branch from 61478cc to a06b32f Compare March 12, 2025 18:25

mergify bot added ci-failure PR has at least one CI failure and removed ci-failure PR has at least one CI failure labels Mar 12, 2025

fix linting, unit tests, docs build

48dc384

Signed-off-by: Jaideep Rao <[email protected]>

jaideepr97 force-pushed the os/third-party-models branch from a06b32f to 48dc384 Compare March 13, 2025 13:26

mergify bot added documentation Improvements or additions to documentation and removed ci-failure PR has at least one CI failure labels Mar 13, 2025

RobotSail commented Mar 13, 2025

View reviewed changes

src/instructlab/cli/model/evaluate.py Outdated Show resolved Hide resolved

src/instructlab/configuration.py Outdated Show resolved Hide resolved

RobotSail marked this pull request as ready for review March 13, 2025 15:37

Update src/instructlab/configuration.py

aef0f63

Signed-off-by: Oleg <[email protected]>

jaideepr97 approved these changes Mar 13, 2025

View reviewed changes

mergify bot added the one-approval PR has one approval from a maintainer label Mar 13, 2025

cdoern reviewed Mar 13, 2025

View reviewed changes

cdoern approved these changes Mar 13, 2025

View reviewed changes

mergify bot removed the one-approval PR has one approval from a maintainer label Mar 13, 2025

fix judge model output dir

88bd0d0

Signed-off-by: Jaideep Rao <[email protected]>

jaideepr97 force-pushed the os/third-party-models branch from 003898e to 88bd0d0 Compare March 13, 2025 20:27

mergify bot added ci-failure PR has at least one CI failure and removed ci-failure PR has at least one CI failure labels Mar 13, 2025

jaideepr97 merged commit 040572b into main Mar 14, 2025
29 checks passed

Adds support for simple global/general model configuration #3197

Adds support for simple global/general model configuration #3197

Uh oh!

Conversation

RobotSail commented Feb 27, 2025

Uh oh!

cdoern left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jaideepr97 Mar 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RobotSail left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

cdoern left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

jaideepr97 Mar 13, 2025 •

edited

Loading