FYI a decision for the project is to NOT bump minimal versions unless the older version won't work at all. That we continue bumping it, ignoring the decision, seems wrong to me.

@booxter , I'm not sure I understand. Are there users of SDG 0.7.0 that would like to stay on that version while also consuming the latest version of instructlab?

I'd like to move this forward because I expect SDG v0.7.3 to come out soon (with backported instructlab/sdg#574), and we would bump instructlab's main requirements.txt again for that.

I'd like to make sure that if users upgrade their InstructLab version, they also get the newer instructlab-sdg version. And I want to make sure all new users installing InstructLab get the fixes from our latest 0.7.x release. Bumping the minimum version of instructlab-sdg in the requirements.txt would be the right way to accomplish this I believe?

I'm referring to instructlab/dev-docs@c4378ce

Specifically, to the following wording: "Specifying the minimum value for the range (foo>=x.y) allows us to declare that we need features that only show up in or after that version of the dependency, which means we won't get bugs from users trying to use instructlab with an old dependency that has an incompatible API or is completely lacking a feature we need." and "If the new dependency requires incompatible changes in our code, then it becomes the new minimum version for our requirements range."

My interpretation of this wording is that we should not bump minimal version whenever we like, but only when we have to. I know this policy was not enforced so far, but this is what our downstream colleagues asked us to do, to allow for some freedom for packagers to pick a particular version off the allowed range.

Are there users of SDG 0.7.0 that would like to stay on that version while also consuming the latest version of instructlab?

I don't know if there are such users, but our policy assumes they are allowed to exist.

By the rules written, it seems like this case could easily fit into the bucket of "or is completely lacking a feature we need." In this case the feature being a usable docling version that we want to ensure all users get upgraded to.

I think the intention of the wording is that we don't want to force users to upgrade when it's not necessary. And it's trying to balance that with we don't want users hitting known bugs/incompatibilities. In the end that comes down to a judgment call of how likely are users to hit the issue vs how painful is the forced upgrade. AFAIK, most of the pain from an upgrade would come from something else being broken with the new changes as the upgrade itself isn't difficult. In this case since it's only the fixes for issues since 0.7.0 included, the risk seems very low and so I don't see any reason not to err on the side of getting the fixes out to a larger audience.

In this case the feature being a usable docling version that we want to ensure all users get upgraded to.

Here's the diff between the two versions: instructlab/sdg@v0.7.0...v0.7.2 Which docling version change do you refer to? (The only change to requirements.txt there is to exclude tesserocr for darwin platform, which AFAIU was done for CI sake only.)

The other changes included seem like straightforward bug fixes that while are useful for those interested in SDG, may not necessarily justify the upgrade of the package for all users (some may not even care about SDG for their use; or not hitting the issue).

In general, the practice of version pinning and minimal version bumps in this project, as practiced, is not packager-friendly, and the general recommendation in Python packaging world - and from our own downstream packagers - is to not bump dependencies without a clear necessity, because Python environments are shared across all other Python packages (in contrast to e.g. JavaScript).

Granted, the instructlab libraries have special status here and are probably less problematic since they are not generally used by other apps except instructlab CLI (at least at this moment); and probably the same packagers will package both CLI and libraries. If that's the argument, it's fine and I'd advise to update the policy with this decision.

FYI I'm not blocking it and you are free to merge it as-is. I only remind that we have a policy and it has an intent to not disrupt packagers' life.

Which docling version change do you refer to?

Apologies. I mixed up the change targeted for 0.7.3. So the reward statement is different than what I stated, but the risk side of the equation still seems very low to ensure users get the targeted fixes.

I agree with your concerns in the general case and that the instructlab libraries don't currently fall into any of the pitfalls. +1 to clarifying further in our documentation.

I expect SDG v0.7.3 to come out soon (with backported instructlab/sdg#574)

To close the loop here: SDG developers revisited this decision today, and we probably won't cherry-pick that to release-v0.7 for now.

We reversed that earlier SDG decision, and instructlab/sdg#574 is now tagged in SDG v0.7.3.

I'm not going to bump instructlab's main requirements.txt again in order to save some work.