This is in context of #2021 but I'm not sure at this point we shouldn't just list() when passing any data coming from click that has to be a list.

This works but ultimately, this is just how click works. We should be aware of this and deal accordingly (either accept tuples everywhere, or cast before passing.)

This PR is still a blocker for 0.18.0, since even with the fix provided in instructlab/training#162, we still get a warning from Pydantic about passing a tuple to a field that expects a list[str]. For this reason, I've added the blocker-proposed tag as this does not directly impede functionality, but it removes one more thing that may confuse users.

@RobotSail do you have a warning message from pydantic to look at?

@booxter

/opt/app-root/lib64/python3.11/site-packages/pydantic/main.py:364: UserWarning: Pydantic serializer warnings:
  Expected `list[str]` but got `tuple` - serialized value may not be as expected
  return self.__pydantic_serializer__.to_python(

This looks like it comes from pydantic-core Rust implementation:

    pub fn on_fallback_ser<S: serde::ser::Serializer>(
        &self,
        field_type: &str,
        value: &Bound<'_, PyAny>,
        extra: &Extra,
    ) -> Result<(), S::Error> {
        // special case for None as it's very common e.g. as a default value
        if value.is_none() {
            Ok(())
        } else if extra.check.enabled() {
            // note: I think this should never actually happen since we use `to_python(..., mode='json')` during
            // JSON serialisation to "try" union branches, but it's here for completeness/correctness
            // in particular, in future we could allow errors instead of warnings on fallback
            Err(S::Error::custom(UNEXPECTED_TYPE_SER_MARKER))
        } else {
            self.fallback_warning(field_type, value);
            Ok(())
        }
    }

    fn fallback_warning(&self, field_type: &str, value: &Bound<'_, PyAny>) {
        if self.mode != WarningsMode::None {
            let type_name = value
                .get_type()
                .qualname()
                .unwrap_or_else(|_| PyString::new_bound(value.py(), "<unknown python object>"));

            let input_str = safe_repr(value);
            let mut value_str = String::with_capacity(100);
            value_str.push_str("with value `");
            crate::errors::write_truncated_to_50_bytes(&mut value_str, input_str.to_cow())
                .expect("Writing to a `String` failed");
            value_str.push('`');

            self.add_warning(format!(
                "Expected `{field_type}` but got `{type_name}` {value_str} - serialized value may not be as expected"
            ));
        }
    }

There's apparently a flag to disable these warnings (though I don't know if it's available or whether it's ok to suppress these messages), but clearly the library prefers that list fields receives lists, even if it will grudgingly accept tuples and other iterables. (I think pydantic should have explained it better in their docs. There, it just says that either of the types is fine, with no caveats.)

We can land this for now, but I think other options that are "lists" that come from click options will also be tuples, and should also be converted to lists (though it assumes we use list types for all models - which may not be the case: using tuple types is as legit - in this case, warnings would be raised if we cast to list!)

I think post-0.18 we should revisit this issue more broadly for all options and all libraries we have. @RobotSail I suggest a tracking issue is reported to follow up later.

@RobotSail any idea why CI was cancelled here?

@booxter It seems like one of the tests in the functional tests is breaking CI - not sure why it was cancelled.

@Mergifyio rebase

rebase

❌ Unable to rebase: user ktam3 is unknown.

@Mergifyio rebase

rebase

✅ Branch has been successfully rebased

This pull request has been automatically marked as stale because it has not had activity within 90 days. It will be automatically closed if no further activity occurs within 30 days.

not stale

This pull request has been automatically marked as stale because it has not had activity within 60 days. It will be automatically closed if no further activity occurs within 30 days.