Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Environment configuration #895

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Jun 26, 2025
Merged

Environment configuration #895

merged 17 commits into from
Jun 26, 2025

Conversation

THardy98
Copy link
Contributor

What was changed

Environment configuration for Python SDK, using core base envconfig module

  1. Closes [Feature Request] Environment configuration #835

  2. How was this tested:
    test_envconfig.py

  3. Any docs updates needed?
    probably

@THardy98 THardy98 requested a review from a team as a code owner June 10, 2025 16:37
@THardy98 THardy98 marked this pull request as draft June 10, 2025 16:37
@THardy98 THardy98 changed the title init impl Environment configuration Jun 11, 2025
@THardy98 THardy98 marked this pull request as ready for review June 11, 2025 22:19
Copy link
Member

@Sushisource Sushisource left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure you get rid of those .DS_Store macOS file nonsense. They should be globally excluded by gitignore.

I dunno if we need to do this in this PR, we can wait and see what @cretz and @dandavison opinions are too, but IMO it would make good sense to add a Client.connect() overload that takes a ClientConfigProfile here for easy use.

Also these guys need experimental notices on them too

@THardy98
Copy link
Contributor Author

THardy98 commented Jun 13, 2025

Make sure you get rid of those .DS_Store macOS file nonsense. They should be globally excluded by gitignore.

Whoops, didn't realize I had committed them. Removed, added to gitignore

I dunno if we need to do this in this PR, we can wait and see what @cretz and @dandavison opinions are too, but IMO it would make good sense to add a Client.connect() overload that takes a ClientConfigProfile here for easy use.

seems reasonable, will wait to chad gets back as i think he'll have more to suggestions anyway

Also these guys need experimental notices on them too

added to each class (not sure if each method is necessary?)

@THardy98 THardy98 requested a review from Sushisource June 16, 2025 15:38
Copy link
Member

@Sushisource Sushisource left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good to me, but have Dan or Chad when he's back take a look

@THardy98 THardy98 requested a review from cretz June 20, 2025 14:04
Comment on lines 18 to 20
DataSource: TypeAlias = Union[
str, bytes
] # str represents a file path, bytes represents raw data
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit confusing. It is very normal to provide PEM file contents as a string in languages. We should differentiate file path better. Maybe Union[pathlib.Path, str, bytes] and only paths are treated as such?

Copy link
Contributor Author

@THardy98 THardy98 Jun 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've changed the typing to your suggestion but I'm unsure whether or not to keep the str type.

The core TOML parser reads the cert configs into:

pub enum DataSource {
    Path(String),
    Data(Vec<u8>),
}

so lang only sees these cert configs as bytes or a path (which we now read as an actual Path type).

Users can provide file contents as string but it would only be possible programmatically, like the (added) test test_read_source_from_string_content (unless I'm misunderstanding something). I am not sure if we wanted envconfig to be used this way

Copy link
Member

@cretz cretz Jun 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The core TOML parser reads the cert configs into:

Rust has a clearer difference between paths and data. It is normal in Python to have a string of data. test_read_source_from_string_content is actually test_read_source_from_bytes_content because you have a b in front. I think it makes sense to accept string content too since PEM data is often a string, but if not, no prob, let's just make sure the exception is clear if the data type is wrong.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_read_source_from_string_content reads in string content, but asserts that it is read correctly into bytes.
So it's still from string content so to speak.

return source.encode("utf-8")
if isinstance(source, bytes):
return source
return None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not silently drop/ignore invalid data types in this case at runtime, should raise here IMO

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

raise a TypeError if invalid

from typing_extensions import TypeAlias, TypedDict

import temporalio.service
from temporalio.bridge.temporal_sdk_bridge import envconfig as _bridge_envconfig
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it may make more sense to have a temporalio/bridge/envconfig.py module that wraps all the bridge stuff. Every other use of the bridge in this repo goes through that approach. But not required.

def load_client_connect_config(
profile: str = "default",
*,
env_vars: Optional[Mapping[str, str]] = None,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we want this here, can this be moved to the bottom of the kwarg list and be called override_env_vars to be clear that it's abnormal to set?

Copy link
Contributor Author

@THardy98 THardy98 Jun 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved/renamed

ClientConfig.load_profile_from_data(toml_config)


def test_load_client_connect_config(base_config_file: Path):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you have somewhere that splats this on to .connect and makes an actual connection (say to the same dev server used in other tests or start your own if you must)? We need something to confirm type checking on the result of the load_client_connect_config call that it matches the connect arg expectations.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

return ClientConfig._from_bridge_profiles(loaded_profiles)

@staticmethod
def load_profiles_from_file(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we can combine all of these load calls into one to reduce surface area since they are not the commonly called methods. I know Go doesn't combine them because it's a bit harder for that language. I figure you have one load call that can accept config file override (can be path or data like cert), disable env, override env, etc.

As for the load_profile stuff, wouldn't that just be load on ClientConfigProfile instead of this class? Most of the same parameters should apply.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, some changes to the bridge to support this



@dataclass
class ClientConfig:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We want users to be able to convert this (and ClientConfigProfile) to/from TOML on their own if they want. However, we don't necessarily need to expose literal TOML conversion, just the ability to convert to/from the dict that represents TOML (they can use some Python TOML library if they want to do the TOML part). Can we expose to_dict and from_dict on this and the ClientConfigProfile class that represents exactly what it looks like in TOML?

Copy link
Contributor Author

@THardy98 THardy98 Jun 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added to_dict methods
the resulting dictionaries are well-typed, not sure if that was necessary or not

one caveat with this, when the given config input is in bytes
to_dict converts bytes data into strings because strings are more suitable for TOML

I wrote it in a test for this, converting from to_dict -> from_dict -> compare:

    # We expect the new profile to be the same, but with bytes-based data
    # sources converted to strings. This is because to_dict converts
    # bytes-based data to a string, suitable for TOML. So we only have
    # a string representation to work with.

Copy link
Member

@cretz cretz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! This is a good model to follow for other SDKs. Only one class name change request (and two other unimportant comments).



# We define typed dictionaries for what these configs look like as TOML.
class ClientTLSConfigDict(TypedDict, total=False):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was originally going to say you could leave these out since users using to/from dict should not really care about the types since that's low level. But I now see that we also are dogfooding the dict serialization stuff (awesome), so can leave even if just for our benefit.

)


class ClientConnectConfig(TypedDict, total=False):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a heads up here, I think we may be having another situation ("plugins") where this actually has value at the client level for others wanting to type a dict for the kwargs of connect/init. So this may move to the client module in the future.

@THardy98 THardy98 merged commit 34bb050 into main Jun 26, 2025
40 of 48 checks passed
@THardy98 THardy98 deleted the env_config branch June 26, 2025 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request] Environment configuration
3 participants