docs: update README to include information about Windows support #213
Conversation
README.md
Outdated
| ``` | ||
|
|
||
| 2. If you run into OOM errors, especially while running evaluations that use LLMs as evaluators like toxicity and | ||
| 2. Due to `fmeval`'s use of [Ray](https://www.ray.io/), and the fact that [Ray on Windows is still in beta](https://docs.ray.io/en/latest/ray-overview/installation.html#windows-support), users running evaluations on Windows machines may encounter Windows-specific issues. For example, `ray.init()` may raise an OS error, as described in [this thread](https://discuss.ray.io/t/ray-init-raises-oserror-assignprocesstojobobject-failed/7018). |
There was a problem hiding this comment.
Suggestion: Could we simplify this sentence and explicitly direct users to the Ray website to view current limitations?
For example, "There are known limitations and issues when running fmeval on Windows due to fmeval's use of Ray and Ray's Windows support in beta. You can view more details on limitations of running Ray on Windows on Ray's webpage."
There was a problem hiding this comment.
Sure. I'll create a separate line item for the specific OS error in that case.
README.md
Outdated
| ``` | ||
|
|
||
| 2. If you run into OOM errors, especially while running evaluations that use LLMs as evaluators like toxicity and | ||
| 2. Due to `fmeval`'s use of [Ray](https://www.ray.io/), and the fact that [Ray on Windows is still in beta](https://docs.ray.io/en/latest/ray-overview/installation.html#windows-support), users running evaluations on Windows machines may encounter Windows-specific issues. For example, `ray.init()` may raise an OS error, as described in [this thread](https://discuss.ray.io/t/ray-init-raises-oserror-assignprocesstojobobject-failed/7018). |
There was a problem hiding this comment.
Question: This seems to be an important limitation that users should know about. Should we move this to the first line item under Troubleshooting?
There was a problem hiding this comment.
Good point. I was actually thinking of whether this deserves its own section, but I think I'll just put it at the top of Troubleshooting, like you've suggested.
Description of changes:
This PR adds information to the troubleshooting section regarding Windows-specific issues that users may run into.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.