UiPath Agents Gym Sample

A comprehensive reference implementation of the Agents Gym loop and agents compatible with the SDK.

What's Included

Two Agents:

• Calculator Agent - Uses add and multiply tools to calculate an expression. Escalates for missing numbers.
• Loan Agent - Loan request processing with mocked escalations.

Dual-Mode Architecture:

• CLI Mode - Supports execution via uipath run, uipath dev and uipath eval
• Evaluation Mode - Custom runtime for batch execution of evaluations

Developer Experience:

• Convenience Scripts - run.sh, eval.sh, export_evals.sh for common operations
• Debugging - The custom runtime facilitates seamless debugging for evaluators

Key Technical Patterns

• Separate Graph Builders - build_cli_graph() for runtime input, build_evaluation_graph() for custom runtime batch execution of evaluations
• Evaluation Export - Convert Python evaluators/datapoints to UiPath eval JSON format

Quick Start

cd samples/gym-sample
uv sync                              # Install package
uv run uipath auth                   # Authenticate
./run.sh calculator                  # Run with `uipath run` CLI using a test input
uipath dev                           # Run with `uipath dev` CLI to inspect the traces and other execution metadata
./export_evals.sh                    # Export datapoints and evaluators to JSON
./eval.sh calculator                 # Run with `uipath eval` CLI to execute + evaluate the datapoints
python -m gym_sample.run --agent_name $AGENT_NAME   # Run the custom evaluations runtime