[Add feature] Implement ClientManager for efficient API client lifecycle management #405
Description
Description
Problem
Currently, API clients (GithubClient, HuggingFaceClient, QdrantClient, SemanticScholarClient, etc.) are instantiated repeatedly in each node function throughout the codebase. This pattern causes several issues:
- Memory Waste
- Each client instantiation creates new
requests.Session()andhttpx.AsyncClient()objects - In a single experiment run with dozens of nodes, dozens of redundant sessions are created
client = github_client or GithubClient()# New instance every time
- Performance Issues
- Connection pools are not shared across nodes
- TLS/TCP handshakes occur repeatedly for the same API endpoints
- HTTP connection reuse benefits are lost
- Unnecessary consumption of API rate limits
- Resource Leaks
httpx.AsyncClientinstances are not explicitly closed- Relies on garbage collection to release connections
- In long-running processes, this can lead to socket/memory leaks
- No explicit lifecycle management
- Missed Optimization Opportunities
- For long-running AIRAS experiments with continuous GitHub/LLM API interactions, maintaining persistent connections would significantly improve performance
- Current implementation negates the benefits of connection pooling
Proposed Solution
Implement a ClientManager singleton pattern to manage API client lifecycle:
src/airas/services/api_client/client_manager.py
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from airas.services.api_client.github_client import GithubClient
from airas.services.api_client.hugging_face_client import HuggingFaceClient
from airas.services.api_client.semantic_scholar_client import SemanticScholarClient
from airas.services.api_client.qdrant_client import QdrantClient
class ClientManager:
"""Singleton manager for API clients with lifecycle management"""
_instance: "ClientManager | None" = None
def __init__(self):
self._github_client: "GithubClient | None" = None
self._hf_client: "HuggingFaceClient | None" = None
self._semantic_scholar_client: "SemanticScholarClient | None" = None
self._qdrant_client: "QdrantClient | None" = None
@classmethod
def get_instance(cls) -> "ClientManager":
"""Get or create singleton instance"""
if cls._instance is None:
cls._instance = cls()
return cls._instance
@classmethod
def reset_instance(cls) -> None:
"""Reset singleton (useful for testing)"""
cls._instance = None
@property
def github(self) -> "GithubClient":
"""Get shared GitHub client"""
if self._github_client is None:
from airas.services.api_client.github_client import GithubClient
self._github_client = GithubClient()
return self._github_client
@property
def hugging_face(self) -> "HuggingFaceClient":
"""Get shared HuggingFace client"""
if self._hf_client is None:
from airas.services.api_client.hugging_face_client import HuggingFaceClient
self._hf_client = HuggingFaceClient()
return self._hf_client
@property
def semantic_scholar(self) -> "SemanticScholarClient":
"""Get shared SemanticScholar client"""
if self._semantic_scholar_client is None:
from airas.services.api_client.semantic_scholar_client import SemanticScholarClient
self._semantic_scholar_client = SemanticScholarClient()
return self._semantic_scholar_client
@property
def qdrant(self) -> "QdrantClient":
"""Get shared Qdrant client"""
if self._qdrant_client is None:
from airas.services.api_client.qdrant_client import QdrantClient
self._qdrant_client = QdrantClient()
return self._qdrant_client
async def aclose_all(self) -> None:
"""Close all async sessions explicitly"""
if self._github_client and hasattr(self._github_client, 'aclose'):
await self._github_client.aclose()
if self._hf_client and hasattr(self._hf_client, 'aclose'):
await self._hf_client.aclose()
if self._semantic_scholar_client and hasattr(self._semantic_scholar_client, 'aclose'):
await self._semantic_scholar_client.aclose()
if self._qdrant_client and hasattr(self._qdrant_client, 'aclose'):
await self._qdrant_client.aclose()Usage in nodes:
# Before
def execute_experiment(..., github_client: GithubClient | None = None):
client = github_client or GithubClient() # New instance
...
# After
from airas.services.api_client.client_manager import ClientManager
def execute_experiment(..., github_client: GithubClient | None = None):
client = github_client or ClientManager.get_instance().github # ReuseBenefits
- Memory Efficiency
- Single client instance per type across entire experiment run
- Shared connection pools reduce memory footprint
- Performance Improvement
- Connection reuse significantly reduces TLS/TCP handshake overhead
- Faster API response times through persistent connections
- More efficient use of API rate limits
- Resource Management
- Explicit lifecycle control via
aclose_all() - Prevents resource leaks in long-running processes
- Clean shutdown capabilities
- Better Testing
- Easy to inject mock clients via dependency injection
reset_instance()method for test isolation- Maintains backward compatibility with existing
client: GithubClient | None = Noneparameters
- Future Extensibility
- Foundation for connection pool configuration (max connections, timeouts, etc.)
- Easier to add monitoring/logging for API usage
- Supports adding new clients without architectural changes
This approach should also work well with issue #358.