Tags: dannykopping/litellm
Tags
fix: prevent memory blowout in LoggingWorker Previously, tasks were executed sequentially, awaiting each task before starting the next. With large queues (10k+ tasks), this caused objects to accumulate in memory, holding references to heavy resources and leading to memory blowouts. This update: 1. Introduces a semaphore to allow a configurable number of concurrent tasks, improving throughput and preventing queue buildup. 2. Implements a configurable cleaning mechanism for when the queue reaches its limit, ensuring tasks are not dropped.
PreviousNext