PatchEval: A New Benchmark for Evaluating LLMs on Patching Real-World Vulnerabilities

This repository contains results of systematic analysis of cloud emulators used in software testing, focusing on their fidelity compared to real cloud services.

R package interfacing the Bloomberg API from https://www.bloomberglabs.com/api/

A collection of postmortems. Sorry for the delay in merging PRs!

An AI-Native Platform for Benchmarking SRE Agents

Code for the paper "VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Use"

Get your documents ready for gen AI

First Latency-Aware Competitive LLM Agent Benchmark

Code repository for scenarios and environment setup as part of ITBench

A holistic framework to enable the design, development, and evaluation of autonomous AIOps agents.

A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training

Code repository for SRE agent as part of ITBench

An open source benchmarking framework for IT automation

ESP AMP

A holistic framework to enable the design, development, and evaluation of autonomous AIOps agents.

stock股票.获取股票数据,计算股票指标,筹码分布,识别股票形态,综合选股,选股策略,股票验证回测,股票自动交易,支持PC及移动设备。

Run your own AI cluster at home with everyday devices 📱💻 🖥️⌚

Awesome things about LLM-powered agents. Papers / Repos / Blogs / ...

Cloud incidents/failures related work.

Systems for GenAI

BCC - Tools for BPF-based Linux IO analysis, networking, monitoring, and more

An implementation of failpoints for Golang.

Byteman Project main repo

Inject python code into a running python process

A curated list of large language model tools for cybersecurity research.

Like Prometheus, but for logs.

A collection of papers on the topic of ``Computer Vision in the Wild (CVinW)''