YCSB-IVS introduces a novel benchmarking technique to evaluate database performance as value sizes vary over time. This enhancement builds on the original YCSB framework, enabling experiments with dynamic value size growth and providing new insights into database behavior under evolving workloads.
Our benchmarking approach evaluates three widely-used databases:
- MongoDB
- MariaDB + InnoDB
- MariaDB + RocksDB
- Analysis of database performance under dynamic value size variations.
- Comparison of latency, throughput, and scalability across different database systems.
- Comprehensive scripts and figures for replicating our analysis.
To get started, clone the YCSB-IVS repository:
git clone https://github.com/dliyanage/YCSB-IVS.git
cd YCSB-IVSFor details on running YCSB (the core tool behind YCSB-IVS), refer to the installation and build guide:
Official YCSB README
The rest of this document outlines the additions made on top of the original YCSB version as of 1 Feb 2025, specifically for YCSB-IVS experiments.
| Directory | Description |
|---|---|
./experiment_scripts |
Bash scripts for running workload experiments. |
./analysis/Data |
Output data files relevant for analysis that are generated from our experiments by running bash scripts in ./experiment_scripts. Refer to ./analysis/README.md. |
./analysis/Scripts |
Analysis scripts written in R (Jupyter notebooks). |
./analysis/Figures |
Generated output figures and visualizations referenced in our paper. |
The ./experiment_scripts directory contains all the necessary bash scripts for running workload experiments and saving the results as CSV files.
- MongoDB Workload:
Use the./experiment_scripts/experiment_mongodb.shscript to execute the benchmarking workloads in MongoDB with varying value sizes. - MongoDB Baseline:
Use the./experiment_scripts/experiment_mongodb_baseline.shscript to run baseline executions with fixed value sizes for comparison.
Please refer to the general instructions on configuring experiments in the README file at ./experiment_scripts/README.md.
All output files generated during experiments are stored in the ./analysis/Data directory. These files are prepared for analysis and visualization.
To understand the analysis process and view results, refer to the README.md within the ./analysis directory. This document provides step-by-step details on our analysis methodology and generates outputs included in our publication.
The R Jupyter notebook located in the ./analysis/Scripts folder includes:
- Step-by-step explanations of our analysis process.
- Intermediate results and final output figures.
-
Clone the repository:
git clone https://github.com/dliyanage/YCSB-IVS.git cd YCSB-IVS -
Explore the data:
Navigate to the./analysis/Datadirectory to view raw data files. -
Run the analysis:
Open the R notebook in the./analysis/Scriptsfolder to reproduce the analysis and figures. -
View results:
Output figures are stored in the./analysis/Figuresdirectory.
If you use this work, please cite:
Benchmarking Databases with Varying Value Sizes [Experiment, Analysis, and Benchmark]." VLDB 2025.
For further information, visit the YCSB-IVS GitHub repository.
YCSB-IVS expands the capabilities of the original YCSB framework to simulate realistic scenarios of value size growth, offering a powerful tool for evaluating database scalability and performance.