-
Notifications
You must be signed in to change notification settings - Fork 3
Overview
This section provides a general understanding of what openVRE is, its key features and benefits, and an overview of the system architecture and core components.
openVRE (open Virtual Research Environment) is an open-source platform designed to facilitate the creation, management, and customization of Virtual Research Environments (VREs).
It provides the frameworks, tools, and compute infrastructure needed to rapidly build cloud-based, collaborative research environments.
-
Customizable Front-End Design
openVRE’s modular architecture allows institutions or research groups to build tailored VREs suited to their specific research needs. Users can add or remove tools, customize the interface, and adapt workflows with minimal configuration. -
Data Management and Integration
openVRE supports complex data handling across diverse data types and formats. Integrated data storage and metadata management make it easier for researchers to upload, organize, and manage datasets securely. -
Integrated Analytical and Computational Tools
The platform supports modular integration of analytical and visualization tools. Execution can occur on local cloud resources, HPC clusters, or external compute backends — enabling scalable computational analysis. -
Interoperability with External Resources
openVRE connects seamlessly with external databases, repositories, and services, allowing researchers to bring external datasets directly into their workflows. -
Data Sharing and Publication Support
Built-in features for publishing and sharing datasets — including support for long-term repositories like B2SHARE — simplify the process of making research outputs FAIR (Findable, Accessible, Interoperable, Reusable).
- Scalability and Flexibility — Deploy openVRE for projects of any size and dynamically scale compute resources based on demand.
- Cost-Effectiveness — As an open-source framework, openVRE significantly reduces licensing costs associated with proprietary systems.
- Enhanced Collaboration — Built-in tools for data and workflow sharing support teamwork across institutions and geographical boundaries.
- Improved Data Governance — Features like data versioning, access control, and provenance tracking enhance reproducibility and compliance with FAIR data principles.
The main repository contains the core source code of openVRE — including a customizable web front-end and the backend code responsible for managing data, metadata, analysis tools, and computational resources.
openVRE’s architecture integrates several services that work together to provide a unified computational environment.
When users initiate an analysis or visualization via the web interface, their requests are processed through supported scheduling systems that dispatch jobs to local or remote compute backends.
Jobs are executed in isolated containers or virtual machines (VMs) that dynamically scale based on workload. Each tool runs within its dedicated execution environment with access to the user’s data volumes. This ensures security, scalability, and reproducibility.
All integrated tools adhere to the common structure defined by the openvre-tool-api, enabling the openVRE core to trigger and manage them uniformly.
The VRE provides a digital workspace supporting the entire research lifecycle. It combines user interfaces, tools, and data management features to enable online collaborative research.
openVRE integrates with well-known workload managers to handle job scheduling and resource allocation in compute clusters.
By default, openVRE supports SGE (Oracle Grid Engine) and SLURM.
The openVRE core acts as a submitter node, dispatching jobs to worker VMs that host tool code.
Each tool has a dedicated queue, with multiple replicas (VMs) managed by Oneflow, an OpenNebula service that automatically provisions or releases VMs based on load — enabling horizontal scaling.
openVRE can also submit remote jobs via message brokers. Integration with RabbitMQ is in progress, while PMES (Programming Model Enactment Service) is currently supported.
PMES handles remote job execution through the Open Cloud Computing Interface (OCCI), managing VM creation, contextualization, execution, and termination independently of the underlying cloud middleware (OpenNebula, OpenStack, etc.).
MongoDB serves as the main database, storing user data, job metadata, and tool configurations.
Its flexible schema supports dynamic data modeling, essential for diverse computational workflows.
MongoDB ensures fast retrieval and scalability across large-scale data analyses and visualizations.
Keycloak manages secure authentication and role-based access control.
It can be deployed locally or integrated with an external identity provider.
Keycloak ensures that only authorized users can access specific functionalities, datasets, and tools.
Vault securely stores API keys, credentials, and tokens required to access compute environments or external databases.
It provides strong encryption, dynamic secrets, and fine-grained access policies to protect sensitive credentials.
Analysis tools and workflows follow the structure defined by the openvre-tool-api.
This Python-based framework standardizes tool integration and execution by providing:
- A consistent command-line interface (CLI)
- Batch and asynchronous job support
- Automatic parsing and result collection
This design allows the openVRE backend to uniformly manage third-party tools developed by researchers, bioinformaticians, and domain experts.
Note: For more details on how to develop or register new tools, see the section Extending openVRE.
| Layer | Technology | Description |
|---|---|---|
| Frontend | React / Angular | Modern, modular web interfaces |
| Backend | Python (Flask / Django) | Core orchestration and API services |
| Database | MongoDB | NoSQL database for tool metadata and jobs |
| Auth | Keycloak | Identity and access management |
| Vault | HashiCorp Vault | Secure credential management |
| Compute | Docker / Singularity / OpenNebula / SLURM | Scalable container and VM orchestration |
| Messaging | RabbitMQ / PMES | Remote job dispatch and orchestration |
📘 Return to the Home page or continue to 2. Getting Started.