Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Overview

Maria Paola Ferri edited this page Oct 24, 2025 · 5 revisions

Overview

This section provides a general understanding of what openVRE is, its key features and benefits, and an overview of the system architecture and core components.

1.1 What is openVRE

openVRE (open Virtual Research Environment) is an open-source platform designed to facilitate the creation, management, and customization of Virtual Research Environments (VREs).
It provides the frameworks, tools, and compute infrastructure needed to rapidly build cloud-based, collaborative research environments.

homepage

1.2 Key Features

  • Customizable Front-End Design
    openVRE’s modular architecture allows institutions or research groups to build tailored VREs suited to their specific research needs. Users can add or remove tools, customize the interface, and adapt workflows with minimal configuration.

  • Data Management and Integration
    openVRE supports complex data handling across diverse data types and formats. Integrated data storage and metadata management make it easier for researchers to upload, organize, and manage datasets securely.

  • Integrated Analytical and Computational Tools
    The platform supports modular integration of analytical and visualization tools. Execution can occur on local cloud resources, HPC clusters, or external compute backends — enabling scalable computational analysis.

  • Interoperability with External Resources
    openVRE connects seamlessly with external databases, repositories, and services, allowing researchers to bring external datasets directly into their workflows.

  • Data Sharing and Publication Support
    Built-in features for publishing and sharing datasets — including support for long-term repositories like B2SHARE — simplify the process of making research outputs FAIR (Findable, Accessible, Interoperable, Reusable).

1.3 Benefits of openVRE

  • Scalability and Flexibility — Deploy openVRE for projects of any size and dynamically scale compute resources based on demand.
  • Cost-Effectiveness — As an open-source framework, openVRE significantly reduces licensing costs associated with proprietary systems.
  • Enhanced Collaboration — Built-in tools for data and workflow sharing support teamwork across institutions and geographical boundaries.
  • Improved Data Governance — Features like data versioning, access control, and provenance tracking enhance reproducibility and compliance with FAIR data principles.

1.4 Architecture

The main repository contains the core source code of openVRE — including a customizable web front-end and the backend code responsible for managing data, metadata, analysis tools, and computational resources.

arch_1

openVRE’s architecture integrates several services that work together to provide a unified computational environment.
When users initiate an analysis or visualization via the web interface, their requests are processed through supported scheduling systems that dispatch jobs to local or remote compute backends.

Jobs are executed in isolated containers or virtual machines (VMs) that dynamically scale based on workload. Each tool runs within its dedicated execution environment with access to the user’s data volumes. This ensures security, scalability, and reproducibility.

All integrated tools adhere to the common structure defined by the openvre-tool-api, enabling the openVRE core to trigger and manage them uniformly.

1.5 Core Components

Virtual Research Environment

The VRE provides a digital workspace supporting the entire research lifecycle. It combines user interfaces, tools, and data management features to enable online collaborative research.

Job Schedulers

openVRE integrates with well-known workload managers to handle job scheduling and resource allocation in compute clusters.

• Queue Systems

By default, openVRE supports SGE (Oracle Grid Engine) and SLURM.
The openVRE core acts as a submitter node, dispatching jobs to worker VMs that host tool code.
Each tool has a dedicated queue, with multiple replicas (VMs) managed by Oneflow, an OpenNebula service that automatically provisions or releases VMs based on load — enabling horizontal scaling.

• Message Brokers

openVRE can also submit remote jobs via message brokers. Integration with RabbitMQ is in progress, while PMES (Programming Model Enactment Service) is currently supported.
PMES handles remote job execution through the Open Cloud Computing Interface (OCCI), managing VM creation, contextualization, execution, and termination independently of the underlying cloud middleware (OpenNebula, OpenStack, etc.).

arch_2

Tools Database (MongoDB)

MongoDB serves as the main database, storing user data, job metadata, and tool configurations.
Its flexible schema supports dynamic data modeling, essential for diverse computational workflows.
MongoDB ensures fast retrieval and scalability across large-scale data analyses and visualizations.

Authentication and Authorization (Keycloak)

Keycloak manages secure authentication and role-based access control.
It can be deployed locally or integrated with an external identity provider.
Keycloak ensures that only authorized users can access specific functionalities, datasets, and tools.

Credentials Vault (HashiCorp Vault)

Vault securely stores API keys, credentials, and tokens required to access compute environments or external databases.
It provides strong encryption, dynamic secrets, and fine-grained access policies to protect sensitive credentials.

Analysis Tools (openvre-tool-api)

Analysis tools and workflows follow the structure defined by the openvre-tool-api.
This Python-based framework standardizes tool integration and execution by providing:

  • A consistent command-line interface (CLI)
  • Batch and asynchronous job support
  • Automatic parsing and result collection

This design allows the openVRE backend to uniformly manage third-party tools developed by researchers, bioinformaticians, and domain experts.

Note: For more details on how to develop or register new tools, see the section Extending openVRE.


1.6 Supported Technologies (optional section)

Layer Technology Description
Frontend React / Angular Modern, modular web interfaces
Backend Python (Flask / Django) Core orchestration and API services
Database MongoDB NoSQL database for tool metadata and jobs
Auth Keycloak Identity and access management
Vault HashiCorp Vault Secure credential management
Compute Docker / Singularity / OpenNebula / SLURM Scalable container and VM orchestration
Messaging RabbitMQ / PMES Remote job dispatch and orchestration

📘 Return to the Home page or continue to 2. Getting Started.

Clone this wiki locally