Latest version: 11:32 (Commvault Platform Release 2023E)
August 15, 2023 to June 15, 2026
The Web Console interface is being phased out and replaced by the Command Center interface, which includes
most of the same functionality. For new 11.32 (Commvault Platform Release 2023E)
Environment details:
Commserve Details: We have Commserve on APAC ,EMEA & US regions
APAC:
1 Commserve + 1 DR server
50 Media Agent + Hyper scale (data movers) + Windows VSA Proxy (The VSA manages backups and restores
for Virtual Machines)
Clients around: 1600
EMEA:
1 Commserve + 1 DR server
221 Media Agent + Hyper scale (data movers) + Windows VSA Proxy (The VSA manages backups and restores
for Virtual Machines)
Clients around: 13478
US:
1 Commserve + 1 DR server
75 Media Agent + Hyper scale (data movers) + Windows VSA Proxy (The VSA manages backups and restores
for Virtual Machines)
Clients around: 5279
Metrics Server: 1 (Reporting): We can use Metrics server for generating backup reports Daily,Weekly,Monthly
for all the regions. Through interface.
We are using hyperscale (Linux 7.9)
ILO version: 5
Hyper scale (hardware support HPE) :
Hyperscale Monitoring: HPE OneView Standard
Model: ProLiant XL420 Gen10
Hyper scale capacity: 3 Node
1 Node: 24 slots: Disk 4 TB = 96 TB
2nd Node: 24 slots: Disk 4 TB = 96 TB
3rd Node: 24 slots: Disk 4 TB = 96 TB
Total: 288 TB.
3 Nodes: DDB hosted on all the 3 nodes
6 Nodes: DDB hosted on all the 6 nodes
9 Nodes: DDB hosted only on 6 nodes.
We used to perform cyclic Firm ware upgrade.
We will the Vulnerability report notification from security team..
Based on the
License:
Once the license has been utilized 110% the jobs will be pending status.
Use the grace period (96hrs for 1reset | Total we do 8 reset) or else need to add the license.
Backup Type:
File system,
VM snapshot backup
SQL backup
Oracle DB backup
We configure backup via work flow. Only if workflow have issue, then we will configure backup manually.
Restores we will perform file level & Full VM restored.
DB restored are performed by DB Team. Incase of any issue if they faced, we will assist the same.
Commserver Overall Health Check:
Observe the successful login to commcell without any errors (If any Java errors and certificate errors, pls engage
L3 team).
Check the overall SLA status of the commserver.
Media Agent Health Check (Goto Storage Resources -> Media Agents). Verify the media agents are ready.
Disk Library and mount path health Check (Goto Storage Resources -> Libraries). Verify the disk library and
mount paths are ready
DDB Status of all policies (Goto Storage Resource -> Deduplication Engines). Verify if all the DDB status is
online and active.
DDB Performance (Click on each of the Deduplication Engine and ensure the Average Q&I time is under
control.
DDB Backup Status (Click on each of the Deduplication Partition and ensure each DB backup is completed on
date)
Goto Admin Job History and ensure the DR backup, data aging jobs are completed successfully.
After Health checkup we will fetch up the backup failure report for 24hours from command center and work on
the failures.
After finishes health checkup we will use work on the emails, incident and task assigned on our group.
This is our daily Work routine.
========================================================
VM issue:
KB0062574: Virtual machine not found
KB0060008 : quiesce
KB0063279: failed to create VM snapshot” due to multiple existing snapshots on virtual machine.
KB0063339: VM’s backups are getting failed with the error independent disks are not supported.
KB0081659: No virtual machines were discovered for this Subclient. Check the subclient content and configured
File System issues:
KB0062314: System state issue
Backup failed since its reached the threshold failed value.
Communication issue.
SQL:
KB0062068: VDI timeout value from Commvault console
KB0062508: Providing the SQL system admin role to Backup
KB0083287: Unable to get the SQL Version for the server
RMAN:
KB0062938: archived redo log not deleted.
KB0063037 : error oracle DB in unknown or down state
KB0063256 : Subclient threshold value
Restore:
KB0064072 to restore the vmdk
KB0081918 - article published for Commvault Linux VM Granular Files & Folders Restore Issue.
Reports:
Health check report for all the regions
Strike report
Backup failure report
Killed failed job
Commserve DR backup and targets:
1. By default, the CommServe software runs Disaster Recovery (DR) backup jobs to protect the
CommServe production database.
2. The DR backups include the CommServe database and other databases (if installed with the CommServe
software).
3. In this method, the databases are backed up to a disk media and moved to a tape media periodically.
In the event of an actual disaster, you can use these backed up database dumps from the remote media and
recover the CommServe database on the same or new host.
Because no standby CommServe exists, the recovery time objective (RTO) will be considerably long.
Commserve DB’s
DR Backup Phases:
DR Backup process:
DR backup will stage the SQL database backup in the Location: Software_installation_directory\contentstore\
CommserveDR (We have enough space on the drive)
To Change DR location: Need to add the key ERStagingDirectory
DR Backup two phases:
1. 0% to 50% is called Export Phase: The DR backup that was stagged temporarily on the installation
directory Copy of the DR backup will create on the Export location.
Destination will be 1) Network share or 2) Local Path Example D:/DR backup.
(Data’s will be written for 5 copies. As soon as the 6th copy created 1st copy will get deleted) and
(We can create an additional copy on Commvault cloud service or any cloud storage)
1. 50 % to 100 % is called backup phase: Copy of the DR backup will be written to one of the libraries
configured on the environment (it can disk or tape or cloud library).
2. When DR policy will get created:
After configuring one Media Agent and one library in the environment automatically there will be storage policy
created with the name CommserveDR .
You can set up the following additional export destinations:
1) Commvault Cloud Services portal: Commvault uses AZURE- block storage
a) We need to register the Commserve with the Commvault cloud service. => getting started=>Initial
setup=>Initial CommCell configuration => Register product=>create account.
b) Additionally, We need to register with cloud metrics reporting.
2) Cloud storage library: We can configure any cloud storage.
====================================
MEDIA EXPLORER TOOL: To convert the Chunk files to .dmp files.
1. In Export phase files are saved as .dmp format and we will use the file to recover the Commserve using
CSRecovery tool.
2. In case Export phase copies are not available. We must use Backup Phase copies.
3. In Backup phase files are saved as chunk format and to read chunk format Commvault provide a tool
name called MEDIA explorer.
4. Cloud.commvault.com => All downloads => Media Explorer=>Download.
From here we can extract from Tape or disk.
CSRecovery Tool:
1. Its exe file and available on Installation directory/Content Store/Base/CSRecoveryAssistant
2. a) Staging / Troubleshooting (We can use for testing environment to test)
b) Recovery / Production (Actual DR)
Browse the DR file:
Data file extraction the path:
Need to start the services.
1. Down server name ABC and restored was XYZ.
2. Clients will refuse to connect since its trying to communicate with the name ABC
3. Go to control panel and there is a tool on control panel: Name Management
4. Update the new Commserve name.
============================================================================
has context menuParagraph
Commvault CVFailover
Production:
Production failover can be performed to switch the CommServe server from the currently active CommServe
host to a standby CommServe host.
A production failover can be one of the following:
Planned Failovers: A planned failover can be performed to alternate between the active and passive
CommServe hosts.
Unplanned Failovers:
When the production CommServe host is inaccessible due to network or hardware failure
When the production CommServe host is partially available
When there is a failure during a planned failover
Production Maintenance:
1. Maintenance failovers can be used to temporarily halt all activities except restores in the CommCell.
2. Use this option to perform maintenance related activities without completely shutting down the
CommServe server (for example, to install platform releases on the production CommServe host).
Test Failover:
A test failover can be performed to ensure that the failover process succeeds, and the services are successfully
started.
Test failovers do not affect the current jobs and activities on the production CommServe host.
Note that backups and restores are not allowed from the standby CommServe host during a test failover.
Procedure:
What to Do Next
After successful completion of the test failover, reset the test failover.
has context menu
