Commands
Restart ESM
# service cpservice stop
# service cpservice start
Restart APM
# /etc/init.d/apm stop
# /etc/init.d/apm start
# NitroStop –nod (no daemon)
# NitroStart –nod (no daemon)
View ESM DB Errors
tailf /usr/local/ess/data/NitroError.Log
less /usr/local/ess/data/NitroError.Log (Use ‘q’ to exit the text editor)
View Rebuild Status of DB Partitions (in addition to ESM System Properties page)
# /data_hd/usr/local/ess/data/watch –d ‘ls –lia *X*’
Viewing Sort Files created by ESM (too large can create slowdowns)
# /data_hd/usr/local/ess/data/du –ch *SRT*
Retrieve APM Crash Logs Location
Advertisements
REPORT THIS AD
# /var/log/ice/crash.log
Retrieve Health Status on SIEM device
# cat /var/lib/HealthStatus.data (tac for newest first, cat for oldest first)
Gather appliance resource information
# grep Resource /var/log/messages
Check Event Receiver for Restart Events
cat /var/log/messages* | grep -i “starting ERC” (tac for newest first, cat for oldest first)
less /var/log/messages (type ‘q’ to exit the editor, “Shift + g” to get to the end of the file)
Check upgrade history
# cat /etc/upgrade.history (tac for newest first, cat for oldest first)
Zip Messages Log
# cd /var/log
# tar –czvf messages.tgz messages*
Checking RAID drives (from /opt/MegaRAID/MegaCli/)
#./MegaCli64 -ldinfo -l0 -a0
#./MegaCli64 -ldinfo –l1 -a0
#./MegaCli64 -pdlist -a0
Most helping CPU processes view
# ps -ejH (shows processes and threads)
# ps -C commandname (search like “ps | grep”)# ps -auxf (most details)
# ps -ax (all processes)
View the Build Version of the McAfee Software on the appliance
# cat /etc/buildstamp
View the build version of the Linux OS
# cat /proc/version
Check Disk Space – General
# df –h (to show space by drive) or du –h (to show space by folder)
Check if logs are being sent to a receiver
# tcpdump –nni eth1 host 10.x.x.x
Check multiple hosts
# tcpdump –nni eth1 ‘(host 10.x.x.x or host 10.y.y.y)’
Finding the largest file on a system
Use the du command to find out how much data is in each directory.
# du -hc –max-depth=1
This will return the base level directories with the size.
NOTE: is best to start in the “/” directory.
The following command will limit the results to only the Gigabyte size directories:
# du -hc –max-depth=1 | grep G
You can continue to use these commands as you drill down in the directories to find the directory that
contains the largest amount of data.
Check the disk utilization
Sar –d 1 10
Checks disk utilization every second over 10 seconds
TCP dump commands when you do not see the expected DBM data
# ifconfig -a (determines which eth port DB traffic is being sent to)
# tcpdump -s0 -ieth3 -wfile1.pcap host 1.2.3.4 and port 1433 (dump file will be called file1.pcap in the
current directory)
Run the dump for 30, and then press CTRL+C to escape out. Check the file size and repeat. Generally
a few hundred MBs worth of files should be good if you see the expected DB traffic in the tcpdump.
To check for vlan tagging
# tcpdump -s0 -ieth3 -wfile1.pcap vlan # host 10.x.x.x and port 1433 (optional to leave the tag number,
for example 130, off if unknown).
# tcpdump -s0 -nnvXi eth1 vlan # and host 172.x.x.x and port 1433 (with vlan tag and shows packet
contents in list form)
# tcpdump -s0 -nnvXi eth1 -wfile3.pcap vlan # and host 172.x.x.x and port 1433 (with vlan tag and
saves tcpdump to pcap file)
Find all Receiver database index files
# ls /var/log/data/inline/
Track Receiver database index progress
# tailf /var/log/data/inline/NitroError.Log
Restarting collectors / parsers on a Receiver
# killall collectorsctl filterctl parsersctl
# /etc/init.d/nitrodbserver restart
# collectorsctl — +laux
# filterctl — +laux
# parsersctl — +laux
Restarting WMI collector / parser only
# killall -9 wmin
# killall -9 wmip
Run checksum on install file
# sha1sum <filename>
Check DAS Virtual Drive Information from ESM or ELM
Watch –d megacli –ldinfo –l0 –a0
“Ctrl + C” to exit
Check DAS Drive Status
megacli -pdlist -a0 | grep Firmware
Check the Controller, RAID, and Drive Status on ESM
Show the Controller:
# tw_cli show
Show the Status of the RAID/Drives
# tw_cli show c# (# shown using show command – c2 for ESM’s, c0 on 2250 Receivers and
APM)
Show individual drive status
# tw_cli info c2 p# (# drive number)
Show individual drive serial number
# tw_cli info c2 p# serial (# drive number)
Check HA or Cluster Status on Receiver or ESM
# ha_status
# crm status
Check agent logs status on Receiver
# tailf /var/log/NPP_c.log
# tailf /var/log/NPP_p.log
Finding and viewing raw logs in stored text file on Receiver (example)
# tail –n 1000 /var/log/NPP_c.log | grep 10.198.12.73
Results: NPP_c[18094]: Got a connection from client IP addr: 10.198.12.73, id = 105
Find the ID of the host and use on the next command to find files to search
# ls /var/log/data/inline/thirdparty.logs/105/in/
# cat /var/log/data/inline/thirdparty.logs/105/in/data.20121115161524000
View ESM DB partitions
# !nsq (shortcut command which will launch the nsql command below – if in the correct directory)
# nsql /usr/local/ess/data/connect_esm.sql (To exit the nsql editor type ‘x’ or ‘exit’ and <enter>)
(Attached = Good, Detached = Call Support, Bad = Awaiting Rebuild)
To view Event data
show partitions from alert
To view Flow data
show partitions from connection
To view Appliance Packet data
Show partitions from packet
View record counts in DB
# nsql /usr/local/ess/data/connect_esm.sql (To exit the nsql editor type ‘x’ or ‘exit’ and <enter>)
To view event data
select count(*) from alert
To view flow data
select count(*) from connection
To view Appliance Packet data
select count(*) from packet
View Event data on a receiver
# nsql /var/log/data/inline/connect.sql (Launch this command when ssh in to a receiver)
To view Event data
show partitions from event
To get the clutter off of the SSH session and your prompt back to the top of the screen (does not
remove scroll back data on screen)
clear
Article for rekeying a SIEM Appliance
# cd /root/.ssh
# cat /etc/NitroGuard/factory-id_rsa.pub >authorized_keys2
Wiping Receiver Data Sources and log files
# rm /etc/NitroGuard/thirdparty.*.*
# rm /var/log/data/inline/thirdparty.logs/(1* thru 9*)
# rm /var/log/data/inline/thirdparty.logs/elm.logs/(1* thru 9*)
# rm /var/log/data/inline/thirdparty.logs/elm.logs/tmp/(1* thru 9*)
ESM Related
Quickest running filters – Very Important!!!!
There are combinations of filters that are specifically tuned to run more quickly. These combinations
have been defined by users who frequently use the filters for quickly drilling down to specific events.
Signature ID + Event Subtype + Protocol
Signature ID + Source IP
Signature ID + Destination IP
Source IP + Destination IP
Additional Enhancements since 8.x:
Normalization ID + Source IP
Normalization ID + Destination IP
Normalization ID + Event Subtype
Normalization ID + Protocol
Normalization ID + Signature ID
Source Zone + Destination Zone
Directory Related
ESM & Receiver Software File Location (for upgrades)
/usr/local/ess/SoftwareUpdates/
/usr/local/NitroGuard
ESM Software File Locations Archives
/usr/local/ess/update/archive/
ESM Default Backup Location
/db1/usr/local/ess/dbbackup/
ESM DB Blob & Index Location
/data_hd/usr/local/ess/data/
ESM Index HD Location (Most recent data)
/index_hd/usr/local/ess/data/
ESM Redundant File Copy Location (For Alert, Connection, and Log files)
/usr/local/ess/dbredund
Commands:
McAfee-ETM-6000 ~ # less /usr/local/ess/data/NitroError.Log
McAfee-ETM-6000 ~ # service cpservice stop
tailf /var/log/messages – Watch what’s happening
Tcpdump -nni eth0 host <ipaddress of host you want to get a dump from> -vvv -w <path to write dump>
NitroTID – get database troubleshooting information
Indications that database needs to be rebuilt:
2014/04/09 21:43:01.429 Error 132 opening table with field Alert.ALERTTIM(partition 2406)(data count
= 77427336, index count = 77427320) Index count does not match record count
Rebuild Database index:
McAfee-ETM-6000 ~ # cd /usr/local/ess/data
McAfee-ETM-6000 /usr/local/ess/data # mkdir copy_ngcp
McAfee-ETM-6000 /usr/local/ess/data # cp ngcp.cfg copy_ngcp
McAfee-ETM-6000 /usr/local/ess/data # cp ngcp.cfd copy_ngcp
McAfee-ETM-6000 /usr/local/ess/data # cd copy_ngcp/
McAfee-ETM-6000 /usr/local/ess/data/copy_ngcp # ls
ngcp.cfd ngcp.cfg
McAfee-ETM-6000 /usr/local/ess/data/copy_ngcp # cd ../
McAfee-ETM-6000 /usr/local/ess/data # rm ngcp.cfg
McAfee-ETM-6000 /usr/local/ess/data # rm ngcp.cfd
McAfee-ETM-6000 /usr/local/ess/data # ls
Check Database:
McAfee-ETM-6000 ~ # service cpservice stop
Stopping ESS cpservice [ OK ]
McAfee-ETM-6000 ~ # DBCheck -d ‘/usr/local/ess/data/ngcp.dfl’ -p ‘LOCDB327|CPDB126’ -t ‘!Alert|!
Connection|!Log|!Packet|!stringmap’ -r
McAfee-ETM-6000 ~ # DBCheck -d ‘/usr/local/ess/data/ngcp.dfl’ -p ‘LOCDB327|CPDB126’ -t -c
Option at position 5 needs an argument : t
McAfee-ETM-6000 ~ # DBCheck -d ‘/usr/local/ess/data/ngcp.dfl’ -p ‘LOCDB327|CPDB126’ -c
DBCheck -d ‘/usr/local/ess/data/ngcp.dfl’ -p ‘LOCDB327|CPDB126’ -c | grep Not
Check database health:
1. Cd /usr/local/ess/data
2. DBCheck -d ngcp.dfl -c | grep not
Check Syslog for Raid errors:
1. Cd /usr/local/ess/data
2. Dmesg | grep RAID
Display Raid Status
1. Cd /usr/local/ess/data
2. MegaCli64 -CfgDsply -aAll | grep fail or grep down
Use grep for “downgraded, or fail”which will indicate raid issues