DevOps Shack
LINUX TROUBLESHOOTING GUIDE
1. System Boot Issues
1.1 Boot Loader Issues (GRUB Issues)
Symptoms:
● System does not boot into Linux.
● Stuck at GRUB prompt.
● Error: grub-rescue>
Causes:
● Corrupt or missing GRUB configuration.
● Wrong boot device.
● Kernel update failure.
Solution:
1. Boot using a Live CD/USB.
Mount the root partition:
sudo mount /dev/sdXn /mnt
Reinstall GRUB:
sudo grub-install --root-directory=/mnt /dev/sdX
Update GRUB:
sudo update-grub
1.2 Kernel Panic
Symptoms:
● Black screen with panic messages.
● Kernel panic – not syncing error.
Causes:
● Incompatible kernel update.
● Corrupt boot files.
Solution:
1. Reboot and select an older kernel from GRUB.
Check logs using:
journalctl -xb
2. Roll back the kernel update if necessary.
2. Login and User Issues
2.1 Forgotten Root Password
Symptoms:
● Cannot log in as root.
Solution:
1. Boot into recovery mode.
Remount root partition:
mount -o remount,rw /
Reset password:
passwd root
2. Reboot.
2.2 SSH Connection Refused
Symptoms:
● Connection refused when trying to SSH.
Causes:
● SSH service not running.
● Firewall blocking SSH.
Solution:
Check SSH status:
sudo systemctl status sshd
Start service if stopped:
sudo systemctl start sshd
Check firewall rules:
sudo ufw allow ssh
3. File System and Disk Issues
3.1 Disk Space Full
Symptoms:
● No space left on device error.
Solution:
Check disk usage:
df -h
Find large files:
du -ah / | sort -rh | head -20
1. Delete unnecessary files.
3.2 Read-Only File System
Symptoms:
● Read-only file system error.
Solution:
Remount the filesystem:
sudo mount -o remount,rw /
Check and repair disk errors:
sudo fsck -y /dev/sdXn
4. Process and Performance Issues
4.1 High CPU Usage
Symptoms:
● System slows down.
● High load average in top output.
Solution:
Identify culprit process:
top
Kill process:
kill -9 <PID>
5. Networking Issues
5.1 No Internet Connection
Solution:
Check network interfaces:
ip a
Restart networking:
sudo systemctl restart NetworkManager
6. Package Management Issues
6.1 Broken Package Dependencies
Symptoms:
● Unable to correct problems, you have held broken
packages
● Package installation fails
Solution:
Identify broken packages:
sudo apt --fix-broken install
Clean package cache:
sudo apt clean && sudo apt autoremove
Reinstall dependencies:
sudo apt install -f
6.2 Slow Package Installation
Symptoms:
● Installing or updating packages takes too long
Solution:
Update repository lists:
sudo apt update
Switch to a faster mirror:
sudo sed -i
's/http:\/\/archive.ubuntu.com/http:\/\/mirror.example.com/'
/etc/apt/sources.list
7. Service and Daemon Issues
7.1 Service Not Starting
Symptoms:
● Failed to start service-name.service
Solution:
Check service logs:
sudo journalctl -xe | grep service-name
Restart the service:
sudo systemctl restart service-name
1. Check configuration files for errors.
7.2 Port Conflicts
Symptoms:
● Service fails to bind to a port
Solution:
Identify the conflicting process:
sudo netstat -tulnp | grep <port-number>
1. Kill the process or reconfigure services.
8. Kernel and Module Issues
8.1 Kernel Update Breaks System
Symptoms:
● System does not boot after update
Solution:
1. Reboot and select an older kernel in GRUB.
Remove faulty kernel:
sudo apt remove linux-image-<version>
8.2 Missing Kernel Modules
Symptoms:
● modprobe: FATAL: Module not found
Solution:
Check available modules:
lsmod | grep module-name
Load the module manually:
sudo modprobe module-name
9. Security and Access Control Issues
9.1 Unauthorized Access Attempts
Symptoms:
● Unusual login attempts in logs
Solution:
Check logs:
sudo cat /var/log/auth.log | grep "Failed password"
Block IP using firewall:
sudo ufw deny from <IP-ADDRESS>
9.2 SELinux Blocking Services
Symptoms:
● Service fails due to permission errors
Solution:
Check SELinux logs:
sudo ausearch -m AVC
Temporarily disable SELinux (for debugging only):
sudo setenforce 0
10. Hardware Issues
10.1 USB Device Not Recognized
Symptoms:
● USB device does not show up in lsusb
● dmesg shows errors related to USB
Solution:
Check if the USB device is detected:
Lsusb
Re-scan USB devices:
echo "1" | sudo tee /sys/bus/usb/devices/usb1/authorized
Restart the USB service:
sudo modprobe -r usb_storage && sudo modprobe usb_storage
10.2 Hard Drive Not Detected
Symptoms:
● fdisk -l does not show the disk
● dmesg shows disk-related errors
Solution:
List available disks:
lsblk
Re-scan the disk:
echo "- - -" | sudo tee /sys/class/scsi_host/host0/scan
11. Time and Date Issues
11.1 System Clock Out of Sync
Symptoms:
● Incorrect system time
Solution:
Sync time with NTP:
sudo timedatectl set-ntp on
Manually set time:
sudo timedatectl set-time "YYYY-MM-DD HH:MM:SS"
12. Firewall and Network Security Issues
12.1 Firewall Blocking Services
Symptoms:
● Cannot access services from another system
Solution:
List active firewall rules:
sudo ufw status
Allow required ports:
sudo ufw allow 80/tcp
12.2 Too Many Failed SSH Attempts
Symptoms:
● Too many authentication failures in SSH
Solution:
Edit SSH configuration:
sudo nano /etc/ssh/sshd_config
Increase allowed authentication retries:
MaxAuthTries 10
Restart SSH:
sudo systemctl restart sshd
13. Virtualization Issues
13.1 Virtual Machine Won't Start
Symptoms:
● Failed to start the virtual machine error
Solution:
Check virtualization support:
egrep -c '(vmx|svm)' /proc/cpuinfo
1. Enable virtualization in BIOS.
14. Miscellaneous Issues
14.1 Command Not Found
Symptoms:
● command not found error
Solution:
Check if the package is installed:
dpkg -l | grep package-name
Install missing package:
sudo apt install package-name
15. Logs and Monitoring Issues
15.1 Log Files Not Updating
Symptoms:
● /var/log/syslog or /var/log/messages is empty or not updating.
Solution:
Restart the logging service:
sudo systemctl restart rsyslog
Check disk space:
df -h
Verify SELinux settings:
sudo ausearch -m AVC
15.2 How to Monitor System Logs in Real-Time
Solution:
Use tail to follow log updates:
tail -f /var/log/syslog
Use journalctl for systemd logs:
journalctl -f
16. SSH Issues
16.1 SSH Connection Timeout
Symptoms:
● SSH session freezes or disconnects after inactivity.
Solution:
Edit SSH config:
sudo nano /etc/ssh/sshd_config
Set ClientAliveInterval and ClientAliveCountMax:
plaintext
ClientAliveInterval 60
ClientAliveCountMax 5
Restart SSH service:
sudo systemctl restart sshd
16.2 Cannot SSH as Root
Symptoms:
● Permission denied when connecting as root.
Solution:
Enable root login in SSH config:
sudo nano /etc/ssh/sshd_config
Set:
plaintext
PermitRootLogin yes
Restart SSH:
sudo systemctl restart sshd
17. Printer and Device Issues
17.1 Printer Not Working
Symptoms:
● Printer not detected.
Solution:
Restart the CUPS service:
sudo systemctl restart cups
Check printer connections:
lpstat -p
18. User Management Issues
18.1 User Cannot Run sudo Commands
Symptoms:
● user is not in the sudoers file error.
Solution:
Add user to the sudo group:
sudo usermod -aG sudo username
18.2 User Cannot Write to a Directory
Symptoms:
● Permission denied error.
Solution:
Change ownership:
sudo chown username:groupname /path/to/directory
Set proper permissions:
sudo chmod 755 /path/to/directory
19. Software and Application Issues
19.1 Application Crashes Unexpectedly
Solution:
Check application logs:
journalctl -xe | grep application-name
1. Run application in debug mode if available.
19.2 Web Server Not Working
Symptoms:
● 502 Bad Gateway or 503 Service Unavailable.
Solution:
Restart the web server:
sudo systemctl restart apache2 # For Apache
sudo systemctl restart nginx # For Nginx
20. Advanced Security Issues
20.1 Detecting Unauthorized Root Access
Solution:
Check recent root logins:
sudo last -n 10
Check for unauthorized sudo commands:
cat /var/log/auth.log | grep sudo
20.2 Blocking Brute-Force Attacks
Solution:
Install Fail2Ban:
sudo apt install fail2ban
Enable SSH protection:
sudo systemctl enable fail2ban
21. Disk and Storage Issues
21.1 Unable to Mount a Disk
Symptoms:
● mount: unknown filesystem type error
● Disk not visible in lsblk or fdisk -l
Solution:
Identify the disk:
lsblk
Check filesystem type:
sudo blkid /dev/sdX
Manually mount the disk:
sudo mount /dev/sdX /mnt
If the filesystem is corrupted, repair it:
sudo fsck -y /dev/sdX
21.2 LVM Volume Not Found
Symptoms:
● lvdisplay does not show expected volumes
Solution:
Scan for missing volumes:
sudo vgscan --mknodes
sudo lvscan
Activate volume group:
sudo vgchange -ay
Mount the logical volume:
sudo mount /dev/mapper/vgname-lvname /mnt
22. Kernel and Driver Issues
22.1 Missing Graphics Drivers
Symptoms:
● GUI does not load
● NVIDIA driver not found error
Solution:
Identify the graphics card:
lspci | grep -i vga
Install the correct driver:
sudo ubuntu-drivers autoinstall
Reboot the system:
sudo reboot
22.2 Sound Not Working
Symptoms:
● No audio output
● No sound device found error
Solution:
Check audio devices:
aplay -l
Restart PulseAudio:
pulseaudio --kill && pulseaudio --start
Unmute sound:
amixer set Master unmute
23. Network Troubleshooting
23.1 Wi-Fi Not Connecting
Symptoms:
● No Wi-Fi adapter found error
Solution:
List available network interfaces:
nmcli device
Restart NetworkManager:
sudo systemctl restart NetworkManager
If the Wi-Fi adapter is missing, install drivers:
sudo apt install firmware-iwlwifi
23.2 IP Address Not Assigned
Symptoms:
● ip a does not show an IP address
Solution:
Restart DHCP client:
sudo dhclient -r && sudo dhclient
Manually assign an IP address:
sudo ip addr add 192.168.1.100/24 dev eth0
24. Performance Optimization
24.1 System Running Slow
Symptoms:
● High load average
● Applications freezing
Solution:
Check system load:
top
Kill high CPU usage processes:
kill -9 <PID>
Clear cached memory:
sudo sync && sudo sysctl -w vm.drop_caches=3
24.2 High Memory Usage
Symptoms:
● Out of memory errors
Solution:
Check memory usage:
free -h
Enable swap:
sudo fallocate -l 2G /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
25. Backup and Recovery
25.1 Restoring Deleted Files
Symptoms:
● Important files deleted accidentally
Solution:
Recover files from the trash:
ls ~/.local/share/Trash/files/
If permanently deleted, use extundelete:
sudo extundelete /dev/sdX --restore-all
25.2 Creating a Full System Backup
Solution:
Use rsync for backup:
sudo rsync -av --progress /home/ /backup/home/
Restore files when needed:
sudo rsync -av /backup/home/ /home/
26. Advanced Security
26.1 Encrypting a Directory
Solution:
Install ecryptfs-utils:
sudo apt install ecryptfs-utils
Encrypt a directory:
ecryptfs-mount-private
26.2 Setting Up Automatic Security Updates
Solution:
Install unattended-upgrades:
sudo apt install unattended-upgrades
Enable automatic updates:
sudo dpkg-reconfigure unattended-upgrades
27. Miscellaneous Troubleshooting
27.1 How to Find Which Process is Using a Port
Solution:
Use netstat:
sudo netstat -tulnp | grep :<port>
Use lsof:
sudo lsof -i :<port>
Kill the process if needed:
sudo kill -9 <PID>
27.2 How to Check System Uptime
Solution:
Display uptime:
uptime
View uptime in human-readable format:
who -b
28. Debugging and Log Analysis
28.1 Checking Kernel Logs for Errors
Solution:
View logs in real-time:
dmesg -w
Filter errors:
dmesg | grep -i error
28.2 Finding Recently Modified Files
Solution:
Search for files modified in the last 24 hours:
find / -type f -mtime -1
29. Essential Linux Commands
29.1 Finding a File by Name
Solution:
Use find:
find / -name "filename"
29.2 Checking Disk Usage per Directory
Solution:
Use du to find large directories:
du -ah / | sort -rh | head -10
30. Common Errors and Fixes
30.1 Fixing command not found for Common Commands
Solution:
Refresh the PATH:
export PATH=$PATH:/usr/local/bin
Reinstall the package:
sudo apt install package-name
30.2 Fixing permission denied When Running a Script
Solution:
Make the script executable:
chmod +x script.sh
Run the script with sudo if needed:
sudo ./script.sh
31. Advanced File System Issues
31.1 Filesystem Goes into Read-Only Mode
Symptoms:
● Unable to write files
● dmesg shows EXT4-fs error
Solution:
Check disk errors:
dmesg | tail -50
Remount the filesystem:
sudo mount -o remount,rw /
Run filesystem check:
sudo fsck -y /dev/sdX
31.2 Recovering Deleted Files from EXT4 Filesystem
Solution:
Unmount the affected partition:
sudo umount /dev/sdX
Use extundelete:
sudo extundelete /dev/sdX --restore-all
32. Advanced Networking Issues
32.1 Identifying Slow Network Connections
Symptoms:
● Websites load slowly
● High latency on SSH
Solution:
Test network speed:
speedtest-cli
Identify packet loss:
ping -c 10 google.com
Check for dropped packets:
sudo netstat -s | grep "dropped"
32.2 Troubleshooting DNS Issues
Symptoms:
● ping google.com fails, but ping 8.8.8.8 works
Solution:
Check DNS settings:
cat /etc/resolv.conf
Change to Google DNS:
echo "nameserver 8.8.8.8" | sudo tee /etc/resolv.conf
33. Docker and Container Issues
33.1 Docker Container Won't Start
Symptoms:
● docker run fails with an error
Solution:
Check container logs:
docker logs container_id
Restart Docker:
sudo systemctl restart docker
Prune unused containers:
docker system prune -a
33.2 Docker Container Can't Access the Internet
Solution:
Restart the Docker network:
sudo systemctl restart docker
Add network to the container:
docker network connect bridge container_id
34. Kernel Panic and Boot Issues
34.1 Kernel Panic After Update
Symptoms:
● System does not boot after an update
Solution:
1. Boot into an older kernel from GRUB.
Reinstall the latest kernel:
sudo apt install --reinstall linux-image-$(uname -r)
34.2 System Stuck at GRUB Rescue Mode
Solution:
Identify the boot partition:
ls
Load the correct partition:
set prefix=(hd0,gpt2)/boot/grub
set root=(hd0,gpt2)
insmod normal
normal
35. High Availability and Load Balancing Issues
35.1 Load Balancer Not Distributing Traffic Evenly
Solution:
Check Nginx load balancer configuration:
sudo nano /etc/nginx/nginx.conf
Verify upstream servers:
sudo systemctl restart nginx
36. Virtualization and Cloud Issues
36.1 VM is Running Slowly
Solution:
Enable CPU virtualization:
egrep -c '(vmx|svm)' /proc/cpuinfo
1. Increase allocated RAM and CPU in the hypervisor.
36.2 Cloud Instance Fails to Boot
Solution:
1. Check the instance console logs.
2. Attach the disk to another instance and repair the OS files.
37. SELinux and AppArmor Issues
37.1 Application Blocked by SELinux
Symptoms:
● Permission denied errors even with correct permissions
Solution:
Check SELinux logs:
sudo ausearch -m AVC
Set the correct SELinux policy:
sudo semanage fcontext -a -t httpd_sys_content_t
"/var/www/html(/.*)?"
sudo restorecon -Rv /var/www/html
37.2 AppArmor Blocking a Service
Solution:
Check AppArmor logs:
sudo journalctl -xe | grep apparmor
Set AppArmor profile to complain mode:
sudo aa-complain /usr/sbin/nginx
38. System Recovery and Rescue Mode
38.1 How to Boot into Single User Mode
Solution:
1. Edit the GRUB boot entry.
2. Add single or init=/bin/ to the kernel parameters.
38.2 Resetting Root Password from Recovery Mode
Solution:
1. Boot into recovery mode.
Mount the root filesystem:
mount -o remount,rw /
Reset password:
passwd root
39. Advanced Performance Tuning
39.1 Identifying Bottlenecks with iostat
Solution:
Install sysstat:
sudo apt install sysstat
Check disk I/O performance:
iostat -x 1 5
39.2 Reducing Swap Usage
Solution:
Check current swap usage:
swapon -s
Lower the swappiness value:
sudo sysctl vm.swappiness=10
40. Debugging and Profiling Applications
40.1 Debugging a Segmentation Fault
Solution:
Run the application with gdb:
gdb ./application
run
40.2 Profiling CPU Usage with perf
Solution:
Install perf:
sudo apt install linux-tools-common
1. Profile CPU usage:
sudo perf top
