Archive

Archive for November, 2011

pfSense: Why can’t I access the latest information of a log file with tail, grep, cat or vi?

November 24th, 2011 Matthias Pölzinger No comments

Problem description:

You want to debug the logfiles of a pfSense firewall, but tail, grep, cat or vi do not provide the latest information and always ends with a “CLOG@?|?”:

# cd /var/log/
# tail -10l system.log 
Nov 24 15:33:14 pfsense99 sshd[45576]: error: PAM: authentication error for root from 192.168.210.198
Nov 24 15:33:14 pfsense99 last message repeated 2 times
Nov 24 15:33:14 pfsense99 sshd[45576]: Failed keyboard-interactive/pam for root from 192.168.210.198 port 47927 ssh2
Nov 24 15:33:14 pfsense99 sshd[45576]: Failed password for root from 192.168.210.198 port 47927 ssh2
Nov 24 15:33:14 pfsense99 last message repeated 2 times
Nov 24 15:33:14 pfsense99 sshd[45576]: Failed password for root from 192.168.210.198 port 47927 ssh2
Nov 24 15:34:18 pfsense99 sshd[45741]: error: PAM: authentication error for root from 192.168.210.198
Nov 24 15:34:18 pfsense99 sshd[45741]: error: PAM: authentication error for root from 192.168.210.198
Nov 24 15:34:18 pfsense99 last message repeated 2 times
Nov 24 15:34:18 pfsense99 sshd[45741]: Failed keyboard-interactive/pam for root fromCLOG@?|?# 
# 
# date
Thu Nov 24 19:15:38 CET 2011
#

 
Problem resolution:
pfSense writes its log information in a circular log format in order to keep a constant size. This prevents filesystem fillup, but also restricts you from using your standard shell tools to access the data inside these logfiles. Instead of using tail, grep, cat or vi directly, you should first access the log file with a command called “clog”:

# clog system.log 
2
Nov 24 12:59:08 pfsense99 sshd[20315]: error: PAM: authentication error for root from 192.168.210.198
...
...
...
Nov 24 19:15:42 pfsense99 sshd[20501]: Accepted publickey for root from 192.168.210.198 port 38451 ssh2
Nov 24 19:15:42 pfsense99 sshd[20503]: Accepted publickey for root from 192.168.210.198 port 38452 ssh2
#

 
clog also provides a follow option like tail:

# clog -f system.log 
...
...
...
Nov 24 19:15:42 pfsense99 sshd[20501]: Accepted publickey for root from 192.168.210.198 port 38451 ssh2
Nov 24 19:15:42 pfsense99 sshd[20503]: Accepted publickey for root from 192.168.210.198 port 38452 ssh2

 
If you want to grep for specific messages, just add a pipe and a grep:

# clog system.log | grep "keyboard-interactive"
Nov 24 19:15:09 pfsense99 sshd[16616]: Accepted keyboard-interactive/pam for root from 192.168.210.198 port 45104 ssh2
#

 

Categories: Uncategorized Tags:

Red Hat: How to check and repair your root-filesystem in rescue mode

November 24th, 2011 Matthias Pölzinger No comments

Problem description:

Your Linux server had some troubles with it’s underlying storage devices and put all your filesystems into read only mode. You corrected the issue and are booting up your Linux server. The boot process determines some problems with your root filesystems and requires a manual intervention by logging in with the root password. Unfortunately your are unable to login because the root password is not accepted (although you are using the correct one for the 25th time ;-) ).

You are unable to boot because of filesystem issues. At the same time your are unable to fix the filesystem, because your root password is not accepted.

 
Problem resolution:

Take a Linux installation media of your distribution and boot into linux rescue mode by typing the following command when asked for boot options:

linux rescue



 
Select your language and keyboard layout:

 



 
Network configuration is not necessary and can be skipped:

 


 
A very important step is to SKIP the mount of your Linux installation. Do not use MOUNT or READ-ONLY. For a filesystem check it is required that the filesystem is unmounted. Once mounted it is nearly unpossible to unmount your root-filesystem in rescue mode.

 


 
If you are not using a Logical Volume Manager for your filesystems you can directly jump to execute the “fsck” command. Otherwise you will have to first scan for your Physical Volumes and review them:

lvm pvscan
lvm pvdisplay

 


 
If your Physical Volumes were listed correctly continue to review the detected Volume Groups:

lvm vgdisplay

 
Now continue to activate your Volume Groups in order to create your Logical Volume devices in the “/dev” filesystem. In this example a Volume Group with the name “VolGroup00″ has to be activated:

lvm vgchange -a y VolGroup00

 


 
After activating your Volume Group(s), you will be able to perform a filesystem check and correct problems:

fsck /dev/VolGroup00/LogVol00

 


 
Once the filesystem check finishes successfully, you should be able to reboot without any further complication.

 

Categories: Uncategorized Tags: