New PCS node post-install tasks

jesse's picture

Adding a new hardware node to a PCS cluster is really easy, as the PCS installer does most of the work. There are a few things that need to be done manually though, and this is my standard post-install task list.

Software Updates

For security and to run consistent kernel versions with other nodes which are already up-to-date.

yum update -y
reboot

Hostname into /etc/hosts

The PCS installer doesn't set this up, but things can sure slow down when DNS is down without this:

echo `ifconfig em1 | grep 'inet addr:' | cut -d: -f2 | cut -d' ' -f1` `hostname -f` `hostname -s` >> /etc/hosts

NTP / Time synchronization

All hardware nodes should use ntp to sync their time, preferably to the same time source. If needed, add your own servers to /etc/ntp.conf.

chkconfig ntpdate on
service ntpdate start
chkconfig ntpd on
service ntpd start
hwclock --systohc --utc

Virtual Networks

If you use Virtual Networks on your hardware node, set those up. The PCS installer creates one called Bridged; I rename that to FrontNet and create a BackNet:

vznetcfg net change Bridged FrontNet
vznetcfg net new BackNet
vznetcfg net addif BackNet em2

Sometimes the installer places em2 on Bridged, if that happens it looks more like:

vznetcfg net change Bridged FrontNet
vznetcfg net new BackNet
vznetcfg net delif em2
vznetcfg net addif FrontNet em1
vznetcfg net addif BackNet em2

You can also use PVA to setup Virtual Networks if you prefer that.

Syslog

Send a copy of your logs through your central logging system. Edit /etc/rsyslog.conf and change the default

# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
#$WorkDirectory /var/lib/rsyslog # where to place spool files
#$ActionQueueFileName fwdRule1 # unique name prefix for spool files
#$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
#$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
#$ActionQueueType LinkedList   # run asynchronously
#$ActionResumeRetryCount -1    # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# ### end of the forwarding rule ###

To something like this (I'm using the hostname logs for the logging server):

### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
$WorkDirectory /var/lib/rsyslog # where to place spool files
$ActionQueueFileName fwdRule1 # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList   # run asynchronously
$ActionResumeRetryCount -1    # infinite retries if host is down
# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
*.* @@logs:514
### end of the forwarding rule ###

Utilities

I like to have locate available to quickly find files, but don't want it indexing everything under /pstorage. sharutils is handy for pasting files correctly through the terminal. strace and tcpdump come installed already.

yum install mlocate sharutils -y
sed -i 's/^PRUNEPATHS = "[^"]*/& \/pstorage/' /etc/updatedb.conf 
updatedb

And that's it for a standard setup. Anything I'm missing?

Tags: 

Add new comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.