This article was published on September 29th 2014 and takes about 5 minutes to read.
Use it with caution — it is probably still valid, but it has not been updated for over a year.
5 bash tips useful for scripted provisioning
Setting up servers with a shell script is a great way to reduce errors, spare yourself from having to carry out the same steps over and over again and provision a machine in minutes.
Table of contents
You don't have to necessarily use a provisioning tool like Puppet or Chef to provision a server. To get started, a shell script is enough to promote your machine from a base installation to a full-fledged box.
But you have to know how to carry out a few common tasks that are crucial for doing system administration tasks from inside a script.
Although this article is written with bash in mind, these tips will also work with other shells with small modifications.
Modifying a configuration file
Modifying a configuration file is one of the most common tasks needed in a provisioning script. It can easily be done using the
As an example, you may want to prohibit logins as root via SSH:
sed -i 's/^#\?PermitRootLogin \(no\|yes\)/PermitRootLogin no/' /etc/ssh/sshd_config /etc/init.d/sshd restart
Keep in mind that while
sed works with regular expressions, you have to escape special characters (in this example
| and the braces) with a backslash. It's also worth noting that in order to replace a whole line, you have to adjust your regular expression accordingly.
In any case you should test your command carefully before using it in a script (if you run
sed without the
-i flag, it prints the modified output to the screen which can be of help).
There are quite a few different versions of
sed and depending on your environment, the above command may or may not work without modifications (its syntax works on CentOS 6.5 but does not on Mac OS X for example).
sed is also a very powerful command capable of a whole lot more than just small modifications like this one. People have written books about it but as always, the man page is a good starting point to dig deeper.
Reloading the environment
If you want to reload the environment for the user which runs the current script, you can do so by sourcing the corresponding configuration file (
source ~/.bash_profile or
Maybe you have installed rbenv and want to reload your environment so that rbenv is initialized:
git clone https://github.com/sstephenson/rbenv.git ~/.rbenv echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile echo 'eval "$(rbenv init -)"' >> ~/.bash_profile source ~/.bash_profile
If you are not sure which file to load (or even which file to put your configuration directives in), take a look at the chapter Bash startup files in the Bash reference manual.
Running a series of commands as another user
Using a combination of the
su command and bash's heredoc syntax, you can easily run a series of commands as another user:
su - app <<END cd git clone https://github.com/sstephenson/rbenv.git ~/.rbenv git clone https://github.com/sstephenson/ruby-build.git ~/.rbenv/plugins/ruby-build echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile echo 'eval "$(rbenv init -)"' >> ~/.bash_profile source ~/.bash_profile rbenv install 2.1.2 rbenv global 2.1.2 gem install bundler END
If you want to print a literal
$ inside a heredoc, you have to escape it with a backslash (
\$) to prevent variable expansion.
In combination with the
cat command, heredocs can also be used to write whole configuration files at once:
cat > /etc/profile.d/rails.sh <<END # Set a system-wide Rails environment export RAILS_ENV=production END
However, using this technique things can get messy when your files are big or if you need to write a lot of them. In this case it may be worth considering to create the files beforehand and simply copy them over their original using
Printing each command in a script before execution
If your script takes some time to execute or you use a lot of commands which do not generate output (
mv for example), then it is useful to have each command printed to screen (or wherever you redirect the output to) before it is executed.
You can achieve this by setting
set -x at the start of your script. If you want this detailed output just for a section of your script, you can turn this behaviour off with
set command has a bunch of other useful switches which are explained in its man page.
Writing a log file while still having output on screen
To redirect all output of a script (including errors printed to
STDERR) to a file, you have to first redirect
sh script.sh 2>&1 > script.log
This way, all of the script's output is captured in the file
script.log. However, when you do this, you will get no output on screen.
You can use the
tee command to capture ouput in a file while still having output sent to
sh script.sh 2>&1 | tee script.log
A pipe only reads from
STDOUT in any case, so leave the redirection of
STDERR in place. If you want to append to the file instead of replacing it, use
tee -a script.log.
Get in the loop
Join my email list to get new articles delivered straight to your inbox and discounts on my products.
No spam — guaranteed.
Got it, thanks a lot!
Please check your emails for the confirmation request I just sent you. Once you clicked the link therein, you will no longer see these signup forms.