How to Debug Bash Scripts

Bash is the default scripting language in most Linux systems. Its usage ranges from an interactive command interpreter to a scripting language for writing complex programs. Debugging facilities are a standard feature of compilers and interpreters, and bash is no different in this regard. In this article, I will explain various techniques and tips for debugging Bash scripts.

Tracing script execution

You can instruct Bash to print debugging output as it interprets you scripts. When running in this mode, Bash prints commands and their arguments before they are executed.

To see how this works, let's try it on an example script. The following simple script greets the user and prints the current date:

#!/bin/bash
echo "Hello $USER,"
echo "Today is $(date +'%Y-%m-%d')"

To trace the execution of the script, use bash -x to run it:

$ bash -x example_script.sh
+ echo 'Hello ayman,'
Hello ayman,
++ date +%Y-%m-%d
+ echo 'Today is 2009-08-24'
Today is 2009-08-24

In this mode, Bash prints each command (with its expanded arguments) before executing it. Debugging output is prefixed with a number of + signs to indicate nesting. This output helps you see exactly what the script is doing, and understand why it is not behaving as expected.

Adding line numbers to tracing output

In large scripts, it may be helpful to prefix this debugging output with the script name, line number and function name. You can do this by setting the following environment variable:

export PS4='+${BASH_SOURCE}:${LINENO}:${FUNCNAME[0]}: '

Let's trace our example script again to see the new debugging output:

$ bash -x example_script.sh
+example_script.sh:2:: echo 'Hello ayman,'
Hello ayman,
++example_script.sh:3:: date +%Y-%m-%d
+example_script.sh:3:: echo 'Today is 2009-08-24'
Today is 2009-08-24

Tracing part of a script

Sometimes, you are only interested in tracing one part of your script. This can be done by calling set -x where you want to enable tracing, and calling set +x to disable it. Let's apply this to our example script:

#!/bin/bash
echo "Hello $USER,"
set -x
echo "Today is $(date %Y-%m-%d)"
set +x

Now, let's run the script:

$ ./example_script.sh
Hello ayman,
++example_script.sh:4:: date +%Y-%m-%d
+example_script.sh:4:: echo 'Today is 2009-08-24'
Today is 2009-08-24
+example_script.sh:5:: set +x

Notice that we no longer need to run the script with bash -x.

Logging

Tracing script execution is sometimes too verbose, especially if you are only interested in a limited number of events, like calling a certain function or entering a certain loop. In this case, it's better to log the events you are interested in. Logging can be achieved with something as simple as a function that prints a string to stderr:

_log() {
  if [ "$_DEBUG" == "true" ]; then
    echo 1>&2 "$@"
  fi
}

Now you can embed logging messages into your script by calling this function:

_log "Copying files..."
cp src/* dst/

Log messages are printed only if the _DEBUG variable is set to true. This allows you to toggle the printing of log messages depending on your needs. You don't need to modify your script in order to change this variable; you can set it on the command line:

$ _DEBUG=true ./example_script.sh

Using the Bash debugger

If you are writing a complex script and you need a full-fledged debugger to debug it, then you can use bashdb, the Bash debugger. The debugger contains all the features that you would expect, like breakpoints, stepping in and out of functions, and attaching to running scripts. Its interface is a bit similar to gdb. You can read the documentation of bashdb for more information.

Comments

Dennis Roberts
Dennis Roberts's gravatar

I liked your write up. I have been using Linux for the last 10 years and not a day goes by that I don't learn something new. The PS4 is new to me. I also like your log function.


Posted at 5:20 a.m. on August 25, 2009

Anonymous
Anonymous's gravatar

Today is 2009-08-25


Posted at 8:30 a.m. on August 25, 2009

Anonymous
Anonymous's gravatar

Excellent - thanks!

  • Andre.

Posted at 9:19 a.m. on August 25, 2009

Abdul Fattah
Abdul Fattah's gravatar

Thanks Ayman, very useful tips :)


Posted at 10:48 a.m. on August 25, 2009

Anonymous
Anonymous's gravatar

I'd highly recommend all your bash scripts start off by:

set -e

This will abort the rest of the script if any of the commands you run return non-0. This will help find and avoid what would have otherwise been silent lurking problems.


Posted at 11:12 a.m. on August 25, 2009

decasm
decasm's gravatar

The logging function you have there is good, but for larger projects, log4sh might be best.


Posted at 1:19 p.m. on August 25, 2009

Drew
Drew's gravatar

Vivid articles. thinks


Posted at 2:08 p.m. on August 25, 2009

Marko
Marko's gravatar

Hey Ayman, thanks a lot! I'm still fairly new to bash scripting and there some really nice tips here...


Posted at 6:17 p.m. on August 25, 2009

omid8bimo
omid8bimo's gravatar

cool notes dude :) pretty useful


Posted at 6:57 p.m. on August 25, 2009

Ayman Hourieh
Ayman Hourieh's gravatar

Thanks for your comment. Glad to know that you found it useful.


Posted at 7:09 p.m. on August 25, 2009

Ayman Hourieh
Ayman Hourieh's gravatar

"set -o x" is the short form of "set -o xtrace".


Posted at 7:11 p.m. on August 25, 2009

Thell
Thell's gravatar

Your sharing this info is greatly appreciated.

That PS4 was just what the doctor ordered.

It has been added to:: http://mywiki.wooledge.org/BashGuide/Practices/Debugging


Posted at 4:47 p.m. on September 3, 2009

missnoerrors
missnoerrors's gravatar

Useful and concise info and comments. Thanks! set -e is what I was looking for to control build script, now it'll get a logging upgrade too!


Posted at 4:58 a.m. on October 3, 2009

Polprav
Polprav's gravatar

Hello from Russia! Can I quote a post in your blog with the link to you?


Posted at 2:06 p.m. on October 20, 2009

Stefan
Stefan's gravatar

Thanks for the helpful suggestions on tracing!

Another cool feature of bash is to redirect the trace output to another file than stderr. That is very helpful for keeping traces completely separate from all the other output:

exec 9>"$__LOG__DIR__/trace"
BASH_XTRACEFD=9

Posted at 9:57 a.m. on February 15, 2011

Rich
Rich's gravatar

You can also enable trace in the hashbang, ie #!/bin/bash +x


Posted at 6:42 p.m. on March 22, 2011

Felix Rabe
Felix Rabe's gravatar

As I have set -o nounset in my script, I had to change PS4 slightly: (note the '-')

export PS4='+${BASH_SOURCE}:${LINENO}:${FUNCNAME[0]-}: '

Posted at 12:36 p.m. on April 14, 2011

Jon
Jon's gravatar

Thank you, Ayman! Very helpful and nice succinct examples.


Posted at 9:46 p.m. on August 5, 2011

GuruM
GuruM's gravatar

@Thell - link is broken. Updated link is http://mywiki.wooledge.org/BashGuide/Practices#Debugging


Posted at 8:22 a.m. on September 6, 2011

avkosinsky
avkosinsky's gravatar

Debugger for Bash version 3(Bourne again shell). Plugin for Eclipse. http://sourceforge.net/projects/basheclipse/


Posted at 9:05 a.m. on October 1, 2011

Ryszard
Ryszard's gravatar

Brilliant!

With PS4 trick I was able to prepare test coverage report for bash. In below report lines prefixed by RED were not executed, and prefixed by OK were executed.

RED:1 #/bin/bash <br />
RED:2 if [ -f ${BASH_SOURCE}.trace ]; then rm ${BASH_SOURCE}.trace; fi
RED:3 exec 4>&2
RED:4 exec 2> >(tee -a ${BASH_SOURCE}.trace | grep -v "^+")
RED:5 exec 3>&1 > >(tee -a ${BASH_SOURCE}.trace)
RED:6 export PS4='+${BASH_SOURCE}:${LINENO}:${FUNCNAME[0]-}:'
RED:7 set -o xtrace
RED:8 
OK :9 x=1
OK :10 echo $x
OK :11 if [ $x -gt 1 ]; then
RED:12         echo big
RED:13 else
OK :14         echo not so big
RED:15 fi
OK :16 y=2;z=1
OK :17 echo $y
OK :18 if [ $y -gt 1 ]; then
OK :19         echo big
RED:20 else
RED:21         echo not so big
RED:22 fi

The report was generated by this script:

cat >coverage.sh <<EOF
#/bin/bash
if [ -f \${BASH_SOURCE}.trace ]; then rm \${BASH_SOURCE}.trace; fi
exec 4>&2
exec 2> >(tee -a \${BASH_SOURCE}.trace | grep -v "^+")
exec 3>&1 > >(tee -a \${BASH_SOURCE}.trace)
export PS4='+\${BASH_SOURCE}:\${LINENO}:\${FUNCNAME[0]-}:'
set -o xtrace

x=1
echo \$x
if [ \$x -gt 1 ]; then
        echo big
else
        echo not so big
fi
y=2;z=1
echo \$y
if [ \$y -gt 1 ]; then
        echo big
else
        echo not so big
fi
EOF
bash coverage.sh
cat coverage.sh.trace | grep '^+coverage.sh:' | cut -d: -f2 | sed 's/^/^/g' >                 coverage.sh.executed
cat coverage.sh |  nl -i 1 -s ' ' -b a | sed 's/^[ ]*//g' | grep -v -f coverage.sh.executed | sed 's/^/RED:/g' > coverage.sh.coverage
cat coverage.sh |  nl -i 1 -s ' ' -b a | sed 's/^[ ]*//g' | grep -f coverage.sh.executed | sed 's/^/OK :/g' >> coverage.sh.coverage
cat coverage.sh.coverage | sort -g -k2 -t :

rm coverage.sh.trace
rm coverage.sh.executed

-Ryszard


Posted at 8:23 p.m. on October 25, 2011

Ray
Ray's gravatar

This is an example of a well written document -- short, simple, to the point, full of examples, and concentrating on what a beginner to bash debugging really wants to know. The Linux/Unix/GNU world is overflowing with huge documents that seem to almost enjoy making it hard to find what you want. Ayman shows how it should be done.


Posted at 5:54 p.m. on November 23, 2011

Bithin A
Bithin A's gravatar

Thanks you :). Your tips are really useful.


Posted at 10:42 a.m. on December 11, 2011

Post a comment

HTML is not allowed. You can use markdown syntax to format your comment.