{"id":6211,"date":"2020-11-09T19:27:50","date_gmt":"2020-11-09T19:27:50","guid":{"rendered":"https:\/\/www.blopig.com\/blog\/?p=6211"},"modified":"2020-11-10T16:11:35","modified_gmt":"2020-11-10T16:11:35","slug":"building-more-useful-bash-scripts","status":"publish","type":"post","link":"https:\/\/www.blopig.com\/blog\/2020\/11\/building-more-useful-bash-scripts\/","title":{"rendered":"Building More Useful Bash Scripts"},"content":{"rendered":"\n

If there is some set of keystrokes that you find yourself using day in, day out, you can almost certainly write something that can automate the process. This could be something simple that could be achieved by putting a bash alias somewhere in your ~\/.bashrc<\/code>:<\/p>\n\n\n\n\n\n\n\n

echo \"alias ls='ls -lh --color=auto'\" >> ~\/.bashrc<\/pre>\n\n\n\n
But what if you’d like to be able to give arguments or use the output of commands? In that case, a bash script is usually the way to go. Anything placed in a directory that appears in the PATH environment variable, with the correct permissions, can be run from the command line:<\/p>\n\n\n\n
$ echo $PATH\n\/usr\/bin:~\/local\/bin\n\n$ echo \"echo \\\"Hello world.\\\"\" > ~\/local\/bin\/hello_world\n$ hello_world\nbash: ~\/local\/bin\/hello_world: Permission denied\n\n$ chmod +x ~\/local\/bin\/hello_world\n$ hello_world\nHello world.<\/pre>\n\n\n\n
The last command (chmod +x<\/code>) gives the current user permission to treat the file as live code rather than just reading and writing from and to it like a text file.<\/p>\n\n\n\n
This is not a particularly useful script; for that we can take arguments passed after the name of the script. Placing this script named add_numbers<\/code> in a PATH-accessible directory and giving it the appropriate permissions, we can perform some simple arithmetic on the inputs:<\/p>\n\n\n\n
echo $(($1 + $2))<\/pre>\n\n\n\n
There are no prizes for guessing the result:<\/p>\n\n\n\n
$ add_numbers 1 3\n4<\/pre>\n\n\n\n
The nth argument to the script is denoted $n, and indexing begins at 1. $(( ... ))<\/code> is the expand arithmetic operator, and echo<\/code> prints the result. One use of arguments that I use a lot is in checking in on Slurm jobs. The following script takes a single argument – the Slurm job ID – prints live updates of the most recent 40 lines of the job output:<\/p>\n\n\n\n
watch -n -0.1 \"tail -n 40 ~\/slurm_logs\/slurm_$1.out\"<\/pre>\n\n\n\n
It may be the case that the console output of a command is useful to us, and we can capture, process and use this in subsequent commands. In the following script, the single argument is the job file; passing this filename to sbatch<\/code> submits the job to Slurm, and prints information about the associated job ID to the console. The job ID can be used to monitor the progress of the job – all of this uses a single command and a single argument:<\/p>\n\n\n\n
OUTPUT=$(sbatch $1)\nJOBID=$(echo \"$OUTPUT\" | awk -F ' ' '{print $4}')\nwatch -n 0.1 \"tail ~\/slurm_logs\/slurm_$JOBID.out -n 50\"<\/pre>\n\n\n\n
If we have a file called slurm_job.sh, running sbatch<\/code> will give the following output:<\/p>\n\n\n\n
$ sbatch slurm_job.sh\nSubmitted batch job 831489<\/pre>\n\n\n\n
In our script, the string Submitted batch job 831489<\/code> is captured in the OUTPUT<\/code> variable (line 1); we then use awk<\/a><\/code> to split this string with the delimiter argument (-F<\/code>) set to [space], and ‘print’ the 4th and final item (a six digit job ID) into the JOBID<\/code> variable (line 2). This can be used in the final line to observe the progress of the job via its Slurm log.<\/p>\n\n\n\n
These are simple examples, but I try to automate any command I use more than a few times per day in this manner. Bash scripting is extremely powerful, and learning the basics<\/a> can save a lot of time in the long run.<\/p>\n","protected":false},"excerpt":{"rendered":"
If there is some set of keystrokes that you find yourself using day in, day out, you can almost certainly write something that can automate the process. This could be something simple that could be achieved by putting a bash alias somewhere in your ~\/.bashrc:<\/p>\n","protected":false},"author":61,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"nf_dc_page":"","wikipediapreview_detectlinks":true,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"ngg_post_thumbnail":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[29,296,14],"tags":[344,345,346],"ppma_author":[541],"class_list":["post-6211","post","type-post","status-publish","format-standard","hentry","category-code","category-hints-and-tips","category-howto","tag-bash","tag-tips","tag-utility"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"authors":[{"term_id":541,"user_id":61,"is_guest":0,"slug":"jack","display_name":"Jack Scantlebury","avatar_url":"https:\/\/secure.gravatar.com\/avatar\/2d30962dbe9d08db0ac110abbf9ffd5bd52f4eb7da79636d286fb584280feb2c?s=96&d=mm&r=g","0":null,"1":"","2":"","3":"","4":"","5":"","6":"","7":"","8":""}],"_links":{"self":[{"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/posts\/6211","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/users\/61"}],"replies":[{"embeddable":true,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/comments?post=6211"}],"version-history":[{"count":5,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/posts\/6211\/revisions"}],"predecessor-version":[{"id":6234,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/posts\/6211\/revisions\/6234"}],"wp:attachment":[{"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/media?parent=6211"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/categories?post=6211"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/tags?post=6211"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.blopig.com\/blog\/wp-json\/wp\/v2\/ppma_author?post=6211"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}