If there is some set of keystrokes that you find yourself using day in, day out, you can almost certainly write something that can automate the process. This could be something simple that could be achieved by putting a bash alias somewhere in your ~/.bashrc:
Category Archives: How To
Improving your Python code quality using git pre-commit hooks
Intro
I recently completed an internship during which I spent a considerable amount of time doing software engineering. One of my main take-aways from this experience was that in industry, a lot more attention is spent on ensuring that code committed to a GitHub repo is clean and bug-free.
This is achieved through several means like code review (get other people to read your code), test-driven development (make sure your code works as you are adding functionality) or paired development (have two people work together on the same piece of code). Here, I will instead focus on a useful tool that is easy to integrate into your existing git workflow: Pre-commit hooks.
Continue readingFrom Jupyter to Slides using RISE
In preparation for remote teaching this year, I’ve spent the last few weeks converting the Doctoral Training Centre’s ‘Introduction to Computer Programming’ course into a series of Jupyter notebooks so that the course can be run entirely using Google Colaboratory.
Continue readingHow to SCP files from a gated server to your personal computer
Jack recently made a blog post in which he provided a script which can transfer your files between your personal computer and a given remote machine via temporarily hosting a file on file.io (blog post here); where you have some sensitive data that you do not want to risk hosting online, you can also fairly easily use SCP to keep business strictly between your local and remote machine.
What I am referring to is described here. This blog post refers to the case where you want to SCP from local host to a remote machine which is only accessible via a gate server (this isn’t necessarily true for the Stats computers as we can use the VPN to directly access our remote machine of choice by the way). I won’t effectively plagiarise the blog post I linked to as the explanation is clear enough in itself, but you just use port forwarding and the localhost address of your local machine!
Best wishes,
Eve
Prerecording Conference Talks and Posters using OBS Studio
Seemingly every conference due to take place this year has either been cancelled or will be run virtually due to the COVID-19 pandemic. Many organisers have decided that running entirely live virtual programmes causes more trouble than it’s worth (e.g. due to unforseeable IT and internet issues disrupting the schedule), and so are asking their presenters to prerecord their talks, which are then broadcast “live” on the day.
I recently “presented” two virtual prerecorded talks at the ISMB conference using Open Broadcast Software Studio (OBS Studio), a free open-source software package most commonly used by live-streamers on Twitch and Youtube. It is super simple to use and achieves a professional output, with video overlaying a presentation slide deck/poster PDF. This blog is a “how-to” on getting started with OBS for conference talks/poster presentations.
Continue readingUploading/downloading small files across systems
Sometimes you just want to quickly move a copy of a script, image or binary from, for example, your local (linux) machine to another (linux) machine. The usual tool would be SCP, but this can get complicated when there are several layers of ssh and sometimes it doesn’t work at all (as is the case for transfers between the Department of Statistics computers and the outside world).
Continue readingStreamlining SSH for remote work
With the university now working remotely, and our group working entirely on linux systems, I figured that now would be a good time to share some useful SSH commands to streamline remote access. This is far from an exhaustive list, but will hopefully serve as a useful starting point for anybody who finds themself needing to work remotely on a linux system.
Continue readingUsing SLURM a little bit more efficiently
Your research group slurmified their servers? You basically have two options now.
Either you install all your necessary things on one of the slurm nodes within an interactive session, e.g.:
srun -p funkyserver-debug --pty --nodes=1 --ntasks-per-node=1 -t 00:10:00 --wait=0 /bin/bashand always specify this node by adding the ‘#SBATCH –nodelist=funkyserver.cpu.do.work’ line to your sbatch scripts or you set up some template scripts that will help you to install all your requirements on multiple nodes so you can enjoy the benefits of the slurm system.
Here is how I did it; comments and suggestions welcome!
Step 1: Create an sbatch template file (e.g. sbatch_job_on_server.template_sh) on the submission node that does what you want. In the ‘#SBATCH –partition’ or ‘–nodelist’ lines use a placeholder, e.g. ‘<server>’, instead of funkyserver.
For example, for installing the same conda environment on all nodes that you want to work on:
Continue readingConsistent plotting with ggplot
Unlike other OPIGlets (looking at you, Claire), I have neither the skill nor the patience to make good figures from scratch. And making good figures — as well as remaking, rescaling and adapting them — is incredibly important, because they play a huge role in the way we communicate our research. So how does an aesthetically impaired DPhil student do her plotting?
Continue readingA few more reasons why UNIX is awesome
One could easily find dozens of reasons for which UNIX — mainly Ubuntu — is simply, the best operating system. Although I remember people in my proximity mentioning this for ages, it’s been only a few months that I’ve realized what are the true advantages. Helpful for this were all the people teaching/demonstrating in various modules during my first year in SABS/DTC: quite often we would be asked to do something in the console rather than by clicking the mouse. In the meanwhile, I’d wonder why using the console can be better from a nice, user-friendly GUI (i.e. Windows…). Tools like sed, grep, tar and of course alias-ing form a quick answer. I will not argue more about these but demonstrate two more tools/tricks.
Continue reading