This is the actual talk notes companion to UsefulScienceSoftware. Most items don't have links to other resources, unless I know exactly where to look for help. Google will usually work, or ask someone.

units

An incredibly powerful unit converter and calculator. Instead of just converting things, it operates as a fully units-aware calculator. It has a ton of units built in. 

You have: 10 feet * 30 inches
You want: meters^2
        * 2.322576
        / 0.43055642

It has tons of constants built in: 

You have: h * c / 400 nm
You want: eV
        * 3.0996046
        / 0.32262179

Adding up atomic masses: 

You have: oxygen + 2*hydrogen
You want: 
        Definition: 18.01528

Notes:

du/df

You probably already use these, but here's a fast way to see where the majority of your disk space is going: 

$ du . | sort -n > tmpfile
$ less tmpfile

tmpfile now contains a list of all directories, sorted by size.

ssh

cron / at

cron runs tasks periodically, at runs programs at a certain time in the future:

To edit crontab: crontab -e 

#m h dom m dow

40 23 * * sun,thu  /home/richard/bin/bin-u/zarankiewicz-pdumpbackup.sh

Each of the first five columns means "minute", "hour", "day of month", "month", "day of week". Use * to match anything. The script given on the rest of the line is run every time every column matches.

Examples: 

00 15 * * fri    echo "" | mail richard clopez@mail.utexas.edu -s "water plants"
59 4 * * *       python -c 'import random; print random.choice(range(30))' | mail rkd@zgib.net -s "The number for today is..."
00 */4 * * *     ( echo "/var/lib/apache/mod-bandwidth/link/" ; ls /var/lib/apache/mod-bandwidth/link/ ) | mail richard -s "mod_bandwidth listing: `ls /var/lib/apache/mod-bandwidth/link/ | wc  -w`"

at runs things at a certain time in the future: 

richard@lefschetz:~$ at may 8
warning: commands will be executed using /bin/sh
at> echo "renew books" | mail richard -s "renew books"
at> <EOT>
job 56 at 2007-05-08 11:22

It can take complex time specifications: 

richard@lefschetz:~ 1 $ at now + 10 minutes
warning: commands will be executed using /bin/sh
at> echo "this will happen ten minutes from now" | mail richard
at> <EOT>
job 57 at 2007-04-23 11:33

Notes:

meld

Graphical diff viewer.

This program (or something like it) is a must.

unison

Sync directories (two-way) on two computers. You probably want to use unison-gtk, which has a good UI.

rsync

rsync is a file synchronization program. It can be used for various purposes, most typically for synchronizing two machines and/or directories. I happen to use it for backing up data.

For example, here's an rsync command that I placed in my crontab to backup my research directory on my work computer:

rsync -av /media/jrd_research/* /media/jrd_research.bkup/

the option 'a' means to synchronize in 'archive mode' which ensures that symbolic links, devices, attributes, permissions, ownerships, etc. are preserved in the transfer. the 'v' option just increases verbosity.

rsync can also be used remotely. Here's a more complicated example where I transfer relevant simulation data from lonestar to my work computer (onsager): 

rsync -av \
   --include "+ */" \
   --include "+ *.steng" \
   --include "+ *.confp" \
   --include "- *" \
   ~/work_directory/replica_exchange/ubiquitin/run/   onsager:/media/jrd_research/replica_exchange/ubiquitin/run/

In this command I have used the --include option to send the entire directory structure of the "run/" directory on lonestar but only with the configuration ".confp" and energy ".steng" files..

Backing up to the archiver (from lonestar): 

rsync -av --rsync-path=/usr/local/bin/rsync ~/work_directory/* archive:~/archive_directory/work/

pdumpfs

This will keep a snapshot of your directories at any time in the past. If you code broke between two days, you can go back in time and look at the changes in it (for example, using meld).

Example script: 

cd /home/richard

function main {
time pdumpfs --exclude-by-size=1G --exclude="^./(noarchive|old_systems|isos|rkd|pdumpfs|tmp|backup).*" . /mnt/backup2/pdump/richard/
}

if tty > /dev/null ; then 
    # is a tty
    main | tee /mnt/backup2/pdump/richard/pdump-log
else 
    main > /mnt/backup2/pdump/richard/pdump-log
fi

Notes:

bash history stuff

I keep per-directory history information.

My big complicated .bashrc information is below. 

shopt -s histreedit
shopt -s histverify
shopt -s histappend
HISTFILESIZE=10000
HISTCONTROL=ignoreboth
#HISTIGNORE="something:something_else:something_else"
histsave() { HISTFILESIZE=1000 history -a "$PWD/_bashhist" ; }
histread () { history -r _bashhist ; }
cd_hook() { test -w . -a -O . -a -G . && histsave ; command cd "$@" ;}
alias cd=cd_hook
# we have to write our history before we shutdown:
#   all shells get SIGHUP before dying.  
hist_exit_hook()   # we might also want to consider using SIGTERM for the trap
{ test -w "$PWD" -a -O "$PWD" -a -G "$PWD" && histsave ; }
exit_hook()
{
    hist_exit_hook
}
trap exit_hook EXIT
trap exit_hook SIGTERM
trap exit_hook SIGHUP

These bindings enable the C-r for searching history backwards and C-s for searching history forward, for lines beginning with what the line begins with already. {{{bind 'C-r:history-search-backward' bind 'C-s:history-search-forward' bind 'M-r:reverse-search-history' bind 'M-s:forward-search-history' }}}

pychecker

less, Richard's "l" script

function l () (
# Function to Do The Right Thing:  
# - If it's a directory, list it
# - Otherwise, open it with the smart less-opener
file=$1
if [ "x$1" = x"" ] ; then
    ls  # list pwd
elif [ -d "$1" ] ; then
    ls "$@"
else  # it's a file.
    LESS="-iM"
    LESSOPEN="|lesspipe.sh %s"
    export LESS ; export LESSOPEN
    if file $1 | sed -r 's/^[^[:blank:]]*[[:blank:]]*(.*)$/\1/' | grep text &>/dev/null
        then
        LESS='-iMR'
        less "$@"
    else
        less "$@"
    fi
fi
)

Some Python utilities

Use pydoc to get help on functions and things.

Regular Expressions

Some people, when confronted with a problem, think “I know, 
I’ll use regular expressions.” Now they have two problems.
   —Jamie Zawinski, in comp.lang.emacs

Applications:

Perhaps more eccentric utilities

screen

Hints:

lsof

Search for what program has a certain file open: 

richard@ehrenfest:~$ lsof | grep \\.ssh
ssh       31019         richard    4u     unix 0xd6639e40               60566 /home/richard/.ssh/mux-ssh-richard@lefschetz.zgib.net

You can also search for network connections: 

richard@ehrenfest:~$ lsof | grep ssh | grep TCP
ssh       31019         richard    3u     IPv4      60557                 TCP w-mob201-128-62-94-167.public.utexas.edu:53285->cpe-24-28-120-59.houston.res.rr.com:ssh (ESTABLISHED)

file

Determines the MIMI type/content type of files. Useful when you don't have a extension on a file extension. 

richard@ehrenfest:~$ file deathstar.jpg 
deathstar.jpg: JPEG image data, JFIF standard 1.01

richard@ehrenfest:~$ file utah2.txt 
utah2.txt: ASCII text

richard@ehrenfest:~$ file track0025.2007-01-28.12\:44\:19.ogg 
track0025.2007-01-28.12:44:19.ogg: Ogg data, Vorbis audio, stereo, 48000 Hz, ~160000 bps, created by: Xiph.Org libVorbis I

richard@ehrenfest:~$ file resume_MichaelQian.pdf 
resume_MichaelQian.pdf: PDF document, version 1.3

latex-beamer

auctex

xmodmap

R

a2ps

pdiff

shttpd.py

g3data

hexedit

UsefulScienceSoftwareTalk (last edited 2009-07-17 18:53:35 by RichardDarst)