%  XXX XXX XXX  THIS IS A NEW FILE XXX XXX XXX
%
% unixsheet.tex
%
% Developed by Em Tonkin <e.tonkin@ukoln.ac.uk>
% Copyright (c) 2006 UNKNOWN Company
% Licensed under terms of GNU General Public License.
% All rights reserved.
%
% Changelog:
% 2006-11-27 - created
%

% $Platon$

\documentclass[a4paper,11pt]{article}
%\usepackage [latin2]{inputenc}
%\usepackage {fontenc}
%\usepackage {t1enc}
\usepackage{hyperref}
% use graphics and floating figures
\usepackage {graphicx}
\usepackage {floatflt}

\begin{document}
\title{UNIX command quick reference}
\author{Emma Tonkin \& Greg Tourte}
\maketitle
\newpage
\tableofcontents
\newpage
\section{Getting to know the command line}

\subsection{What is a shell?}

When you type commands on a command line, they are processed and interpreted by an application known as a {\em shell}. There are many shells available for UNIX systems (many of which are also available for Windows systems using a compatibility layer like Cygwin). Essentially, a shell is a simple scripting language. A set of shell commands can be typed into a shell manually, or they can be saved into a file and run at a later time -- just like any other interpreted scripting language, such as Perl or Python (in fact, it is possible to use both of these languages as rather overcomplicated shells). \\

The most popular shell in the Linux/Unix world these days is {\em bash}~; on some very old or very cut-down systems you might find yourself using {\em sh}.\\

\subsection{Typing in shell commands}

The majority of command-line applications accept parameters and options. In UNIX, most programs accept parameters using either a verbose ``--option-name'' syntax, or a rather terser ``-o'' syntax.\\

\subsection{Inbuilt help text}

Some commands come with inbuilt help text. This is usually accessed by
typing the command name, plus the string ``--help'' or, rather more
unusually, `` -h ''. \\

For example, here is the result of trying this with the small word count
application {\em wc}, which exists on most UNIX systems.\\

\begin{verbatim}
em@cyclops:~$ wc --help
Usage: wc [OPTION]... [FILE]...
Print newline, word, and byte counts for each FILE, 
and a total line if more than one FILE is specified.  
With no FILE, or when FILE is -, read standard input.

  -c, --bytes            print the byte counts
  -m, --chars            print the character counts
  -l, --lines            print the newline counts
  -L, --max-line-length  print the length of the longest line
  -w, --words            print the word counts
      --help     display this help and exit
      --version  output version information and exit

\end{verbatim}

\subsection{Man: Reading The Fine Manual}
Most UNIX systems come with an application called {\em man}, which offers a set of documentation for the majority of available applications. {\em man} suffers from a number of limitations, principally that it tends to be written in an extremely telegraphic style. It is useful if you simply need a little help to jog your memory. Learning to read {\em man} pages is something of an art, but you will find that it becomes easier with time. The first thing to remember is not to be intimidated!\\

Here is an example of using {\em man}~:\\

\begin{verbatim}
em@cyclops:~$ man wc
WC(1)           User Commands   	WC(1)

NAME
       wc - print the number of newlines, words, 
	   and bytes in files

SYNOPSIS
       wc [OPTION]... [FILE]...

DESCRIPTION
       Print  newline, word, and byte counts for 
	   each FILE, and a total line if more than 
	   one FILE is specified.  With no FILE, or 
	   when FILE is -, read standard input.

...
\end{verbatim}

\subsubsection{Using man}

As you saw above, {\em man} is easy to query~; just type {\em man commandname}.\\

\subsubsection{The structure of man pages}

Each man page comes with an introductory summary, a synopsis in which the basic use of the application is described, a detailed description of command line options, if any, and a little additional information such as the name of the application's author(s) and copyright information. \\ 

\newpage

\section{Introducing the UNIX file system}

Windows file systems are essentially based around the idea that every
partition on a hard disk and every ``device'', which is to say, every
CD-ROM, DVD burner, etc, gets its own drive letter. Similarly, if you
view a network drive on Windows, it can be mapped to its own drive
letter (alternatively, you can view it from the other computer; however,
the point remains; drive letters are the way Windows likes to view the
world).\\

UNIX, being much older and designed long before anybody really expected
just one or two drives to do the trick, was designed with something else
in mind. The UNIX authors wanted the filesystem to be spread over a large
number of computers and an extended network, but they did not want this
configuration to be obvious for the user -- if it was, then changing the
network at all would mean that the filesystem would apparently change!\\

So UNIX is designed with the idea that every computer has precisely one
filesystem, and its name is {\bf /}. {\bf /}, pronounced ``root'', as in
``the filesystem root'', is exactly what it sounds like - the root of
your filesystem tree.\\

In general, Unix/Linux shares a simple basic structure~:\\

\subsubsection{/bin}

Really necessary programs live in /bin -- like the command line shell
{\em bash}, and handy utilities such as {\em cp} and {\em mv}.\\

\subsubsection{/etc}

System configuration files live in /etc.\\

\subsubsection{/home}

The area in which users' home directories are (typically) stored.\\

\subsubsection{/lib}

Contains library files (code that is reused within other
applications).\\

\subsubsection{/sbin}

Where /sbin exists it is used for (more) really necessary programs.
However, only programs that are used for system administration purposes
live in /sbin -- such as shutdown, kernel module manipulation functions
and so on.\\

\subsubsection{/tmp}

Anybody can write files to /tmp; it's a general-purpose temporary file
store.\\

\subsubsection{/usr}

Programs for users -- not necessarily essential programs -- live here in
/usr/bin. Similarly, handy but not necessary resources live in
/usr/share or /usr/local.\\

\subsubsection{/var}

/var is so named because it is the disk partition in which email and
news data was traditionally stored. Because this sort of data appears
and disappears, it is a file system of {\bf var}iable size.\\

\subsubsection{About . and ..}

In UNIX, '.' is your current directory; {\em cd .} will simply change
your directory\dots to the directory you are already in\dots

{\em cd ..} will change your directory to the directory above the one
you are already in - that is, one directory nearer to the root
directory.\\

\subsubsection{About $\sim$}

{\em $\sim$/ } is your own home directory. If you type 'cd $\sim$' you will be
returned to your home directory. Similarly, if you type 'cd $\sim$em/'  you
will be taken to em's home directory (you probably won't be able to read
it, though!)


\newpage
\section{Pipes, stdin and stdout}

When you type into the command line, you are introducing input into what is known as {\bf stdin}, otherwise known as ``standard input''. When an application provides information back to you, it sends information to what is referred to as {\bf stdout}. The important point to recognise here is that both stdin and stdout can be {\em redirected}.\\

\subsection{Redirection of output to a file with $>$}

To redirect the output of a command to a file, make use of the
greater-than sign, ``$>$''. Here is an example of using ``$>$''.

\begin{verbatim}
ls > myfilelist.txt
\end{verbatim}

Later, when you look into the file called ``myfilelist.txt'', you will find that it contains a file listing of the directory that you were in when you typed this command.\\

\subsection{Piping output to another application using $|$}

The interesting thing about stdin and stdout is that they can be sent,
not only to file, but also to other applications. This is done using the
``$|$'' (pronounced 'pipe') redirector. \\

\subsubsection{Example 1: how many files in a directory?}

Using the application ``ls'', we can list all of the files in a directory. However, perhaps we only want to know how many files there are in that directory. Using the pipe redirector, we can accomplish this easily.

\begin{verbatim}
ls -l lists all of the files in that directory, 
with one filename on each line.

wc -l counts all of the lines in a file.

We can chain them together as follows:

em@cyclops:~$ ls -l | wc -l
384
\end{verbatim} 

Analysing this example, {\em ls -l} actually gave the following output to stdout~:

\begin{verbatim}
-rw-r--r--   1 em   users   3558095 2006-06-15 10:15 13.ogg
-rw-------   1 em   users    464704 2006-10-19 16:21 1467-8659.pdf
...etc...
\end{verbatim}

When the pipe was used to redirect this to {\em wc -l}, wc read precisely this same text from stdin, counted each line and output a number when it ran out of anything more to count.\\

This is exactly equivalent to the following~:\\

\begin{verbatim}
ls -l > temporaryfile.txt
wc -l temporaryfile.txt
\end{verbatim}

%\subsubsection{Example 2: manipulating input from a network}

%Pipes are surprisingly powerful. Here is an example of downloading a web
%page, extracting some useful data and formatting it for display or
%storage.\\


% something with wget (awk?) and sed
\newpage
\section{Basic Linux command reference}

\subsection{A note about your \$PATH}

Linux makes use of a number of variables to customise your environment.
One of them is the PATH variable. This is simply the number of
directories into which Bash will look to find whatever application name
you have typed on the command line. For example, on my laptop I have put
a number of directories in my PATH for convenience, so that I do not
have to type the full ``address'' of the application every time.\\

\begin{verbatim}
em@cyclops:~$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/usr/games:
/usr/lib/java/bin:/usr/lib/java/jre/bin:/usr/lib/java/bin:
/usr/lib/java/jre/bin:/opt/kde/bin:/usr/lib/qt/bin:
/usr/share/texmf/bin:.:/home/em/tree-tagger/bin:
/home/em/tree-tagger/cmd:
\end{verbatim}

To find out what's in your own \$PATH, type {\em echo \$PATH}.\\

If you need to add a directory to your \$PATH temporarily, you can do so
by typing~:\\

\begin{verbatim}
em@cyclops:~$ PATH=$PATH:/the/additional/directory/name:
\end{verbatim}


\subsection{Reading files with cat}

The ``cat'' application is a very simple little application that simply
opens a file and prints its contents to stdout.\\

\begin{verbatim}
cat myfile.txt
\end{verbatim}

\subsection{Reading files with less}

Less is a very simple little application known as a pager. It succeeds slightly more basic pagers such as ``more'' (note~: UNIX programmers tend to have a very strange sense of humour. Less was named this way because, as the designers said, ``less is more''...\\

The major difference between {\em less} and {\em more} is that {\em
less} allows you to page up and down using the Page Up and Down keys,
and the arrow keys -- {\em more} only allows you to page downwards. This
is still a lot better than trying to read a file with {\em cat} as it
flashes at super-speed down the screen, but you will find that {\em
less} is relatively painless to use. Sometimes, though, older
installations or those more challenged for space will not have {\em
less} installed at all, and you will have to use {\em more} after all.

\subsection{Listing files with ls}

For one of the simplest-sounding functions on Unix, {\em ls} has one of
the longest man pages imaginable. This is partly because it gets used
all over the place, in all sorts of scripts and interfaces; {\em ls}
will produce output in colour, in any format, with any amount of useful
or not-so-useful metadata. Nonetheless, the basic application of {\em
ls} is really pretty simple~:\\

\subsubsection{Basic listing of all files in directory}
\begin{verbatim}
ls 
\end{verbatim}

\subsubsection{All files that finish in ``.txt''}
\begin{verbatim}
ls *.txt
\end{verbatim}

\subsubsection{A detailed file listing}
\begin{verbatim}
ls -lisa
\end{verbatim}

\subsubsection{Reading {\em ls} output with a pager}
\begin{verbatim}
ls -lisa | less
\end{verbatim}

\subsection{Editing files on the command line with pico}

Pico is a relatively user-friendly little command line text editor. To
edit a file with pico, simply type:\\

\begin{verbatim}
pico filename.txt
\end{verbatim}

You will notice that pico provides a little help text along the lines of

\begin{verbatim}
^X Exit                
\end{verbatim}

This simply means ``press control and X to exit the program''.\\

\subsection{Deleting files with rm}

{\em rm} is a fantastically dangerous application. Apply with care!

Basic use of {\em rm} looks like the following~:\\

\begin{verbatim}
rm filename.txt
\end{verbatim}

Recursive use (that is, deletion of a directory) looks like~:\\

\begin{verbatim}
rm -rf foldername
\end{verbatim}

{\bf Be warned}: rm is not as friendly as Windows' delete function, and
generally it will not ask you whether you are sure, certain or completely happy
with what you have done. There is no undelete function!\\

\subsection{Creating empty files with touch}

Touch is a very simple little application; if a file doesn't exist, it
creates it. If it does, {\em touch} does nothing at all. This may sound
pretty useless, and it is, until you realise that lots of programs
demand files to exist, even if they're empty. You probably won't find
this one very useful for your own work at the moment, but it is as well
to know what it does in the event that you find it in a script or
similar.\\


\subsection{Finding out your login with whoami}

You'll probably find this more useful when writing scripts than anywhere
else, unless your short-term memory is a little in need of work.
However, it's a very simple command~:\\

\begin{verbatim}
em@cyclops:~$ whoami
em
\end{verbatim}


\subsection{Finding out your current directory with pwd}

\begin{verbatim}
em@cyclops:~$ pwd
/home/em
\end{verbatim}

\subsection{File permissions with chmod}

UNIX makes use of a permissions system to work out whether files should
be considered~:\\

\begin{itemize}
\item executable
\item readable
\item writable
\end{itemize}

{\em chmod} allows you to change these values for a given file or
folder.\\

\begin{verbatim}
chmod 777 filename

equivalent:

chmod a+rwx filename
\end{verbatim}

As you can see, there are two equivalent syntaxes. Each one is explained
below.\\

\begin{tabular}{l|l}
	0 & no permissions\\
	1 & executable \\
	2 & writable\\
	3 & writable + executable\\
	4 & readable \\
	5 & readable and executable\\
	6 & readable and writable\\
	7 & readable, writable and executable\\
\end{tabular}

Why, one might ask, are there three 7s in the command {\em chmod 777
filename}? Because there are three distinct cases; the first is yourself,
the owner of the file, the second is your user group, and the third is everybody who ever uses
the machine. Similarly, then, if one typed {\em chmod 754 filename}, the
result would be that you yourself see the file as readable, writable and
executable; your user group sees it as readable and executable but not
writable, and everybody else sees it as readable but not executable.\\

The alternative syntax for chmod is the following~:\\

\begin{tabular}{l|l}
	u & your own permissions \\
	g & user group\\
	o & the other users\\
	a & all the users\\
	+r & make the file readable\\
 	+w & make the file writable \\
	+x & make the file executable \\
\end{tabular}

In other words, chmod u+x makes the file executable for the user; chmod
o+rw makes the file readable and writable for all the other users (eg
not yourself or your user group), chmod g+x makes
the file executable for the user group. So u corresponds to the 7 in
chmod 754 filename; g corresponds to the 5, and a corresponds to the
4. Which of these syntaxes you choose to use is entirely a question of
which you personally feel more comfortable with.\\

It is worth bearing in mind that the latter syntax does not reset,
whilst the former does; in other words, if you type 'chmod 644 *.txt',
you will completely replace the permissions previously set. If you type
'chmod u+rw *.txt' you will not affect the group and general permissions
at all. \\

\subsubsection{Using chmod on web sites}

In general, you want HTML files to be readable and writable for
yourself, but you do not want others to be able to write them.
So you might set them with the following~: {\em chmod 644 *.html}.

\subsubsection{Chmod and directories}

Unlike files, directories have to be executable for you to change
directory into them. In other words, if a directory was given the
following permissions~:\\

\begin{verbatim}
chmod 644 mydirectory
\end{verbatim}

you would no longer be able to {\em cd} into that directory.\\

Directories must be set as executable and readable for you to be able to
read them!\\


\subsection{File ownership with chown}

Every file belongs to somebody, and to some group. To see what user and
what group a file belongs to, use {\em ls -l}~:

\begin{verbatim}
em@cyclops:~$ ls -l
total 84
-rw-r--r--  1 em users  2991 2006-08-24 15:28 Author.java
\end{verbatim}

As you can see, the permissions on this file (see {\em chmod}) are
-rw-r--r--, otherwise known as 644; I can read and write the file, my
user group and everybody else can only read it. However, you can also
see that it is owned by ``em'' and that its group is ``users''. \\

As a user, you will not find yourself using chown. However, as a system
administrator you might find yourself needing to do this; you would do
so using the following command.\\

\begin{verbatim}
chown username:usergroup filename
\end{verbatim}

So for example in my example, if I wished to change the owner to
``bob'', I would do~:\\

\begin{verbatim}
chown bob:users Author.java 
\end{verbatim}

Both chown and chgrp have a ``recursive'' (-R) option. Use with
caution!\\

\subsection{Gaining administrator privileges with su}

You may not need to do much of this for the time being. However, if you
are given the root password of a server and need to do something you
can't do as a user, like installing files into /usr/bin, then you will
need to change user to root using {\em su}.\\

\begin{verbatim}
em@cyclops:~$ su -
Password:
root@cyclops@~# 
\end{verbatim}

That little - after su is important -- it tells the system to provide a
full root environment. If you do not do this you will find that you have
very few applications in your \$PATH.


\subsection{Finding out what's running with ps}

{\em ps} lists the currently running processes. For example (note that
this particular usage of ps does not require a - before the options)~:\\

\begin{verbatim}
ps ax | less 
\end{verbatim}

If you can't see the application running in here, the chances are that
it has crashed!\\

\subsection{If something goes horribly wrong -- using kill}

Sometimes applications get confused, and you won't be able to shut them
down using ordinary means. In this case you have two options. If you
know the exact name of the process, try~:\\

\begin{verbatim}
killall -9 processname
\end{verbatim}

If you do not, look for it using {\em ps}, and then kill it either by
process name (as above) or by process id, using {\em kill}~:\\

\begin{verbatim}
kill -9 5987
\end{verbatim}

\begin{quotation}
Be careful with killall -- use it only on Linux! It works completely
differently on Linux than it does on classic UNIX systems.
\end{quotation}

\subsection{Copying files with cp}

cp is fairly simple~:\\

\begin{verbatim}
cp filename.txt filename2.txt
\end{verbatim}

If you need to copy folders, you need~:\\

\begin{verbatim}
cp -a folder folder2
\end{verbatim}

\subsection{Moving files with mv}

{\em mv} is even simpler than {\em cp}\dots

\begin{verbatim}
mv filename.txt filename2.txt
mv foldername foldername2
\end{verbatim}

\subsection{Date and time with date}

{\em date} is a surprisingly useful program, especially if you are
producing any sort of script; you will want to use it for getting a
timestamp with which to log results, for example.\\


\subsubsection{Human-readable date}

\begin{verbatim}
em@cyclops:~$ date
Tue Nov 28 01:54:11 GMT 2006
\end{verbatim}

\subsubsection{ISO 8601}

\begin{verbatim}
em@cyclops:~$ date --iso-8601
2006-11-28
\end{verbatim}

\subsubsection{Seconds since the epoch}

\begin{verbatim}
em@cyclops:~$ date +%s
1164679060
\end{verbatim}

This last is particularly interesting. The ``seconds since the epoch''
refers to the way in which UNIX systems internally measure time -- the
number of seconds that have elapsed since 1970-01-01 00:00:00 UTC.
Whilst not very useful for a human, it is very helpful as a simple way
of working with time. As a piece of trivia, this value also has the fun characteristic that, on older
UNIX systems at least, this value is internally represented using a
32-bit integer. Thus, these systems will suffer a UNIX equivalent of the
Millennium Bug in 2038-01-19 03:14:07 UTC, when they will roll back to
1970. \\

\subsection{Compressing and uncompressing with gzip}

Instead of PKZIP or similar, UNIX native compression format is gzip. It
provides reasonably efficient compression, but can only compress one
file at a time (Windows zips can, of course, contain many files). See
``tar'' for the Unix answer to this problem.\\

Basic use of gzip~:\\

\begin{verbatim}
gzip filename.ext
gunzip filename.ext
\end{verbatim}

\subsection{Compressing and uncompressing with bzip2}

Gzip is relatively old, and is designed for computational efficiency
(speed) as well as for efficiency in terms of compression. Later, bzip2
was designed to make better use of the power of modern processors for
more effective compression. It is used in very much the same way~:\\

\begin{verbatim}
bzip2 filename.ext
bunzip2 filename.ext
\end{verbatim}


\subsection{Archiving with tar}

Tar is a very old utility designed for use in the days that people still
kept all their backups on magnetic tape. From this fact comes the name~;
{\em tar}, the tape archiver. At heart, {\em tar} does a very simple job
-- it reads the contents of each file into one simple bytestream, which
it then records serially onto tape (or, in this day and age, onto disk).
This is a complicated way of saying that tar can turn an entire
directory of files into one large lump, which can then be
compressed and stored as just one big file. \\


Gzip-compressing a directory with tar~:\\

\begin{verbatim}
tar cvfz mydirectory.tgz mydirectory/
\end{verbatim}

Uncompressing a gzip compressed directory with tar~:\\

\begin{verbatim}
tar xvfz mydirectory.tgz
\end{verbatim}

Finding out what's in a gzip compressed tar file~:\\

\begin{verbatim}
tar tvfz mydirectory.tgz
\end{verbatim}

Using bzip2, replace the ``z'' in each of these examples with a ``j'';
thus~:

\begin{verbatim}
   tar cvfj mydirectory.tgz mydirectory/
\end{verbatim}
	  
\subsubsection{Tar options explained}

tar options~:\\

\begin{tabular}{l|l}
	c & create file\\
	t & list files in archive \\
	x & extract files\\
	v & be verbose about what you are doing\\
	f & operate on a file (as opposed to a stream)\\
	z & make use of gzip compression\\
	j & make use of bzip2 \\
\end{tabular}

\subsection{Checksums using md5sum}

{\em md5sum} calculates and displays md5 hashes. This is generally used
to check that the contents of a file have not changed; for example, that
a downloaded disk image still contains what it does on the server. For
this purpose, md5s are often published along with files. To calculate an
md5sum~:\\

\begin{verbatim}
em@cyclops:~$ md5sum review.doc
bf159362a24dc55a3b02ee5c4117d431  review.doc
\end{verbatim}

\subsection{Searching for matches using grep}

Grep searches for strings and returns matches. Like most applications,
it can either work from standard input (stdin) or directly from a
file.\\

A couple of basic uses of grep~:\\

Given that the file filename.txt contains the words "foo, bar, baz, boo,
for, fork", the following uses would return~:\\

\begin{tabular}{|l|l|}
	\hline
grep -e 'f.o' filename.txt & foo\\
grep -e 'fo.' filename.txt & foo, for,fork \\
grep -e 'r\$' filename.txt & for, bar \\
grep -e '\^for\$' filename.txt & for\\
\hline
\end{tabular}

Grep is a reasonably fast way of turning large, irrelevant files into
small, useful result sets.\\

See the wildcard section for further info.\\

\subsection{Finding files using locate}

Locate is a database-backed system designed to help you find files on a
large filesystem, quickly, by storing information in an centralised
database that can be queried in seconds. Whilst this is much faster than
searching all the way through the filesystem each time, it is also
dependent on regular sweeps through the filesystem to refresh the
database. Therefore, you will find locate useful only if you are looking
for a file that is older than about a day -- on most Linux systems, the
process to rewrite the database runs about once a day, generally around
4am.\\

\begin{verbatim}
locate A*.java
locate Author*.java
locate *.pm
\end{verbatim}

\subsection{Finding files using find}

Find is the slow way of finding files. You should not use it unless it
is really necessary -- it is much slower than locate. It is also
something of a complex tool; however, here is a simple formula for
finding files using find.\\

\begin{verbatim}
find . -iname 'Author.java'
\end{verbatim}

The . means 'start in the current directory'. The -iname means
'something with a case-insensitive name like this' -- so author.java or
Author.java or AUThOR.jAVA all match.\\

\subsection{Stream editing using sed}

Sed provides you with a way of editing streams on the fly. For example,
to change all uses of the word 'Blair' to the word 'Bliar' in a given
piece of text, you would use the following command line~:\\

\begin{verbatim}
cat filename.txt | sed -e 's/Blair/Bliar/g'
\end{verbatim}


\subsection{Downloading from the Web with wget}

wget provides you with a means of downloading files directly from the
command line. The simplest use of wget~:\\

\begin{verbatim}
wget http://some.web.address/filename.txt
\end{verbatim}

You can have much more fun with wget than this, however; for example, a
surrealist viewpoint on the BBC website's HTML can be gained by doing
the following~:\\

\begin{verbatim}
wget -O - http://news.bbc.co.uk | sed -e 's/\ news/\ frogs/g'|less
\end{verbatim}

Note: {\bf wget -O - } causes wget to write directly to stdout.\\

An excerpt from the resulting HTML~:\\

\begin{quotation}
$<$meta name="description" content="Visit BBC News for up-to-the-minute
frogs, breaking frogs, video, audio and feature stories. BBC News provid
es trusted World and UK frogs as well as local and regional
perspectives. Also entertainment, business, science, technology and
health frogs."
 /$>$
 $<$meta name="keywords" content="BBC, News, BBC News, frogs online,
 world, uk, international, foreign, british, online, service" /$>$
\end{quotation}


\subsection{Making Bash scripts out of commands}
\subsubsection{The hash-bang}

A Bash script is simply a set of commands, as we have discussed above.
However, to execute them, the system needs to know what interpreter it
should be using. Therefore, you provide this information using the
following code as the first line of your script:\\

\begin{verbatim}
#!/bin/bash
\end{verbatim}

Here for example is a very simple bash script~:\\

\begin{verbatim}
#!/bin/bash
# this replaces ls -lisat
ls -lisat $1 
\end{verbatim}

Similarly, a Perl script has the following line~:\\

\begin{verbatim}
#!/usr/bin/perl
print "I am a perl script!\n";
\end{verbatim}

\subsubsection{Example script: My age relative to the Epoch}

\begin{verbatim}
#!/bin/bash
# this little script calculates the difference
# between my age and the current age of UNIX

MYBIRTH=`date -d "1978-10-30" +%s`
NOW=`date +%s`
echo $((${NOW}-${MYBIRTH}))
\end{verbatim}

Instead of hardcoding my age into this script, I can use the \$1 variable -- which is set to the first argument on the command line.\\

\begin{verbatim}
#!/bin/bash
# this little script calculates the difference
# between any age and the current age of UNIX
# Invoke it with something like:
# ./myscript.sh 1982-01-01

MYBIRTH=`date -d $1 +%s`
NOW=`date +%s`
echo $((${NOW}-${MYBIRTH}))
\end{verbatim}

\subsubsection{Example script: Creating a wordlist from a file}

\begin{verbatim}
#!/bin/bash
# Note: the following command is essentially one long line
# the \ tells bash to ignore the newline

cat $1 | tr 'A-Z' 'a-z' |  sed -e 's/\ /\n/g'| \ 
sed -e 's/[^A-Za-z\].//g' | sed -e 's/[\.,?]//g' \
| sort | uniq | less
\end{verbatim}

\subsubsection{Example script: A basic spellchecker}

Many UNIX systems come with some form of word list; this often lives in
/usr/share/dict/words. This script demonstrates a use of the previous
example script, combined with a search of /usr/share/dict/words, to
identify words that do not appear in /usr/share/dict/words and may
therefore be misspelt.\\

\begin{verbatim}

for foo in `cat $1 | tr 'A-Z' 'a-z' \
| sed -e 's/\ /\n/g' | \
sed -e 's/[\.,?]//g' | \
sed -e 's/[^a-z\ ].//g' `;

do
        MYRESPONSE=`grep /usr/share/dict/words -e "^$foo$" | wc -l `
        if [ $MYRESPONSE -gt 0 ] ; then
                #echo "Word $foo OK"
                true
        else
                echo "Word $foo misspelt?"
        fi
done
\end{verbatim}

Shell scripting (and scripting in general) is a powerful technique, even though it combines extremely simple elements.\\

\newpage
\section{Configuring and compiling software}

Linux comes with a set of tools for the compilation, linking and installing of software. Each flavour of Linux (Red Hat, Slackware and so on) have their own package management system for installation of precompiled software; however, here, we are looking at how to compile software from scratch. \\

\subsection{The compilation process}

The process of compiling a piece of software from scratch is complex;
software depends on libraries (kept in /lib, incidentally, on UNIX
systems -- on Windows systems, dlls do a similar job). Because of this, scripts are produced to simplify the
compilation process.\\
% the use of linkers, compilers, how libraries fit in, why it is important to have libraries and where dlls fit in

Two pieces of software are used for this; {\em autoconf} and {\em make}.

\subsection{autoconf}

You will almost never find yourself using autoconf directly, unless you one day decide to produce and distribute a large piece of software. However, you will often make use of the result of it. Usually, when you download a set of source files to be compiled on Linux, you will find a file in that source directory called 'configure'. This is a script generated by autoconf that, when you call it by means of './configure' will automagically sort out all of the configurable aspects of the software -- such as the location of required libraries, the preferred libraries against which to link and where to place the resulting files.\\

Sometimes you will want to give some command-line preferences to the ./configure script; for example, if you want it to try to install a program in a directory within your own home directory rather than attempting a system-wide install, you will want to tell the ./configure script to do so. For example, in this case you would write the following~:\\

\begin{verbatim}
./configure --prefix=/home/em/bin/
\end{verbatim}

To find out what command-line preferences exist, type {\em ./configure --help}. \\

\subsection{make}

Generally, if a file called 'Makefile' is present in a directory, then
the process of compiling and installing the software is as simple as~:\\

\begin{verbatim}
make
make install
\end{verbatim}

In practice, all sorts of things can go wrong. Makefiles are
human-editable, however, so if the system is using the wrong compiler or
any of the other non-fatal annoyances that frequently occur, it is
possible to make some changes and try again.

\section{Using Perl}

\subsection{What is Perl?}

It is sometimes said to stand for the Practical Extraction and Reporting
Language. In fact, this is a backronym.\\


\subsection{Writing simple Perl}

A beginner's introduction to Perl is available at~:\\

{\url http://www.perl.com/pub/a/2000/10/begperl1.html}


\subsection{Perl modules from CPAN}

Perl is a nice scripting language, especially for manipulation of text
files, web-based work and so on. It is something of a Swiss Army Knife,
in that somebody with Perl experience can look at almost any problem in
Web development and say something along the lines of ``we could try
using the left-handed corkscrew/awl attachment, I'm almost sure it'd do
the trick'' and be right most of the time. This being the case,
attachments (known as {\em modules} in Perl parlance) have been
developed for pretty much every task. \\

Since it is a waste of everybody's
time to reinvent the wheel every time the task comes up, modules are
generally shared online for everybody to test, reuse and mess around
with. The website used for storing Perl modules is known as {\em CPAN},
which stands for something along the lines of ``the Central Perl Archive
Network'', although I might be wrong about the details.\\

\subsection{Manually installing Perl modules}


Perl modules come packaged as a neat little tar.gz. First, extract the
data using the following command~:\\

\begin{verbatim}
tar xvfz filename.tar.gz
\end{verbatim}

Then change directory into the directory created by this command
(generally called something like module-name-1.1/). You will now have to
prepare and install the Perl module. Perl comes with a simple system for
doing this, which is based on the following three commands~:\\

\begin{verbatim}
perl Makefile.PL
\end{verbatim}

Perl will respond by writing something like ``Writing Makefile for
EPrints:: QueryBuilder''. If you then type 'ls' in this directory, you
will see that a new file has been created. Logically, it's called
``Makefile''. You will recognise this as a configuration file for the
GNU Make utility explained earlier in this document. As ever with Make,
the sensible thing to do when you see this file is to see what happens
when you type~:\\

\begin{verbatim}
make
make test
make install
\end{verbatim}

If any of these stages fail, you will not be able to install the module
without working out why (in fact you can install the module even if
``make test'' fails, but there is not a great deal of point since the
result will be that you will have installed a module that is broken in
some way). To explain what is going on underneath the surface here, best
practice for authors of Perl modules is to provide a set of ``unit
tests'', which is to say, small tests of the behaviour/functionality of
the compiled module on your system. If any of it does not work as
planned, Perl knows that there is something wrong somewhere.\\

\subsection{Auto-installing Perl modules from CPAN}

This is a very easy way of doing what was just discussed in the previous
section, automatically and somewhat faster -- when it works. 

\subsubsection{How to auto-install Perl modules}

\begin{verbatim}
perl -mCPAN -e 'install module::name'

For example:

perl -mCPAN -e 'install HTML::Mason'
\end{verbatim}

\subsubsection{What can go wrong?}

Sometimes, Perl modules will depend on other modules that are not
installed. Generally, the Perl module installer is smart enough to test
for this. However, sometimes, if the author of the module has forgotten
to place this information in the module configuration files, it will
not. When this happens, Perl will not install the module (this would
lead to having a broken module!) Instead, it will generally come up with
a slightly cryptic-looking message about failed unit tests.\\

Detailed debugging is beyond the scope of this document. However, there
are two things that you may wish to try~; a forced install, and a forced
upgrade of installed Perl modules.\\

\subsubsection{Further information}

CPAN itself is at {\url http://www.cpan.net}. Additionally, try typing
{\em man CPAN} for further information about the {\em perl -mCPAN}
commands.\\
\newpage
\section{Wildcard/Regular expression cheat sheet}
\subsection{What are wildcards?}

Wildcards are special characters interpreted by the shell. \\

\subsection{Basic wildcards}

\begin{tabular}{l|l}
* & matches anything \\
? & matches a single character\\
$[$abc$]$ & matches a,b or c \\
$[$a-z$]$ & matches any character from a to z \\
$[$0-9$]$ & matches any number from 0-9 \\
*.\{txt,pdf,doc\} & matches any .txt, .pdf or .doc files \\
\end{tabular}
	
\subsection{Grep-style regular expressions}

You can test these out on /usr/share/dict/words\dots
for example, try~: \\

\begin{verbatim}
grep /usr/share/dict/words -e 'cat'
grep /usr/share/dict/words -e '^bure'
grep /usr/share/dict/words -e 'ity$'
grep /usr/share/dict/words -e '^fl.ng'
grep /usr/share/dict/words -e '[au]ggy$' 
grep /usr/share/dict/words -e '[a-e]lity$'  
grep /usr/share/dict/words -e '^[^a-y]' 
\end{verbatim}

\begin{tabular}{l|l}
\textasciicircum  & matches the beginning of a word\\
\$ & matches the end of a word\\
. & matches any single character\\
$[$abc$]$ & matches a, b or c\\
$[$a-z$]$ & matches a-z\\
$[$A-Z$]$ & matches A-Z\\
$[$0-9$]$ & matches 0-9\\
$[$\textasciicircum a-y$]$ & matches every line that does not start with
letters a-y\\
\end{tabular}


Note: Regular expressions within Perl have a slightly different syntax
and we will not cover them in this session! 

For more about grep-style regular expressions, try~:\\

{\url
http://www.robelle.com/library/smugbook/regexpr.html}.
%\subsection{Wildcards and regular expressions}
%\newpage
%\section{The regular expression cheat sheet}
%\subsection{What are regular expressions?}
%\subsection{A few example regular expressions}
\end{document}

% vim: ts=4
% vim600: fdl=0 fdm=marker fdc=3

