Linux is a new (and better), open source implementation of the
commercial UNIX® Operating System (OS).
An OS translates your high level commands into the actual series of operations
that the hardware (CPU, memory, hard disk, etc..) in your computer has to
perform to implement that command and return a —hopefully
meaningful— result.
Command-line based operating systems (such as
DOS®) interpret the text commands you type.
Graphical User Interface (GUI) based operating systems (such as
MacOS® or Microsoft®
Windows®) allow you to interact with the
computer through pictures and menus, via mouse-clicks or by pressing a key
on the keyboard.
The larger flexibility of Unix and Linux generally gives you the option of
using both command-lines and GUIs. The former offer more
control and flexibility — but you need to learn and remember a fairly
large set of commands and command options. The latter can be easier and more
intuitive for complex operations that don't need to be frequently modified
or repeated many times — but are often not flexible enough for the task
at hand.
Unix has been in use for more than four decades and resulted from the efforts of two groups, at Bell Labs and at UC Berkeley, to develop a reliable, multi-user, true multi-tasking (none of which apply to, e.g., DOS or Windows) operating system. It originally ran mainly on large mainframe computers and workstations, but was exceptional in that the OS itself was written in a high-level programming language (C) and could therefore be made to run different types of computers (hardware architectures). Windows, on the other hand, gained popularity in the mid-1980's as the only major operating system designed specifically for Intel-compatible processors as found in the, then new, Personal Computers (PC's). As a mass-marketed, single user OS, it was designed to look good and be intuitive to use by people with little knowledge of computers, not for stability, flexibility, and multi-tasking.
Starting around 1991, a new implementation of Unix called Linux (named after originator Linus Torvalds) began gaining popularity with PC users, particularly for high performance and mission-critical operations (e.g., Internet Service Providers, hospital administrations) and for users with serious computational needs (in the exact sciences, PC's running Linux have replaced many of the more expensive Unix workstations and their recurring software license fees). The Linux® kernel and GNU C compiler and suite of application software form the basis of the Linux OS, which is available for a large variety of architectures and has in many ways improved upon UNIX. The unique feature of Linux, and reason for its rapid spread, is that all source code is freely available (see: GNU General Public License) to anyone who wishes to modify or add functionality to existing programs or write and integrate new ones (as opposed to proprietary operating systems, where the source code remains a well-guarded trade secret, preventing users from modifying programs to suit their needs, or solve common problems with existing programs). This means that the Linux OS benefits from the work of thousands of users who help in further developing the kernel and application programs. Despite claims to the contrary by proprietary OS vendors, this large number of Linux developers has yielded an operating system of unprecedented efficiency and robustness, with many CDROM's worth of freely available applications software packages for both business and pleasure. Juergen Haas nicely sums up why Linux is the Ultimate Unix.
Dozens of companies package the Linux kernel and C compiler, the core of the OS, with large quantities of applications software and a convenient helper program that unpacks and installs everything onto your computer's hard-disk. The market leaders of these are Redhat (including its offshoot Fedora; particularly in the US) and SuSE (particularly in Europe), but Debian (including its offshoot Ubuntu), Mandrake, and Slackware also have a sizeable followings. Ubuntu may be the easiest entry into the Linux world for people migrating from Windows. Although the packages of application software (including the desktop GUI) and/or the place where they wind up on your hard-disk may differ between these distributions, each generation runs the same Linux kernel (give or take a minor version number).
A shell is a program that acts as an intermediary between the user and the guts of the OS (much like the OS acts as an intermediary between the shell and the nuts and bolts of the hardware). A shell provides a command line. In DOS, command.com acts as the shell (albeit a limited one). Linux shells (e.g., tcsh, csh, bash, sh, ksh) perform the same function, but besides translating your commands into something the kernel can understand and act upon, they also add some important functions that the base OS doesn't supply. Whenever you open a new terminal window (like xterm), a tcsh shell is automatically started for normal users; the root account for the system administrator typically runs a bash shell. Some basic features of all Linux shells are:
Prompts A prompt is a character or string of characters (such as
a "%", or ">" for csh and tcsh; "$" for the other shells)
that the shell displays when it is ready to receive a new command. The prompt
may be customized (to, e.g., "user@machine>").
Command resolution When you enter a command, the shell must
determine which program to run in order to perform that command. This command
resolution process can be customized to suit the user's needs.
Job control Linux lets you multi-task (run more than one command
at the same time). Individual tasks may be started, stopped, suspended, and
executed in either the foreground or background. One can start
many shells, in each of which different tasks can be running simultaneously.
Command history and completion Sometimes you want to repeat a
command you've executed before, or issue a similar one. Previously executed
commands can be recalled and modified. Also, the shell can automatically
complete commands and file names for you.
Wildcards and aliases Wildcards let you process a whole
bunch of files at once, instead of having to repeat the same command for each
file individually. Commonly used commands, or modifications of commands with
different sets of command options can be stored as aliases.
Piping and I/O redirection Connecting programs together by
sending the output of one program directly to the input of another
(piping), or reading input from and/or sending output to a file
(I/O redirect) can save time and keystrokes.
Although the user may remain oblivious to this fact (since a shell is started when we open a terminal window), there are two types of commands: Unix/Linux commands and Shell commands.
Shell commands are commands that are built into a shell program. Shell commands are entered at the shell-prompt in a terminal window program (e.g., xterm, kterm, gnome-terminal). Although the command set built into different shells differs, the commands within a given shell are the same across Unix/Linux distributions.
Unix/Linux commands are not part of a shell, but correspond to separate executable programs. Each such program is written in a programming language like C (or C++, Fortran, Java, etc..). These programs tend to be located in system directories where shells can automatically find them, but where only the system administrator can create or modify files. The shell program itself is a Unix/Linux command. The set of available commands tends to vary from one Unix/Linux distribution to the next, and depends on which software packages the system administrator chose to install.
Most commands (both of the shell and Unix/Linux variety) accept one or more options and/or arguments. Options modify the default actions performed by a command, or the format of their output. Usually, single character options need to be preceded by a single dash (-), while most multi-character options need to be preceded by a double dash (--). For example:
user@machine> du --version
user@machine> du -h
user@machine> du -s -h
In the latter command, we could have merged the two single character options (this is why the double dash is needed for multi-character options):
user@machine> du -sh
Arguments are typically used to specify what file(s) the command should operate on and/or what file(s) to create. For example:
user@machine> du -sh /home/user/
user@machine> du -sh *
where, in the latter command, we used the wildcard *, which matches any and all files or directories.
cd,chdir | change directory | go to your home directory:
cd or cd ~ or cd /home/user go to your data directory: cd /data#/user return to previous directory: cd - |
echo | write arguments to the standard output of the shell (usually the screen) | echo "Hello, world!" echo -n "Hello, " ; echo "world!" |
exit | exits the shell | |
history | displays the history list of prior commands | show last 10 commands: history 10 |
kill | sends a termination signal to a process | kill process# dangerous: don't mistype process number! |
printenv,setenv | print or set environment variables | printenv HOSTTYPE |
source | reads a series of (shell)commands from a script | source ~/my_macro.txt |
which | shows the full path of (shell) commands | which arg narg range |
acroread | display and read PDF documents (see also gv, xpdf, and evince) | acroread document.pdf |
apropos | search for available commands that match one or more search string[s] | apropos 'search_keyword[s]' |
awk | pattern scanning and processing language | excise columns 3 and 4 from an ASCII table: cat table.txt | awk '{print $3,$4}' > mytable.txt excise cols 1, 3 and 4 from ASCII table, and reformat: cat table.txt | \ awk '{printf("%-12s %4d %7.4f\n",tolower($1),$3,$4)}' \ > mytable.txt |
cat | concatenate files and print on the standard output | cat ~/AA_README |
chmod | change file access permissions | make a script executable: chmod +x myscript or chmod 755 myscript set reasonable file permissions for a text file copied from a Microsoft® file system (VFAT,FAT32,NTFS) to a linux file system (ext2,ext3,ext4): chmod 644 some_file.txt |
clear | clear the terminal screen | |
cp | copy files and directories | move large downloaded data files to your data directory: cp ~/Desktop/bigfile*.fits /data#/user/Project/. |
csh, tcsh | start a C shell (when starting 'xterm' using the button on the desktop, 'tcsh' will be started up automatically) | |
cut | remove sections from each line in files | cut -d/ -f2-4 some_file_list | sort -u |
date | print (or set) the system date and time | print the current local and UTC time with default format: date ; date -u print date+time as a Y2K-compliant FITS date string: date -u +%Y-%m-%dT%T |
df | report filesystem disk space usage | report on all mounted file systems, use human-readable sizes: df -ah |
diff | find differences between two files | diff file1.txt file2.txt |
du | estimate file space usage | summarize file space usage, use human-readable sizes: du -sh ~/ ; du -sh /data#/user/ |
dvips | convert a LaTeX DVI file to Postscript | convert a LaTeX DVI file to PS suitable for later conversion
to PDF, while including all fonts and specifying the paper type to be
US Letter: dvips -Ppdf -Pamz -Pcmz -t letter ms1.dvi \ -o ms1_20130614.ps |
emacs | GNU Emacs text editor See: tutorial, beginners, survival.pdf and refcard.pdf. |
create and edit a ASCII text file in current 'xterm' window: emacs -nw newfile.txt open existing text file and use Emacs' own GUI window: emacs oldfile.txt |
enscript | print plain text files (nicer formatting than lpr) | print ASCII text (e.g., source code) in a 2-pages per page
landscape format, printing in duplex mode: enscript -2r program.c -p program.c.ps lpr -Pps -o Duplex=DuplexNoTumble program.c.ps |
evince | display, read, and print PDF documents | evince document.pdf (as of CentOS 6.x) |
find | search for files in a directory hierarchy | find all PDF files newer than 30 days in your home dir: find ~/ -type f -mtime -30 -name '*.pdf' |
g77, gfortran | GNU Fortran77 (+ some newer Fortran extensions) compiler | g77 -W -Wall -o program program.f -L/home/lib -lmytools (assuming some dependency on a nonstandard library "libmytools.a" that can be found in your ~/lib/ directory) |
gcc | GNU C compiler — without it, no Linux | gcc -W -Wall -o program program.c |
grep, fgrep, egrep | search for expressions in a file | search for upper-, lower-, or mixed case instances of "flatR"
in all files with names of the form "flat*.lis": grep -i 'flatR' `find /data#/user/ -type f -name 'flat*.lis'` |
gs | Ghostscript (PostScript and PDF language interpreter) | use as a basic PostScript viewer to view file "plot.ps": gs -sDEVICE=x11 plot.ps use as a filter to merge multiple PDF files into one: gs -dNOPAUSE -sDEVICE=pdfwrite \ -sOUTPUTFILE=file123.pdf -dBATCH \ file1.pdf file2.pdf file3.pdf |
gtar | GNU version of the tar archiving utility | create a gzip-compressed backup in archive mode (preserve
permissions and time stamps) of directory "Project": gtar -cpzf /tmp/backup.tgz Project inspect the contents of the tar-ball just created: gtar -tvpzf /tmp/backup.tgz restore this backup (some place else): gtar -xvpzf backup.tgz |
gv | Ghostview (PostScript and PDF viewer) | view a (possibly gzip-compressed) Postscript or PDF file: gv plot.ps or gv plot.ps.gz or gv plot.pdf |
gzip, gunzip | compress or expand files | compress all Postscript files and raw (integer valued) FITS
files of a project: gzip -v `find /data#/user/Project/ -type f -name '*.ps' \ -or -name '*.eps' -or '-name '*.cps' -or -name '*.fit' \ -or '-name '*.fits'` compress all files larger than 1 Mb in size, except files that are poorly compressible or already compressed: gzip -v `find /data#/user/Project/ -type f -size +1M | \ egrep -ve '(.pdf$|.bin$|.jar$|.exe$|.gz$|.tgz$|.zip$|.bz$)'` |
head | output the first few lines of files | display the first 22 lines in file "img.lis": head -22 img.lis display the 3rd batch of 10 lines in file "img.lis": head -30 img.lis | tail -10 |
info | read Info documents | info command |
latex | LaTeX, macro package for the TeX typesetting program (see the online LaTeX manual or wikibook); produces DVI file. (see also dvips). | latex ms1.tex ; latex ms1.tex (Note: latex needs to be run twice, or even trice, when you request automatic generation of a table of contents, figure and table numbering, and/or citations) |
pdflatex | PDFLaTeX (see latex), produces PDF output directly and can handle PDF,PNG,JPG embedded graphics | pdflatex ms1.tex ; pdflatex ms1.tex (Note: still needs to be run twice, or even trice, when you request automatic generation of a table of contents, figure and table numbering, and/or citations) |
less, more | page through a text file | less ~/AA_README |
ln | make links between files | create a symbolic link to a project directory that resides on
your data partition in your home directory: cd ; ln -s /data#/user/Project Project |
lpq | show printer queue status | lpq -Pps |
lpr | print plain text or Postscript files | ASCII text: better to use enscript; Postscript files: lpr -Pps -o Duplex=DuplexNoTumble program.c.ps |
lprm | cancel print jobs | find Job number (#) using lpq, then: lprm -Pps # |
ls | list directory contents | print a listing of all files in the current directory, for
both block and human-readable file sizes (include hidden files): ls -la and ls -lah the same in a directory other than where you are now (no need to change directories first): ls -lh ~/Project/src/python/ or ls -lh ../../data/new/ print a single-column listing of file names without colors or other markup (needed when creating input file lists): \ls -1 /data#/user/Project/*.fits |
man command | format and display the on-line manual pages for a command — singlemost important Unix/Linux command. | man ls |
mkdir | make directories | mkdir /data#/user/Project2 |
mozilla, firefox | web-browsers (offshoots of Netscape) | (usually started by clicking the icon on your desktop) |
mv | move (rename) files | mv /data#/user/Project/whatIdid20130614.txt ~/Notes/. |
paste | merge lines of files | paste the lines from two input files using a "/" as delimiting
character: paste -d/ paths.lis files.lis |
ps2pdf | distill a PDF from a Postscript document | convert your manuscript to PDF before sending it as an
email attachment: ps2pdf ms1_20130614.ps |
pwd | print name of current/working directory | where am i? pwd |
rm, rmdir | remove (irrecoverably!) files or directories | remove a file with confirmation: rm testimage.fits recursively remove the contents of a directory and that directory without any confirmation (any typo and you may be royally screwed): rm -fr ./testreduction |
rsync | copy/sync files to/from another machine via Secure Shell | copy all files that exist in the local directory but do not
exist in the corresponding remote directory, and update all files in
the remote directory that differ from the ones in the local directory.
Use compression (-z) to speed up transfers over slow networks,
and show (-v) what is going on: rsync -avz --rsh "ssh -l user" ./ hostname.hostdomain:/destination_path/. update local files using remote directory as source: rsync -avz --rsh "ssh -l user" hostname.hostdomain:/source_path/ . synchronise all but the hidden (configuration) files from your home directory on a remote machine: rsync -avz --rsh "ssh -l user" hostname.hostdomain:/home/user/\[0-9A-Za-z\]\* . |
sed | unix command-line editor | append the file extension ".fits" to all image root names in
a list of such names, and prepend a directory path: sed 's:$:.fits:g' imroot.lis | \ sed 's:^:/path/:g' > files.lis |
sleep | delay for a specified number of seconds | Test data transfer in Mb per minute: du -sm ./ ; sleep 60 ; du -sm ./ |
sort | sort lines of text files | Find unique file sizes: du -m `find . -type f` | awk '{print $1}' | sort -u |
ssh, scp | connect to or copy files to/from another machine via Secure Shell | login to a remote machine via a securely encripted connection: ssh username@hostname.hostdomain (you may need to specify the -X or -Y options to ssh to allow X11 forwarding, i.e., spawning applications from the remote host onto the local display) securely copy a local file to a remote host, preserving attributes: scp -p /source_path/filename user@hostname.hostdomain:/destination_path/. securely copy a remote file to current directory on local host: scp -p user@hostname.hostdomain:/source_path/filename . |
tail | output the last few lines of files | |
top | display top CPU processes | |
vi, vim | Unix text editor | |
w | show who is logged on and what they are doing | |
wc | print the number of bytes, words, and lines in files | count # of objects (lines) in list: wc -l objects.lis |
who | shows who is logged on | |
xpdf | display, read, and print PDF documents | xpdf document.pdf |
xv, eog | view and/or edit pictures in various image formats (e.g., gif,jpg,png,bmp,ppm,tiff,ras,ico,svg) | xv picture.jpg or eog picture.png |
The Unix/Linux directory path structure differs somewhat from Windows'.
Note, that Unix/Linux (contrary to, e.g., DOS) is case sensitive. So beware of inadvertently hitting your [Caps Lock] key (on most keyboards, there is a LED indicating whether it's on or off). In short, LS differs from Ls, lS and ls.
(file name) completion
Often, it's not necessary to type the
full name of a file. Type the first few characters, then hit the
[Tab] key. If you typed enough
characters to provide a unique match, the shell will add the remaining
characters of the name. If there are no files, or if there is more than one
file that starts with these first few characters, the shell will beep at you.
In that case, hit the [D] key while
depressing the [Ctrl] key —
commonly denoted as ^D and pronounced
"Control D" — and the shell will give you a list of all possible
matches. It is also possible that the shell will append additional
characters to the characters already typed, up to the character where more
than one possibility exists.
Example:
I want to edit a text file with a long name, that starts with an "i", and that
I know resides in a directory with absolute path /data1/me/ .
me@mymachine> vi /data1/me/iHitting [Tab] sounds a beep, so either the file name didn't start with an "i" after all, or there are more than one files or subdirectories with names that start with an "i". Hitting ^D produces:
me@mymachine> vi /data1/me/i idl/ iraf/ itinerary info_ast494_bck050901v1.txt me@mymachine> vi /data1/me/iSo to edit the "info" file with the long name, I only have to add an "n" to what I already typed and then hit [Tab], i.e.:
me@mymachine> vi /data1/me/in[Tab]Had there been also a file named "info_ast494_bck050901v2.txt" in the same directory, and if that were the file I wished to edit, then I would only need to type in the last step above:
me@mymachine> vi /data1/me/in[Tab]2[Tab]Similarly, if you know the first few characters of a command, then hitting ^D will give a listing of all available commands starting with these characters.
Copy/Paste using your mouse
To copy text from either the terminal
window you are working in right now or from another window (like the browser
you're viewing this web-page with) to your command-line or to a text file
that you are editing, you can use your mouse in several ways. A Unix/Linux
mouse generally has three (or more) buttons. If the mouse is a wheel-mouse,
depressing (rather than rotating) the wheel serves the function of a middle
button.
Method 1: Drag your mouse with the left-button depressed over
the portion of the text that you want to copy. The selected text will become
highlighted.
Method 2: Left-click on the first character, then right-click
on the last character of the portion of text to copy. The selected text will
become highlighted.
NB: Method 2 does not work within
applications in which the right-click has a special meaning (e.g., to pop-up
a menu)
Method 3: Double-clicking the left mouse button anywhere in a
word will select it. What is considered a word may depend on
the application.
Method 4: To select an entire line, triple-click with the left
mouse button.
Then move you mouse to the window were the text should be copied to, and hit the middle mouse button. Note that, if the text should be copied into a text editor, then (depending on the editor program) you either have to move the text cursor or the mouse pointer to the location where the text should appear before hitting the middle mouse button.
Command-line editing
Why re-type a previous command that was almost
what you want to do now? You can browse through the command history using the
up-arrow [↑] and down-arrow
[↓] keys. Instead of the arrow
keys, one can also use ^P and
^N (remember: ^P denotes hitting the [Ctrl]
and [P] keys simultaneously).
If the command was issued a while back, try searching the command history
using grep (note the pipe operation — the vertical bar
— that allows us to pass the output of the history command to
the input of the grep command):
me@mymachine> history | grep scp 1292 22:08 scp -p linux_intro.html me@myothermachine:/afs/asu.edu/users/me/www/linux/.
This command may be copied using the mouse. If I actually want to scp another file than the one I copied before, try hitting the ^A. This results in the text cursor jumping from the end to the start (left) of the command-line. You can now move the cursor forward to the begin of the file name one character at a time with either the right-arrow [→] key (opposite: left-arrow [←]) or by hitting ^F (opposite: ^B), or one "word" at a time using the [Meta]F (opposite: [Meta]B) keystroke combination. NB: Depending on your keyboard and/or brand of computer, one of [Alt], [Esc], or [Option] will serve as the [Meta] key. To delete the old file name hit ^D repeatedly. Note, that anywhere except at the end of the command line, ^D means delete-next-character. Now you can type the name of the file you do want to copy (typing the first few unique characters followed by [Tab] suffices). You can now move to the end of the command-line by hitting ^E. If the original destination directory differs from the one you want to copy to now, hit [Backspace] to delete the original name and then type the correct destination directory. Since the destination directory resides on another machine, directory name completion using [Tab] won't work in this example. Finally, hit [Enter] or [Return] to execute the modified command. Once you have a little practice with this, you'll find this way faster than re-typing every command or editing the command line with only the arrow and [Backspace] keys.
In case of a vi scare
I inadvertently started the vi editor and opened up a file; I
don't know vi commands yet and was just hitting some keys and
suddenly some text changed and the file may have been ruined. How can I
get out, hopefully restoring the file to its original content?
Don't worry. Start by hitting the [ESC] key once or twice to clear any of the modes that you may have inadvertently entered. Then hit the colon key, [ : ]. If a colon appears as the first character in the bottom line of the window, type q (i.e., hit the [Q] key). If a colon did not appear, hit [ESC] again, then type :q. This will exit vi without changing the original — all editing up to this point had only been performed in a copy of the file in the computer's memory.
Exiting editor programs
Exit editor programs correctly using the commands/keystrokes appropriate
for that editor. Many editor programs open up their own window instead of
running within a terminal window. For such programs, the warning below may
not apply. But never ever close an editor
program that runs within a terminal window by killing that window,
e.g., by clicking the "cross button" at the top right of the window bar.
The communication of the editor program with the window/shell from
which it was launched is lost that way and the now orphaned editor program
starts to take up all CPU time (try the top command; hit q to
exit). If this happens, carefully note the process number (leftmost column
in top) of your editor program (e.g., vi, emacs,
mem) and issue a kill process_number command. If
thereafter, the process still runs, issue a
kill −9 process_number command.
How can I read data from CDROM
Normal users, in general, will not have permission to change the hardware
configuration of a linux PC. Since mounting a CDROM falls technically in this
category, on some computers users are not allowed to mount and
umount a CDROM, while on others a special program automatically
detects when a CDROM has been inserted and will transparently mount it for
the user. On most, you can right-click on the CDROM icon on your desktop and
select the "Mount" option. If you wish (or need) to manually mount the CDROM,
users can temporarilly gain the extra privileges required using the sudo
command:
me@mymachine> sudo mount -t iso9660 /dev/cdrom /mnt/cdromdepending on the machine and version of Linux. You will be prompted for your user-password. After the CDROM is mounted on /mnt/cdrom, you can cd, ls and cp your data to a data directory on the hard disk. When you're done, type:
or: me@mymachine> sudo mount -t iso9660 /dev/scd0 /mnt/cdrom
me@mymachine> sudo umount /mnt/cdromor you can right-click on the CDROM icon on you desktop and select "Eject", which will first automatically dismount the CDROM. In either case, make sure you are no longer in the /mnt/cdrom/ directory, or a "Device busy" error will be returned.
For more info, tutorials, tips and tricks, see:
William Knottenbelt's "Introduction to UNIX course"
Juergen Haas', "Linux 101: A Newcomer's Guide"
Rafael Irizarry's, "Introduction to unix/linux"
Doctor Bob's, "Linux Tutorial"
Rolf Jansen's,
"Using
(C)shell scripts to make life more manageable"
And various guides available from www.codecoffee.com/tipsforlinux/ :
Using Emacs – part I and
Using Emacs – part II
Wildcards, Quotes, Back Quotes and Apostrophes in shell commands
How to use pipes – get the most out of your shell
Input/Output redirection in Unix
How to use the 'grep' command
How to find files using 'find'
Last updated: Oct 13, 2020