Unix FAQ's

Fundamental Unix Commands

2009-02-24T23:29:00.000-08:00

So now you're logged in. Great. Now what? Unix does not present itself to you through an interface that is graphical (GUI). You know, the screen full of icons and pictorial symbols and menus for you to interact with using your mouse. While using linux on the remote server, you won't even touch your mouse. I'm not bragging, mind you, I'm not saying that's good. Just pointing it out. What the operating system does present is merely a prompt. A symbol that mutely stares at you waiting. For anything to happen, you're supposed to type in a command. In its language, not yours. So you have to know one. That requires you to learn the commands that Unix has. As opposed to a graphical interface, this style of interaction is called a command-driven or character-based interface.

By the way, note I didn't say that Unix does not have a graphical user interface (because it does). I only said it doesn't present itself to you that way. Unix has an optional GUI, called the X windowing system, that can be started by issuing a certain command while in the character-based interface. But not while using telnet or ssh. So in this course, primary focus is on Unix used through its command-driven interface.

The point is that to get beyond the prompt, you have to start learning some commands. Right now. So you won't be stuck. Using Unix effectively in command mode means mastering the use of certain commands.

There are a whole lot of commands. Every beginning Unix tutorial seems to pick its own shortlist. The ones it thinks are the top priority. A kind of "top 10." While no particular pick is sacrosanct, it's not too hard for experienced Unix users to agree on certain high-frequency commands you should know.

What does "know" mean? How deeply? Commands generally have a lot of formal "options." These appear in their documentation. As an example, the ls command is for the purpose of listing the contents of a directory. At its simplest people use it by typing "ls, " nothing more, and pressing the enter key. A list of filenames typically appears (try it). However, ls has an option called -l (the letter ell) which prints out a little more information about each file, in a different format. And another, the -t option, that sorts the file list into chronological order. And forty-eight other options that appear in the documentation for the ls command. You don't need to know them all. But among them there are just a few that make life easier if you know them (or harder if you don't). And it's the same for the other commands.

I want to call to your attention a list of arguably "most important" commands. And for each, to its "most important" options. And I want you to familiarize yourself with each command option through actual use. Here's my list:

ls - list directory contents (like MS-DOS dir)
cat - send file content to screen (like MS-DOS type)
cd - change current directory (like MS-DOS cd)
chmod - change file permissions
cp - copy files and directories (like MS-DOS copy)
echo - write characters to the screen
find - find files (slow but fresh)
locate - find files (faster but stale)
grep - print lines matching a pattern
less - file filter for viewing
man - display on-line manual pages for individual commands
mkdir - make directories (like MS-DOS md)
mv - rename/move files (like MS-DOS ren or move)
ps - give a process status report
pwd - print name of the current, working directory
rm - remove files and directories (like MS-DOS del)
rmdir - remove empty directories (like MS-DOS rd)

Here are the several variations of each command you should become familiar with:

ls
list directory contents

[david@EMACH2 fish]$ ls
salmon trout

ls -l
long listing

[david@EMACH2 fish]$ ls -l
total 8
-rw-r--r-- 1 david david 7 Jan 9 2000 salmon
-rw-r--r-- 1 david david 6 Jan 9 2000 trout

ls -F
displays a slash after each directory, asterisk after each executable file, at sign after symbolic links

ls -a
displays all files, including invisible ones (those whose filenames begin with a period)

ls -t
displays files in order by the time of last modification

ls -R
recursively lists subdirectories

[david@EMACH2 animal]$ ls -R
.:
amphibian bird fish insect mammal reptile

amphibian:
frog newt

bird:
crow robin

fish:
salmon trout

insect:
ant mosquito

mammal:
cat horse

reptile:
lizard turtle

cd
makes your home directory the current, working directory (see pwd)

[david@EMACH2 log]$ pwd
/var/log
[david@EMACH2 log]$ cd
[david@EMACH2 david]$ pwd
/home/david

cd
makes the specified directory the current, working directory

[david@EMACH2 david]$ cd /usr/src/linux
[david@EMACH2 linux]$ pwd
/usr/src/linux

cd
makes the specified directory, which must be under the initial working directory, the new working directory

[david@EMACH2 taxonomy]$ pwd
/home/david/taxonomy
david@EMACH2 taxonomy]$ ls -F
animal/ mineral/ vegetable/
[david@EMACH2 taxonomy]$ cd mineral/
[david@EMACH2 mineral]$ pwd
/home/david/taxonomy/mineral

cp
creates a copy of the source file, applying the given destination file name to it

cp
creates a copy of the source file, without changing its name, in the given destination directory

cp -r
recursive. If any of the source files is a directory, all its contents plus those of any of its subdirectories are copied into the given destination directory

cp -p
preserves each file's characteristics when copying it (owner, group, permissions, modifications times)

mv Afile B
renames a file (Afile is an existing file, B is a new name for it)

mv Xdir Y
renames a directory (Xdir is an existing directory, Y is a new name for it)

mv Afile Xdir
moves a file into a specified directory (Afile is an existing file, Xdir is an existing directory)

rm -i
removes files, interactively (asks before removing each file)

rm -f
removes files with "force" (doesn't ask)

rm -r
removes files recursively, including those found in all subdirectories of the one specified (or current working directory if none specified)

rm -rf *
removes all files in the current and subordinate directories, and also those directories. Potentially dangerous if executed from within the wrong directory.

cat Afile Bfile > Cfile
concatenate (paste together end to end) contents of two files, send result (>) to a new file (result: C = A + B)

cat /etc/lilo.conf
extract contents of the file, send result to the terminal

cat -n
number the lines as they are output

cat --show-all
show non-printing as well as printing characters (e.g., tabs, linefeeds)

man
display documentation for the named command, explaining all its options just as we're doing here

less
prints contents of a file (or other input) one screenful at a time to prevent scrolling out of sight. Press enter to advance a line, spacebar to advance a full screen, : q to exit.

grep
searches for a pattern in files. Frequently used to print all lines in file containing a certain word or phrase

| grep
searches the output produced by a command (in place of the contents of a file) for a pattern

grep -v
reverses search outcome, searching for lines the do NOT contain the target pattern

ps
process status. Displays status information about active processes that your terminal controls including PID (process id) numbers

[david@EMACH2 david]$ ps
PID TTY TIME CMD
1235 pts/0 00:00:00 bash
1820 pts/0 00:00:00 ps

ps ax
displays status information about all processes

chmod u+r
add to or remove from (+ or -) a file(s) any of 3 different kinds of permission (r or w or x) for any of 3 different user constituencies (u or g or o)
used to restrict/extend access to others, and to enable script execution

chmod 777
same, simultaneously setting multiple permissions using a numeric notation

locate
search for files on the computer (fast and stale)

find / -name -print
search for files on the computer (slow and fresh)

Note that there are a number of different kinds of "Unix." And there are some command differences among them. They have the great majority of the commands in common, but sometimes a command with which you are familiar in one version of Unix isn't included in another one. One of the kinds of Unix that is widely popular is called FreeBSD. The one used on the remote server is Linux. Are such different versions the same?? They are like different dialects of the same language (English in New Zealand, Canada, Scotland, South Africa, ...). There is both a little difference and a lot of similarity. But a lot more same than different.

Below are some links to other peoples' "top 10" command sets, and a linux command "cheatsheet" you might print out. Spend some time exploring them.

One

... and another

Linux command cheatsheet:
page1 page 2

An Overview of the UNIX* Operating System

2008-10-15T04:35:00.000-07:00

The UNIXError! Hyperlink reference not valid. operating system was designed to let a number of programmers access the computer at the same time and share its resources.

The operating system coordinates the use of the computer's resources, allowing one person, for example, to run a spell check program while another creates a document, lets another edit a document while another creates graphics, and lets another user format a document -- all at the same time, with each user oblivious to the activities of the others.

The operating system controls all of the commands from all of the keyboards and all of the data being generated, and permits each user to believe he or she is the only person working on the computer.

This real-time sharing of resources make UNIX one of the most powerful operating systems ever.

Although UNIX was developed by programmers for programmers, it provides an environment so powerful and flexible that it is found in businesses, sciences, academia, and industry. Many telecommunications switches and transmission systems also are controlled by administration and maintenance systems based on UNIX.

While initially designed for medium-sized minicomputers, the operating system was soon moved to larger, more powerful mainframe computers. As personal computers grew in popularity, versions of UNIX found their way into these boxes, and a number of companies produce UNIX-based machines for the scientific and programming communities.

The uniqueness of UNIX

The features that made UNIX a hit from the start are:

Multitasking capability
Multiuser capability
Portability
UNIX programs
Library of application software

Multitasking

Many computers do just one thing at a time, as anyone who uses a PC or laptop can attest. Try logging onto your company's network while opening your browser while opening a word processing program. Chances are the processor will freeze for a few seconds while it sorts out the multiple instructions.

UNIX, on the other hand, lets a computer do several things at once, such as printing out one file while the user edits another file. This is a major feature for users, since users don't have to wait for one application to end before starting another one.

Multiusers

The same design that permits multitasking permits multiple users to use the computer. The computer can take the commands of a number of users -- determined by the design of the computer -- to run programs, access files, and print documents at the same time.

The computer can't tell the printer to print all the requests at once, but it does prioritize the requests to keep everything orderly. It also lets several users access the same document by compartmentalizing the document so that the changes of one user don't override the changes of another user.

System portability

A major contribution of the UNIX system was its portability, permitting it to move from one brand of computer to another with a minimum of code changes. At a time when different computer lines of the same vendor didn't talk to each other -- yet alone machines of multiple vendors -- that meant a great savings in both hardware and software upgrades.

It also meant that the operating system could be upgraded without having all the customer's data inputted again. And new versions of UNIX were backward compatible with older versions, making it easier for companies to upgrade in an orderly manner.

UNIX tools

UNIX comes with hundreds of programs that can divided into two classes:

Integral utilities that are absolutely necessary for the operation of the computer, such as the command interpreter, and
Tools that aren't necessary for the operation of UNIX but provide the user with additional capabilities, such as typesetting capabilities and e-mail.

Tools can be added or removed from a UNIX system, depending upon the applications required.

UNIX Communications

E-mail is commonplace today, but it has only come into its own in the business community within the last 10 years. Not so with UNIX users, who have been enjoying e-mail for several decades.

UNIX e-mail at first permitted users on the same computer to communicate with each other via their terminals. Then users on different machines, even made by different vendors, were connected to support e-mail. And finally, UNIX systems around the world were linked into a world wide web decades before the development of today's World Wide Web.

Applications libraries

UNIX as it is known today didn't just develop overnight. Nor were just a few people responsible for it's growth. As soon as it moved from Bell Labs into the universities, every computer programmer worth his or her own salt started developing programs for UNIX.

Today there are hundreds of UNIX applications that can be purchased from third-party vendors, in addition to the applications that come with UNIX.

How UNIX is organized

The UNIX system is functionally organized at three levels:

The kernel, which schedules tasks and manages storage;
The shell, which connects and interprets users' commands, calls programs from memory, and executes them; and
The tools and applications that offer additional functionality to the operating system

The three levels of the UNIX system: kernel, shell, and tools and applications.

The kernel

The heart of the operating system, the kernel controls the hardware and turns part of the system on and off at the programer's command. If you ask the computer to list (ls) all the files in a directory, the kernel tells the computer to read all the files in that directory from the disk and display them on your screen.

The shell

There are several types of shell, most notably the command driven Bourne Shell and the C Shell (no pun intended), and menu-driven shells that make it easier for beginners to use. Whatever shell is used, its purpose remains the same -- to act as an interpreter between the user and the computer.

The shell also provides the functionality of "pipes," whereby a number of commands can be linked together by a user, permitting the output of one program to become the input to another program.

Tools and applications

There are hundreds of tools available to UNIX users, although some have been written by third party vendors for specific applications. Typically, tools are grouped into categories for certain functions, such as word processing, business applications, or programming.

References

Want to know more about UNIX?

Unix File System

2008-10-15T04:30:00.000-07:00

UNIX Unleashed, System Administrator's Edition

- 18 -

File System and Disk Administration

By Steve Shah

This chapter discusses the trials and tribulations of creating, maintaining, and repairing file systems. While these tasks may appear simple from a user's standpoint, they are, in fact, intricate and contain more than a handful of nuances . In the course of this chapter, we'll step through many of these nuances and, hopefully, come to a strong understanding of the hows and whys of file systems.

Before we really jump into the topic, you should have a good understanding of UNIX directories, files, permissions, and paths. These are the key building blocks in understanding how to administer your file systems, and I assume you already have a mastery of them. If the statement, "Be sure to have /usr/bin before /usr/local/bin in your $PATH" confuses you in any way, you should be reading something more fundamental first. Refer to Part I, "Introduction to UNIX," for some basic instructions in UNIX.

This chapter goes about the explanation of file systems a bit differently than other books. We first discuss the maintenance and repair of file systems, then discuss their creation. This was done because it is more likely that you already have existing file systems you need to maintain and fix. Understanding how to maintain them also helps you better understand why file systems are created the way they are.

The techniques we cover here are applicable to most UNIX systems currently in use. The only exceptions are when we actually create the file systems. This is where the most deviation from any standard (if there ever was one) occurs. We cover the creation of file systems under the SunOS, Solaris, Linux, and IRIX implementations of UNIX. If you are not using one of these operating systems, you should check the documentation that came with your operating system for details on the creation of file systems.

--------------------------------------------------------------------------------

CAUTION: Working with file systems is inherently dangerous. You may be surprised at how quickly and easily you can damage a file system beyond repair. In some instances, it is even possible to damage the disk drive as well. BE CAREFUL. When performing the actions explained in this chapter, be sure you have typed the commands in correctly and you understand the resulting function fully before executing it. When in doubt, consult the documentation that came from the manufacturer. Most importantly, the documentation that comes from the manufacturer is always more authoritative than any book.

--------------------------------------------------------------------------------

NOTE: You should read the entire chapter before actually performing any of the tasks below. This will give you a better understanding of how all the components work together, thereby giving you more solid ground when performing potentially dangerous activities.

--------------------------------------------------------------------------------

What Is a File System

The file system is the primary means of file storage in UNIX. Each file system houses directories, which, as a group, can be placed almost anywhere in the UNIX directory tree. The topmost level of the directory tree, the root directory, begins at /. Subdirectories nested below the root directory may traverse as deep as you like so long as the longest absolute path is less than 1,024 characters.

With the proliferation of vendor-enhanced versions of UNIX, you will find a number of "enhanced" file systems. From the standpoint of the administrator, you shouldn't have to worry about the differences too much. The two instances where you will need to worry about vendor-specific details are in the creation of file systems and when performing backups. We will cover the specifics of:

SunOS 4.1.x, which uses 4.2

Solaris, which uses ufs

Linux, which uses ext2

IRIX, which uses efs and xfs

Note that the ufs and 4.2 file systems are actually the same.

A file system, however, is only a part of the grand scheme of how UNIX keeps its data on disk. At the top level, you'll find the disks themselves. These disks are then broken into partitions, each varying in size depending on the needs of the administrator. It is on each partition that the actual file system is laid out. Within the file system, you'll find directories, subdirectories, and, finally, the individual files.

Although you will rarely have to deal with the file system at a level lower than the individual files stored on it, it is critical that you understand two key concepts: inodes and the superblock. Once you understand these, you will find that the behavior and characteristics of files make more sense.

inodes

An inode maintains information about each file. Depending on the type of file system, the inode can contain upwards of 40+ pieces of information. Most of it, however, is only useful to the kernel and doesn't concern us. The fields that do concern us are

mode The permission mask and type of file.

link count The number of directories that contain an entry with this inode number.

user ID The ID of the file's owner.

group ID The ID of the file's group.

size Number of bytes in this file.

access time The time at which the file was last accessed.

mod time The time at which the file was last modified.

inode time The time at which this inode structure was last modified.

block list A list of disk block numbers which contain the first segment of the file.

indirect list A list of other block lists.

The mode, link count, user ID, group ID, size, and access time are used when generating file listings. Note that the inode does not contain the file's name. That information is held in the directory file (see below for details).

Superblocks

This is the most vital information stored on the disk. It contains information on the disk's geometry (number of heads, cylinders, and so on), the head of the inode list, and free block list. Because of its importance, the system automatically keeps mirrors of this data scattered around the disk for redundancy. You only have to deal with superblocks if your file system becomes heavily corrupted.

Types of Files

Files come in 8 flavors:

Normal Files

Directories

Hard Links

Symbolic links

Sockets

Named Pipes

Character Devices

Block Devices

Normal Files These are the files you use the most. They can be either text or binary files; however, their internal structure is irrelevant from a System Administrator standpoint. A file's characteristics are specified by the inode in the file system that describes it. An ls -l on a normal file will look something like this:

-rw------- 1 sshah admin 42 May 12 13:09 hello

Directories These are a special kind of file that contains a list of other files. Although there is a one-to-one mapping of inode to disk blocks, there can be a many-to-one mapping from directory entry to inode. When viewing a directory listing using the ls -l command, you can identify directories by their permissions starting with the d character. An ls -l on a directory looks something like this:

drwx------ 2 sshah admin 512 May 12 13:08 public_html

Hard Links

A hard link is actually a normal directory entry except instead of pointing to a unique file , it points to an already existing file . This gives the illusion that there are two identical files when you do a directory listing. Because the system sees this as just another file, it treats it as such. This is most apparent during backups because hard-linked files get backed up as many times as there are hard links to them. Because a hard link shares an inode, it cannot exist across file systems. Hard links are created with the ln command. For example, given this directory listing using ls -l, we see:

-rw------- 1 sshah admin 42 May 12 13:04 hello

When you type ln hello goodbye and then perform another directory listing using ls -l, you see:

-rw------- 2 sshah admin 42 May 12 13:04 goodbye

-rw------- 2 sshah admin 42 May 12 13:04 hello

Notice how this appears to be two separate files that just happen to have the same file lengths. Also note that the link count (second column) has increased from one to two. How can you tell they actually are the same file? Use ls -il. Observe:

13180 -rw------- 2 sshah admin 42 May 12 13:04 goodbye

13180 -rw------- 2 sshah admin 42 May 12 13:04 hello

You can see that both point to the same inode, 13180.

--------------------------------------------------------------------------------

WARNING: Be careful when creating hardlinks, especially when hardlinking to a directory. It is possible to corrupt a filesystem by doing so since the hardlink does not contain the fact that the i-node being pointed to needs to be treated as a directory.

--------------------------------------------------------------------------------

Symbolic Links

A symbolic link (sometimes referred to as a symlink) differs from a hard link because it doesn't point to another inode but to another filename. This allows symbolic links to exist across file systems as well as be recognized as a special file to the operating system. You will find symbolic links to be crucial to the administration of your file systems, especially when trying to give the appearance of a seamless system when there isn't one. Symbolic links are created using the ln -s command. A common thing people do is create a symbolic link to a directory that has moved. For example, if you are accustomed to accessing the directory for your home page in the subdirectory www but at the new site you work at, home pages are kept in the public_html directory, you can create a symbolic link from www to public_html using the command ln -s public_html www. Performing an ls -l on the result shows the link.

drwx------ 2 sshah admin 512 May 12 13:08 public_html

lrwx------ 1 sshah admin 11 May 12 13:08 www -> public_html

Sockets

Sockets are the means for UNIX to network with other machines. Typically, this is done using network ports; however, the file system has a provision to allow for interprocess communication through socket files. (A popular program that uses this technique is the X Windows system.) You rarely have to deal with this kind of file and should never have to create it yourself (unless you're writing the program). If you need to remove a socket file, use the rm command. Socket files are identified by their permission settings beginning with an s character. An ls -l on a socket file looks something like this:

srwxrwxrwx 1 root admin 0 May 10 14:38 X0

Named Pipes

Similar to sockets, named pipes enable programs to communicate with one another through the file system. You can use the mknod command to create a named pipe. Named pipes are recognizable by their permissions settings beginning with the p character. An ls -l on a named pipe looks something like this:

prw------- 1 sshah admin 0 May 12 22:02 mypipe

Character Devices

These special files are typically found in the /dev directory and provide a mechanism for communicating with system device drivers through the file system one character at a time. They are easily noticed by their permission bits starting with the c character. Each character file contains two special numbers, the major and minor. These two numbers identify which device driver that file communicates with. An ls -l on a character device looks something like this:

crw-rw-rw- 1 root wheel 21, 4 May 12 13:40 ptyp4

Block Devices

Block devices also share many characteristics with character devices in that they exist in the /dev directory, are used to communicate with device drivers, and have major and minor numbers. The key difference is that block devices typically transfer large blocks of data at a time versus one character at a time. (A hard disk is a block device, whereas a terminal is a character device.) Block devices are identified by their permission bits starting with the b character. An ls -l on a block device looks something like this:

brw------- 2 root staff 16, 2 Jul 29 1992 fd0c

Managing File Systems

Managing file systems is relatively easy. That is, once you can commit to memory the location of all the key files in the directory tree on each major variation of UNIX as well as your own layout of file systems across the networkÉ

In other words, it can be a royal pain.

From a technical standpoint there isn't much to deal with. Once the file systems have been put in their correct places and the boot time configuration files have been edited so that your file systems automatically come online at every start up, there isn't much to do besides watch your disk space.

From a management standpoint, it's much more involved. Often you'll need to deal with existing configurations, which may not have been done "the right way," or you're dealing with site politics such as, "I won't let that department share my disks." Then you'll need to deal with users who don't understand why they need to periodically clean up their home directories. Don't forget the ever exciting vendor-specific nuisances and their idea of how the system "should be" organized.

This section covers the tools you need to manage the technical issues. Unfortunately, managerial issues are something that can't be covered in a book. Each site has different needs as well as different resources, resulting in different policies. If your site lacks any written policy, take the initiative to write one yourself.

Mounting and Unmounting File Systems

As I mentioned earlier in this chapter, part of the power in UNIX stems from its flexibility in placing file systems anywhere in the directory tree. This feat is accomplished by mounting file systems.

Before you can mount a file system, you need to select a mount point. A mount point is the directory entry in the file system where the root directory of a different file system will overlay it. UNIX keeps track of mount points, and accesses the correct file system, depending on which directory the user is currently in. A mount point may exist anywhere in the directory tree.

--------------------------------------------------------------------------------

NOTE: While it is technically true that you can mount a file system anywhere in the directory tree, there is one place you will NOT want to mount it: the root directory. Remember that once a file system is mounted at a directory, that directory is overshadowed by the contents of the mounted file system. Hence, by mounting on the root directory, the system will no longer be able to see its own kernel or local configuration files. How long your system goes on before crashing depends on your vendor.

There is an exception to the rule. Some installation packages will mount a network file system to the root directory. This is done to give the installation software access to many packages that may not be able to fit on your boot disk. Unless you fully understand how to do this yourself, don't.

--------------------------------------------------------------------------------

Mounting and Unmounting File Systems Manually To mount a file system, use the mount command:

mount /dev/device /directory/to/mount

where /dev/device is the device name you want to mount and /directory/to/mount is the directory you want to overlay in your local file system. For example, if you wanted to mount /dev/hda4 to the /usr directory, you would type:

mount /dev/hda4 /usr

Remember that the directory must exist in your local file system before anything can be mounted there.

There are options that can be passed to the mount command. The most important characteristics are specified in the -o option. These characteristics are:

rw read/write

ro read only

bg background mount (if the mount fails,

place the process into the background

and keep trying until success.)

intr interruptible mount (if a process is

pending I/O on a mounted partition, it

will allow the process to be interrupted

and the I/O call dropped)

An example of these parameters being used is:

mount -o rw,bg,intr /dev/hda4 /usr

See the man page on your system for vendor specific additions.

To unmount a file system, use the umount command. For example:

umount /usr

This unmounts the /usr file system from the current directory tree, unveiling the original directory underneath it.

There is, of course, a caveat. If users are using files on a mounted file system, you cannot unmount it. All files must be closed before this can happen, which on a large system can be tricky to say the least. There are three ways to handle this:

Use the lsof program (available at ftp://vic.cc.purdue.edu/pub/tools/unix/lsof) to list the users and their open files on a given file system. Then either wait until they are done, beg them to leave, or kill their processes off. Then unmount the file system. Often, this isn't very desirable.

Use the -f option with umount command to force the unmount. This is often a bad idea because it leaves the programs (and users) accessing the partition confused. Files which are in memory that have not been committed to disk may be lost.

Bring the system to single user mode, then unmount the file system. While the largest inconvenience, it is the safest way because no one loses any work.

Mounting File Systems Automatically At boot time, the system automatically mounts the root file system with read-only privileges. This enables it to load the kernel and read critical startup files. However, once it has bootstrapped itself, it needs guidance. Although it is possible for you to mount all the file systems by hand, it isn't realistic because you would then have to finish bootstrapping the machine yourself, and worse, the system could not come back online by itself. (Unless, of course, you enjoy coming into work at 2 a.m. to bring a system back up.)

To get around this, UNIX uses a special file called /etc/fstab (/etc/vfstab under Solaris). This file lists all the partitions that need to be mounted at boot time and the directory where they need to be mounted. Along with that information you can pass parameters to the mount command.

Each file system to be mounted is listed in the fstab file in the following format:

/dev/device /dir/to/mount ftype parameters fs_freq fs_passno

where:

/dev/device Is the device to be mounted, for instance, /dev/hda4.

/dir/to/mount Is the location at which the file system should be mounted on your directory tree.

ftype Is the file system type. This should be 4.2 under SunOS, ufs under Solaris, ext2 under Linux, efs or xfs in IRIX (depending on your version), nfs for NFS mounted file systems, swap for swap partitions, and proc for the /proc file system. Some operating systems, such as Linux, support additional filesystem types, although they are not as likely to be used.

parameters Are the parameters we passed to mount using the -o option. They follow the same comma-delineated format. An example entry would look like rw,intr,bg.

fs_freq Is used by dump to determine whether a file system needs to be dumped.

fs_passno Is used by the fsck program to determine the order to check disks at boot time.

Any lines in the fstab file that start with the pound symbol (#) are considered comments.

If you need to mount a new file system while the machine is live, you must perform the mount by hand. If you wish to have this mount automatically active the next time the system is rebooted, you should be sure to add the appropriate entry to your fstab file.

There are two notable partitions that don't follow the same set of rules as normal partitions. They are the swap partition and /proc. (Note that SunOS does not use the /proc file system.)

Mounting the swap partition is not done using the mount command. It is instead managed by the swap command under Solaris and IRIX, and by the swapon command under SunOS and Linux. In order for a swap partition to be mounted, it must be listed in the appropriate fstab file. Once it's there, use the appropriate command (swap or swapon) with the -a parameter followed by the partition on which you've allocated swap space.

The /proc file system is even stranger because it really isn't a file system. It is an interface to the kernel abstracted into a file system style format. This should be listed in your fstab file with file system type proc.

--------------------------------------------------------------------------------

TIP: If you need to remount a file system that already has an entry in the fstab file, you don't need to type in the mount command with all the parameters. Instead, simply pass the directory to mount as a parameter like this:

mount /dir/to/mount

mount automatically looks to the fstab file for all the details, such as which partition to mount and which options to use.

If you need to remount a large number of file systems that are already listed in the fstab file (in other words, you need to remount directories from a system that has gone down), you can use the -a option in the mount command to try and remount all the entries in the fstab file like this:

mount -a

If mount finds that a file system is already mounted, no action is performed on that file system. If, on the other hand, mount finds that an entry is not mounted, it automatically mounts it with the appropriate parameters.

--------------------------------------------------------------------------------

Here is a complete fstab file from a SunOS systems:

# Sample /etc/fstab file for a SunOS machine

# Local mounts

/dev/sd0a / 4.2 rw 1 1

/dev/sd0g /usr 4.2 rw 1 2

/dev/sd0b swap swap rw 0 0

/dev/sd0d /var 4.2 rw 0 0

# Remote mounts

server1:/export/home /home nfs rw,bg,intr 0 0

server1:/export/usr/local /usr/local nfs rw,bg,intr 0 0

server2:/export/var/spool/mail /var/spool/mail nfs rw,bg,intr 0 0

Common Commands for File System Management

In taking care of your system, you'll quickly find that you can use these commands and many of their parameters without having to look them up. This is because you're going to be using them all the time. I highly suggest you learn to love them.

--------------------------------------------------------------------------------

NOTE: In reading this book you may have noticed the terms program and command are used interchangably. This is because there are no "built in" commands to the system, each one is invoked as an individual program. However, you will quickly find that both the people who use UNIX, as well as UNIX related texts (such as this one), use the both terms to mean the same thing. Confusing? A bit. But it's tough to change 25+ years of history.

--------------------------------------------------------------------------------

NOTE: At the end of each command description, I mention the GNU equivalent. Linux users shouldn't worry about getting them, because Linux ships with all GNU tools. If you are using another platform and aren't sure whether you're using the GNU version, try running the command with the --version option. If it is GNU, it will display its title and version number. If it isn't GNU, it'll most likely reject the parameter and give an error.

--------------------------------------------------------------------------------

df The df command summarizes the free disk space by file system. Running it without any parameters displays all the information about normally mounted and NFS mounted file systems. The output varies from vendor to vendor (under Solaris, use df -t) but should closely resemble this:

Filesystem 1024-blocks Used Available Capacity Mounted on

/dev/hda3 247871 212909 22161 91% /

/dev/hda6 50717 15507 32591 32% /var

/dev/hda7 481998 15 457087 0% /local

server1:/var/spool/mail

489702 222422 218310 50% /var/spool/mail

The columns reported show: Filesystem Which file system is being shown. File systems mounted using NFS are shown as hostname:/dir/that/is/mounted

1024-blocks The number of 1 KB blocks the file system consists of. (Its total size.)

Used The number of blocks used.

Available The number of blocks available for use.

Capacity Percentage of partition currently used.

Mounted on The location in the directory tree this partition has been mounted on.

Common parameters to this command are:

directory Show information only for the partition on which the specified directory exists.

-a Show all partitions including swap and /proc.

-i Show inode usage instead of block usage.

The GNU df program, which is part of the fileutils distribution, has some additional print formatting features you may find useful. You can download the latest fileutils package at ftp://ftp.cdrom.com/pub/gnu.

du The du command summarizes disk usage by directory. It recurses through all subdirectories and shows disk usage by each subdirectory with a final total at the end. Running it without any parameters shows the usage like so:

409 ./doc

945 ./lib

68 ./man

60 ./m4

391 ./src

141 ./intl

873 ./po

3402 .

The first column shows the blocks of disk used by the subdirectory, and the second column shows the subdirectory being evaluated. To see how many kilobytes each subdirectory consumes, use the -k option. Some common parameters to this command are

directory Show usage for the specified directory. The default is the current directory.

-a Show usage for all files, not just directories.

-s Show only the total disk usage.

Like the df program, this program is available as part of the GNU fileutils distribution. The GNU version has expanded on many of the parameters which you may find useful. The fileutils package can be downloaded from ftp://ftp.cdrom.com/pub/gnu.

ln The ln program is used to generate links between files. This is very useful for creating the illusion of a perfect file system in which everything is in the "right" place when, in reality, it isn't. This is done by making a link from the desired location to the actual location.

The usage of this program is

ln file_being_linked_to link_name

where file_being_linked_to is the file that already exists, and you wish to have another file point to it called link_name. The command above generates a hard link, meaning that the file link_name will be indistinguishable from the original file. Both files must exist on the same file system.

A popular parameter to ln is the -s option, which generates symbolic links instead of hard links. The format of the command remains the same:

ln -s file_being_linked_to link_name

the difference being that the link_name file is marked as a symbolic link in the file system. Symbolic links may span file systems and are given a special tag in the directory entry.

--------------------------------------------------------------------------------

TIP: Unless there is an explicit reason not to, you should always use symbolic links by specifying the -s option to ln. This makes your links stand out and makes it easy to move them from one file system to another.

--------------------------------------------------------------------------------

tar The tar program is an immensely useful archiving utility. It can combine an entire directory tree into one large file suitable for transferring or compression.

The command line format of this program is:

tar parameters filelist

Common parameters are:

c Create an archive

x Extract the archive

v Be verbose

f Specify a tar file to work on

p Retain file permissions and ownerships

t View the contents of an archive.

Unlike most other UNIX commands, the parameters do not need to have a dash before them.

To create the tarfile myCode.tar, I could use tar in the following manners:

tar cf myCode.tar myCode

where myCode is a subdirectory relative to the current directory where the files I wish to archive are located.

tar cvf myCode.tar myCode

Same as the previous tar invocation, although this time it lists all the files added to the archive on the screen.

tar cf myCode.tar myCode/*.c

This archives all the files in the myCode directory that are suffixed by .c

tar cf myCode.tar myCode/*.c myCode/*.h

This archives all the files in the myCode directory that are suffixed by .c or .h

To view the contents of the myCode.tar file, use:

tar tf myCode.tar

To extract the files in the myCode.tar file, use:

tar xf myCode.tar

If the myCode directory doesn't exist, tar creates it. If the myCode directory does exist, any files in that directory are overwritten by the ones being untarred.

tar xvf myCode.tar

Same as the previous invocation of tar, but this lists the files as they are being extracted.

tar xpf myCode.tar

Same as the previous invocation of tar, but this attempts to set the permissions of the unarchived files to the values they had before archiving (very useful if you're untarring files as root).

--------------------------------------------------------------------------------

TIP: The greatest use of tar for Systems Administrators is to move directory trees around. This can be done using the following set of commands:

(cd /src;tar cf - *) | (cd /dest;tar xpf -)

where /src is the source directory and /dest is the destination directory.

This is better than using a recursive copy because symbolic links and file permissions are kept. Use this and amaze your friends.

--------------------------------------------------------------------------------

While the stock tar that comes with your system works fine for most uses, you may find that the GNU version of tar has some nicer features. You can find the latest version of GNU tar at ftp://ftp.cdrom.com/pub/gnu.

find Of the commands that I've mentioned so far, you're likely to use find the most. Its purpose is to find files or patterns of files. The parameters for this tool are

find dir parameters

where dir is the directory where the search begins, and parameters define what is being searched for. The most common parameters you will use are:

-name Specify the filename or wildcards to look for. If you use any wildcards, be sure to place them inside of quotes so that the shell doesn't parse them before find does.

-print Typically turned on by default, it tells find to display the resulting file list.

-exec Executes the specified command on files found matching the -name criteria.

-atime n File was last accessed n days ago.

-mtime n File's data was last modified n days ago.

-size n[bckw] File uses n units of space where the units are specified by either b,c,k, or w. b is for 512 byte blocks, c is bytes, k is kilobytes, and w is two-byte words.

-xdev Do not traverse down nonlocal file systems.

-o Logical or the options.

-a Logical and the options.

Some examples of the find command are

find / -name core -mtime +7 -print -exec /bin/rm {} \;

This starts its search from the root directory and finds all files named core that have not been modified in seven days.

find / -xdev -atime +60 -a -mtime +60 -print

This searches all files, from the root directory down, on the local file system, which have not been accessed for at least 60 days and have not been modified for at least 60 days, and prints the list. This is useful for finding those files that people claim they need but, in reality, never use.

find /home -size +500k -print

This searches all files from /home down and lists them if they are greater than 500 KB in size. A handy way of finding large files in the system.

The GNU version of find, which comes with the findutils package, offers many additional features you will find useful. You can download the latest version from ftp://ftp.cdrom.com/pub/gnu.

Repairing File Systems with fsck

Sooner or later, it happens: Someone turns off the power switch. The power outage lasts longer than your UPS's batteries and you didn't shut down the system. Someone presses the reset button. Someone overwrites part of your disk. A critical sector on the disk develops a flaw. If you run UNIX long enough, eventually a halt occurs where the system did not write the remaining cached information (sync'ed) to the disks.

When this happens, you need to verify the integrity of each of the file systems. This is necessary because if the structure is not correct, using the file systems could quickly damage them beyond repair. Over the years, UNIX has developed a very sophisticated file system integrity check that can usually recover the problem. It's called fsck.

The fsck Utility

The fsck utility takes its understanding of the internals of the various UNIX file systems and attempts to verify that all the links and blocks are correctly tied together. It runs in five passes, each of which checks a different part of the linkage and each of which builds on the verifications and corrections of the prior passes.

fsck walks the file system, starting with the superblock. It then deals with the allocated disk blocks, pathnames, directory connectivity, link reference counts, and the free list of blocks and inodes.

--------------------------------------------------------------------------------

NOTE: The xfs filesystem now shipped with all IRIX based machines no longer need the fsck command.

--------------------------------------------------------------------------------

The Superblock Every change to the file system affects the superblock, which is why it is cached in RAM. Periodically, at the sync interval, it is written to disk. If it is corrupted, fsck checks and corrects it. If it is so badly corrupted that fsck cannot do its work, find the paper you saved when you built the file system and use the -b option with fsck to give it an alternate superblock to use. The superblock is the head of each of the lists that make up the file system, and it maintains counts of free blocks and inodes.

Inodes fsck validates each of the inodes. It makes sure that each block in the block allocation list is not on the block allocation list in any other inode, that the size is correct, and that the link count is correct. If the inodes are correct, then the data is accessible. All that's left is to verify the pathnames.

What Is a Clean (Stable) File System?

Sometimes fsck responds

/opt: stable (ufs file systems)

This means that the superblock is marked clean and that no changes have been made to the file system since it was marked clean. First, the system marks the superblock as dirty; then it starts modifying the rest of the file system. When the buffer cache is empty and all pending writes are complete, it goes back and marks the superblock as clean. If the superblock is marked clean, there is normally no reason to run fsck, so unless fsck is told to ignore the clean flag, it just prints this notice and skips over this file system.

Where Is fsck?

When you run fsck, you are running an executable in either the /usr/sbin or /bin directory called fsck, but this is not the real fsck. It is just a dispatcher that invokes a file system type-specific fsck utility.

When Should I Run fsck?

Normally, you do not have to run fsck. The system runs it automatically when you try to mount a file system at boot time that is dirty. However, problems can creep up on you. Software and hardware glitches do occur from time to time. It wouldn't hurt to run fsck just after performing the monthly backups.

--------------------------------------------------------------------------------

CAUTION: It is better to run fsck after the backups rather than before. If fsck finds major problems, it could leave the file system in worse shape than it was prior to running. Then you can just build an empty file system and reread your backup, which also cleans up the file system. If you did it in the other order, you would be left with no backup and no file system.

--------------------------------------------------------------------------------

How Do I Run fsck?

Because the system normally runs it for you, running fsck is not an everyday occurrence for you to remember. However, it is quite simple and mostly automatic.

First, to run fsck, the file system you intend to check must not be mounted. This is a bit hard to do if you are in multiuser mode most of the time, so to run a full system fsck you should bring the system down to single user mode.

In single user mode you need to invoke fsck, giving it the options to force a check of all file systems, even if they are already stable.

fsck -f (SunOS)

fsck -o f (Solaris)

fsck (Linux and IRIX)

If you wish to check a single specific file system, type its character device name. (If you aren't sure what the device name is, see the section on adding a disk to the system for details on how to determine this information.) For example:

fsck /dev/hda1

Stepping Through an Actual fsck fsck occurs in five to seven steps, depending on your operating system and what errors are found, if any. fsck can automatically correct most of these errors and does so if invoked at boot time to automatically check a dirty file system.

The fsck we are about to step through was done on a ufs file system. While there are some differences between the numbering of the phases for different file systems, the errors are mostly the same, requiring the same solutions. Apply common sense liberally to any invocation of fsck and you should be okay.

Checking ufs File Systems For ufs file systems, fsck is a five-phase process. fsck can automatically correct most of these errors and does so if invoked at boot time to automatically check a dirty file system. However, when you run fsck manually, you are asked to answer the questions that the system would automatically answer.

--------------------------------------------------------------------------------

CAUTION: Serious errors reported by the ufs fsck at the very beginning, especially before reporting the sta8rt of phase 1, indicate an invalid superblock. fsck should be terminated and restarted with the -b option specifying one of the alternate superblocks. Block 32 is always an alternate and can be tried first, but if the front of the file system was overwritten, it too may be damaged. Use the hard copy you saved from the mkfs to find an alternate later in the file system.

--------------------------------------------------------------------------------

Phase 1: Check Blocks and Sizes This phase checks the inode list, looking for invalid inode entries. Errors requiring answers include

UNKNOWN FILE TYPE I=inode number (CLEAR)

The file type bits are invalid in the inode. Options are to leave the problem and attempt to recover the data by hand later or to erase the entry and its data by clearing the inode.

PARTIALLY TRUNCATED INODE I=inode number (SALVAGE)

The inode appears to point to less data than the file does. This is safely salvaged, because it indicates a crash while truncating the file to shorten it.

block BAD I=inode number

block DUP I=inode number

The disk block pointed to by the inode is either out of range for this inode or already in use by another file. This is an informational message. If a duplicate block is found, phase 1b is run to report the inode number of the file that originally used this block.

Phase 2: Check Pathnames This phase removes directory entries from bad inodes found in phase 1 and 1b and checks for directories with inode pointers that are out of range or pointing to bad inodes. You may have to handle

ROOT INODE NOT DIRECTORY (FIX?)

You can convert inode 2, the root directory, back into a directory, but this usually means there is major damage to the inode table.

I=OUT OF RANGE I=inode number NAME=file name (REMOVE?)

UNALLOCATED I=inode number OWNER=O MODE=M SIZE=S MTIME=T TYPE=F

(REMOVE?)

BAD/DUP I=inode number OWNER=O MODE=M SIZE=S MTIME=T TYPE=F (REMOVE?)

A bad inode number was found, an unallocated inode was used in a directory, or an inode that had a bad or duplicate block number in it is referenced. You are given the choice to remove the file, losing the data, or to leave the error. If you leave the error, the file system is still damaged, but you have the chance to try to dump the file first and salvage part of the data before rerunning fsck to remove the entry.

fsck may return one of a variety of errors indicating an invalid directory length. You will be given the chance to have fsck fix or remove the directory as appropriate. These errors are all correctable with little chance of subsequent damage.

Phase 3: Check Connectivity This phase detects errors in unreferenced directories. It creates or expands the lost+found directory if needed and connects the misplaced directory entries into the lost+found directory. fsck prints status messages for all directories placed in lost+found.

Phase 4: Check Reference Counts This phase uses the information from phases 2 and 3 to check for unreferenced files and incorrect link counts on files, directories, or special files.

UNREF FILE I=inode number OWNER=O MODE=M SIZE=S MTIME=T (RECONNECT?)

The filename is not known (it is an unreferenced file), so it is reconnected into the lost+found directory with the inode number as its name. If you clear the file, its contents are lost. Unreferenced files that are empty are cleared automatically.

LINK COUNT FILE I=inode number OWNER=O MODE=M SIZE=S MTIME=T COUNT=X

(ADJUST?)

LINK COUNT DIR I=inode number OWNER=O MODE=M SIZE=S MTIME=T COUNT=X

(ADJUST?)

In both cases, an entry was found with a different number of references than what was listed in the inode. You should let fsck adjust the count.

BAD/DUP FILE I=inode number OWNER=O MODE=M SIZE=S MTIME=T (CLEAR)

A file or directory has a bad or duplicate block in it. If you clear it now, the data is lost. You can leave the error and attempt to recover the data, and rerun fsck later to clear the file.

Phase 5: Check Cylinder Groups This phase checks the free block and unused inode maps. It automatically corrects the free lists if necessary, although in manual mode it asks permission first.

What Do I Do After fsck Finishes?

First, relax, because fsck rarely finds anything seriously wrong, except in cases of hardware failure where the disk drive is failing or where you copied something on top of the file system. UNIX file systems are very robust.

However, if fsck finds major problems or makes a large number of corrections, rerun it to be sure the disk isn't undergoing hardware failure. It shouldn't find more errors in a second run. Then, recover any files that it may have deleted. If you keep a log of the inodes it clears, you can go to a backup tape and dump the list of inodes on the tape. Recover just those inodes to restore the files.

Back up the system again, because there is no reason to have to do this all over again.

Dealing with What Is in lost+found

If fsck reconnects unreferenced entries, it places them in the lost+found directory. They are safe there, and the system should be backed up in case you lose them while trying to move them back to where they belong. Items in lost+found can be of any type: files, directories, special files (devices), and so on. If it is a named pipe or socket, you may as well delete it. The process which opened it is long since gone and will open a new one when it is run again.

For files, use the owner name to contact the user and have him look at the contents and see if the file is worth keeping. Often, it is a file that was deleted and is no longer needed, but the system crashed before it could be fully removed.

For directories, the files in the directory should help you and the owner determine where they belong. You can look on the backup tape lists for a directory with those contents if necessary. Then just remake the directory and move the files back. Then remove the directory entry in lost+found. This re-creation and move has the added benefit of cleaning up the directory.

Creating File Systems

Now that you understand the nuances of maintaining a file system, it's time to understand how they are created. This section walks you through the three steps of:

Picking the right kind of disk for your system

Creating partitions

Creating the file system

Disk Types

Although there are many different kinds of disks, UNIX systems have come to standardize on SCSI for workstations. Many PCs also sport SCSI interfaces, but because of the lower cost and abundance, you'll find a lot of IDE drives on UNIX PC's as well.

SCSI itself comes in a few different flavors now. There is regular SCSI, SCSI-2, SCSI-Wide, SCSI-Fast and Wide, and now SCSI-3. Although it is possible to mix and match these devices with converter cables, you may find it both easier on your sanity and your performance if you stick to one format. As of this writing, SCSI-2 is the most common interface.

When attaching your SCSI drive, there are many important points to remember.

Terminate your SCSI chain. Forgetting to do this causes all sorts of non-deterministic behavior (a pain to track down). SCSI-2 requires active termination, which is usually indicated by terminators with LEDs on them.

If a device claims to be self-terminating, you can take your chances, but you'll be less likely to encounter an error if you put a terminator on anyway.

There is a limit of eight devices on a SCSI chain with the SCSI card counting as a device. Some systems may have internal SCSI devices, so be sure to check for those.

Be sure all your devices have unique SCSI IDs. A common symptom of having two devices with the same ID is their tendency to frequently reset the SCSI chain. Of course, many devices simply won't work under those conditions.

When adding or removing a SCSI disk, be sure to power the system down first. There is power running through the SCSI cables, and failing to shut them down first may lead to problems in the future.

Although SCSI is king of the workstation, PCs have another choice: IDE. IDE tends to be cheaper and more available than SCSI devices with many motherboards offering direct IDE support. The advantage of using this kind of interface is its availability as well as lower cost. They are also simpler and require less configuration on your part.

The down side to IDEs is that their simplicity comes at the cost of configurability and expandability. The IDE chain can only hold two devices, and not all motherboards come with more than one IDE chain. If your CD-ROM is IDE, you only have space for one disk. This is probably okay with a single person workstation, but as you can imagine, it's not going to fly well in a server environment. Another consideration is speed. SCSI was designed with the ability to perform I/O without the aid of the main CPU, which is one of the reasons it costs more. IDE, on the other hand, was designed with cost in mind. This resulted in a simplified controller; hence, the CPU takes the burden for working the drive.

While IDE did manage to simplify the PC arena, it did come with the limitation of being unable to handle disks greater than 540M. Various tricks were devised to circumvent this, however, the clean solution is now predominantly available. Known as EIDE (Enhanced IDE), it is capable of supporting disks up to 8G and can support up to 4 devices on one chain.

In weighing the pros and cons of EIDE versus SCSI in the PC environment, don't forget to think about the cost-to-benefit ratio. Having a high speed SCSI controller in a single person's workstation may not be as necessary as the user is convinced it is. Plus, with disks being released in 2+ gigabyte configurations, there is ample room on the typical IDE disk.

Once you have decided on the disk subsystem to install, read the documentation that came with the machine for instructions on physically attaching the disk to the system.

What Are Partitions and Why Do I Need Them?

Partitions are UNIX's way of dividing the disk into usable pieces. UNIX requires that there be at least one partition; however, you'll find that creating multiple partitions, each with a specific function, is often necessary.

The most visible reason for creating separate partitions is to protect the system from the users. The one required partition mentioned earlier is called the root partition. It is here that critical system software and configuration files (the kernel and mount tables) must reside. This partition must be carefully watched so that it never fills up. If it fills up, your system may not be able to come back up in the event of a system crash. Because the root partition is not meant to hold the users' data, you must create separate partitions for the users' home directories, temporarily files, and so forth. This enables their files to grow without the worry of crowding out the key system files.

Dual boot configurations are becoming another common reason to partition, especially with the ever-growing popularity of Linux. You may find your users wanting to be able to boot to either Windows or Linux; therefore, you need to keep at least two partitions to enable them to do this.

The last, but certainly not least, reason to partition your disks is the issue of backups. Backup software often works by dumping entire partitions onto tape. By keeping the different types of data on separate partitions, you can be explicit about what gets backed up and what doesn't. For example, daily backup of the system software isn't necessary, but backups of home directories are. By keeping the two on separate partitions, you can be more concise in your selection of what gets backed up and what doesn't.

Another example relates more to company politics. It may be possible that one group does not want their data to be backed up to the same tape as another group's. (Note: common sense doesn't always apply to inter-group politicsÉ) By keeping the two groups on separate partitions, you can exclude one from your normal backups and exclude the others during your special backups.

Which Partitions To Create As I mentioned earlier, the purpose of creating partitions is to separate the users from the system areas. So how many different partitions need to be created? While there is no right answer for every installation, here are some guidelines to take into account.

You always need a root partition. In this partition, you'll have your /bin, /etc, and /sbin directories at the very least. Depending on your version of UNIX, this could require anywhere from 30 to 100 megabytes. /tmp The /tmp directory is where your users, as well as programs, store temporarily files. The usage of this directory can quickly get out of hand, especially if you run a quota-based site. By keeping it a separate partition, you do not need to worry about its abuse interfering with the rest of the system. Many operating systems automatically clear the contents of /tmp on boot. Size /tmp to fit your site's needs. If you use quotas, you will want to make it a little larger, whereas sites without quotas may not need as much space.

Under Solaris, you have another option when setting up /tmp. Using the tmpfs filesystem, you can have your swap space and /tmp partition share the same physical location on disk. While it appears to be an interesting idea, you'll quickly find that it isn't a very good solution, especially on a busy system. This is because as more users do their work, more of /tmp will be used. Of course, if there are more users, there is a greater memory requirement to hold them all. The competition for free space can become very problematic.

/var The /var directory is where the system places its spool files (print spool, incoming/outgoing mail queue, and so on) as well as system log files. Because of this, these files constantly grow and shrink with no warning. Especially the mail spool. Another possibility to keep in mind is the creation of a separate partition just for mail. This enables you to export the mail spool to all of your machines without having to worry about your print spools being exported as well. If you use a backup package that requires its own spool space, you may wish to keep this a separate partition as well.

/home The /home directory is where you place your users' account directories. You may need to use multiple partitions to keep your home directories (possibly broken up by department) and have each partition mount to /home/dept where dept is the name of the respective department.

/usr The /usr directory holds noncritical system software, such as editors and lesser used utilities. Many sites hold locally compiled software in the /usr/local directory where they either export it to other machines, or mount other machines' /usr/local to their own. This makes it easy for a site to maintain one /usr/local directory and share it amongst all of its machines. Keeping this a separate partition is a good idea since local software inevitably grows.

swap This isn't a partition you actually keep files on, but it is key to your system's performance. The swap partition should be allocated and swapped to instead of using swap files on your normal file system. This enables you to contain all of your swap space in one area that is out of your way. A good guideline for determining how much swap space to use is to double the amount of RAM installed on your system.

--------------------------------------------------------------------------------

TIP: Several new versions of UNIX are now placing locally compiled software in the /opt directory. Like /usr/local, this should be made a separate partition as well. If your system does not use /opt by default, you should make a symbolic link from there to /usr/local. The vice versa is true as well, if your system uses /opt, you should create a symbolic link from /usr/local to /opt.

To add to the confusion, the Redhat Distribution of Linux has brought the practice of installing precompiled software (RPMs) in the /usr/bin directory. If you are using Redhat, you may want to make your /usr directory larger since locally installed packages will consume that partition.

--------------------------------------------------------------------------------

The Device Entry

Most implementations of UNIX automatically create the correct device entry when you boot it with the new drive attached. Once this entry has been created, you should check it for permissions. Only root should be given read/write access to it. If your backups run as a nonroot user, you may need to give group read access to the backup group. Be sure that no one else is in the backup group. Allowing world read/write access to the disk is the easiest way to have your system hacked, destroyed, or both.

Device entries under Linux IDE disks under Linux use the following scheme to name the hard disks:

/dev/hd[drive][partition]

Each IDE drive is lettered starting from a. So the primary disk on the first chain is a; the slave on the first chain is b; the primary on the secondary chain is c; and so on. Each disk's partition is referenced by number. For example, the third partition of the slave drive on the first chain is /dev/hdb3.

SCSI disks use the same scheme except instead of using /dev/hd as the prefix, /dev/sd is used. So to refer to the second partition of the first disk on the SCSI chain, you would use /dev/sda2.

To refer to the entire disk, specify all the information except the partition. For example, to refer to the entire primary disk on the first IDE chain, you would use /dev/hda.

Device entries under IRIX SCSI disks under IRIX are referenced in either the /dev/dsk or /dev/rdsk directories. The following is the format:

/dev/[r]dsk/dksCdSP

where C is the controller number, S is the SCSI address, and P is the partition, s0,s1,s2, and so on. The partition name can also be vh for the volume header or vol to refer to the entire disk.

Device entries under Solaris The SCSI disks under Solaris are referenced in either the /dev/dsk or /dev/rdsk directories. The following is the format:

/dev/[r]dsk/cCtSd0sP

where C is the controller number, S is the SCSI address, and P is the partition number. Partition 2 always refers to the entire disk and label information. Partition 1 is typically used for swap.

Device entries under SunOS Disks under SunOS are referenced in the /dev directory. The following is the format:

/dev/sdTP

where T is the target number and P is the partition. Typically, the root partition is a, the swap partition is b, and the entire disk is referred to as partition c. You can have partitions from a through f.

An important aspect to note is an oddity with the SCSI target and unit numbering: Devices that are target three need to be called target zero, and devices that are target zero need to be called target three.

A Note About Formatting Disks

"Back in the old days," disks needed to be formatted and checked for bad blocks. The procedure of formatting entailed writing the head, track, and sector numbers in a sector preamble and a checksum in the postamble to every sector on the disk. At the same time, any sectors that were unusable due to flaws in the disk surface were marked and, depending on the type of disk, an alternate sector mapped into its place.

Thankfully, we have moved on.

Both SCSI and IDE disks now come pre-formatted from the factory. Even better, they transparently handle bad blocks on the disk and remap them without any assistance from the operating system.

--------------------------------------------------------------------------------

CAUTION: You should NEVER attempt to low level format an IDE disk.

Doing so will make your day very bad as you watch the drive quietly kill itself. Be prepared to throw the disk away should you feel the need to low level format it.

--------------------------------------------------------------------------------

Partitioning Disks and Creating File Systems

In this section, we will cover the step by step procedure for partitioning disks under Linux, IRIX, SunOS, and Solaris. Since the principles are similar across all platforms, each platform will also cover another method of determining how a disk should be partitioned up depending on its intended usage.

Linux To demonstrate how partitions are created under Linux, we will setup a disk with a single user workstation in mind. It will need not only space for system software, but for application software and the user's home directories.

Creating Partitions For this example, we'll create the partitions on a 1.6 GB IDE disk located on /dev/hda. This disk will become the boot device for a single user workstation. We will create the boot /usr, /var, /tmp, /home, and swap partitions.

During the actual partitioning, we don't name the partitions. Where the partitions are mounted is specified with the /etc/fstab file. Should we choose to mount them in different locations later on, we could very well do that. However, by keeping the function of each partition in mind, we have a better idea of how to size them.

A key thing to remember with the Linux fdisk command is that it does not commit any changes made to the partition table to disk until you explicitly do so with the w command.

With the drive installed, we begin by running the fdisk command:

# fdisk /dev/hda

This brings us to the fdisk command prompt. We start by using the p command to print what partitions are currently on the disk.

Command (m for help): p

Disk /dev/hda: 64 heads, 63 sectors, 786 cylinders

Units = cylinders of 4032 * 512 bytes

Device Boot Begin Start End Blocks Id System

Command (m for help):

We see that there are no partitions on the disk. With 1.6 GB of space, we can be very liberal with allocating space to each partition. Keeping this policy in mind, we begin creating our partitions with the n command:

Command (m for help): n

e extended

p primary partition (1-4)

Partition number (1-4): 1

First cylinder (1-786): 1

Last cylinder or +size or +sizeM or +sizeK ([1]-786): +50M

Command (m for help):

The 50 MB partition we just created becomes our root partition. Because it is the first partition, it is referred to as /dev/hda1. Using the p command, we see our new partition:

Command (m for help): p

Disk /dev/hda: 64 heads, 63 sectors, 786 cylinders

Units = cylinders of 4032 * 512 bytes

Device Boot Begin Start End Blocks Id System

/dev/hda1 1 1 26 52384+ 83 Linux native

Command (m for help):

With the root partition out of the way, we will create the swap partition. Our sample machine has 32 MB of RAM and will be running X-Windows along with a host of development tools. It is unlikely that the machine will get a memory upgrade for a while, so we'll allocate 64 MB to swap.

Command (m for help): n

Command action

e extended

p primary partition (1-4)

Partition number (1-4): 2

First cylinder (27-786): 27

Last cylinder or +size or +sizeM or +sizeK ([27]-786): +64M

Command (m for help):

Because this partition is going to be tagged as swap, we need to change its file system type to swap using the t command.

Command (m for help): t

Partition number (1-4): 2

Hex code (type L to list codes): 82

Changed system type of partition 2 to 82 (Linux swap)

Command (m for help):

Because of the nature of the user, we know that there will be a lot of local software installed on this machine. With that in mind, we'll create /usr with 500 MB of space.

Command (m for help): n

Command action

e extended

p primary partition (1-4)

Partition number (1-4): 3

First cylinder (60-786): 60

Last cylinder or +size or +sizeM or +sizeK ([60]-786): +500M

If you've been keeping your eyes open, you've noticed that we can only have one more primary partition to use, but we want to have /home, /var, and /tmp to be in separate partitions. How do we do this?

Extended partitions.

The remainder of the disk is created as an extended partition. Within this partition, we can create more partitions for use. Let's create this extended partition:

Command (m for help): n

Command action

e extended

p primary partition (1-4)

Partition number (1-4): 4

First cylinder (314-786): 314

Last cylinder or +size or +sizeM or +sizeK ([314]-786): 786

Command (m for help):

We can now create /home inside the extended partition. Our user is going to need a lot of space, so we'll create a 500 MB partition. Notice that we are no longer asked whether we want a primary or extended partition.

Command (m for help): n

First cylinder (314-786): 314

Last cylinder or +size or +sizeM or +sizeK ([314]-786): +500M

Command (m for help):

Using the same pattern, we create a 250 MB /tmp and a 180 MB /var partition.

Command (m for help): n

First cylinder (568-786): 568

Last cylinder or +size or +sizeM or +sizeK ([568]-786): +250M

Command (m for help): n

First cylinder (695-786): 695

Last cylinder or +size or +sizeM or +sizeK ([695]-786): 786

Command (m for help):

Notice on the last partition we created that I did not specify a size, but instead specified the last track. This is to ensure that all of the disk is used.

Using the p command, we look at our final work:

Command (m for help): p

Disk /dev/hda: 64 heads, 63 sectors, 786 cylinders

Units = cylinders of 4032 * 512 bytes

Device Boot Begin Start End Blocks Id System

/dev/hda1 1 1 26 52384+ 83 Linux native

/dev/hda2 27 27 59 66528 82 Linux swap

/dev/hda3 60 60 313 512064 83 Linux native

/dev/hda4 314 314 786 953568 5 Extended

/dev/hda5 314 314 567 512032+ 83 Linux native

/dev/hda6 568 568 694 256000+ 83 Linux native

/dev/hda7 695 695 786 185440+ 83 Linux native

Command (m for help):

Everything looks good. To commit this configuration to disk, we use the w command:

Command (m for help): w

The partition table has been altered!

Calling ioctl() to re-read partition table.

(Reboot to ensure the partition table has been updated.)

Syncing disks.

Reboot the machine to ensure that the partition has been updated and you're done creating the partitions.

Creating File Systems in Linux Creating a partition alone isn't very useful. In order to make it useful, we need to make a file system on top of it. Under Linux, this is done using the mke2fs command and the mkswap command.

To create the file system on the root partition, we use the following commands:

mke2fs /dev/hda1

The program only takes a few seconds to run and generates output similar to this:

mke2fs 0.5b, 14-Feb-95 for EXT2 FS 0.5a, 95/03/19

128016 inodes, 512032 blocks

25601 blocks (5.00%) reserved for the super user

First data block=1

Block size=1024 (log=0)

Fragment size=1024 (log=0)

63 block groups

8192 blocks per group, 8192 fragments per group

2032 inodes per group

Superblock backups stored on blocks:

8193,16385,24577,32769,40961,49153,57345,65537,73729,

81921,90113,98305,106497,114689,122881,131073,139265,147457,

155649,163841,172033,180225,188417,196609,204801,212993,221185,

229377,237569,245761,253953,262145,270337,278529,286721,294913,

303105,311297,319489,327681,335873,344065,352257,360449,368641,

376833,385025,393217,401409,409601,417793,425985,434177,442369,

450561,458753,466945,475137,483329,491521,499713,507905

Writing inode tables: done

Writing superblocks and file system accounting information: done

You should make a note of these superblock backups and keep them in a safe place. Should the day arise that you need to use fsck to fix a superblock gone bad, you will want to know where the backups are.

Simply do this for all of the partitions, except for the swap partition.

To create the swap file system, you need to use the mkswap command like this:

mkswap /dev/hda2

Replace /dev/hda2 with the partition you chose to make your swap space.

The result of the command will be similar to:

Setting up swapspace, size = 35090432 bytes

And the swap space is ready.

To make the root file system bootable, you need to install the lilo boot manager. This is part of all the standard Linux distributions, so you shouldn't need to hunt for it on the Internet.

Simply modify the /etc/lilo.conf file so that /dev/hda1 is set to be the boot disk and run:

lilo

The resulting output should look something like:

Added linux *

where linux is the name of the kernel to boot, as specified by the name= field in /etc/lilo.conf.

SunOS In this example, we will be preparting a Seagate ST32550N as an auxiliary disk to an existing system. The disk will be divided into three partitions: one for use as a mail spool, one for use as a /usr/local, and the third as an additional swap partition.

Creating the partitions

--------------------------------------------------------------------------------

CAUTION: The procedure for formatting disks is not the same for SunOS and Solaris. Read each section to note the differences.

--------------------------------------------------------------------------------

Once a disk has been attached to the machine, you should verify its connection and SCSI address by running the probe-scsi command from the PROM monitor if the disk is attached to the internal chain, or the probe-scsi-all command to see all the SCSI devices on the system. When you are sure the drive is properly attached and verified to be functioning, you're ready to start accessing the drive from the OS.

After the machine has booted, run the dmesg command to collect the system diagnostic messages. You may want to pipe the output to grep so that you can easily find the information on disks. For example:

dmesg | grep sd

On our system this generated the following output:

sd0:

sd1 at esp0 target 1 lun 0

sd1: corrupt label - wrong magic number

sd1: Vendor 'SEAGATE', product 'ST32550N', 4194058 512 byte blocks

root on sd0a fstype 4.2

swap on sd0b fstype spec size 32724K

dump on sd0b fstype spec size 32712K

This result tells us that we have an installed disk on sd0 that the system is aware of and using. The information from the sd1 device is telling us that it found a disk, but it isn't usable because of a corrupt label. Don't worry about the error. Until we partition the disk and create file systems on it, the system doesn't know what to do with it, hence the error.

If you are using SCSI address 0 or 3, remember the oddity we mentioned earlier where device 0 needs to be referenced as 3 and device 3 needs to be referenced as 0.

Even though we do not have to actually format the disk, we do need to use the format program that come with SunOS because it also creates the partitions and writes the label to the disk.

To invoke the format program, simply run:

format sd1

where sd1 is the name of the disk we are going to partition.

The format program displays the following menu:

FORMAT MENU:

disk - select a disk

type - select (define) a disk type

partition - select (define) a partition table

current - describe the current disk

format - format and analyze the disk

repair - repair a defective sector

show - translate a disk address

label - write label to the disk

analyze - surface analysis

defect - defect list management

backup - search for backup labels

quit

format>

We need to enter type at the format> prompt so that we can tell SunOS the kind of disk we have. The resulting menu looks something like:

AVAILABLE DRIVE TYPES:

0. Quantum ProDrive 80S

1. Quantum ProDrive 105S

2. CDC Wren IV 94171-344

3. SUN0104

...

13. other

Specify disk type (enter its number):

Because we are adding a disk this machine has not seen before, we need to select option 13, other. This begins a series of prompts requesting the disk's geometry. Be sure to have this information from the manufacturer before starting this procedure.

The first question, Enter number of data cylinders: is actually a three-part question. After you enter the number of data cylinders, the program asks for the number of alternative cylinders and then the number of physical cylinders. The number of physical cylinders is the number your manufacturer provided you. Subtract two from there to get the number of data cylinders, and then just use the default value of 2 for the number of alternate cylinders. For our Seagate disk, we answered the questions as follows:

Enter number of data cylinders: 3508

Enter number of alternate cylinders [2]: 2

Enter number of physical cylinders [3510]: 3510

Enter number of heads: 11

Enter number of data sectors/track: 108

Enter rpm of drive [3600]:

Enter disk type name (remember quotes): "SEAGATE ST32550N"

selecting sd1:

[disk formatted, no defect list found]

No defined partition tables.

Note that even though our sample drive actually rotates at 7200 rpm, we stick with the default of 3600 rpm because the software will not accept entering a higher speed. Thankfully, this doesn't matter because the operating system doesn't use the information.

Even though format reported that the disk was formatted, it really wasn't. It only acquired information needed to later write the label.

Now we are ready to begin preparations to partition the disk.

These preparations entail computing the amount each cylinder holds and then approximating the number of cylinders we want in each partition.

With our sample disk, we know that each cylinder is composed of 108 sectors on a track, with 11 tracks composing the cylinder.

From the information we saw in dmesg, we know that each block is 512 bytes long. Hence, if we want our mail partition to be 1 GB in size, we perform the following math to compute the necessary blocks:

1 gigabyte = 1048576 kilobytes

One cylinder = 108 sectors * 11 heads = 1188 blocks

1188 blocks = 594 kilobytes

1048576 / 594 = 1765 cylinders

1765 * 1188 = 2096820 blocks

Obviously, there are some rounding errors since the exact one GB mark occurs in the middle of a cylinder and we need to keep each partition on a cylinder boundary. 1,765 cylinders is more than close enough. The 1,765 cylinders translates to 2,096,820 blocks.

The new swap partition we want to make needs to be 64 MB in size. Using the same math as before, we find that our swap needs to be 130,680 blocks long. The last partition on the disk needs to fill the remainder of the disk. Knowing that we have a 2 GB disk, a 1 GB mail spool, and a 64 MB swap partition, this should leave us with about 960 MB for /usr/local.

Armed with this information, we are ready to tackle the partitioning. From the format> prompt, type partition to start the partitioning menu. The resulting screen looks something like this:

format> partition

PARTITION MENU:

a - change 'a' partition

b - change 'b' partition

c - change 'c' partition

d - change 'd' partition

e - change 'e' partition

f - change 'f' partition

g - change 'g' partition

h - change 'h' partition

select - select a predefined table

name - name the current table

print - display the current table

label - write partition map and label to the disk

quit

partition>

To create our mail partition, we begin by changing partition a. At the partition> prompt, type a.

partition> a

This brings up a prompt for entering the starting cylinder and the number of blocks to allocate. Because this is going to be the first partition on the disk, we start at cylinder 0. Based on the math we did earlier, we know that we need 2,096,820 blocks.

partition a - starting cyl 0, # blocks 0 (0/0/0)

Enter new starting cyl [0]: 0

Enter new # blocks [0, 0/0/0]: 2096820

partition>

Now we want to create the b partition, which is traditionally used for swap space. We know how many blocks to use based on our calculations, but we don't know which cylinder to start from.

To solve this, we simply display the current partition information for the entire disk using the p command:

partition> p

Current partition table (unnamed):

partition a - starting cyl 0, # blocks 2096820 (1765/0/0)

partition b - starting cyl 0, # blocks 0 (0/0/0)

partition c - starting cyl 0, # blocks 0 (0/0/0)

partition d - starting cyl 0, # blocks 0 (0/0/0)

partition e - starting cyl 0, # blocks 0 (0/0/0)

partition f - starting cyl 0, # blocks 0 (0/0/0)

partition g - starting cyl 0, # blocks 0 (0/0/0)

partition h - starting cyl 0, # blocks 0 (0/0/0)

partition>

We can see that partition a is allocated with 2,096,820 blocks and is 1,765 cylinders long. Because we don't want to waste space on the disk, we start the swap partition on cylinder 1765.

(Remember to count from zero!)

partition> b

partition b - starting cyl 0, # blocks 0 (0/0/0)

Enter new starting cyl [0]: 1765

Enter new # blocks [0, 0/0/0]: 130680

partition>

Before we create our last partition, we need to take care of some tradition first, namely partition c. This is usually the partition that spans the entire disk. Before creating this partition, we need to do a little math.

108 cylinders x 11 heads x 3508 data cylinders = 4167504 blocks

Notice that the number of blocks we compute here does not match the number actually on the disk. This number was computed based on the information we entered when giving the disk type information.

It is important that we remain consistent.

Since the c partition spans the entire disk, we specify the starting cylinder as 0. Creating this partition should look something like this:

partition> c

partition c - starting cyl 0, # blocks 0 (0/0/0)

Enter new starting cyl [0]: 0

Enter new # blocks [0, 0/0/0]: 4167504

partition>

We have only one partition left to create: /usr/local. Because we want to fill the remainder of the disk, we need to do one last bit of math to compute how many blocks are still free.

This is done by taking the size of partition c (the total disk) and subtracting the sizes of the existing partitions. For our example, this works out to be:

4167504 - 2096820 - 130680 = 1940004 remaining blocks

Now we need to find out which cylinder to start from.

To do so, we run the p command again:

partition> p

Current partition table (unnamed):

partition a - starting cyl 0, # blocks 2096820 (1765/0/0)

partition b - starting cyl 1765, # blocks 130680 (110/0/0)

partition c - starting cyl 0, # blocks 4167504 (3508/0/0)

partition d - starting cyl 0, # blocks 0 (0/0/0)

partition e - starting cyl 0, # blocks 0 (0/0/0)

partition f - starting cyl 0, # blocks 0 (0/0/0)

partition g - starting cyl 0, # blocks 0 (0/0/0)

partition h - starting cyl 0, # blocks 0 (0/0/0)

partition>

To figure out which cylinder to start from, we add the number of cylinders used so far. Remember not to add the cylinders from partition c since it encompasses the entire disk.

1765 + 110 = 1875

Now that we know which cylinder to start from and how many blocks to make it, we create our last partition.

partition> d

partition d - starting cyl 0, # blocks 0 (0/0/0)

Enter new starting cyl [0]: 1875

Enter new # blocks [0, 0/0/0]: 1940004

partition>

Congratulations! You've made it through the ugly part. Before we can truly claim victory, we need to commit these changes to disk using the label command. When given the prompt, Ready to label disk, continue? simply answer y.

partition> label

Ready to label disk, continue? y

partition>

To leave the format program, type quit at the partition> prompt, and then quit again at the format> prompt.

Creating File Systems Now comes the easy part. Simply run the newfs command on all the partitions we created except for the swap partition and the entire disk partition . Your output should look similar to this:

# newfs sd1a

/dev/rsd1a: 2096820 sectors in 1765 cylinders of 11 tracks, 108 sectors

1073.6MB in 111 cyl groups (16 c/g, 9.73MB/g, 4480 i/g)

superblock backups (for fsck -b #) at:

32, 19152, 38272, 57392, 76512, 95632, 114752, 133872, 152992,

172112, 191232, 210352, 229472, 248592, 267712, 286832, 304160, 323280,

342400, 361520, 380640, 399760, 418880, 438000, 457120, 476240, 495360,

514480, 533600, 552720, 571840, 590960, 608288, 627408, 646528, 665648,

684768, 703888, 723008, 742128, 761248, 780368, 799488, 818608, 837728,

856848, 875968, 895088, 912416, 931536, 950656, 969776, 988896, 1008016,

1027136, 1046256, 1065376, 1084496, 1103616, 1122736, 1141856, 1160976, 1180096,

1199216, 1216544, 1235664, 1254784, 1273904, 1293024, 1312144, 1331264, 1350384,

1369504, 1388624, 1407744, 1426864, 1445984, 1465104, 1484224, 1503344, 1520672,

1539792, 1558912, 1578032, 1597152, 1616272, 1635392, 1654512, 1673632, 1692752,

1711872, 1730992, 1750112, 1769232, 1788352, 1807472, 1824800, 1843920, 1863040,

1882160, 1901280, 1920400, 1939520, 1958640, 1977760, 1996880, 2016000, 2035120,

2054240, 2073360, 2092480,

Be sure to note the superblock backups. This is critical information when fsck discovers heavy corruption in your file system. Remember to add your new entries into /etc/fstab if you want them to automatically mount on boot.

If you created the first partition with the intention of making it bootable, you have a few more steps to go. First, mount the new file system to /mnt.

# mount /dev/sd1a /mnt

Once the file system is mounted, you need to clone your existing boot partition using the dump command like this:

# cd /mnt

# dump 0f - / | restore -rf -

With the root partition cloned, use the installboot command to make it bootable:

# /usr/kvm/mdec/installboot /mnt/boot /usr/kvm/mdec/bootsd /dev/rsd1a

Be sure to test your work by rebooting and making sure everything mounts correctly. If you created a bootable partition, be sure you can boot from it now. Don't wait for a disaster to find out whether or not you did it right.

Solaris For this example, we are partitioning a disk that is destined to be a web server for an intranet. We need a minimal root partition, adequate swap, tmp, var, and usr space, and a really large partition, which we'll call /web. Because the web logs will remain on the /web partition, and there will be little or no user activity on the machine, /var and /tmp will be set to smaller values. /usr will be a little larger because it may be destined to house web development tools.

Creating partitions

--------------------------------------------------------------------------------

TIP: In another wondrous effort on its part to be just a little different, Sun has decided to call partitions "slices." With the number of documents regarding the file system so vast, you'll find that not all of them have been updated to use this new term, so don't be confused by the mix of "slices" with "partitions"--they are both the same.

--------------------------------------------------------------------------------

Once a disk has been attached to the machine, you should verify its connection and SCSI address by running the probe-scsi command from the PROM monitor if the disk is attached to the internal SCSI chain, probe-scsi-all to list all the SCSI devices on the system Once this shows that the drive is properly attached and verified to be functioning, you're ready to start accessing the drive from the OS. Boot the machine and login as root.

In order to find the device name, we are going to use for this, we again use the dmesg command.

# dmesg | grep sd

...

sd1 at esp0: target 1 lun 0

sd1 is /sbus@1,f8000000/esp@0,800000/sd@1,0

WARNING: /sbus@1,f8000000/esp@0,800000/sd@1,0 (sd1):

corrupt label - wrong magic number

Vendor 'SEAGATE', product 'ST32550N', 4194058 512 byte blocks

...

From this message, we see that our new disk is device /dev/[r]dsk/c0t1d0s2. The disk hasn't been set up for use on a Solaris machine before, which is why we received the corrupt label error.

If you recall the layout of Solaris device names, you'll remember that the last digit on the device name is the partition number. Noting that, we see that Solaris refers to the entire disk in partition 2, much the same way SunOS refers to the entire disk as partition c.

Before we can actually label and partition the disk, we need to create the device files. This is done with the drvconfig and disks commands. They should be invoked with no parameters:

# drvconfig ; disks

Now that the kernel is aware of the disk, we are ready to run the format command to partition the disk.

# format /dev/rdsk/c0t1d0s2

This brings up the format menu as follows:

FORMAT MENU:

disk - select a disk

type - select (define) a disk type

partition - select (define) a partition table

current - describe the current disk

format - format and analyze the disk

repair - repair a defective sector

label - write label to the disk

analyze - surface analysis

defect - defect list management

backup - search for backup labels

verify - read and display labels

save - save new disk/partition definitions

inquiry - show vendor, product and revision

volname - set 8-character volume name

quit

format>

To help the format command with partitioning, we need to tell it the disk's geometry by invoking the type command at the format> prompt. We will then be asked to select what kind of disk we have. Because this is the first time this system is seeing this disk, we need to select other. This should look something like this:

format> type

AVAILABLE DRIVE TYPES:

0. Auto configure

1. Quantum ProDrive 80S

2. Quantum ProDrive 105S

3. CDC Wren IV 94171-344

. . .

16. other

Specify disk type (enter its number): 16

The system now prompts for the number of data cylinders. This is two less than the number of cylinders the vendor specifies because Solaris needs two cylinders for bad block mapping.

Enter number of data cylinders: 3508

Enter number of alternate cylinders[2]: 2

Enter number of physical cylinders[3510]: 3510

The next question can be answered from the vendor specs as well.

Enter number of heads: 14

The followup question about drive heads can be left as default.

Enter physical number of heads[default]:

The last question you must answer can be pulled from the vendor specs as well.

Enter number of data sectors/track: 72

The remaining questions should be left as default.

Enter number of physical sectors/track[default]:

Enter rpm of drive[3600]:

Enter format time[default]:

Enter cylinder skew[default]:

Enter track skew[default]:

Enter tracks per zone[default]:

Enter alternate tracks[default]:

Enter alternate sectors[default]:

Enter cache control[default]:

Enter prefetch threshold[default]:

Enter minimum prefetch[default]:

Enter maximum prefetch[default]:

The last question you must answer about the disk is its label information. Enter the vendor name and model number in double quotes for this question. For our sample disk, this would be:

Enter disk type name (remember quotes): "SEAGATE ST32550N"

With this information, Solaris makes creating partitions easy. Dare I say, fun?

After the last question from the type command, you will be placed at the format> prompt. Enter partition to start the partition menu.

format> partition

PARTITION MENU:

0 - change '0' partition

1 - change '1' partition

2 - change '2' partition

3 - change '3' partition

4 - change '4' partition

5 - change '5' partition

6 - change '6' partition

7 - change '7' partition

select - select a predefined table

modify - modify a predefined partition table

name - name the current table

print - display the current table

label - write partition map and label to the disk

quit

partition>

At the partition> prompt, enter modify to begin creating the new partitions. This brings up a question about what template to use for partitioning. We want the All Free Hog method.

partition> modify

Select partitioning base:

0. Current partition table (unnamed)

1. All Free Hog

Choose base (enter number)[0]? 1

The All Free Hog method enables you to select one partition to receive the remainder of the disk once you have allocated a specific amount of space for the other partitions. For our example, the disk hog would be the /web partition because you want it to be as large as possible.

As soon as you select option 1, you should see the following screen:

Part Tag Flag Cylinders Size Blocks

0 root wm 0 0 (0/0/0)

1 swap wu 0 0 (0/0/0)

2 backup wu 0 - 3507 1.99GB (3508/0/0)

3 unassigned wm 0 0 (0/0/0)

4 unassigned wm 0 0 (0/0/0)

5 unassigned wm 0 0 (0/0/0)

6 usr wm 0 0 (0/0/0)

7 unassigned wm 0 0 (0/0/0)

Do you wish to continue creating a new partition

table based on above table [yes]? yes

Because the partition table appears reasonable, agree to use it as a base for your scheme. You will now be asked which partition should be the Free Hog Partition, the one that receives whatever is left of the disk when everything else has been allocated.

For our scheme, we'll make that partition number 5.

Free Hog Partition[6]? 5

Answering this question starts the list of questions asking how large to make the other partitions. For our web server, we need a root partition to be about 200 MB for the system software, a swap partition to be 64 MB, a /tmp partition to be 200 MB, a /var partition to be 200 MB, and a /usr partition to be 400 MB. Keeping in mind that partition 2 has already been tagged as the "entire disk" and that partition 5 will receive the remainder of the disk, you will be prompted as follows:

Enter size of partition '0' [0b, 0c, 0.00mb]: 200mb

Enter size of partition '1' [0b, 0c, 0.00mb]: 64mb

Enter size of partition '3' [0b, 0c, 0.00mb]: 200mb

Enter size of partition '4' [0b, 0c, 0.00mb]: 200mb

Enter size of partition '6' [0b, 0c, 0.00mb]: 400mb

Enter size of partition '7' [0b, 0c, 0.00mb]: 0

As soon as you finish answering these questions, the final view of all the partitions appears looking something like:

Part Tag Flag Cylinders Size Blocks

0 root wm 0 - 344 200.13mb (345/0/0)

1 swap wu 345 - 455 64.39mb (111/0/0)

2 backup wu 0 - 3507 1.99GB (3508/0/0)

3 unassigned wm 456 - 800 200.13mb (345/0/0)

4 unassigned wm 801 - 1145 200.13mb (345/0/0)

5 unassigned wm 1146 - 2817 969.89mb (1672/0/0)

6 unassigned wm 2818 - 3507 400.25mb (690/0/0)

7 unassigned wm 0 0 (0/0/0)

This is followed by the question:

Okay to make this the correct partition table [yes]? yes

Answer yes since the table appears reasonable. This brings up the question:

Enter table name (remember quotes): "SEAGATE ST32550N"

Answer with a description of the disk you are using for this example. Remember to include the quote symbols when answering. Given all of this information, the system is ready to commit this to disk. As one last check, you will be asked:

Ready to label disk, continue? y

As you might imagine, we answer yes to the question and let it commit the changes to disk. You have now created partitions and can quit the program by entering quit at the partition> prompt and again at the format> prompt.

Creating file systems To create a file system, simply run:

# newfs /dev/c0t1d0s0

where /dev/c0t1d0s0 is the partition on which to create the file system. Be sure to create a file system on all the partitions except for partitions 2 and 3, the swap, and entire disk, respectively. Be sure to note the backup superblocks that were created. This information is very useful when fsck is attempting to repair a heavily damaged file system.

After you create the file systems, be sure to enter them into the /etc/vfstab file so that they are mounted the next time you reboot.

If you need to make the root partition bootable, you still have two more steps. The first is to clone the root partition from your existing system to the new root partition using:

# mount /dev/dsk/c0t1d0s0 /mnt

# ufsdump 0uf - / | ufsrestore -rf -

Once the file root partition is cloned, you can run the installboot program like this:

# /usr/sbin/installboot /usr/lib/fs/ufs/bootblk /dev/rdsk/c0t1d0s0

Be sure to test your new file systems before you need to rely on them in a disaster situation.

IRIX For this example, we are creating a large scratch partition for a user who does modeling and simulations. Although IRIX has many GUI-based tools to perform these tasks, it is always a good idea to learn the command line versions just in case you need to do any kind of remote administration.

Creating partitions Once the drive is attached, run a program called hinv to take a "hardware inventory." On the sample system, you saw the following output:

...

Integral SCSI controller 1: Version WD33C93B, revision D

Disk drive: unit 6 on SCSI controller 1

Integral SCSI controller 0: Version WD33C93B, revision D

Disk drive: unit 1 on SCSI controller 0

...

Our new disk is external to the system, so we know it is residing on controller 1. Unit 6 is the only disk on that chain, so we know that it is the disk we just added to the system.

To partition the disk, run the fx command without any parameters. It prompts us for the device name, controller, and drive number. Choose the default device name and enter the appropriate information for the other two questions.

On our sample system, this would look like:

# fx

fx version 6.2, Mar 9, 1996

fx: "device-name" = (dksc)

fx: ctlr# = (0) 1

fx: drive# = (1) 6

fx: lun# = (0)

...opening dksc(1,6,0)

...controller test...OK

Scsi drive type == SEAGATE ST32550N 0022

----- please choose one (? for help, .. to quit this menu)-----

[exi]t [d]ebug/ [l]abel/

[b]adblock/ [exe]rcise/ [r]epartition/

fx>

We see that fx found our Seagate and is ready to work with it. From the menu we select r to repartition the disk. fx displays what it knows about the disk and then presents another menu specifically for partitioning the disk.

fx> r

----- partitions-----

part type cyls blocks Megabytes (base+size)

7: xfs 3 + 3521 3570 + 4189990 2 + 2046

8: volhdr 0 + 3 0 + 3570 0 + 2

10: volume 0 + 3524 0 + 4193560 0 + 2048

capacity is 4194058 blocks

----- please choose one (? for help, .. to quit this menu)-----

[ro]otdrive [u]srrootdrive [o]ptiondrive [re]size

fx/repartition>

Looking at the result, we see that this disk has never been partitioned in IRIX before. Part 7 represents the amount of partitionable space, part 8 the volume header, and part 10 the entire disk.

Because this disk is going to be used as a large scratch partition, we want to select the optiondrive option from the menu. After you select that, you are asked what kind of file system you want to use. IRIX 6 and above defaults to xfs, while IRIX 5 defaults to efs. Use the one appropriate for your version of IRIX.

Our sample system is running IRIX 6.3, so we accept the default of xfs:

fx/repartition> o

fx/repartition/optiondrive: type of data partition = (xfs)

Next we are asked whether we want to create a /usr log partition. Because our primary system already has a /usr partition, we don't need one here. Type no.

fx/repartition/optiondrive: create usr log partition? = (yes) no

The system is ready to partition the drive. Before it does, it gives one last warning allowing you to stop the partitioning before it completes the job. Because you know you are partitioning the correct disk, you can give it "the go-ahead":

Warning: you must reinstall all software and restore user data from backups after changing the partition layout. Changing partitions causes all data on the drive to be lost. Be sure you have the drive backed up if it contains any user data. Continue? y

The system takes a few seconds to create the new partitions on the disk. Once it is done, it reports what the current partition list looks like.

----- partitions-----

part type cyls blocks Megabytes (base+size)

7: xfs 3 + 3521 3570 + 4189990 2 + 2046

8: volhdr 0 + 3 0 + 3570 0 + 2

10: volume 0 + 3524 0 + 4193560 0 + 2048

capacity is 4194058 blocks

----- please choose one (? for help, .. to quit this menu)-----

[ro]otdrive [u]srrootdrive [o]ptiondrive [re]size

fx/repartition>

Looks good. We can exit fx now by typing .. at the fx/repartition> prompt and exit at the fx> prompt.

Our one large scratch partition is now called /dev/dsk/dks1d6s7.

Creating the filesystem To create the file system, we use the mkfs command like this:

# mkfs /dev/rdsk/dks1d6s7

This generates the following output:

meta-data=/dev/dsk/dks1d6s7 isize=256 agcount=8, agsize=65469 blks

data = bsize=4096 blocks=523748, imaxpct=25

log =internal log bsize=4096 blocks=1000

realtime =none bsize=65536 blocks=0, rtextents=0

Remember to add this entry into the /etc/fstab file so that the system automatically mounts the next time you reboot.

Summary

As you've seen in this chapter, creating, maintaining, and repairing filesystems is not a trivial task. It is, however, a task which should be well understood. An unmaintained file system can quickly lead to trouble and without its stability, the remainder of the system is useless.

Let's make a quick rundown of the topics we covered:

Disks are broken into partitions (sometimes called slices).

Each partition has a file system.

A file system is the primary means of file storage in UNIX.

File systems are made of inodes and superblocks.

Some partitions are used for raw data such as swap.

The /proc file system really isn't a file system, but an abstraction to kernel data.

An inode maintains critical file information.

Superblocks track disk information as well as the location of the heads of various inode lists.

In order for you to use a file system, it must be mounted.

No one must be accessing a file system in order for it to be unmounted.

File systems can be mounted anywhere in the directory tree.

/etc/fstab (vfstab in Solaris) is used to by the system to automatically mount file systems on boot.

The root file system should be kept away from users.

The root file system should never get filled.

Be sure to watch how much space is being used.

fsck is the tool to use to repair file systems.

Don't forget to terminate your SCSI chain!

In short, file systems administration is not a trivial task and should not be taken lightly. Good maintenance techniques not only help maintain your uptime, but your sanity as well.

What Is System Administration?

2008-08-28T23:57:00.002-07:00

By Eric Goebelbecker

System Administration is planning, installing, and maintaining computer systems. If that seems like a very generalized answer, it's because "What is System Administration" is a broad question.

In this chapter we'll describe what is expected from a System Administrator, and how she might approach her responsibilities.

Help Wanted
Administer Sun and IBM UNIX systems and control Internet access. Assist with some database administration (Oracle). Administer Sun server and network: system configuration, user id/creation/maintenance, security administration, system backup, and ensuring all applications are running on the system properly. Administer Internet access and services: internal/external security, user identification creation and maintenance, and Web server maintenance.

This is a typical "Help Wanted" advertisement for a UNIX System Administration position. From this description you might guess that a System Administrator has to install and configure the operating system, add new users, back up the system(s), keep the systems secure, and make sure they stay running. Setting up Internet access and keeping it running is part of the job too.

Unfortunately, you'd only be considering about half of the job. Being an expert on installing, running, and maintaining all of the major UNIX variants isn't enough to be a truly good system administrator. There is a significant nontechnical component to being a system administrator, especially in terms of planning, organizational, and people skills.

As computers become more and more pervasive in business, system administration becomes a mission critical position in more and more organizations. The administrator has to understand the systems that he is responsible for, the people who use them, and the nature of the business that they are used for. A key skill in administration is planning, because at the rate that systems are being created, overhauled, and expanded, trying to improvise and design a network "on the fly" just doesn't work.

Companies are moving more and more processes not just to computers, but to point of sale systems, such as Web commerce and sophisticated in-store systems, such as electronic catalogs and cash registers that are directly connected to both inventory and credit systems. Companies that may have moved their inventory control to computers five years ago are scrambling to get order entry computerized and on the Web, while companies that haven't automated their inventory and ordering systems yet are scrambling to do so in order to remain competitive. E-mail is now regarded as just as important as faxes and telephones, while every part of customer service that can be manned by a computer and fax retrieval system already is. These companies are almost completely dependent on their computers, and their system administrators need to understand a lot more than how to partition a disk drive, add memory, and install Adobe PhotoShop.

This chapter introduces some of the basic technical and organizational concepts that a system administrator needs to know in order to perform his job well. It also covers a few key system administration tools that are already found with most UNIX variants or are included on the UNIX Unleashed CD that accompanied this book.

The chapter is divided into the following sections:

Technical Concepts for System Administrators--in this section we introduce some of UNIX's important characteristics and how they differ from operating systems like Windows and Macintosh.
UNIX is Heterogeneous--this section describes UNIX's diverse and sometimes self-contradictory user interfaces and how those interfaces ended up that way.
System Administration Tasks--this part of the chapter is where basic administration tasks and responsibilities are introduced.
Administration Resources--both the operating system itself and the Internet provide a wide variety of information for administrators. This section provides a few pointers to these resources.
Tools of the Trade--UNIX provides users and administrators with an amazing set of tools. We briefly cover a few of them in the end of this chapter.

TIP: No one, regardless of how long they've "been in the business" can remember all of the configuration options, command line switches, oddities, and outright bugs in the UNIX tool chest. Experienced users soon learn the value of UNIX's online documentation, the manual pages. Before you use any tool, read the man page(s) for it; many of them include useful examples. If you install a utility from the CD, be sure you read the installation instructions that accompany it and install the man page too. Later in this chapter we'll cover some advanced features found in the manual page system.

Technical Concepts for New System Administrators

UNIX differs from Windows and Macintosh at a fundamental level. UNIX was originally intended for multiple users running multiple simultaneous programs. (At least it was by the time it was distributed outside of Bell Labs.) The phenomenon of a user having a low cost personal workstation, running Linux or any other variant of UNIX is actually a recent development in terms of the operating system's history. The fact that UNIX is designed for use by more than a single user is reflected throughout the operating system's file systems, security features, and programming model.

Networking is not an afterthought or a recent development for UNIX, the way it seems to be for Windows and Macintosh. Support for sharing files, moving from workstation to workstation, and running applications on remote computers that are controlled and viewed on a local workstation is not only intuitive and natural on UNIX, but was more powerful and stable on past versions of UNIX than the latest versions of Windows and Windows NT.

Multiple Users and Multiple Accounts

The DOS/Windows and Macintosh environments are "computer centric" in terms of security and customization. The ownership of files and processes (programs that are currently running) is more or less governed by the computer where they are located as opposed to the notion of a user id or session. If a person can physically get to a computer, he has access to all of the files and programs on it. If a truly malicious person wants access to the contents of a PC, even a password-protected screen saver can be overcome by restarting the system, because the files and programs belong to the PC, not a user, and the computer will boot ready for use. (Even the login prompt on Windows 95 can be bypassed with the Cancel button, bringing the user interface up with no network resources connected.)

Add-ons are available for DOS and Macintosh systems that enable users to identify themselves and save files in protected areas. But these are applications, not a part of the operating system, and come with their own set of problems, limitations, and rules. The fact is, as anyone who has ever had to share a computer with a family member or coworker will agree, personal computer operating systems are single user and geared toward supporting one user and one set of configuration options.

UNIX systems are multi-user. Users must log in to the system. Each has his own areas for saving files, the home directory, and files have properties that determine who can or cannot access them, their mode. All running programs are associated with a user, and, similar to the way files have access control, programs can only be started or stopped by certain users. Unless a user can log in as the super-user, she cannot access another user's files unless the owner gives her permission through one of several direct or indirect means. Only the super-user can reboot the computer (without the power switch) or stop another user's processes. Even if a system is rebooted, all of the security features will be remain effect, so restarting is not a valid means of subverting security.

Network Centricity

Networking has become an integral part of UNIX. The ability to share files, support network logins, share network configuration information, and run applications across a network is included in all of the major UNIX distributions, and more importantly is a natural extension of the base operating system, not an application that was designed by an individual vendor, with their idea of networking and administration, and that has to be purchased separately.

When configured to allow it, anything that can be done at the console (main keyboard and monitor) of a UNIX system can be done at another system via a network connection. (We'll refer to these other systems as remote nodes or remote systems.) Actually many server systems, such as Web servers and file servers, have consoles with very limited capabilities (such as a text-only terminal), and are deliberately designed with the idea of doing as much administration as possible from remote nodes outside the data center.

Two of the mechanisms that provide these capabilities are remote (or pseudo) terminals and remote shells. Remote terminals emulate an actual terminal session, just as if the user were logged into a terminal connected to the system via a serial line, where the network is the line and the application (usually telnet) is the terminal (Remote terminals are usually referred to as telnet sessions, since telnet is by far the most common application.) This is similar, in theory at least, to dialing into a system and using an application like Procomm.

Remote shells (or remote logins) are sessions on remote computers that are centered around the execution of a shell (or shell command) instead of a terminal on a remote system. A remote shell executes a command on a remote host, while a remote login runs a shell; both usually appear to be running on the local system. Remote shells are frequently used by administrators in shell scripts, allowing them to execute commands on several systems and consolidate the results in one place, such as collecting disk utilization statistics or automating backups.

The differences between a remote shell and a telnet session are very important. Telnet sessions use terminal emulation and are run as separate applications. So the results cannot be shared with other applications unless a mechanism such as cut and paste is used, while remote shells allow the results of commands to be interspersed with local commands.

So a directory on one workstation could be backed up to a tape drive on another via a simple shell command:

tar cvfb  - tgt_dir | rsh -n bilbo dd of=/dev/rmt/0

The tar command creates an archive of tgt_dir and sends it to standard output. This stream of data is redirected to the rsh command. rsh connects to the host bilbo, executes dd, and passes the output of tar to it. The dd command just happens to know how to save the archive to the tape drive on /dev/rmt/0. (This may seem complicated. By the end of this chapter, it will make perfect sense.) So two programs on two different computers are linked as one, without any special configuration requirements or extra software.

With remote logins and shells being so powerful, why would anyone use telnet? One reason is that telnet is much more robust over slow links, such as Internet and WAN connections between different sites than remote logins, The other is security. We'll cover some of the risks posed by remote shells later.

X-windows is another example of UNIX's networking prowess. The X environment is a graphical user interface, much like the Windows or Macintosh environments. A key difference is that it is segmented into a client (the application) and a server (the display.) The server is a process that manages the keyboard, mouse, and screen, and accepts connections from applications over a socket (a network connection). Because a network connection is the method of communication between the two programs, the application can be running on a different workstation than the display that is controlling it. (When applications are running on the same workstation as the server, the network connection is frequently bypassed in favor of faster methods. These enhancements are proprietary extensions and differ widely from vendor to vendor.)

The remote functionality of X-windows is a very powerful advantage, especially when considered in terms of large database, Web, or file servers that tend to be installed in data centers or equipment rooms, but have graphical management and configuration applications. Another possibility is installing slower, less expensive workstations on desktops while still being able to run more demanding applications on shared, more powerful computers.

File sharing is another extension that has become a natural and integral part of UNIX computing. All of the major UNIX variants support NFS (Network File System) and can share files seamlessly between themselves and other UNIX versions.

Because NFS file systems are treated like any other type of disk, the same way SCSI, IDE, and floppy disks are implemented, network drives fully support user accounts, file permissions, and all of the mechanisms UNIX already uses for adding, removing, and managing other types of files and file systems, even when shared between different UNIX variants.

This means that two UNIX systems, regardless of what version of UNIX or on what type of computer they happen to be, can share files, maybe even from a common file server running on a third system type. Only the user IDs have to be coordinated in order for the files to be securely shared. No additional software or special modifications are required.

Any UNIX system can export or share file systems. This is one of the ways that the difference between servers and clients is blurred with UNIX systems; any system can provide or consume file system resources. When file systems are shared, the workstation can specify which systems can and cannot use it, and also whether or not users may write to it.

Clients specify where the file system will be mounted in their directory tree. For example, a system can mount the /export/applications directory from another system to the /mnt/apps directory.

UNIX Networking: Sharing files and Information

Consider the following practical applications to illustrate how UNIX networking works, and how some common tools and planning can make administration very easy.

A system that is frequently used to administer NFS is the automounter. This program (or set of programs) allows administrators to specify file systems to be mounted (attached) and unmounted (detached) as needed, based on when clients refer to them. This process happens completely in the background, without any intervention after the automount program(s) are configured properly.

LINUX
Linux systems use a publicly available application called amd instead of automount. Although the configuration files differ slightly, the functionality amd provides Linux is the same. amd is provided with all of the major Linux distributions. As always, see the manual page for specifics.

automount is configured with maps (a fancy term used for referring to configuration files), where each map specifies a top level directory. So, if the automount system is to create five directories, the master map, which is contained in the file /etc/auto_master might look like this:

/share         auto_share
/home          auto_home
/gnu           auto_gnu
/apps          auto_apps
/opt           auto_opt

This file provides the automount with the names of the directories it will create and the files that contain what the directories will contain.

For example, let's look at an organization that decided that all user directories belong under the directory /home. (Therefore, the user dan would find his files in /home/dan.) The first thirty home directories were located on the file server gandalf in the /export/home directory.

When the next user comes along, the administrator realizes that gandalf does not have enough drive space for a new user, and, as a matter of fact, moving a couple of users over to the other file server, balrog, which has been used for applications until now, would probably be a good idea.

automount simplifies this distribution by allowing us to create a virtual /home directory on every workstation that runs the automount daemon. An excerpt from the home directory map, /etc/auto_home, as named in /etc/auto_master above, would look like this:

eric  gandalf:/export/home/eric
dan   gandalf:/export/home/dan
mike  balrog:/export/home/michael

In this map, eric, dan, and mike are referred to as directory keys. The directories are called values. When a program refers to /home/eric for the first time, the system recognizes the key eric and arranges for /export/home/eric on gandalf to be mounted at /home/eric. Because automount does a simple key/value replacement, Eric and Dan's directories can be on gandalf while Mike's is on balrog. Also note that the directory names in balrog and the automounter virtual directory do not have to agree. After a period of inactivity, automount unmounts the directory.

automount alleviates the tedious task of mounting all of the directories that a user or group of users will need in advance. Permanently mounting all of those file systems leads to unnecessary system overhead and requires excessive human intervention. This system also makes it easier for users to travel from workstation to workstation, since they can always expect to find important application and data files in the same place.

Solaris
Sun Microsystems defined autofs as a new file system type in Solaris 2.x, and created a multi-threaded automounter system. This makes automounted file systems perform extraordinarily well under Solaris, since the kernel is now aware that automounted file systems exist and by preventing bottlenecks in the automount program through the use of threads.

HP-UX
AIX
Hewlett Packard's HP-UX and IBM's AIX provide a version of automount. Since automount uses NFS, it is compatible across different UNIX variants.

Newer versions of automount (and amd) also enable us to specify more than one file server for a directory, in order to provide a backup for when one file server is unavailable or overloaded. This system, with a little planning and thought, can simplify a distributed network and even make it a little more reliable.

The maps, however, still must be kept up to date and distributed to each workstation. This could be done using ftp, or even nfs, but on a large network this can become tedious. There are two very elegant solutions that are examples of how experienced system administrators tend to solve these problems.

The first is a tool called rdist. It is a tool for maintaining identical copies of software between remote hosts. It can accept filenames from the command line or use a configuration file (usually referred to as a distfile) that lists what files to copy to which hosts.

In order to distribute a set of automounter maps we might use a distfile like this simple example:

HOSTS = (bilbo frodo thorin snowball)
FILES = /etc/auto_master /etc/auto_home /etc/auto_apps
${FILES}->${HOSTS}
       install;

The distfile is a very powerful mechanism. In this example, we take advantage of the ability to create variables. By specifying the hosts to be updated and the files to send in lists, we can easily add to them. After we define $FILES and $HOSTS, we indicate that the lists of files should be kept up to date on the hosts that are listed. install (the one line command after the dependency line) is one of many rdist commands, it indicates the rdist should keep the files up to date with the local copies. See the man page for more information on the numerous directives and distfile options.

To use this we could add the distfile to the /etc directory on the host where the master set of automounter maps are created. Whenever we make a change, we would execute:

rdist -f /etc/auto_distfile

rdist can be used for much more complicated tasks, such as synchronizing user directories between redundant file servers or distributing new versions of software packages where nfs is not being used.

The disadvantage behind rdist is that it has the same security requirement as any other remote shell command. The user executing the rdist command must be able to attach to the remote host and execute commands without a password. (Refer to the following "Network Security Issues" section.)

If the security issues associated with rdist are unacceptable, how could we conveniently distribute these maps?

Sun Microsystems's Network Information Service (NIS) (also frequently called "yp" or yellow pages) may be an acceptable alternative. NIS provides common configuration information, such as IP addresses, service port numbers, and automount maps to hosts on a network from a server or set of servers.

The NIS server(s) have master copies of the files that are made available. Instead of copying the files to the clients, they are distributed as needed across the network. So the clients retrieve the information from the server instead of reading files.

NIS is used for hosts, services, passwd, automount and a few other configuration files. Configuring a network for NIS is beyond the scope of this chapter, and is not the same for each UNIX variant, since many vendors made their own "improvements" on Sun's system. Thoroughly consult the documentation for the platform(s) that you are using before you try to implement it, and make sure you understand any changes that the vendor's have made. NIS does have some advantages and drawbacks that we can cover.

The main advantage to NIS is convenience. Changes can be made to a map and made available to clients almost instantly, usually by executing a single command after the file is edited. Being able to keep all of this information in one place (or a few places if secondary servers are used) is obviously convenient, especially since synchronizing user IDs, IP addresses, and service ports is crucial to keeping a network working well.

There are however, some significant disadvantages.

A workstation that uses NIS to resolve IP addresses cannot use DNS for internet addresses without some important modifications. Sun's NIS+ solves this problem, but is not yet widely supported by other versions of UNIX and is considered to be a proprietary and very hard to use system. (Which is why it is not covered here.)

If the passwd file is distributed by NIS, the encrypted password field can be read by anyone who connects to NIS. This can be used to subvert security by many hackers, because cracking the encrypted password field is susceptible to a brute force attack with publicly available software.

NIS has no mechanism for controlling who connects to the server and reads the maps. This adds to the security issues, and is one of the reasons you won't find any NIS servers on the Internet. (NIS+ also addresses this issue with a complex authentication system.)

If a client cannot contact a server to handle a request, the request tends to wait until a server is found instead of trying to find a way to continue.

Obviously, NIS comes with its own set of issues, and is a system that requires considerable examination before being selected as an administration tool.

Network Security Issues

We've already mentioned a few security issues regarding UNIX and networking. These issues are very serious for administrators, and frequently have a huge impact on how a network is configured.

Earlier we mentioned that telnet is still used extensively in favor of remote shells. Remote shells allow noninteractive logins, such as the examples above using tar, dd, and rdist. While this is a very convenient feature, it's also a very dangerous one when not carefully administered.

The automatic login feature is implemented with a pair of text files. One of them, /etc/hosts.equiv, controls users on the system level. The other, .rhosts, controls access for individual users. Each file lists the systems by name that a user can login from without supplying a password if the user ID exists on the target host. All of the major UNIX variants treat these files the same way.

hosts.equiv provides this access for an entire workstation, except for the root account. Obviously this file should be very carefully used, if at all. The .rhosts file provides access for individual users, and is located in the user's home directory. It is consulted instead of hosts.equiv. If the root account has one of these files, then the root account from the listed hosts may enter the workstation without any authentication at all, since the .rhosts effectively supercedes the hosts.equiv file.

So the convenience of the remote logins and commands comes with a high price. If a set of workstations were configured to allow root to travel back and forth without authentication than a malicious or, maybe even worse, ill-informed user only needs to compromise one of them in order to wreak havoc on them all.

Some possible precautions are:

Use root as little as possible. Root should never be used for remote operations is a simple enough general rule.
Avoid using rlogin where telnet will do.
If you must use rlogin, try to get by without using .rhosts or hosts.equiv.
If you need to use noninteractive logins for operations such as backups or information collection, create a special account for it that only has access to the files and devices necessary for the job.
Remote logins are just about out of the question on any systems that are directly connected to the Internet. (Please note that a home system that is dialing into the Internet through an ISP is not truly directly connected.)

A little bit of explanation regarding the first rule is in order. The root account should be used as little as possible in day-to-day operations. Many UNIX neophytes feel that root access is necessary to accomplish anything worthwhile, when that's not true at all. There is no reason for common operations, such as performing backups, scanning logs, or running network services to be done as root. (Other than the services that use ports numbered less than 1024. For historical reasons UNIX only allows processes run as root to monitor these ports.) The more situations where root is used, the more likely it is that something unexpected and quite possibly disastrous will occur.

Even beyond remote logins, UNIX offers a lot of network services. File sharing, e-mail, X-windows, and information services, such as DNS and NIS, comprise only part of the flexibility and extensibility offered by networking. However many of these services represent risks that are not always necessary and sometimes are unacceptable.

The right way to handle these services is to evaluate which ones are needed and enable only them. Many network services are managed by inetd. This daemon process listens for requests for network services and executes the right program in order to service them.

For example, the ftp service is administered by inetd. When a request for the ftp service (service port number 21) is received, inetd consults its configuration information and executes ftpd. Its input and output streams are connected to the requester.

Network services are identified by ports. Common services such as ftp and telnet have well-known ports, numbers that all potential clients need to know. inetd binds and listens to these ports based on its configuration data, contained in the file inetd.conf.

A typical configuration file looks like this:

# Configuration file for inetd(1M).  See inetd.conf(4).
#
# To re-configure the running inetd process, edit this file, then
# send the inetd process a SIGHUP.
#
# Syntax for socket-based Internet services:
#        
#
ftp     stream  tcp     nowait  root    /usr/sbin/in.ftpd       in.ftpd
telnet  stream  tcp     nowait  root    /usr/sbin/in.telnetd    in.telnetd
#
# Tnamed serves the obsolete IEN-116 name server protocol.
#
name    dgram   udp     wait    root    /usr/sbin/in.tnamed     in.tnamed
#
# Shell, login, exec, comsat and talk are BSD protocols.
#
shell   stream  tcp     nowait  root    /usr/sbin/in.rshd       in.rshd
login   stream  tcp     nowait  root    /usr/sbin/in.rlogind    in.rlogind
exec    stream  tcp     nowait  root    /usr/sbin/in.rexecd     in.rexecd
comsat  dgram   udp     wait    root    /usr/sbin/in.comsat     in.comsat
talk    dgram   udp     wait    root    /usr/sbin/in.talkd      in.talkd

As the comment states, information for inetd is available on two different manual pages.

Each configuration entry states the name of the service port, which is resolved by using the /etc/services file (or NIS map), some more information about the connection, the user name that the program should be run as, and, finally, the program to run. The two most important aspects of this file, from a security standpoint, are what user the services are run as and what services are run at all. (Details on network connection types are covered in Chapter 20, Networking.")

Each service name corresponds to a number. Ports that are numbered less than 1024, which ftp, telnet, and login all use, can only be attached to as root, so inetd itself does have to be run as root, but it gives us the option of running the individual programs as other users. The reason for this is simple: If a program that is running as root is somehow compromised, the attacker will have root privileges. For example, if a network service that is running as root has a "back door" facility that allows users to modify files, an attacker could theoretically use the program to read, copy or modify any file on the host under attack.

Some of the most serious and effective Internet security attacks exploited undocumented features and bugs in network services that were running as root. Therefore the best protection against the next attack is to avoid the service completely, or at least provide attackers with as little power as possible when they do get find a weakness to take advantage of.

Most UNIX variants come from the vendor running unneeded and, in some cases, undesirable services, such as rexecd, which is used for the remote execution of programs, frequently with no authentication. As you see above, this service came from the vendor configured as root. Many organizations configure key systems to deny all network services except the one service that they are built to provide, such as Internet Web and ftp servers.

Well written network software also takes these issues into consideration. For example the Apache Web Server is usually configured to listen to the http port, which is number 80 and therefore can only be bound to by root. Instead of handling client requests as root and posing a significant security risk, Apache accepts network connections as root, but only handles actual requests as nobody, a user with virtually no rights except to read Web pages. It does this by running multiple copies of itself as nobody and utilizing interprocess communication to dispatch user requests to the crippled processes.

Sharing files on the network poses another set of security issues. Files should be shared carefully, with close attention to not only who can write to them, but who can read them, because e-mail and other forms of electronic communication have become more and more common and can contain important business information.

See Chapters 20, "Networking," and 21, "System Accounting," for more detailed information and instructions on how to properly secure your systems.

UNIX Is Heterogeneous

Linux
UNIX is frequently criticized for a lack of consistency between versions, vendors and even applications. UNIX is not the product of any single corporation or group, and this does have a significant impact on its personality. Linux is probably the ultimate expression of UNIX's collective identity. After Linus Torvalds created the Linux kernel and announced it to the Internet, people from all over the world began to contribute to what has become called the Linux Operating System. While there is a core group of a few developers who were key in its development, they do not all work for the same company or even live in the same country. Obviously, Linux reflects a few different views on how computers should work. UNIX does too.

Administration Tools

UNIX vendors all offer their own GUI administrative tools that are generally useful, provided that you do not have to do something that the vendor did not anticipate. These tools vary widely in how they work and how they are implemented.

AIX
IBM's UNIX operating system, AIX, comes with a sophisticated tool called SMIT. Administrators can use SMIT to configure the system, add and remove users, and upgrade software among other things. SMIT is widely considered to be the best and most mature of the system administration systems, because it can be customized and run in either X-windows or a terminal session. It also allows the user to view the command line equivalent of each task before it is performed. The downside (in at least some system administrator's opinions) is that use of SMIT is just about mandatory for some basic administration tasks.

HP-UX
Hewlett Packard's HP/UX has a similar tool called SAM, which provides much of the functionality offered by SMIT but is not quite as powerful or sophisticated. Its use is not required to administer the system, however.

Solaris
Sun's Solaris does not come with a tool comparable to SMIT or SAM. However, Sun's individual tools for upgrading and installing software and administering NIS+ are functional and intuitive. Unfortunately, the tool supplied with Solaris for administering users, printers, and NIS/NIS+ requires X-windows. It is not, however required to administer the system at all.

Linux
Linux distributions vary widely when it comes to administrative tools. RedHat offers a powerful desktop environment for adding software, administering users, configuring printers, and other everyday administrative tasks. Unlike the commercial tools, it's based on scripts, not binary code, and therefore can be examined and customized by administrators. The Slackware distribution comes with management tools for upgrading and adding software also.

In addition to the different administrative tools and environments provided by the different UNIX vendors, each vendor has felt obligated to provide its own improvements to UNIX over the years. Fortunately, the threat of Windows NT and its homogeneous look and feel has made the UNIX vendors sensitive to these differences, and the tide has turned toward standardization.

UNIX has historically been divided into two major variants, AT&T's UNIX System V and The University of California's BSD UNIX. Most of the major vendors are now moving toward a System V system, but many BSD extensions will always remain.

Regardless of what the vendors do (and claim to do) UNIX's heterogeneous nature is a fact of life, and is probably one of its most important strengths, since that nature is what's responsible for giving us some of the Internet's most important tools, such as perl, e-mail, the Web, and Usenet News. It also provides us with a lot of choices and how to administer our systems. Few problems in UNIX have only one answer.

Graphical Interfaces

Macintosh and Microsoft Windows benefit from a user interface that is designed and implemented by a single vendor. For the most part, a set of applications running on one of these computers shares not only a common look and feel but the same keyboard shortcuts, menus, and mouse movements. X-windows does not have this advantage.

X-windows is a collection of applications, not an integral part of the operating system. Because it is structured this way, it differs greatly from the windowing systems on a Macintosh or Microsoft Windows. As a result, the relationship between X applications and the operating system tends to be a bit more loose than on those platforms.

One of the things that gives an X desktop its "personality" is the window manager. This is the application that provides each window with a border and allows them to be moved and overlapped. However, it is just an application, not part of the X-window system. This separation enables users to select the manager they want, just like a shell. Individual users on the same system can also use different window managers. The differences between window managers are significant, having a significant impact on the look of the desktop, the way the mouse acts, and sometimes, what applications can be run.

The OpenLook window manager, which is distributed by Sun and also accompanies many Linux distributions, has many proprietary extensions and is very lightweight and fast. It bears little resemblance, however, to Macintosh or Microsoft Windows in look or feel. (For one thing, it makes heavy use of the right mouse button, which Microsoft Windows only recently started to do and does not even exist on a Macintosh.) The Sun OpenWindows package comes with OpenLook and a set of tools for reading mail, managing files, and a few other common tasks.

Motif, which has many similarities with Microsoft Windows, has become more popular in the past few years, with most of the major vendors having agreed to standardize on the Common Desktop Environment (CDE), which is based largely on Motif. The CDE also has additional features, such as a graphical file manager, a toolbar, and support for virtual screens. The CDE also comes with a set of user applications.

Window managers also come with programming libraries for creating menus and other programming tasks. As a result, it is possible to create an application that runs poorly under some window managers or requires libraries that a UNIX version does not have. This is a common problem for Motif applications on Linux, because the libraries are not free. For this reason, many applications are available in a non-Motif version or with the libraries statically linked, which means they are built into the application. (This is generally considered undesirable because it makes the application larger and requires more memory and more disk space.)

Windows 3.x has the win.ini and .ini file scheme; Windows 95 and Windows NT have the system registry. Both serve as central repositories for application configuration information. X-windows has its own standard interface for configuration information, called X resources. X resources are more flexible in many ways than the Windows mechanisms.

X resources support wildcards with parameters, which allows administrators (and users) to standardize behavior between diverse applications. Most X-windows application developers recognize this ability and tend to use standard naming schemes for configuration parameters, which makes the use of wildcards even more convenient.

Users are able to customize application to reflect their personal preferences without affecting others by maintaining personal resource files, unlike single user systems that maintain one set of parameters for an entire system. However, administrators still have the ability to set default parameters, so users that do not wish to customize an application still have minimal functionality.

Command Line Interfaces

The command line is where UNIX's long history and diversity really show. One would think that every utility was written by a different person for a different purpose, and one might be right.

There is little cohesion between command line tools as far as the interface goes. The directory list command ls covers just about the whole alphabet when it comes to command line options. The find command, however, uses abbreviations for options instead of letters and almost looks like a new language. Most commands require a "-" to delineate different arguments; others do not. Several commands use subtly different dialects of regular expressions (wildcards).

There are also frequently two versions of the same command: one from the System V world and one from the BSD world. My favorite example of this is the mail command. The mail command is a simple, text-based, virtually feature-free mail reading and sending tool. There are two versions of this command though, so there must have been something worth disagreeing about when it came time to create the most basic of mail readers.

Both mail commands can be used to send e-mail from the command line and are invaluable for unattended scripting jobs since they can alert administrators of problems via e-mail. The BSD version allows you to specify a subject on the command line, like this:

mail -s "Backup Results" {hyperlink mailto:eric@niftydomain.com } < backup.txt

where the -s is an option for the mail subject. The System V version of mail (which is the default for Solaris) quietly ignores the -s option. This has led to my actually troubleshooting why my mail had no subject on at least three different occasions.

Another endearing difference between the two major variants is the ps command. Both commands provide the same information about running programs. The command line arguments are completely different.

The best way to avoid problems with these pointless and gratuitous differences is to take full advantage of the wealth of information contained in the man pages. (In the "Administration Resources" portion of this chapter we cover a few ways to get more information from them.) And keep these tips in mind:

UNIX has been around for more than 25 years. What are the chances that you have a problem that hasn't been solved by someone yet? Before you spend a lot of time and effort solving a problem, spend some time reading man pages about the problem and the tools you are using.
Many of the tools on a UNIX system have been around for those 25+ years. Software that lasts that long must have something going for it.
Experiment with the tools and try to solve the same problem using different methods every once in awhile. Sometimes you can pick up new tricks that will save you hours of time in the future.

System Administration Tasks

System administration can generally be divided into two broad categories: supporting users and supporting systems.

Supporting Users

Users are your customers. Without them the network would be a single computer, probably running a frivolous application like Doom and generating no business or creating no new sales. (Sounds awful doesn't it?) We support users by creating their logins and providing them with the information they need to use their computers to get something done, without forcing them to become computer experts or addicts. (Like us.)

Creating User Accounts The most fundamental thing that an administrator can do for a user is create her account. UNIX accounts are contained in the /etc/passwd file with the actual encrypted password being contained in either the passwd file or the /etc/shadow file if the system implements shadow passwords.

When a user is created, the account obviously has to be added to the passwd file. (The specifics behind user accounts are covered in Chapter 17, "User Administration.") This is a very easy task that can be performed in less than a minute with any text editor. However, if the UNIX version you are using has a tool for adding users, it may not be a bad idea to just take advantage of it.

These tools will frequently:

Add the user to the passwd file. (and shadow file if it is used) Since all users require a unique user ID, letting the computer assign the number when you are administering a large number of users can be very convenient.
Create a home directory for the user with the proper file permissions and ownership.
Copy generic skeleton files to the account, which give the user a basic environment to work with. These files can be created by the administrator, so the paths to shared applications and necessary environment variables can be distributed to users as they are created.
Register the new account with network systems, such as NIS and NFS, if the home directory needs to be accessed on other hosts.

So these tools can prevent common errors and save a bit of time. Most UNIX variants also provide a command line tool named useradd that will perform all or most of these steps. If using the GUI tool is uncomfortable or unworkable, useradd can be incorporated into a shell or perl script.

Providing Support Providing users with documentation and support is another key system administration task and potential headache. The best system administrator is both invisible and never missed. She also realizes that expecting users to get as excited about learning how to use a computer as she is is asking too much.

A famous proverb talks about teaching a man to fish instead of buying him a McFish sandwich. (Or something like that.) The idea behind this expression can be applied to users with amazing results. However, few users will go out of their way to learn how to use their systems. How can an administrator teach his users how to help themselves without fighting a never ending battle?

All user populations are different, and there is no universal solution to user training, but here are a couple of ideas that may help.

Try to provide and gently enforce a formal method for requesting changes to systems and getting help. If users get accustomed to instant answers to questions, they will become angry when you can't drop what you're doing to help them. Try to set up an e-mail or Web-based help system. It may seem bureaucratic to force users to use these systems, but it helps prevent your day from becoming "interrupt-driven" and also provides you with a log of requests. (Have you ever forgotten to take care of a request because you were too overloaded? Give yourself an automatic to do list.)

Provide as much documentation as you can through easy-to-use interfaces, such as Web browsers. The time you spend developing this information in user friendly formats will be paid back in phone calls and conversations that never happen, and may help you learn some more about your systems as you do it.

If you have Internet access now, I'm sure your users spend hours glued to their browsers; take advantage of that. You may also find that a lot of the information your users need is already out there, so you may be able to link your users to some of the information they need, without generating it yourself. (We'll go over some of these resources later in the "Administration Resources" portion of this chapter.)

Supporting Systems

The other half of the job is supporting your systems. Systems have to be built, backed up, upgraded, and, of course, fixed.

Adding Nodes A frequent system administration task is adding new nodes to the network. It's also one of the parts of the job that can truly benefit from some planning and insight.

Not all systems are created equal, and not all of them are used for the same thing. Spending some time understanding what your network is really used for and then applying that to systems is key in network planning. Workstations should have well defined roles and should be configured in accordance with those roles.

When systems are designed and evaluated, some of the questions an administrator can ask are:

Will users be able to access all or some of the systems? Do users need to access more than one system? Are there system that users should never access?
What network file systems will each workstation need to access? Are there enough that automount would help?
What network services, such as telnet, remote logins, sharing file systems, and e-mail, do workstations need to provide? Can each service be justified?
What networks will workstations need to access? Are there networks that should be inaccessible from others?

These questions should help us develop a profile for each workstation. Following a profile makes workstations easier to build, maintain, and troubleshoot, not to mention making them more reliable since they tend to be less complex.

Backups Files get corrupted, lost, accidentally overwritten, or deleted. Our only protection against these situations is backups, since UNIX does not have an undelete command.

UNIX provides several backup tools, and deciding which tool(s) to use can be a difficult.

Earlier in the chapter we had an example of backing up a directory, using tar and dd, to a remote tape drive. In that example, dd was just be used as a way to copy a stream of data to a tape, while tar was the command actually performing the backup.

tar (tape archive) is a commonly used backup tool.

tar -c -f /dev/rmt/0 /home/eric

The above command would back up the contents of the /home/eric directory to the first tape drive installed on a Solaris system. tar automatically traverses the directory, so all files in /home/eric and its subdirectories are archived.

The device name for tape drives on systems differs from variant to variant. BSD derivatives tend to refer to them as /dev/rst1 where 0 is the first, 1 is the second, and so on. (Linux, HP/UX, and SunOS 4.1.x use this nomenclature.) System V derivatives usually use /dev/rmt1 the same way. (AIX uses this model.) However, Solaris 2.x, which is System V based, adds a directory to the path, so it is /dev/rmt/1.

The -f option tells tar which tape drive to use, while the -c option is telling it to create a new archive instead of modifying an existing one. One of tar's idiosyncrasies is that when a path is given to it as the back up specification, it is added to the archive, so when it time to restore /home/eric, that is where it will be restored to. A more flexible way to back up the directory is to do this:

cd /home/eric
tar -cf /dev/rmt/0 .

Tar recognizes . as meaning backup the current directory. When the archive is extracted, it will be placed in the current directory, regardless of where that is.

cpio is another standard UNIX tool for backups. Its interface is a little more difficult than tar's, but has several advantages.

cpio is usually used with ls or find to create archives.

find . -print | cpio -o > /dev/rst0

The find command prints the full path of all of the files in its current directory to standard out. cpio accepts these filenames and archives them to standard output. This is redirected to the tape, where it is archived. This command is an excellent example of the UNIX way of combining commands to create a new tool.

find is a tool for finding files. They can be located by name, size, creation date, modification date, and a whole universe of other criteria too extensive to cover here. (As always, see the man page!) This example takes this tool for locating files and makes it the user interface to our backup system.

cpio is the file copying and archiving "Swiss Army Knife." In addition to streaming files in a format suitable for tape it can:

Backup special files, such as device drive stubs like /dev/rst0.
Place data on tapes more efficiently than tar or dd.
Skip over bad areas on tapes or floppies when restoring, when tar would simply die. With cpio, you can at least restore part of a damaged archive.
Perform backups to floppies, including spread a single archive over more than one disk. tar can only put a single archive on one disk.
Swap bytes during the archive or extraction in order to aid in transferring files from one architecture to another.

This example also illustrates how we can redirect standard output to a device name and expect the device driver to place it on the device.

Our example could also be stretched into a full-featured backup system. Since find allows us to pick files based on creation and modification dates, we could perform a full backup to tape once a week by telling find to name all files, and perform incremental backups to floppy every other day. The UNIX command line is a very powerful tool, especially since programs have been designed for it steadily during the past three decades.

Solaris
There are other backup tools, most notably dump or ufsdump as it is named on Solaris 2.x. These tools are oriented toward backing up entire disk partitions, instead of files. They are covered in Chapter 28, "Backing Up and Restoring Your System."

Larger sites with more comprehensive backup requirements may need a commercial package. Legato's NetWorker provides an advanced backup and restore system that support the automated and unattended backup of UNIX systems, Windows and NT workstations, Macintosh PCs and even database servers. The process of scheduling backups, selecting file for backup and confirming the integrity of archives is all done automatically. Restoring files and file systems is also very simple due to the GUI.

Solaris
Sun bundles NetWorker with the Solaris 2.x server edition.

Just as important as picking a backup tool is designing a backup strategy.

Backing up the entire system daily is not acceptable, since it would take too long and would be too ponderous to restore if only one file is needed.

Here is another area where it pays to know your users and understand their business. It's also important to design the network with backup and recovery in mind.

Do the users have important files that they modify daily?
Are the users clustered into one of two directories on one system, or will systems have to be backed up across the network?
If only a single file has to be restored, will it have to be done quickly? Will our current backup system allow us to find one file.
How often and for how long are users not using the system? (It is better to back up directories when they are not in use.)

Depending on the requirements, a commercial backup tool may be a good investment. The commercial tools, such as NetWorker, do excel in allowing you to locate and restore one specific file quickly, which may be necessary in your situation.

System Load and Performance The task of monitoring system load and system performance falls on the shoulders of the system administrator. This is yet another area where planning and anticipation are superior to waiting for something to happen and reacting to it.

Daily monitoring of usage statistics is a good idea, especially on mission-critical systems and systems that users interact with regularly, such as Web servers and X displays. These statistics can be gathered with automated scripts.

Some of the things to monitor are disk usage, CPU utilization, swap, and memory. The tools used for getting this information, such as du and df for disk information and top and vmstat for the rest, are covered in the next few chapters in Part III.

Administration Resources

A system administrator needs every shred of information and help he can get, and as much as UNIX vendors would hate to admit, the documentation that accompanies is sometimes lacking. Fortunately, it's a big world out there.

The Manual Pages

The famous rejoinder RTFM (Read The Fine Manual) refers to the manual pages installed on (hopefully) every UNIX system. The man pages, as they are frequently called, contain documentation and instructions on just about every UNIX command, C function call and data file on your system.

The man command searches for documentation based on a command or topic name. So the command

man ls

provides us with documentation on the ls command, which happens to be in section one.

As simple as they may appear, the man pages actually have a sophisticated structure. The pages are divided into sections, with some of the sections being further divided into subsections.

The section layout resembles this:

User commands--commands like ls, tar, and cpio.
System calls--C programming functions that are considered system calls, like opening and closing files.
C library functions--C programming functions that are not considered system calls, like printing text.
File formats--descriptions of file layouts, such as hosts.equiv and inetd.conf.
Headers, Tables and Macros--miscellaneous documentation, such as character sets and header files, not already covered.
Games and Demos--games and demo software. (Even Doom for UNIX has a man page!)

This is a generalized table of contents. BSD and System V started out with slightly different section schemes, and vendors tend to add their own sections and make their own "improvements." (How could it be any other way?)

In order to view information about each section, we can view the intro page for it. In order to see information about section one, we would execute the following command.

man -s 1 intro

The -s option selects which section of the man pages to search with the System V version of the man command. BSD versions accept the section number as the first argument with no switch, while the Linux version will select the section from an environment variable or from a -s option.

All of the versions accept the -a option, which will force the man command to search all of the sections and display all pages that match. Since there are different pages with the same name, understanding the different sections and what belongs in each of them is helpful. However, there are additional tools, and it is not necessary to memorize the section layout for each system.

Man pages can be indexed and pre-formatted with a tool called catman. The pre-formatting part make the pages display faster and is less important than the indexing, which allows us to use more powerful information retrieval tools, namely whatis and apropos.

apropos displays the section number, name and short description of any page that contains the specified keyword. whatis gives us the man page name and section number.

For example, let's try the apropos command for the keyword nfs:

apropos nfs

A portion of the response would be this:

automount (8)	Automatically mount NFS file systems
exportfs (8)	Export and unexport directories to NFS clients
exports, xtab (5)	Directories to export to NFS clients
nfs, NFS (4P)	Network file system
nfsd, biod (8)	NFS daemons

(The output was edited, because the actual request returned 12 responses.)

Now let's try whatis:

whatis nfs

nfs, NFS (4P) Network file system

With whatis we only see the name of the actual nfs manual page and its section number.

Solaris
In addition to the manual pages, Sun supplies Answerbook with Solaris. It's a GUI based online help system that supports full text search and just about completely removes the need for hard copy documentation.

AIX
IBM supplies Infoviewer which is also a GUI-based online help system. As a matter of fact, the default AIX installation program installs Infoviewer instead of the manual pages, which can be quite an inconvenience for users that do not use X-windows.

Internet Information resources

The internet provides administrators with a wealth of resources too.

1. Usenet News--While not as useful as it was in the past (due to overcrowding and a plummeting signal to noise ratio) usenet news offers discussion groups about all of the UNIX variants and covers various aspects of them. Some examples are comp.sys.sun.admin, comp.os.linux.setup and comp.unix.admin. Because of the high traffic on usenet news, some sites do not carry it. If you cannot get access, try using a search service such as http://www.dejanews.com to find what you need. Sometimes searching for articles that interest you instead of browsing is a faster way to get what you need anyway. Dejanews offers searching and browsing.

2. FAQ Lists--Frequently Asked Question Lists hold a wealth of information. Most of the computer-related usenet groups have their own FAQ lists. They can be found posted periodically in the groups and at the rtfm.mit.edu ftp server. Many of the FAQS are also available in html format.

3. The Web--Documentation is available from the UNIX vendors and from people and groups involved with Linux. Many users and developers also post a wealth of information just to be helpful. Use the search services to get your research going. UNIX was one of the first operating systems to colonize the Internet, and, as a result, there is a huge number of UNIX users on it.

There is a large amount of documentation and help available free on the Internet, and there is a lot more to be learned from seeking out an answer than simply paying a consultant to tell you or leaning on someone else to do what has to be done for you.

Tools of the Trade

A successful administrator takes full advantage of the tools provided with a UNIX system. To the uninitiated, UNIX seems difficult and unwieldy, but once you get the idea you'll never want to use another system.

The Shell

Earlier, we demonstrated how cpio uses the output of the find command to learn what files to archive. This was a demonstration of shell pipes, which redirect the output of one command to another. We also used this to back up files to a tape drive that is located on another host.

We demonstrated mailing a file to a user on the command line using redirection, which opens a file and passes it to a command as if it was provided in the command line or typed in as a program requested it.

UNIX shells also support sophisticated programming constructs, such as loops, and provide comparison operators, such as equality, greater than, and less than.

Shell programming is essential to system administration, especially as networks grow larger and, as a result, more time consuming to maintain. File backups, adding users and nodes, collecting usage statistics, and a whole host of other administrative tasks are candidates for unattended scripts.

Perl and Other Automation Tools

Perl has become more and more popular over the past few years, and not without good reason. Many tasks that would have required the use of C in the past can now be done by a novice programmer in Perl. System administrators can benefit greatly from a little working knowledge of this language, since it can be used for tasks such as:

Analyzing log files and alerting the administrator of trouble via e-mail or pager
Automatically converting systems statistics into Web pages.
Automating the process of creating user accounts, adding and distributing automounter maps, backing up systems, and creating html content.
Creating and communicating over network connections.

These are only a few examples of what this language can do. Chapter 5, "Perl," in Volume 2 has in-depth coverage of Perl.

Some other tools worth noting are TCL/TK, which most of the RedHat administrative tools are written in and awk, which is covered extensively in Chapter 4, "Awk," of Volume 2.

Intranet Tools

As mentioned earlier, consider using intranet tools like Web servers and e-mail to communicate with your users.

There are many Web discussion and guest book applications, all written in perl, that could be modified and used to allow customers to enter requests for support. An internal home page could be used to announce scheduled outages and system enhancements. The Web also allows you to link clients to vendor-provided support resources and documentation.

Summary

What is system administration? This chapter kind of meandered along and tried to give you an idea of what administrators do and what they need to know.

A literal attempt at answering the question may say that administrators are responsible for:

Understanding how the systems they are responsible interact over the office or organizational LAN/WAN.
Supporting users by creating their accounts, protecting their data and making their sometimes bewildering systems easier to use.
Supporting systems by keeping the secure, up to date and well tuned.
Planning the efforts behind support and growth.
Anticipating and resolving problems.

But it may be better to generalize and say that system administration is the process and effort behind supporting a system or systems so that a company or group can attain its goals.

The rest of this section delves much more deeply into the details of system support. Chapters 15 and 16 describe how to install a UNIX system and start it up and shut it down.

In Chapter 17 we cover supporting users much more in-depth than was covered here. Chapter 18 covers file systems and disk configuration , while 19 provides coverage on kernel configuration.

Chapter 20 is where the complete coverage on UNIX networking is located. In Chapter 21 we cover system accounting, which is key for administrators who are concerned with supported large groups of users or that need to come up with a comprehensive system for charging users for access.

For information on system performance and how to improve it, see Chapter 22. In addition, Chapter 23 provides information on how to add and maintain new system components, such as tape drives and modems.

Managing mail, Usenet news and other network services is covered in Chapters 23 through 27. Our last chapter in this section, 28, covers system backup and restore.

What Is a Shell?

2008-08-28T23:57:00.001-07:00

By William A. Farra

Nearly every human-usable invention has an interface point with which you interact. Whether you are in the front seat of a horse and buggy, in the cockpit of a plane, or at the keyboard of a piano, this position is where you manipulate and manage the various aspects of the invention to achieve a desired outcome. The human interface point for UNIX is the shell, which is a program layer that provides you with an environment in which to enter commands and parameters to produce a given result. As with any invention, the more knowledge and experience you have with it, the greater the accomplishment you make with it.

To meet varying needs, UNIX has provided different shells. Discussed in Chapters 9 through 13 are Bourne, Bourne Again, Korn, and C shells. Each of these offers features and ways to interact with UNIX. Topics discussed in this chapter are the following:

How shells works with you and UNIX
The features of a shell
Manipulating the shell environment

How the Kernel and the Shell Interact

When a UNIX system is brought online, the program unix (the Kernel) is loaded into the computer's main memory, where it remains until the computer is shut down. During the bootup process, the program init runs as a background task and remains running until shutdown. This program scans the file /etc/inittab, which lists what ports have terminals and their characteristics. When an active, open terminal is found, init calls the program getty, which issues a login: prompt to the terminal's monitor. With these processes in place and running, the user is ready to start interacting with the system.

UNIX Calls the Shell at Login

Figure 8.1 shows the process flow from the kernel through the login process. At this point the user is in an active shell, ready to give command to the system.

Figure 8.1.
How a shell is started from login.

During login, when you type your user name, getty issues a password: prompt to the monitor. After you type your password, getty calls login, which scans for a matching entry in the file /etc/passwd. If a match is made, login proceeds to take you to your home directory and then passes control to a session startup program; both the user name and password are specified by the entry in /etc/passwd. Although this might be a specific application program, such as a menu program, normally the session startup program is a shell program such as /bin/sh, the Bourne shell.

From here, the shell program reads the files /etc/profile and .profile, which set up the system-wide and user-specific environment criteria. At this point, the shell issues a command prompt such as $.

When the shell is terminated, the kernel returns control to the init program, which restarts the login process. Termination can happen in one of two ways: with the exit command or when the kernel issues a kill command to the shell process. At termination, the kernel recovers all resources used by the user and the shell program.

The Shell and Child Processes

In the Unix system, there are many layers of programs starting from the kernel through a given application program or command. The relationship of these layers is represented in Figure 8.2.

Figure 8.2.
UNIX system layers.

After you finish logging on, the shell program layer is in direct contact with the kernel, as shown in Figure 8.2. As you type a command such as $ ls, the shell locates the actual program file, /bin/ls, and passes it to the kernel to execute. The kernel creates a new child process area, loads the program, and executes the instructions in /bin/ls. After program completion, the kernel recovers the process area and returns control to the parent shell program. To see an example of this, type the following command:

$ps

This lists the processes you are currently running. You will see the shell program and the ps program. Now type the following:

$sleep 10 &
$ps

The first command creates a sleep child process to run in background, which you see listed with the ps command. Whenever you enter a command, a child process is created and independently executes from the parent process or shell. This leaves the parent intact to continue other work.

Auto-Execution of the Shell

Some UNIX resources, such as cron, can execute a shell program without human interaction. When using this feature, the user needs to specify which shell to run in the first line of the shell program, like this:

#! /bin/sh

This specifies the Bourne shell.

You should also redirect any output, because no terminal is associated with auto-execution. This is described in the "File Handling: Input/Output Redirection and Pipes" section later in this chapter.

The Functions and Features of a Shell

It doesn't matter which of the standard shells you choose, because they all have the same purpose: to provide a user interface to UNIX. To provide this interface, all the shells offer the same basic characteristics:

Command-line interpretation
Reserved words
Shell meta-characters (wild cards)
Access to and handling of program commands
File handling: input/output redirection and pipes
Maintenance of variables
Environment control
Shell programming

Command-Line Interpretation

When you log in, starting a special version of a shell called an interactive shell, you see a shell prompt, usually in the form of a dollar sign ($), a percent sign (%), or a pound sign (#). When you type a line of input at a shell prompt, the shell tries to interpret it. Input to a shell prompt is sometimes called a command line. The basic format of a command line is

command arguments

command is an executable UNIX command, program, utility, or shell program. arguments are passed to the executable. Most UNIX utility programs expect arguments to take the following form:

options filenames

For example, in the command line

$ ls -l file1 file2

there are three arguments to ls; the first is an option, and the last two are filenames.

One of the things the shell does for the kernel is to eliminate unnecessary information. For a computer, one type of unnecessary information is whitespace; therefore, it is important to know what the shell does when it sees whitespace. Whitespace consists of space characters, horizontal tabs, and newline characters. Consider this example:

$ echo part A     part B     part C
part A part B part C

Here, the shell has interpreted the command line as the echo command with six arguments and has removed the whitespace between the arguments. For example, if you were printing headings for a report and wanted to keep the whitespace, you would have to enclose the data in quotation marks, as in

$ echo 'part A     part B     part C'
part A     part B     part C

The single quotation mark prevents the shell from looking inside the quotes. Now the shell interprets this line as the echo command with a single argument, which happens to be a string of characters including whitespace.

Reserved Words

All shell versions have words that have special meaning. In shell programming, words such as do, done, for, and while provide loop control--and if, then, else, and fi provide conditional control. Each shell version has different reserved word pertaining to its specific features.

Shell Meta-Character (Wild Cards)

All shell versions have meta-characters, which allow the user to specify filenames. The following are wild cards:

Wild Card	Description
`*`	Matches any portion
`?`	Matches any single character
`[]`	Matches a range or list of characters

Wild cards can be useful when processing a number of specific files. The following are some examples:

$ls t*

This lists all files starting with t.

$ls test?5.dat

This lists all files starting with test, any single character and ends with 5.dat.

$ls [a-c]*

This lists all files starting with a through c.

$ls [e,m,t]*

This lists all files starting with e, m, or t.

Program Commands

When a command is typed, the shell reads the environment variable $path, which contains a list of directories containing program files. The shell looks through this set of directories to find the program file for the command. The shell then passes the true filename to the kernel.

File Handling: Input/Output Redirection and Pipes

In previous chapters, you learned about standard input and output. Unless otherwise specified with arguments, most UNIX commands take input from the terminal keyboard and send output to the terminal monitor. To redirect output to a file, use the > symbol. For example,

$ls > myfiles

lists the files in your current directory and places them in a file called myfiles. Likewise, you can redirect input with the < symbol. For example,

$wc -l < myfiles

feeds the command wc with input from the file myfiles. Although you could obtain the same output by having the filename as an argument, the need for input redirection becomes more apparent in shell programming.

To string the output from one command to the input of the next command, you can use the | (pipe) symbol. For example,

$ls -s | sort -nr | pg

This lists the files in the current directory with blocksize and then pipes the output to the sort, which sorts the files in numeric descending order and pipes that output to the paging command pg for final display on the terminal's monitor. The pipe command is one of the most useful tools when creating command constructs.

Command Substitution

Command substitution is similar to redirection except that is used to provide arguments to a command from the output of another. For example,

$grep 'wc -l myfiles' *

takes the number of lines in the file myfiles from the wc command and places the number as an argument to the grep command to search all files in the current directory for that number.

Maintenance of Variables

The shell is capable of maintaining variables. Variables are places you can store data for later use. You assign a value to a variable with an equal (=) sign:

$ LOOKUP=/usr/mydir

Here, the shell establishes LOOKUP as a variable and assigns it the value /usr/mydir. Later, you can use the value stored in LOOKUP in a command line by prefacing the variable name with a dollar sign ($). Consider these examples:

$ echo $LOOKUP
/usr/mydir
$ echo LOOKUP
LOOKUP

To make a variable available to child processes, you can use the export command--for example:

$ LOOKUP=/usr/mydir
$export LOOKUP

NOTE: Assigning values to variables in the C shell differs from doing so in the Bourne and Korn shells. To assign a variable in the C-shell, use the set command:
% set LOOKUP = /usr/mydir

Notice that spaces precede and follow the equal sign.

Like filename substitution, variable name substitution happens before the program call is made. The second example omits the dollar sign ($). Therefore, the shell simply passes the string to echo as an argument. In variable name substitution, the value of the variable replaces the variable name.

For example, in

$ ls $LOOKUP/filename

the ls program is called with the single argument /usr/mydir/filename.

Shell Startup--Environment Control

When a user begins a session with UNIX and the shell is executed, the shell creates a specified environment for the user. The following sections describe these processes.

Shell Environment Variables When the login program invokes your shell, it sets up your environment variables, which are read from the shell initialization files /etc/profile and .profile. These files normally set the type of terminal in the variable $TERM and the default path that is searched for executable files in the variable $PATH. Try these examples:

$ echo $TERM
$ echo $PATH

You can easily change the variables the same way you assign values to any shell variable.

NOTE: C shell assigns values to environment variables using the setenv command:
% setenv TERM vt100

Shell Startup Files The file .profile is the local startup file for the Bourne shell. The Korn shell uses .kshrc, and the C shell uses .cshrc. You can edit these files to manipulate your startup environment. You can add additional variables as the need arises. You also can add shell programming to have conditional environment settings, if necessary.

Shell Startup Options When invoking the shell either from /etc/passwd or the command line, you can set several options as arguments to the shell program. For example, the Bourne shell has a -x option that displays commands and their arguments before they are executed. This is useful for debugging a shell program. These options are described in detail in the following chapters.

Shell Programming

You've seen that the shell is used to interpret command lines, maintain variables, and execute programs. The shell is also a programming language. You can store a set of shell commands in file. This is known as a shell script or shell programming. By combining commands and variable assignments with flow control and decision making, you have a powerful programming tool. Using the shell as a programming language, you can automate recurring tasks, write reports, and build and manipulate your own data files. The remaining chapters in Part II discuss shell programming in more detail.

Summary

The shell provides an interface between the user and the heart of UNIX--the kernel. The shell interprets command lines as input, makes filename and variable substitution, redirects input and output, locates the executable file, and initiates and interfaces programs. The shell creates child processes and can manage their execution. The shell maintains each user's environment variables. The shell is also a powerful programming language.

While this chapter gives an overview of the UNIX shell, Chapters 9 through 13 describe in detail the various shells, their features, and language specifics. Also described are the fundamentals of shell programming and execution. Continued reading is highly recommended.

Unix commands

2008-08-28T23:55:00.001-07:00

cd matlab change to the subdirectory named matlab
cd change to the home directory
cp file1 file2 make a copy of file1 called file2
cp dir2/file1 . copy file1 from directory dir2 into current directory
echo $param show idefinition of param
env show values of environment variables
history show previous commands by number
lpr -Pworkman54 f1 print f1 (notice capital P)
lpq -Pworkman54 look at printer queue for sage
ls *dwg list names of all files ending in the letters dwg
man cp learn more about cp command (manual)
mkdir ansys make a new directory called ansys (change to root
first with cd command)
more file1 look at file1 page by page (usually aliased to just m)
mv file1 file2 move or rename file1 to file2
pwd show current directory name
rm *bak remove all files ending in the letters bak
abbr=orig define abbreviation (usually shorter)
tail -15 file1 look at last 15 lines of file1

You can rerun previous commands and make changes with emacs commands:
^P redisplay previous command (then use commands to change it)
^Rls repeat most recent command that begins with ls
!5 repeat command 5 (use history or h to find the command number)
!^ first option of previous command
!$ last option of previous command
!* all options of previous command
!:2 second option of previous command

Before erasing files, check that the parameters are correct. For example, first give:
ls fname* # list corresponding files
rm !$ # erase those files

To copy files from my class directory, define a short symbol a, then check th

rm': Remove files or directories
=================================

`rm' removes each given FILE. By default, it does not remove
directories. Synopsis:

rm [OPTION]... [FILE]...

If a file is unwritable, standard input is a terminal, and the `-f'
or `--force' option is not given, or the `-i' or `--interactive' option
_is_ given, `rm' prompts the user for whether to remove the file. If
the response does not begin with `y' or `Y', the file is skipped.

The program accepts the following options. Also see XRef Common
options.

`-d'
`--directory'
Remove directories with `unlink' instead of `rmdir', and don't
require a directory to be empty before trying to unlink it. Only
works if you have appropriate privileges. Because unlinking a
directory causes any files in the deleted directory to become
unreferenced, it is wise to `fsck' the filesystem after doing this.

`-f'
`--force'
Ignore nonexistent files and never prompt the user. Ignore any
previous `--interactive' (`-i') option.

`-i'
`--interactive'
Prompt whether to remove each file. If the response does not begin
with `y' or `Y', the file is skipped. Ignore any previous
`--force' (`-f') option.

`-r'
`-R'
`--recursive'
Remove the contents of directories recursively.

`-v'
`--verbose'
Print the name of each file before removing it.

One common question is how to remove files whose names begin with a
`-'. GNU `rm', like every program that uses the `getopt' function to
parse its arguments, lets you use the `--' option to indicate that all
following arguments are non-options. To remove a file called `-f' in
the current directory, you could type either:

rm -- -f

or:

rm ./-f

The Unix `rm' program's use of a single `-' for this purpose
predates the development of the getopt standard syntax.

Prev (fileutils) mv invocation Up (fileutils) Basic operations
--------------------------------------------------------------------------------
automatically generated by info2html

FTP Options
===========

`--retr-symlinks'
Retrieve symbolic links on FTP sites as if they were plain files,
i.e. don't just create links locally.

`-g on/off'
`--glob=on/off'
Turn FTP globbing on or off. Globbing means you may use the
shell-like special characters ("wildcards"), like `*', `?', `['
and `]' to retrieve more than one file from the same directory at
once, like:

wget ftp://gnjilux.cc.fer.hr/*.msg

By default, globbing will be turned on if the URL contains a
globbing character. This option may be used to turn globbing on
or off permanently.

You may have to quote the URL to protect it from being expanded by
your shell. Globbing makes Wget look for a directory listing,
which is system-specific. This is why it currently works only
with Unix FTP servers (and the ones emulating Unix `ls' output).

`--passive-ftp'
Use the "passive" FTP retrieval scheme, in which the client
initiates the data connection. This is sometimes required for FTP
to work behind firewalls.

Shell Scripting

2008-08-12T04:59:00.000-07:00

Variables in Shell

To process our data/information, data must be kept in computers RAM memory. RAM memory is divided into small locations, and each location had unique number called memory location/address, which is used to hold our data. Programmer can give a unique name to this memory location/address called memory variable or variable (Its a named storage location that may take different values, but only one at a time).

In Shell, there are two types of variable:
(1) System variables - Created and maintained by Linux itself. This type of variable defined in CAPITAL LETTERS.
(2) User defined variables (UDV) - Created and maintained by user. This type of variable defined in lower letters.

You can see system variables by giving command like $ set, some of the important System variables are:

System Variable	Meaning
BASH=/bin/bash	Our shell name
BASH_VERSION=1.14.7(1)	Our shell version name
COLUMNS=80	No. of columns for our screen
HOME=/home/vivek	Our home directory
LINES=25	No. of columns for our screen
LOGNAME=students	students Our logging name
OSTYPE=Linux	Our Os type
PATH=/usr/bin:/sbin:/bin:/usr/sbin	Our path settings
PWD=/home/students/Common	Our current working directory
SHELL=/bin/bash	Our shell name
USERNAME=vivek	User name who is currently login to this PC

How to define User defined variables (UDV)

To define UDV use following syntax
Syntax:
variable name=value

'value' is assigned to given 'variable name' and Value must be on right side = sign.

Example:
$ no=10# this is ok
$ 10=no# Error, NOT Ok, Value must be on right side of = sign.
To define variable called 'vech' having value Bus
$ vech=Bus
To define variable called n having value 10
$ n=10

How to print or access value of UDV (User defined variables)

To print or access UDV use following syntax
Syntax:
$variablename

Define variable vech and n as follows:
$ vech=Bus
$ n=10
To print contains of variable 'vech' type
$ echo $vech
It will print 'Bus',To print contains of variable 'n' type command as follows
$ echo $n

Exercise
Q.1.How to Define variable x with value 10 and print it on screen.
Q.2.How to Define variable xn with value Rani and print it on screen
Q.3.How to print sum of two numbers, let's say 6 and 3?
Q.4.How to define two variable x=20, y=5 and then to print division of x and y (i.e. x/y)
Q.5.Modify above and store division of x and y to variable called z
Q.6.Point out error if any in following script

$ vi variscript
#
#
# Script to test MY knowledge about variables!
#
myname=Vivek
myos = TroubleOS
myno=5
echo "My name is $myname"
echo "My os is $myos"
echo "My number is myno, can you see this number"

echo Command

Use echo command to display text or value of variable.

echo [options] [string, variables...]
Displays text or variables value on screen.
Options
-n Do not output the trailing new line.
-e Enable interpretation of the following backslash escaped characters in the strings:
\a alert (bell)
\b backspace
\c suppress trailing new line
\n new line
\r carriage return
\t horizontal tab
\\ backslash

For e.g. $ echo -e "An apple a day keeps away \a\t\tdoctor\n"

Shell Arithmetic

Use to perform arithmetic operations.

Syntax:
expr op1 math-operator op2

Examples:
$ expr 1 + 3
$ expr 2 - 1
$ expr 10 / 2
$ expr 20 % 3
$ expr 10 \* 3
$ echo `expr 6 + 3`

Note:
expr 20 %3 - Remainder read as 20 mod 3 and remainder is 2.
expr 10 \* 3 - Multiplication use \* and not * since its wild card.

For the last statement not the following points

(1) First, before expr keyword we used ` (back quote) sign not the (single quote i.e. ') sign. Back quote is generally found on the key under tilde (~) on PC keyboard OR to the above of TAB key.

(2) Second, expr is also end with ` i.e. back quote.

(3) Here expr 6 + 3 is evaluated to 9, then echo command prints 9 as sum

(4) Here if you use double quote or single quote, it will NOT work
For e.g.
$ echo "expr 6 + 3" # It will print expr 6 + 3
$ echo 'expr 6 + 3' # It will print expr 6 + 3

There are three types of quotes

Quotes	Name	Meaning
"	Double Quotes	"Double Quotes" - Anything enclose in double quotes removed meaning of that characters (except \ and $).
'	Single quotes	'Single quotes' - Enclosed in single quotes remains unchanged.
`	Back quote	`Back quote` - To execute command

Example:
$ echo "Today is date"
Can't print message with today's date.
$ echo "Today is `date`".
It will print today's date as, Today is Tue Jan ....,Can you see that the `date` statement uses back quote?

Exit Status

By default in Linux if particular command/shell script is executed, it return two type of values which is used to see whether command or shell script executed is successful or not.

(1) If return value is zero (0), command is successful.
(2) If return value is nonzero, command is not successful or some sort of error executing command/shell script.

This value is know as Exit Status.

But how to find out exit status of command or shell script?
Simple, to determine this exit Status you can use $? special variable of shell.

For e.g. (This example assumes that unknow1file doest not exist on your hard drive)
$ rm unknow1file
It will show error as follows
rm: cannot remove `unkowm1file': No such file or directory
and after that if you give command
$ echo $?
it will print nonzero value to indicate error. Now give command
$ ls
$ echo $?
It will print 0 to indicate command is successful.

The read Statement

Use to get input (data from user) from keyboard and store (data) to variable.
Syntax:
read variable1, variable2,...variableN

Following script first ask user, name and then waits to enter name from the user via keyboard. Then user enters name from keyboard (after giving name you have to press ENTER key) and entered name through keyboard is stored (assigned) to variable fname.

$ vi sayH
#
#Script to read your name from key-board
#
echo "Your first name please:"
read fname
echo "Hello $fname, Lets be friend!"

Run it as follows:
$ chmod 755 sayH
$ ./sayH
Your first name please: vivek
Hello vivek, Lets be friend!

Wild cards (Filename Shorthand or meta Characters)

Wild card /Shorthand	Meaning	Examples
*	Matches any string or group of characters.	$ ls *	will show all files
		$ ls a*	will show all files whose first name is starting with letter 'a'
		*$ ls .c**	will show all files having extension .c
		*$ ls ut.c**	will show all files having extension .c but file name must begin with 'ut'.
?	Matches any single character.	$ ls ?	will show all files whose names are 1 character long
?	Matches any single character.	$ ls fo?	will show all files whose names are 3 character long and file name begin with fo
[...]	Matches any one of the enclosed characters	$ ls [abc]*	will show all files beginning with letters a,b,c

Note:
[..-..] A pair of characters separated by a minus sign denotes a range.

Example:
$ ls /bin/[a-c]*

Will show all files name beginning with letter a,b or c like

/bin/arch /bin/awk /bin/bsh /bin/chmod /bin/cp
/bin/ash /bin/basename /bin/cat /bin/chown /bin/cpio
/bin/ash.static /bin/bash /bin/chgrp /bin/consolechars /bin/csh

But
$ ls /bin/[!a-o]
$ ls /bin/[^a-o]

If the first character following the [ is a ! or a ^ ,then any character not enclosed is matched i.e. do not show us file name that beginning with a,b,c,e...o, like

/bin/ps /bin/rvi /bin/sleep /bin/touch /bin/view
/bin/pwd /bin/rview /bin/sort /bin/true /bin/wcomp
/bin/red /bin/sayHello /bin/stty /bin/umount /bin/xconf
/bin/remadmin /bin/sed /bin/su /bin/uname /bin/ypdomainname
/bin/rm /bin/setserial /bin/sync /bin/userconf /bin/zcat
/bin/rmdir /bin/sfxload /bin/tar /bin/usleep
/bin/rpm /bin/sh /bin/tcsh /bin/vi

More command on one command line

Syntax:
command1;command2
To run two command with one command line.

Examples:
$ date;who
Will print today's date followed by users who are currently login. Note that You can't use
$ date who
for same purpose, you must put semicolon in between date and who command.

Why Command Line arguments required

Telling the command/utility which option to use.
Informing the utility/command which file or group of files to process (reading/writing of files).

Let's take rm command, which is used to remove file, but which file you want to remove and how you will tail this to rm command (even rm command don't ask you name of file that you would like to remove). So what we do is we write command as follows:
$ rm {file-name}
Here rm is command and filename is file which you would like to remove. This way you tail rm command which file you would like to remove. So we are doing one way communication with our command by specifying filename Also you can pass command line arguments to your script to make it more users friendly. But how we access command line argument in our script.

Lets take ls command
$ Ls -a /*
This command has 2 command line argument -a and /* is another. For shell script,
$ myshell foo bar

Shell Script name i.e. myshell
First command line argument passed to myshell i.e. foo
Second command line argument passed to myshell i.e. bar

In shell if we wish to refer this command line argument we refer above as follows

myshell it is $0
foo it is $1
bar it is $2

Here $# (built in shell variable ) will be 2 (Since foo and bar only two Arguments), Please note at a time such 9 arguments can be used from $1..$9, You can also refer all of them by using $* (which expand to `$1,$2...$9`). Note that $1..$9 i.e command line arguments to shell script is know as "positional parameters".

Exercise
Try to write following for commands
Shell Script Name ($0),
No. of Arguments (i.e. $#),
And actual argument (i.e. $1,$2 etc)
$ sum 11 20
$ math 4 - 7
$ d
$ bp -5 myf +20
$ Ls *
$ cal
$ findBS 4 8 24 BIG

Shell Script Name	No. Of Arguments to script	Actual Argument ($1,..$9)
$0	$#	$1	$2	$3	$4	$5
Sum	2	11	20
Math	3	4	-	7
D	0
Bp	3	-5	myf	+20
Ls	1	*
Cal	0
FindBS	4	4	8	24	BIG

Following script is used to print command ling argument and will show you how to access them:

$ vi demo
#!/bin/sh
#
# Script that demos, command line args
#
echo "Total number of command line argument are $#"
echo "$0 is script name"
echo "$1 is first argument"
echo "$2 is second argument"
echo "All of them are :- $* or $@"

Run it as follows

Set execute permission as follows:
$ chmod 755 demo

Run it & test it as follows:
$ ./demo Hello World

If test successful, copy script to your own bin directory (Install script for private use)
$ cp demo ~/bin

Check whether it is working or not (?)
$ demo
$ demo Hello World

Also note that you can't assigne the new value to command line arguments i.e positional parameters. So following all statements in shell script are invalid:
$1 = 5
$2 = "My Name"

Redirection of Standard output/input i.e. Input - Output redirection

Mostly all command gives output on screen or take input from keyboard, but in Linux (and in other OSs also) it's possible to send output to file or to read input from file.

For e.g.
$ ls command gives output to screen; to send output to file of ls command give command

$ ls > filename
It means put output of ls command to filename.

There are three main redirection symbols >,>>,<

(1) > Redirector Symbol
Syntax:
Linux-command > filename
To output Linux-commands result (output of command or shell script) to file. Note that if file already exist, it will be overwritten else new file is created. For e.g. To send output of ls command give
$ ls > myfiles
Now if 'myfiles' file exist in your current directory it will be overwritten without any type of warning.

(2) >> Redirector Symbol
Syntax:
Linux-command >> filename
To output Linux-commands result (output of command or shell script) to END of file. Note that if file exist , it will be opened and new information/data will be written to END of file, without losing previous information/data, And if file is not exist, then new file is created. For e.g. To send output of date command to already exist file give command
$ date >> myfiles

(3) < Redirector Symbol
Syntax:
Linux-command < filename
To take input to Linux-command from file instead of key-board. For e.g. To take input for cat command give
$ cat <>

Pipes

A pipe is a way to connect the output of one program to the input of another program without any temporary file.

Pipe Defined as:
"A pipe is nothing but a temporary storage place where the output of one command is stored and then passed as the input for second command. Pipes are used to run more than two commands ( Multiple commands) from same command line."

Syntax:
command1 | command2

Examles:

Command using Pipes

Meaning or Use of Pipes

$ ls | more

Output of ls command is given as input to more command So that output is printed one screen full page at a time.

$ who | sort

Output of who command is given as input to sort command So that it will print sorted list of users

$ who | sort > user_list

Same as above except output of sort is send to (redirected) user_list file

$ who | wc -l

Output of who command is given as input to wc command So that it will number of user who logon to system

$ ls -l | wc -l

Output of ls command is given as input to wc command So that it will print number of files in current directory.

$ who | grep raju

Output of who command is given as input to grep command So that it will print if particular user name if he is logon or nothing is printed (To see particular user is logon or not)

Filter

If a Linux command accepts its input from the standard input and produces its output on standard output is know as a filter. A filter performs some kind of process on the input and gives output. For e.g.. Suppose you have file called 'hotel.txt' with 100 lines data, And from 'hotel.txt' you would like to print contains from line number 20 to line number 30 and store this result to file called 'hlist' then give command:
$ tail +20 <>hlist

Here head command is filter which takes its input from tail command (tail command start selecting from line number 20 of given file i.e. hotel.txt) and passes this lines as input to head, whose output is redirected to 'hlist' file.

Consider one more following example
$ sort <> u_sname

Here uniq is filter which takes its input from sort command and passes this lines as input to uniq; Then uniqs output is redirected to "u_sname" file.

Command Related with Process

Following tables most commonly used command(s) with process:

For this purpose

Use this Command

Examples*

To see currently running process

ps

$ ps

To stop any process by PID i.e. to kill process

kill {PID}

$ kill 1012

To stop processes by name i.e. to kill process

killall {Process-name}

$ killall httpd

To get information about all running process

ps -ag

$ ps -ag

To stop all process except your shell

kill 0

$ kill 0

For background processing (With &, use to put particular command and program in background)

linux-command &

$ ls / -R | wc -l &

To display the owner of the processes along with the processes

ps aux

$ ps aux

To see if a particular process is running or not. For this purpose you have to use ps command in combination with the grep command

ps ax | grep process-U-want-to see

For e.g. you want to see whether Apache web server process is running or not then give command

$ ps ax | grep httpd

To see currently running processes and other information like memory and CPU usage with real time updates.

top
See the output of top command.

$ top

Note that to exit from top command press q.

To display a tree of processes

pstree

$ pstree

* To run some of this command you need to be root or equivalnt user.

if condition

if condition which is used for decision making in shell script, If given condition is true then command1 is executed.
Syntax:

if condition
then
command1 if condition is true or if exit status
of condition is 0 (zero)
...
...
fi

Condition is defined as:
"Condition is nothing but comparison between two values."

For compression you can use test or [ expr ] statements or even exist status can be also used.

Expreession is defined as:
"An expression is nothing but combination of values, relational operator (such as >,<, <> etc) and mathematical operators (such as +, -, / etc )."

Following are all examples of expression:
5 > 2
3 + 6
3 * 65
a < b
c > 5
c > 5 + 30 -1

Type following commands (assumes you have file called foo)
$ cat foo
$ echo $?
The cat command return zero(0) i.e. exit status, on successful, this can be used, in if condition as follows, Write shell script as

$ cat > showfile
#!/bin/sh
#
#Script to print file
#
if cat $1
then
echo -e "\n\nFile $1, found and successfully echoed"
fi

Run above script as:
$ chmod 755 showfile
$./showfile foo
Shell script name is showfile ($0) and foo is argument (which is $1).Then shell compare it as follows:
if cat $1 which is expanded to if cat foo.

Detailed explanation
if cat command finds foo file and if its successfully shown on screen, it means our cat command is successful and its exist status is 0 (indicates success), So our if condition is also true and hence statement echo -e "\n\nFile $1, found and successfully echoed" is proceed by shell. Now if cat command is not successful then it returns non-zero value (indicates some sort of failure) and this statement echo -e "\n\nFile $1, found and successfully echoed" is skipped by our shell.

Exercise
Write shell script as follows:

cat > trmif
#
# Script to test rm command and exist status
#
if rm $1
then
echo "$1 file deleted"
fi

Press Ctrl + d to save
$ chmod 755 trmif

Answer the following question in referance to above script:
(A) foo file exists on your disk and you give command, $ ./trmfi foo what will be output?
(B) If bar file not present on your disk and you give command, $ ./trmfi bar what will be output?
(C) And if you type $ ./trmfi What will be output?

test command or [ expr ]

test command or [ expr ] is used to see if an expression is true, and if it is true it return zero(0), otherwise returns nonzero for false.
Syntax:
test expression OR [ expression ]

Example:
Following script determine whether given argument number is positive.

$ cat > ispostive
#!/bin/sh
#
# Script to see whether argument is positive
#
if test $1 -gt 0
then
echo "$1 number is positive"
fi

Run it as follows
$ chmod 755 ispostive

$ ispostive 5
5 number is positive

$ispostive -45
Nothing is printed

$ispostive
./ispostive: test: -gt: unary operator expected

Detailed explanation
The line, if test $1 -gt 0 , test to see if first command line argument($1) is greater than 0. If it is true(0) then test will return 0 and output will printed as 5 number is positive but for -45 argument there is no output because our condition is not true(0) (no -45 is not greater than 0) hence echo statement is skipped. And for last statement we have not supplied any argument hence error ./ispostive: test: -gt: unary operator expected, is generated by shell , to avoid such error we can test whether command line argument is supplied or not.

test or [ expr ] works with
1.Integer ( Number without decimal point)
2.File types
3.Character strings

For Mathematics, use following operator in Shell Script

Mathematical Operator in Shell Script

Meaning

Normal Arithmetical/ Mathematical Statements

But in Shell

For test statement with if command

For [ expr ] statement with if command

-eq

is equal to

5 == 6

if test 5 -eq 6

if [ 5 -eq 6 ]

-ne

is not equal to

5 != 6

if test 5 -ne 6

if [ 5 -ne 6 ]

-lt

is less than

5 <>

if test 5 -lt 6

if [ 5 -lt 6 ]

-le

is less than or equal to

5 <= 6

if test 5 -le 6

if [ 5 -le 6 ]

-gt

is greater than

5 > 6

if test 5 -gt 6

if [ 5 -gt 6 ]

-ge

is greater than or equal to

5 >= 6

if test 5 -ge 6

if [ 5 -ge 6 ]

NOTE: == is equal, != is not equal.

For string Comparisons use

Operator

Meaning

string1 = string2

string1 is equal to string2

string1 != string2

string1 is NOT equal to string2

string1

string1 is NOT NULL or not defined

-n string1

string1 is NOT NULL and does exist

-z string1

string1 is NULL and does exist

Shell also test for file and directory types

Test

Meaning

-s file

Non empty file

-f file

Is File exist or normal file and not a directory

-d dir

Is Directory exist and not a file

-w file

Is writeable file

-r file

Is read-only file

-x file

Is file is executable

Logical Operators

Logical operators are used to combine two or more condition at a time

Operator

Meaning

! expression

Logical NOT

expression1 -a expression2

Logical AND

expression1 -o expression2

Logical OR

if...else...fi

If given condition is true then command1 is executed otherwise command2 is executed.
Syntax:

if condition
then
condition is zero (true - 0)
execute all commands up to else statement

else
if condition is not true then
execute all commands up to fi
fi

For e.g. Write Script as follows:

$ vi isnump_n
#!/bin/sh
#
# Script to see whether argument is positive or negative
#
if [ $# -eq 0 ]
then
echo "$0 : You must give/supply one integers"
exit 1
fi

if test $1 -gt 0
then
echo "$1 number is positive"
else
echo "$1 number is negative"
fi

Try it as follows:
$ chmod 755 isnump_n

$ isnump_n 5
5 number is positive

$ isnump_n -45
-45 number is negative

$ isnump_n
./ispos_n : You must give/supply one integers

$ isnump_n 0
0 number is negative

Detailed explanation
First script checks whether command line argument is given or not, if not given then it print error message as "./ispos_n : You must give/supply one integers". if statement checks whether number of argument ($#) passed to script is not equal (-eq) to 0, if we passed any argument to script then this if statement is false and if no command line argument is given then this if statement is true. The echo command i.e.
echo "$0 : You must give/supply one integers"
| |
| |
1 2
1 will print Name of script
2 will print this error message
And finally statement exit 1 causes normal program termination with exit status 1 (nonzero means script is not successfully run).

The last sample run $ isnump_n 0 , gives output as "0 number is negative", because given argument is not > 0, hence condition is false and it's taken as negative number. To avoid this replace second if statement with if test $1 -ge 0.

Nested if-else-fi

You can write the entire if-else construct within either the body of the if statement of the body of an else statement. This is called the nesting of ifs.

$ vi nestedif.sh
osch=0

echo "1. Unix (Sun Os)"
echo "2. Linux (Red Hat)"
echo -n "Select your os choice [1 or 2]? "
read osch

if [ $osch -eq 1 ] ; then

echo "You Pick up Unix (Sun Os)"

else #### nested if i.e. if within if ######

if [ $osch -eq 2 ] ; then
echo "You Pick up Linux (Red Hat)"
else
echo "What you don't like Unix/Linux OS."
fi
fi

Run the above shell script as follows:
$ chmod +x nestedif.sh
$ ./nestedif.sh
1. Unix (Sun Os)
2. Linux (Red Hat)
Select you os choice [1 or 2]? 1
You Pick up Unix (Sun Os)

$ ./nestedif.sh
1. Unix (Sun Os)
2. Linux (Red Hat)
Select you os choice [1 or 2]? 2
You Pick up Linux (Red Hat)

$ ./nestedif.sh
1. Unix (Sun Os)
2. Linux (Red Hat)
Select you os choice [1 or 2]? 3
What you don't like Unix/Linux OS.

Note that Second if-else constuct is nested in the first else statement. If the condition in the first if statement is false the the condition in the second if statement is checked. If it is false as well the final else statement is executed.

You can use the nested ifs as follows also:
Syntax:

if condition
then
if condition
then
.....
..
do this
else
....
..
do this
fi
else
...
.....
do this
fi

Multilevel if-then-else

Syntax:

if condition
then
condition is zero (true - 0)
execute all commands up to elif statement
elif condition1
then
condition1 is zero (true - 0)
execute all commands up to elif statement
elif condition2
then
condition2 is zero (true - 0)
execute all commands up to elif statement
else
None of the above condtion,condtion1,condtion2 are true (i.e.
all of the above nonzero or false)
execute all commands up to fi
fi

For multilevel if-then-else statement try the following script:

$ cat > elf
#
#!/bin/sh
# Script to test if..elif...else
#
if [ $1 -gt 0 ]; then
echo "$1 is positive"
elif [ $1 -lt 0 ]
then
echo "$1 is negative"
elif [ $1 -eq 0 ]
then
echo "$1 is zero"
else
echo "Opps! $1 is not number, give number"
fi

Loops in Shell Scripts

Loop defined as:
"Computer can repeat particular instruction again and again, until particular condition satisfies. A group of instruction that is executed repeatedly is called a loop."

Bash supports:

for loop
while loop

Note that in each and every loop,

(a) First, the variable used in loop condition must be initialized, then execution of the loop begins.

(b) A test (condition) is made at the beginning of each iteration.

(c) The body of loop ends with a statement that modifies the value of the test (condition) variable.

for Loop

Syntax:

for { variable name } in { list }
do
execute one for each item in the list until the list is
not finished (And repeat all statement between do and done)
done

Before try to understand above syntax try the following script:

$ cat > testfor
for i in 1 2 3 4 5
do
echo "Welcome $i times"
done

while loop

Syntax:

while [ condition ]
do
command1
command2
command3
..
....
done

Loop is executed as long as given condition is true. For e.g.. Above for loop program (shown in last section of for loop) can be written using while loop as:

$cat > nt1
#!/bin/sh
#
#Script to test while statement
#
#
if [ $# -eq 0 ]
then
echo "Error - Number missing form command line argument"
echo "Syntax : $0 number"
echo " Use to print multiplication table for given number"
exit 1
fi
n=$1
i=1
while [ $i -le 10 ]
do
echo "$n * $i = `expr $i \* $n`"
i=`expr $i + 1`
done

Save it and try as
$ chmod 755 nt1
$./nt1 7
Above loop can be explained as follows:

n=$1

Set the value of command line argument to variable n. (Here it's set to 7 )

i=1

Set variable i to 1

while [ $i -le 10 ]

This is our loop condition, here if value of i is less than 10 then, shell execute all statements between do and done

do

Start loop

echo "$n * $i = `expr $i \* $n`"

Print multiplication table as
7 * 1 = 7
7 * 2 = 14
....
7 * 10 = 70, Here each time value of variable n is multiply be i.

i=`expr $i + 1`

Increment i by 1 and store result to i. ( i.e. i=i+1)
Caution: If you ignore (remove) this statement than our loop become infinite loop because value of variable i always remain less than 10 and program will only output
7 * 1 = 7
...
...
E (infinite times)

done

Loop stops here if i is not less than 10 i.e. condition of loop is not true. Hence
loop is terminated.

The case Statement

The case statement is good alternative to Multilevel if-then-else-fi statement. It enable you to match several values against one variable. Its easier to read and write.
Syntax:

case $variable-name in
pattern1) command
...
..
command;;
pattern2) command
...
..
command;;
patternN) command
...
..
command;;
*) command
...
..
command;;
esac

The $variable-name is compared against the patterns until a match is found. The shell then executes all the statements up to the two semicolons that are next to each other. The default is *) and its executed if no match is found. For e.g. write script as follows:

$ cat > car
#
# if no vehicle name is given
# i.e. -z $1 is defined and it is NULL
#
# if no command line arg

if [ -z $1 ]
then
rental="*** Unknown vehicle ***"
elif [ -n $1 ]
then
# otherwise make first arg as rental
rental=$1
fi

case $rental in
"car") echo "For $rental Rs.20 per k/m";;
"van") echo "For $rental Rs.10 per k/m";;
"jeep") echo "For $rental Rs.5 per k/m";;
"bicycle") echo "For $rental 20 paisa per k/m";;
*) echo "Sorry, I can not gat a $rental for you";;
esac

Save it by pressing CTRL+D and run it as follows:
$ chmod +x car
$ car van
$ car car
$ car Maruti-800

First script will check, that if $1(first command line argument) is given or not, if NOT given set value of rental variable to "*** Unknown vehicle ***",if command line arg is supplied/given set value of rental variable to given value (command line arg). The $rental is compared against the patterns until a match is found.
For first test run its match with van and it will show output "For van Rs.10 per k/m."
For second test run it print, "For car Rs.20 per k/m".
And for last run, there is no match for Maruti-800, hence default i.e. *) is executed and it prints, "Sorry, I can not gat a Maruti-800 for you".
Note that esac is always required to indicate end of case statement.

How to de-bug the shell script?

While programming shell sometimes you need to find the errors (bugs) in shell script and correct the errors (remove errors - debug). For this purpose you can use -v and -x option with sh or bash command to debug the shell script. General syntax is as follows:
Syntax:
sh option { shell-script-name }
OR
bash option { shell-script-name }
Option can be
-v Print shell input lines as they are read.
-x After expanding each simple-command, bash displays the expanded value of PS4 system variable, followed by the command and its expanded arguments.

Example:

$ cat > dsh1.sh
#
# Script to show debug of shell
#
tot=`expr $1 + $2`
echo $tot

Press ctrl + d to save, and run it as
$ chmod 755 dsh1.sh
$ ./dsh1.sh 4 5
9
$ sh -x dsh1.sh 4 5
#
# Script to show debug of shell
#
tot=`expr $1 + $2`
expr $1 + $2
++ expr 4 + 5
+ tot=9
echo $tot
+ echo 9
9

See the above output, -x shows the exact values of variables (or statements are shown on screen with values).

$ sh -v dsh1.sh 4 5

Use -v option to debug complex shell script.

/dev/null - Use to send unwanted output of program

This is special Linux file which is used to send any unwanted output from program/command.
Syntax:
command > /dev/null

Example:
$ ls > /dev/null
Output of above command is not shown on screen its send to this special file. The /dev directory contains other device files. The files in this directory mostly represent peripheral devices such disks like floppy disk, sound card, line printers etc. See the file system tutorial for more information on Linux disk, partition and file system.

BASIC UNIX COMMANDS COMMANDS

2008-08-06T02:42:00.001-07:00
Commands are what you type at the prompt. Com-
mands have arguments on which they operate. For
example, in rm temp, the command is rm and the ar-
gument is temp; this command removes the le called
temp. Here I put arguments in UPPER CASE. Thus,
words such as FILE are taken to stand for some other
word, such as temp. In the following list, I use [ ] for
optional arguments that are not typicaly used.
Commands have options that are controlled with
switches, which are usually letters following a single
dash. Usually you can write several letters after one
dash. For example ls -l lists les in a long format,
with more information. ls -a lists all the les, in-
cluding those that begin with ., which are usually
les used by various programs. ls -al lists all the
les in the long format. The following list omits most
options. Many programs will give you a list of their
options if you type the name of the program followed
by -h or -?.
Other options are controlled in special les usu-
ally beginning with ., such as .emacs, .mailrc, and
.newsrc. You can edit these. The le .cshrc con-
tains general options, plus aliases that you make up
for commands you use often.
|, the pipe symbol, takes the output of one command
and gives it to the second command as input, for
example, ls -l | more allows you to view a long
le list through \more".
> directs the output to a le, e.g., ls -l >
listing.tmp puts the listing into a le named list-
ing.tmp. > overwrites an old le of the same name.
>> appends to a le.
* stands for any string of letters. For example, ls t*
lists all the les beginning with t.
" recalls previous commands, and you can edit these
commands using emacs editing.
The tab key completes commands if you type the rst
few letters.
ctrl-z suspends most programs. fg resumes. ctrl-c
stops most programs.
alias abbreviates a series of commands, separated
by semicolons. Useful in .cshrc.
GETTING HELP
apropos KEYWORD: Looks for commands related to
KEYWORD.
man COMMAND: Shows manual pages on COMMAND.
This is the authoritative source on the items de-
scribed here. The commands can do much more
than is listed here.
whatis COMMAND: Tells what COMMAND does.
FILES AND DIRECTORIES
All information is stored in les. File names and
commands are case sensitive. Case matters. Files
are contained in directories. You start out in your
own home directory, and your prompt usually tells
its name. At any given time, one of these directories
is your working directory, the one you are in.
You can refer to les in your working directory by
just their names. You can refer to a le that is in a
subdirectory by giving a subdirectory name, a slash,
and the le name, e.g., Mail/baron. You can refer to
any le on the computer by giving its full name, start-
ing with a slash, such as /home7/b/baron/mbox.
If the le is a program, typing its name will run it.
(That is what commands do.) If the program is some-
thing you have just written and is in the director you
are in, put ./ before the name. If the le is not a
program, typing its name will give you an error mes-
sage. If you want to see its contents, for example, you
must use a command such as \more" before the le
name.
ls [DIRECTORY]: Lists les. (Also try: ls -f, ls -s, ls
-a.)
rm FILE: Removes FILE.
more FILE or less FILE: View FILE. (? or h for
help.)
cd DIRECTORY: Change the directory you are in to
DIRECTORY.
cd: Change to your home directory.
cd ..: Change to the next directory up in the hier-
archy.
mkdir DIRECTORY: Make DIRECTORY.
rmdir DIRECTORY: Remove DIRECTORY.
rm -rf: Recursively remove a directory and anything
in it.
mv FILE1 FILE2: Moves or renames FILE1 to
FILE2.
cp FILE1 FILE2: Makes a copy of FILE1, called
FILE2.
cat FILE1 FILE2 > FILE3: Concatenate FILE1
and FILE2, calling the result FILE3.
chmod 644 FILE: Unprotect FILE for others to read
or copy.
chmod 755 FILE: Unprotect program or \script."
chmod 755 DIRECTORY: Unprotect DIRECTORY,
needed for web page directories.
head and tail: Print the top and bottom of a text
le.
TEXT FILE MANIPULATION
These commands operate on les. The output goes
to the \standard output," which is your terminal dis-
play. If you want to \redirect" the output to a le,
use > FILENAME at the end of the command (with
FILENAME being the name of the le). Use >> instead
of > if you want to append to the le rather than
write it from scratch.
diff: Find the dierences between two text les.
grep LETTER-STRING FILE1 [> FILE2]: Prints (or
puts in FILE2) all lines of FILE1 that contain
LETTER-STRING. Use the -v switch to get lines
not containing the string.
sed s/STRING1/STRING2/g: replace STRING1 with
STRING2 throughout a le. See also the y switch
for replacing characters.
cut -d" " -f2 FILE: Extract the second column
from a le, where the columns are delimited by
spaces. Use paste to put such cuttings back to-
gether.
sort FILE: Sort the lines alphabetically.
uniq FILE: Remove adjacent duplicate lines. Typi-
cally used with the output of sort, e.g., sort FILE
| uniq. Use the -c to count the number of ad-
jacent examples of each line.
wc FILE: Count characters, words, and lines.
WHAT'S GOING ON?
w [USER]: Who is using the computer. This is useful
to see whether you are logged on twice. (See ps and
kill, below, in case this happens.) Also try who and
finger.
finger USER: Gives information about the user, in-
cluding the les .project and .plan, if you have
these les. You can sometimes use this for people
on other computers. You can keep useful infor-
mation in your .plan, such as your schedule, your
phone number, etc.
quota -v: Tells you how much of your quota for les
is used up.
ps -fu USERNAME: Lists the processes that you are
running, if you put in your username. You can use
this to nd the number of processes that you want
to kill, such as those left over when you did not
log out properly. It is the rst number listed.
kill PROCESS-NUMBER: Kills the process you don't
want. If this doesn't work, try kill -9
PROCESS-NUMBER.
last -22: Shows you the last 22 users who logged
in.
THE INTERNET
mutt and elm: Read and write electronic mail. Each
has its own \help". See the discussion of these in
the psychology web page computer section.
slrn and tin: Read news (and respond to postings).
Pnews: Post to newsgroups. (You can
also say mutt psych-general@psych or
elm psych-general@psych to post to
upenn.psych.general.)
lynx [URL]: Reads web pages as text les.
ssh HOST: Connects to a remote computer.
talk USERNAME: Allows you to talk with someone
logged on. The full username must be specied
for remote computers, and you must use ntalk
instead of talk. It is not a good idea to use this
casually unless you know that the other person will
not be annoyed.
TRANSFERRING FILES
Many computers on the internet have ftp or ssh, but
all of the programs for transferring les depend on
having the relevant software on both computers.
ftp and scp: Fast way of uploading and down-
loading, or moving any le from one com-
puter to another. For example, scp myfile
baron@psych.upenn.edu: - you need the colon
at the end.
rsync: synchronize les or directories on two dier-
ent computers. Good for backing up.
EDITING
pico FILE or emacs FILE xemacs FILE or vi FILE:
Edit FILE. One of these editors may be specied
as the default for mailing and news programs. Pico
is easiest because it has all the commands listed at
the bottom of the screen, but it is the least useful
because it has few commands.
COMPRESSING, ENCRYPTING, and SE-
CURITY
tar cvf FILE.tar and tar xvf FILE.tar: Create
and extract an archive le. Useful for backing up
directories. On Linux xvfz will extract and unzip
in one step.
gzip FILE: Reduce the size of les for storage. use
gunzip FILE to unzip them. Files \zipped" with
gzip have the sux .gz.
crypt <> FILE2: Encrypts or decodes
le1 so that you need a password to read it. But if
you want to be absolutely sure beyond any doubt
that nobody will read your les, do not leave them
on cattell.
antiword FILE1 > FILE2: Decodes a Word le
into text. (That is all most of them are.) Save
storage space. You can use this with Mutt or Elm
to read doc les sent in email messages.
passwd: Change your password.
Jon Baron, September, 2003

Command using Pipes	Meaning or Use of Pipes
$ ls \| more	Output of ls command is given as input to more command So that output is printed one screen full page at a time.
$ who \| sort	Output of who command is given as input to sort command So that it will print sorted list of users
$ who \| sort > user_list	Same as above except output of sort is send to (redirected) user_list file
$ who \| wc -l	Output of who command is given as input to wc command So that it will number of user who logon to system
$ ls -l \| wc -l	Output of ls command is given as input to wc command So that it will print number of files in current directory.
$ who \| grep raju	Output of who command is given as input to grep command So that it will print if particular user name if he is logon or nothing is printed (To see particular user is logon or not)

For this purpose	Use this Command	Examples*
To see currently running process	ps	$ ps
To stop any process by PID i.e. to kill process	kill {PID}	$ kill 1012
To stop processes by name i.e. to kill process	killall {Process-name}	$ killall httpd
To get information about all running process	ps -ag	$ ps -ag
To stop all process except your shell	kill 0	$ kill 0
For background processing (With &, use to put particular command and program in background)	linux-command &	$ ls / -R \| wc -l &
To display the owner of the processes along with the processes	ps aux	$ ps aux
To see if a particular process is running or not. For this purpose you have to use ps command in combination with the grep command	ps ax \| grep process-U-want-to see	For e.g. you want to see whether Apache web server process is running or not then give command $ ps ax \| grep httpd
To see currently running processes and other information like memory and CPU usage with real time updates.	top See the output of top command.	$ top Note that to exit from top command press q.
To display a tree of processes	pstree	$ pstree

Mathematical Operator in Shell Script	Meaning	Normal Arithmetical/ Mathematical Statements	But in Shell
			For test statement with if command	For [ expr ] statement with if command
-eq	is equal to	5 == 6	if test 5 -eq 6	if [ 5 -eq 6 ]
-ne	is not equal to	5 != 6	if test 5 -ne 6	if [ 5 -ne 6 ]
-lt	is less than	5 <>	if test 5 -lt 6	if [ 5 -lt 6 ]
-le	is less than or equal to	5 <= 6	if test 5 -le 6	if [ 5 -le 6 ]
-gt	is greater than	5 > 6	if test 5 -gt 6	if [ 5 -gt 6 ]
-ge	is greater than or equal to	5 >= 6	if test 5 -ge 6	if [ 5 -ge 6 ]

Operator	Meaning
string1 = string2	string1 is equal to string2
string1 != string2	string1 is NOT equal to string2
string1	string1 is NOT NULL or not defined
-n string1	string1 is NOT NULL and does exist
-z string1	string1 is NULL and does exist

Test	Meaning
-s file	Non empty file
-f file	Is File exist or normal file and not a directory
-d dir	Is Directory exist and not a file
-w file	Is writeable file
-r file	Is read-only file
-x file	Is file is executable

Operator	Meaning
! expression	Logical NOT
expression1 -a expression2	Logical AND
expression1 -o expression2	Logical OR

n=$1	Set the value of command line argument to variable n. (Here it's set to 7 )
i=1	Set variable i to 1
while [ $i -le 10 ]	This is our loop condition, here if value of i is less than 10 then, shell execute all statements between do and done
do	Start loop
echo "$n * $i = `expr $i \* $n`"	Print multiplication table as 7 * 1 = 7 7 * 2 = 14 .... 7 * 10 = 70, Here each time value of variable n is multiply be i.
i=`expr $i + 1`	Increment i by 1 and store result to i. ( i.e. i=i+1) Caution: If you ignore (remove) this statement than our loop become infinite loop because value of variable i always remain less than 10 and program will only output 7 * 1 = 7 ... ... E (infinite times)
done	Loop stops here if i is not less than 10 i.e. condition of loop is not true. Hence loop is terminated.