Tuesday 7 April 2009

Tutorial: An Introduction to Linux CLI

The shell
A shell is a command interpreter which allows the user to interact with the computer. The way things work is simple: the user types in commands, the shell interprets them and performs the tasks it was asked to do, and finally it sends the result to the standard output (which is usually the screen). For example, the output of the ls command can be something like:

$ ls
bin dev home lost+found opt sbin sys var
boot etc initrd.img media proc selinux tmp vmlinuz
cdrom floyd lib mnt root srv usr

This is a list of files inside the root directory. The root directory is the first location in the filesystem hierarchy, and it is represented by a slash sign: /.

Some of the most popular shells are:
bash Bourne-Again Shell, the default shell on most Linux systems
sh Bourne Shell, an older shell which is not so widely used anymore and it has been replaced by Bash
csh C Shell, with a syntax that resembles the C programming language syntax
tcsh an improved version of the C Shell
ksh Korn Shell
dash Debian Almquist Shell

To access a shell-like application you can use, for example, GNOME Terminal (the GNOME terminal application) or Konsole (the KDE terminal application). To open either of those, press Alt+F2, and type either gnome-terminal or konsole in the Run dialogue that appears. You can also press Alt+Ctrl+F1 to get a shell login prompt, and you can return to the graphical interface with Alt+Ctrl+F7. Some other terminal applications include Yakuake, xterm or rxvt, but I recommend to stay with the default one which comes with your distribution for now. Here is a full article about those.

Konsole, default KDE terminal application

Bash, the default shell in most Linux distributions, including Debian and Ubuntu, is a modern implementation of the older Bourne Shell (sh). Bash was developed by the GNU Project, which is also responsible for most of the important command-line tools widely used on a Linux system. In fact, a more appropriate name for Linux is GNU/Linux, since any Linux-based distribution includes the Linux kernel (the core of the operating system) and the GNU tools and utilities. On a Debian system, the default shell used by the system is specified in the /etc/passwd file.

Basic commands
Commands in Linux are executable files which can be ran to perform tasks, like listing the files in a directory or creating and renaming files for example. Examples of frequently used commands are ls, pwd, date, cat, less, top and so on. The general form of Linux commands is:

command option(s) filename(s)

A command may or may not have arguments. An argument (also called a parameter) can be an option or a filename. Here is a list of basic, most used commands, with the exact description specified in the manual pages:

ls list directory contents
pwd print name of current/working directory
date print or set the system date and time
mkdir make directories
rm remove files or directories
mv move (rename) files
who show who is logged on
whoami print effective userid
cat concatenate files and print on the standard output
more file perusal filter for crt viewing
less opposite of more

Standard GNU tools usually have two ways of specifying an option, the short and the long syntax, so ls -a and ls --all will do the same thing, list files including the ones that start with a dot (.).

Options specified between the [ and ] brackets are optionally, for example the synopsis ls [OPTION]... [FILE]... tells that neither OPTION or FILE needs to be necessarily specified.

The cat command can be used to read text files, and it is especially useful for small ones. cat will display on the screen the last page of the text file(s) passed as arguments, without the ability to scroll it in order to see the rest of the file. For larger files which are longer than one page, use the less and more commands, or a text editor like nano. For example, to read the /etc/motd (message of the day) file, one can use:

$ less /etc/motd

Omit the dollar sign ($), which is the prompt in this case. A file with some text message in it should be now displayed on the screen. To navigate in a text file using less, use the following keyboard shortcuts:

^N to go forward one line (same effect as J)
^P to go backward one line (same effect as K)
^V to go forward one page (same effect as SPACE)
Alt+V to go backward one page

To quit and return to the prompt, type Q. In the examples above, the ^ character stands for the CONTROL character (labeled Ctrl), so ^N means 'press Ctrl+N'. The above shortcuts are widely used on a Linux-based OS by applications like Emacs (an advanced text editor and IDE - integrated development environment), the man command and many others. The arrow keys can also be used, by the above method is the preferred one due to the fact that it is faster, and the user doesn't have to move the fingers to the arrows then back to the typing position.

The man command
man is used to see detailed help about a command. You can use the keyboard shortcuts mentioned above to navigate through a manual page.

Type Q to quit the manual page and return to the shell prompt. The Q character is usually used to quit most Linux command-line applications, like more, less, man and many others. If you type H while reading a manual page, a help about the command less will be provided (man uses the less tool in order to format the manual pages). Those are the commands that apply in the manual page.

The cd and pwd commands
The commands cd, which is used to change the current directory, and pwd, used to print the current working directory, are both shell built-ins. This means that they are included in the Bash program rather than a standalone command (file), like ls or date. Some examples of shell built-ins are cd, echo, read, exit, logout, alias, fg or bg.

You can see what cd does by changing the current working directory to a new one and see the new path using the pwd command:

$ pwd
/home/embryo
$ cd /usr/share
$ pwd
/usr/share

You can now type ls to see the available files in /usr/share. As a note, cd without any arguments changes the current directory to the home directory of the current user, located by default in /home/username:

$ pwd
/usr/share
$ cd
$ pwd
/home/embryo

Control characters
A shell uses certain key combinations to perform certain tasks. These key combinations are called control characters. Here are examples of control characters:

^A - go to the start of line
^E - go to the end of line
^H - erase one character to the left (same effect as Backspace)
^D - erase one character to the right; it also ends the session if there is nothing to delete
^U - erase entire line
^K - erase everything to the right until the end of line
^P - bring the previous command in history
^N - bring the next command in history
^C - interrupt character: this usually terminates programs

Those are very important because they make working with the shell faster and easier. Although a little tough to learn for a beginner, they prove fast once they are learned.

Regarding ^N and ^P: for example if you type ls, then date and pwd, and after that ^P, the pwd command will be brought back and you only have to type Enter to issue it again. If you type ^P again, the date command is brought and so on.

Regarding ^D, also called the EOF (end-of-file) character: this key combination can also be used to tell a program that the input has ended and it can return its output. For example, you can type wc -l, which is the command to count the lines in a text file, without providing the text file. Now you can start filling in lines and typing Enter after each line. When you're done, just type Ctrl+D and the wc tool will return the total number of lines you entered:

$ wc -l
There is a house in New Orleans
They call the rising sun
And it's been the ruin of many a poor boy
And god I know I'm one
4

As you can see, there were 4 lines typed in.

The ls command
The ls command is used to list directory contents. With no arguments, it returns a list of files and directories in the current directory. It also offers many arguments, so you can list more information about the existing files. For example:

ls -l use a long listing format
ls -X sort alphabetically by extension
ls -h display sizes in human readable format (transforms in KB, MB or GB when necessary); use this with the -l switch to see the effect
ls -a list all files; this includes files starting with a dot character

You can nest arguments to obtain the desired result:

$ ls -lhX
total 96K
drwxr-xr-x 2 root root 4.0K 2009-03-12 16:58 bin
drwxr-xr-x 3 root root 4.0K 2009-03-20 23:08 boot
lrwxrwxrwx 1 root root 11 2009-02-18 18:22 cdrom -> media/cdrom
drwxr-xr-x 13 root root 4.2K 2009-04-07 06:25 dev
drwxr-xr-x 120 root root 12K 2009-04-07 02:18 etc
...

Notice that this way ls will return the current directory contents, however you can use it to list files located in some other directory:

$ ls -lh /bin
total 4.3M
-rwxr-xr-x 1 root root 685K 2008-05-12 22:02 bash
-rwxr-xr-x 1 root root 121K 2007-07-14 00:52 bsd-csh
-rwxr-xr-x 3 root root 26K 2008-08-05 14:19 bunzip2
-rwxr-xr-x 1 root root 368K 2008-09-08 02:29 busybox
-rwxr-xr-x 3 root root 26K 2008-08-05 14:19 bzcat
...

The . and .. files
When using ls -a, you will notice two special files, . (one dot) and .. (two dots). These are virtual files. The single dot stands for the current working directory, while the double dot represents the parent directory of the current directory. For example if your current directory is /home/chris/ and you want to change it to its parent /home, you can either use cd /home (which is the absolute path) or cd .. (which is the relative path). Or if you want to copy, say, the file /etc/motd in the current directory, you can use:

$ cp /etc/motd .

Instead of:

$ cp /etc/motd /home/chris

The rm, mkdir and touch commands
To remove a file, use the rm command:

$ rm file

To remove a directory, use:

$ rm -r directory

The mkdir command is used to create directories:

$ mkdir mydir

touch is a tool used to change file timestamps. That means it updates the date a file was last accessed and/or modified. Example:

$ ls -lh ubuntupocketguide-v1-1.pdf
-rw-r--r-- 1 embryo embryo 2.2M 2009-01-20 21:51 ubuntupocketguide-v1-1.pdf

The file was last modified on January 20, 2009, at 21:51. To update its modification time, type:

$ touch ubuntupocketguide-v1-1.pdf
$ ls -lh ubuntupocketguide-v1-1.pdf
-rw-r--r-- 1 embryo embryo 2.2M 2009-04-07 16:41 ubuntupocketguide-v1-1.pdf

Now, the date the file was last modified was changed to April 7, 2009, at 16:41. If the file specified as argument for touch does not exist, an empty file is created:

$ ls myfile.txt
ls: cannot access myfile.txt: No such file or directory
$ touch myfile.txt
$ file myfile.txt
myfile.txt: empty

We used the file command to see what what kind of file myfile.txt is.

Getting help
One of the most powerful ways to get help is to read the manual pages, usually available for any GNU utility and application via the man command. These tools also have two standard arguments, --version (or -v) and --help (or -h). They return the version of the program and a brief help on how to use the respective command. You can also use info command.

On a usual Linux system, the manual pages are located in the directory /usr/share/man, in directory names like man1, man2, man3 and so on.

$ bash --version
GNU bash, version 3.2.39(1)-release (i486-pc-linux-gnu)
Copyright (C) 2007 Free Software Foundation, Inc.

Shell built-ins
Shell built-ins are commands included in the Bash program. To see the available built-ins in your Bash application, you can type help in any shell. Some examples of built-ins include:

. (or source) - read and execute commands from a filename
alias - define aliases (read here how to create aliases)
bg - place each job specified as argument in the background
fg - place the job specified as argument in the foreground and make it the current task
read - read one line from the standard input
if - execute commands if the condition specified is true
while - execute commands as long as a condition is true
help - display helpful information about built-in commands

You can use the help command to see a complete list of shell built-ins, and help to see detailed help about each command. In order to see if a command is a standalone program or a shell built-in, you can use the type command:

$ type cd
cd is a shell builtin
$ type bash
bash is hashed (/bin/bash)

If the specified command is an alias, the output will be something like:

$ type ls
ls is aliased to `ls --color=auto'

Linux directory structure
Usually, the standard Linux directory structure is as follows:

/bin contains essential programs for booting and maintaining the system; examples: bash, chmod, chown, cp, grep, kill, ps, rm, tar
/boot contains the Linux kernel image used to boot the system, and also the /boot/grub/menu.lst file, for configuring the GRUB boot menu
/dev special files pointing to system devices
/etc configuration files for various applications
/home contains directories for all the users a system has; examplse: /home/embryo, /home/pink
/lib contains shared libraries available for all programs throughout the system
/media mount points for devices like hard disks or optical drives
/mnt used for mount points like external filesystems
/opt additional applications go here
/proc virtual files which provide information about the system; try cat /proc/cpuinfo for example
/root home for the super user (root)
/tmp temporary files are stored here
/sbin programs for system administration, generally used by root
/usr contains almost all the programs which are not located in /bin or /sbin, documentation, manual pages, icons etc
/var files like logs, emails

Using tar, gzip and bzip2
tar is a tool which creates archives known as tarfiles, while gzip is a compressing tool which uses an advanced compression algorithm. bzip2 is an advanced compression tool which takes more time to compress/uncompress files but offers a better compression rate, which results in smaller files. bzip2 is used especially to save disk space and bandwidth.

To uncompress a .tar.gz file you would issue a command like:

tar -xzf archive.tar.gz

x stands for extract
z specifies that the compressed file is a gzip file
f stands for the filename

To uncompress a .tar.bz2 file you can use:

tar -xjf archive.tar.bz2

To create a .tar.gz file from a directory, you can use:

tar -czf my_archive.tar.gz directory_name

This will create the compressed file my_archive.tar.gz from the contents of directory_name, which will be the root directory.

More Linux commands

who
The command who is used to show who is logged on, the number of the tty (teletype terminal) or display they are using, and the date and time they logged in:

$ who
embryo tty1 2009-04-07 17:08
embryo :0 2009-04-06 23:30

uptime
uptime will show how long the system has been running:

$ uptime
17:09:03 up 17:39, 2 users, load average: 0.14, 0.17, 0.11

uname
This command is used to print system information:

$ uname -a
Linux debian 2.6.26-1-686 #1 SMP Fri Mar 13 18:08:45 UTC 2009 i686 GNU/Linux

echo
This built-in will display a line of text to the screen:

$ echo 'Hello, world!'
Hello, world!

$ echo $PATH
/home/embryo/bin:/usr/local/bin:/usr/bin:/bin:/usr/games

Search for commands
In order to see the path in which a command is located, you can use the whereis command, which will search in all the standard locations:

$ whereis bash
bash: /bin/bash /etc/bash.bashrc /usr/share/man/man1/bash.1.gz

Notice that whereis will not look into non-standard paths (e.g. /home/user/bin), instead, you can use the which command for that:

$ which q3ut4.bash
/home/embryo/bin/q3ut4.bash

When you type in a command like uptime or uname, Bash searches for it in the directories in your $PATH variable (see echo $PATH), and it executes the first one it finds in that order. Aliases have priority.

Additional resources
There is a huge number of tutorials for Linux CLI, but I'd like to recommend two particular ones, specially created for beginners:

TuxFiles.org - Tutorials for Linux newbies
LinuxCommand.org - An introduction to Linux command-line and basic scripting for beginners

Do you have any suggestions or corrections for this tutorial? Please feel free to discuss in the comments section below.

This is a later, revised version of the tutorial which I initially published here.

Updated: April 9, 2009

5 comments:

Unknown said...

I think you have done a pretty good job of covering a very difficult and complex issue, basic linux commands. Congrats.

Craciun Dan said...

Thank you!

Tommy.S said...

"In fact, a more appropriate name for Linux is GNU/Linux, since any Linux-based distribution includes the Linux kernel (the core of the operating system) and the GNU tools and utilities. On a Debian system, the default shell used by the system is specified in the /etc/passwd file."

Heh, bretty bad propaganda.

The Linux kernel is the operating system. It is monolith kernel == operating system by old fashion way.

Please do not spread lies about monolith kernel ain't operating system and GNU is someway part of OS. GNU has not got their OWN operating system work yet (2009).

GNU/Linux is actually a _development platform_ You do not need any part of GNU to have an operating system.
And you dont even need an operating system to get the application work on the hardware, but it is just much harder to code because you need to know all the hardware calls how to control them. OS is bridge between hardware and all other software like C-library, commandline and system programs like cp, ls, free etc. And those build together a new layer what can be used to build other layer until you end up to situation that you have complete software system what is OS + System programs + Middleware + application programs and middleware and/or system programs offers user interface for the user.

Operating System just runs all those and that is in this case the Linux. There is two different kinds of kernels (monolith and microkernel and few variants of microkernel, but they are all microkernel-based like so called "hybrid" kernel) and that is the difference is the kernel alone an operating system or not. Monolith is, microkernel ain't.

Accurate technical fact what just explains OS functionality and the whole software system meaning.
http://tinyurl.com/mum9x
http://tinyurl.com/532kb8

Can you use your basic logic?
http://tinyurl.com/3uaq48

This is very simple..
http://www.osl.iu.edu/~pgottsch/swc2/lec/shell01.html

When you study operating systems on university, you only come to situation that Linux kernel is same as the Linux operating system and there ain't such OS like GNU/Linux (or Ubuntu, fedora etc).

Anonymous said...

in 2004 when I first began using
Freebsd, I was put off by the
irrevocability of the rm command
VS dos where "undelete" does not
exist. Thus approx 2004 to NOW
I've avoided possibly one hundred
erroneous deletions by the few
extra seconds to type
# /bin/rm
since rm is aliased to an echoed
something or other message. since
/bin/rm is learned, one can easily
to always learn -iv with it. That
gives one maybe 3 seconds to recognize
an error before on hits the ENTER
key.

lowell said...

the layout of this place sucks. it's confined to one side of the page and there's tons of whitespace all over the place. i suspect, thought, that the whitespace is due to a ridiculous number of adsense advertisements placed throughout this post. i can't confirm this of course, as i'm using adblock.