W1002 The Client, the Server, and the Shell

From Coder Merlin

Prerequisites[edit]

The Client and the Server[edit]

Client-server-model

A client-server model is a means of distributing resources between the provider of a service and a consumer of that service, the client. You're likely very familiar with this concept though less familiar with the name. One common example of a service provided in this manner is email. You, a user, can open an email client (for example, an app on your iPhone or Outlook on a PC) which then connects to a remote server. The server is responsible for providing several services, including receiving mail on your behalf from other servers, transmitting email on your behalf to others, and storing your email for your later use.

ObserveIcon.png
Observe, Ponder, and Journal : Section 1
  1. Name at least two other services that you personally use which are implemented using a Client/Server Model

There are several advantages to this model:

  • Users don't need to personally possess the hardware required to fulfill a particular service, only a means of connecting and communicating with a server which provides the service
  • Users don't need to worry about installing and configuring the server hardware
  • Users are able to access their data from anywhere in the world (as long as they have internet access)
  • The state of a users data remains consistent, regardless of how and from where they access the server

Merlin is built upon this Client/Server Model, so you won't need any particular type of hardware or software, only a device capable of running a web-browser and a terminal emulator (available in Google Chrome).

First Connection[edit]

Coder Merlin is a cloud-based server where we’ll be doing most of our exercises. To access Merlin, you’ll need a secure shell client.

If you’re using a Mac or a Linux based device you’ll probably have a built-in client that you can access from the terminal.

If you’re using a Windows-based device, you can access a terminal by installing Google Chrome, and then within Google Chrome typing “Secure Shell Extension” and then installing the extension.

Either way, you’ll need a connection string to access the server. The connection string will begin with your first name and last name separated by a hyphen, for example: john-williams. The rest of the connection string will be @ssh.codermerlin.com

For John Williams, the entire connection string would be:

 john-williams@ssh.codermerlin.com

After you’re connected, you’ll be prompted to enter your password. After a successful logon, you'll land in the "shell".


HintIcon.png
Helpful Hint
You won't see your password being echoed to the terminal as you type it. This occurs to keep others from easily seeing your password. Just pretend that your keypresses are being received correctly.


WarnIcon.png
Caution
The correct connection string, like many identifiers that you'll encounter in computer science, must be entered exactly. There is no tolerance for any variance. As such, pay close attention to the requirements. If you make a mistake, you may be locked out of the server for a period of time. If this happens, verify the connection string and your password, wait a while, then try again.

The Shell[edit]

A shell is a user interface which provides access to an operating system’s services. There are two basic types of shells: a “command-line interface” (CLI) and a “graphical user interface” (GUI). You are likely familiar with common GUI’s from Microsoft Windows and perhaps Apple’s macOS. This document will focus on a CLI known as bash, a shell for the Unix operating system by Brian Fox. The shell is an acronym for Bourne-again shell, named for the Bourne shell that it replaces. Login to the Merlin Server as discussed previously. After some introductory text, you’ll see a prompt (letting you know that the server is ready and waiting for your input) which will look something like this:

john-williams@codermerlin:~$ 

You’ll see your login name, an at (@) sign, the name of the server (“codermerlin”), a colon (:), a tilde (~), and a $. This will be followed by a flashing block called a cursor. The cursor indicates where new characters will appear as you type. The cursor will continue to flash patiently as it awaits input. We’ll get back to the meaning of the tilde soon.

We'll be using a server running Linux, a Unix-like operating system. Almost everything in Unix is either a file or a process. A file is either a destination or source for a stream of data. A process is a program that is being executed by the operating system. While printers and screens are considered to be “files” in Unix, a more general way of thinking of files is a collection of data that can be referred to by name. Files can be created by users directly (for example, via a text editor) or indirectly, such as the result of running a program like a compiler.

Terminating a Partial Command[edit]

If you begin a command in error or change your mind about issuing a command, you can use the CONTROL-C key sequence to cancel what you've typed so far and continue with a new command prompt.

Try it now:

john-williams@codermerlin:~$  echo iThoughtIKnewWhatIWasDoingBut CONTROL-C


HintIcon.png
Helpful Hint

If you began issuing a command but need to do something else in the shell before actually issuing it, an alternative to the above is to temporarily store what you've typed so far, do something else, then return to the original command. You can do that with:

CONTROL-U Cut backwards
CONTROL-Y Yank the original text back

Directories[edit]

Directory Hierarchy[edit]

Linux File Hierarchy

All files are integrated into a single, hierarchical directory structure that appears as an inverted tree. A directory provides a logical grouping of related files (and potentially other directories). The top of this tree is called the root and is referred to by a slash (/). In the diagram, we see that the root contains five subdirectories (i.e., a directory below the parent directory): bin, etc, home, lib, and tmp. The home directory contains two subdirectories, jack-williams and john-williams. Note that this differs from a Windows file structure as Unix integrates all files into a single directory structure rather than listing different physical storage devices. Thus, the file structure is logical rather than physical. A path describes an ordered transition through one or more directories on the way to a destination (either a directory or a file). As such, note that while both jack-williams and john-williams each have a subdirectory named "projects" these are different directories. The path to the former is "/home/jack-williams/projects" while the path to the latter is "/home/john-williams/projects". Paths which begin at the root (symbolized by the initial slash) are called absolute paths. All other paths are relative paths.


ObserveIcon.png
Observe, Ponder, and Journal : Section 2
  1. It appears that both Jack Williams and John Williams can have a directory of the same name. Do you think the ability to have two different users create a directory of the same name is important? Why?

Display the Working Directory (Where am I?)[edit]

The shell uses a concept known as your working directory, which can be thought of as your current directory, i.e. where you are currently located in the file system. If you forget where you are, you can always use the command pwd to print the working directory. Try it now. Type “pwd” followed by <RETURN>. Note that you’ll need to press <RETURN> after any command before the shell will execute the command.

john-williams@codermerlin:~$  pwd
/home/john-williams 

The command will print something similar to “/home/john-williams” and then present you with a new prompt and cursor to indicate that it’s ready and awaiting your input. This directory is your home directory and it has a special symbol, the tilde (~).

ObserveIcon.png
Observe, Ponder, and Journal : Section 3
  1. Why do you think the home directory has a special symbol to represent it?

Create a New Directory[edit]

In order to keep our Merlin Missions organized, let’s place them all under a directory named “Merlin”. To make a new directory, we’ll use the command mkdir to which we’ll provide a single argument, the name of the directory that we want to create. Try it now. Type “mkdir Merlin” followed by <RETURN>.

john-williams@codermerlin:~$  mkdir Merlin

Display the Contents of a Directory[edit]

How do we find out what’s contained in our current directory? We can use the ls (that’s a lowercase ‘L’ followed by a lowercase ’S’) command to list the directory contents. Try it now. Type “ls” followed by return.

john-williams@codermerlin:~$  ls

The command will print the contents of the current directory (in this case our home directory) which will include the single directory, “Merlin”, that we created earlier.

Change to a New Directory[edit]

When we want to change our current directory we use the cd command followed by the name of the directory to which we want to change. Try it now. Type “cd Merlin”.

john-williams@codermerlin:~$  cd Merlin
john-williams@codermerlin:~/Merlin$  


ObserveIcon.png
Observe, Ponder, and Journal : Section 4
  1. Did the prompt change? If so, how? Why is this useful?

Note that our prompt has now changed. Before the command was executed, the prompt contained a single tilde indicating that our home directory was our current directory. After executing the command our prompt now includes a tilde, followed by a slash, followed by “Merlin”. This indicates that we are in the directory named “Merlin” below our home directory. The slash is a delimiter separating the levels of our hierarchy.

Let’s try the pwd command again.

john-williams@codermerlin:~/Merlin$  pwd

We can now see that from the root, we’re three levels down:
/ (root)
home
john-williams
Merlin

The cd command has a useful shortcut. If we type the command without an argument, it will take us back to our home directory. It’s the equivalent of "cd ~". Let’s try it now:

john-williams@codermerlin:~/Merlin$  cd
john-williams@codermerlin:~$  

Now, move back to the Merlin directory by issuing the appropriate command.

Change to a New Directory Temporarily[edit]

When we want to change our current directory temporarily (meaning we want to return later to where we are now), we use the pushd command followed by the name of the directory to which we want to change. Try it now. Type “pushd Merlin”.

john-williams@codermerlin:~$  pushd Merlin
john-williams@codermerlin:~/Merlin$  

We can now perform any actions we'd like in this directory and when we're done, we can easily return from whence we came without needing to type in the original directory name. The popd command will look up where we came (when we used pushd) and return us to there. Try it now. Type popd.

john-williams@codermerlin:~/Merlin$  popd
john-williams@codermerlin:~$  

Tab Completion[edit]

Very often, the shell can help us to complete file and directory names when we begin to type the name of a file (or directory) and then type TAB. Try it now. Type "cd M" and then press TAB. The shell will complete the rest of the name by appending "erlin" to what you’ve already typed. Then press ENTER. Because the completion is not ambiguous (i.e. there is only one solution) the shell will complete the name. If there were multiple possibilities, the shell would complete as much as possible and then beep. At this point, we could press TAB twice in succession to see the possible completions. Let’s try this. Within the Merlin directory, create three new directories: hello, hello2, and hello3:

john-williams@codermerlin:~/Merlin$  mkdir hello
john-williams@codermerlin:~/Merlin$  mkdir hello2
john-williams@codermerlin:~/Merlin$  mkdir hello3

Then, type “cd h”:

john-williams@codermerlin:~/Merlin$  cd h

Now, press TAB. The shell will complete as much of the name as possible and then beep. (Depending on your terminal you may not hear the beep or you may see a flash.)

john-williams@codermerlin:~/Merlin$  cd hello

Now press TAB twice in succession. The shell will display matching possibilities and then provide a new prompt followed by the previous entry:

hello/ hello2/ hello3/ 
john-williams@codermerlin:~/Merlin$  cd hello

You now have the option of pressing ENTER to accept "hello" or adding some more characters (in this case “2” to form “hello2” or “3” to form “hello3”) and then pressing ENTER to execute the command as-is or TAB to search for additional completions. In our case, let’s press ENTER to descend into the "hello" directory:

john-williams@codermerlin:~/Merlin$  cd hello
john-williams@codermerlin:~/Merlin/hello$  

Recall Previous Commands[edit]

Sometimes it's expedient to quickly recall a previous command without the need to re-type it. The most common means of accomplishing this is to use the up-arrow. Let's try it:

john-williams@codermerlin:~$  ls

Now, let's repeat the command, but rather than type ls again, try typing the up-arrow:

john-williams@codermerlin:~$  

Try pressing the and several times.

ObserveIcon.png
Observe, Ponder, and Journal : Section 5
  1. What happens each time you press the ?
  2. What happens each time you press the ?


Files[edit]

Echo to the Screen[edit]

The echo command echoes the argument to the output. Let’s try it. Type: echo "Hello, World!" The echo command dutifully echoes the argument to the screen. Try it now:

john-williams@codermerlin:~/Merlin/hello$  echo "Hello, World!"
Hello, World! 

Create an Empty File[edit]

The touch command creates a new, empty file, if such a file does not already exist. If such a file does already exist, the access and modification times will be set to the current time. Try it now:

john-williams@codermerlin:~/Merlin/hello$  touch newFile.txt
john-williams@codermerlin:~/Merlin/hello$  ls
newFile.txt 

We'll see the cat command more below, but for now it's enough to understand that it will display the contents of a file. Because this is a new, empty file, nothing will be displayed. Try it now:

john-williams@codermerlin:~/Merlin/hello$  cat newFile.txt
john-williams@codermerlin:~/Merlin/hello$  

Redirection[edit]

Remember that we said that almost everything in Unix is either a file or a process and that the screen could be considered as a file. We can redirect the output from echo to another file, one that we'll store in the file system. We do this with the > (greater-than) symbol. Normally, commands will send their output to a device called stdout. This is what the echo command did when we used it earlier. In our case, stdout is printed to our terminal. Let’s try to redirect our output to a file. Type: echo "Hello, World!" > out.txt:

john-williams@codermerlin:~/Merlin/hello$  echo "Hello, World!" > out.txt


ObserveIcon.png
Observe, Ponder, and Journal : Section 6
  1. Was "Hello, World!" printed to the screen as before? If not, why not? What do you think happened to the text?


No output was displayed; we just received a new prompt. Let’s use the ls command to take a look:

john-williams@codermerlin:~/Merlin/hello$  ls
newFile.txt out.txt 

We can see that the directory contains a single file, "out.txt" which was created by the echo command. How can we see what's contained within this file?

Quickly View File Contents[edit]

To quickly view the contents of a file we can use the cat command. The cat command concatenates files together and then sends the output to stdout. (Remember that in our case this is the screen.) Let’s try it. Type "cat out.txt".

john-williams@codermerlin:~/Merlin/hello$  cat out.txt
Hello, World! 

Append a Small Amount of Text to an Existing File[edit]

If we already have an existing file with some text we can append (add text to the end) of this file by using echo and redirection as before but with two "greater-than" symbols adjacent to one another. Try it now. Type: echo "Hello, again!" >> out.txt

john-williams@codermerlin:~/Merlin/hello$  echo "Hello, again!" >> out.txt
john-williams@codermerlin:~/Merlin/hello$  

Let’s move back up to our home directory. Remember that we can do this by typing "cd" without any arguments.

john-williams@codermerlin:~/Merlin/hello$  cd
john-williams@codermerlin:~$  


Deleting a File[edit]

We can delete a file with the rm (remove) command. Let's try it:

john-williams@codermerlin:~$  echo "Hello" > hello.txt
john-williams@codermerlin:~$  ls
john-williams@codermerlin:~$  rm hello.txt
john-williams@codermerlin:~$  ls

Renaming a File[edit]

We can rename a file with the mv (move) command. Let's try it:

john-williams@codermerlin:~$  echo "Hello" > hello.txt
john-williams@codermerlin:~$  ls
john-williams@codermerlin:~$  mv hello.txt goodbye.txt
john-williams@codermerlin:~$  ls

Directories (Revisited)[edit]

There are useful shortcuts that you can use when specifying a path.

  • "." (a single period or dot) refers to the current directory
  • ".." (a double period, or two adjacent dots) refers to the parent directory, relative to wherever you happen to be in the path up to this point

Thus:

  • To move to a parent directory, we can type cdSpace..
  • To move to a grandparent directory, we can type cdSpace../..

Assume we have a directory named 'parent' with two children, 'childA' and 'childB', and we are currently inside the 'childB' directory and want to enter the 'childA' directory.

  • To move to a sibling directory, we can type cdSpace../childA

View the Directory Hierarchy[edit]

Sometimes it’s helpful to obtain an overview of our directory hierarchy. We can do this with the tree command. Let's try:

john-williams@codermerlin:~$  tree

.
└─ Merlin
   ├─ hello
   │  ├─ newFile.txt
   │  └─ out.txt
   ├─ hello2
   └─ hello3

ObserveIcon.png
Observe, Ponder, and Journal : Section 7
  1. How does the ls command differ from the tree command?
  2. How do you choose to use one over the other?

Remove a Directory[edit]

Let's remove the two directories that we won't be using anymore. We can do this with the rmdir command. Move to the Merlin directory and then remove the hello2 and hello3 directories. Then do a directory listing to see what’s left.

john-williams@codermerlin:~$  cd Merlin
john-williams@codermerlin:~/Merlin$  rmdir hello2
john-williams@codermerlin:~/Merlin$  rmdir hello3
john-williams@codermerlin:~/Merlin$  ls
hello 

Clear the Screen[edit]

In some cases it’s useful to clear the screen. (For example, this might be useful if you sense your guide creeping up behind you when you were doing something that you weren’t supposed to be doing. 😁) We can do this with the clear command. Type clear and then press RETURN.

john-williams@codermerlin:~$  clear

Jobs[edit]

A program is an executable file which contains a series of instructions which can be executed by the computer. A process is a program which is being executed on the computer. The shell manages processes using the concept of a job. You can obtain a list of jobs and suspend and resume jobs. This functionality enables you to maximize your use of time rather than wait for the prompt to return as the shell waits for the execution of a job to complete.

Identifying Jobs[edit]

The shell provides several different ways to identify a job:

  • %n - The specific job number is preceded by a percentage sign
  • %+ - A shortcut to specify the current job
  • %- - A shortcut to specify the previous job

Listing Jobs[edit]

Jobs may be listed with the jobs command. Let's try an example by executing a few long-running jobs. The sleep command simply waits before returning. Let's try it:

john-williams@codermerlin:~$ sleep 3

Let's try it again:

john-williams@codermerlin:~$ sleep 5


ObserveIcon.png
Observe, Ponder, and Journal : Section 8
  1. What do you think the argument to sleep specifies?
  2. Are you able to prove your hypothesis? How?


These same jobs can be executed in the background (more on this below) simply by specifying an ampersand (&) after the command.


Let's try it:

john-williams@codermerlin:~$ sleep 60 &


ObserveIcon.png
Observe, Ponder, and Journal : Section 9
  1. How is the behavior different from what you expected?
  2. What do you think it means to run a job in the background?
  3. What do you think the number displayed in brackets specifies?


Let's run several jobs in the background and then view a list of our jobs:

john-williams@codermerlin:~$ sleep 40 &
john-williams@codermerlin:~$ sleep 30 &
john-williams@codermerlin:~$ sleep 20 &

Now, let's use the jobs command:

john-williams@codermerlin:~$ jobs

Repeatedly execute the jobs command.

ObserveIcon.png
Observe, Ponder, and Journal : Section 10
  1. What do you observe as you execute the jobs command?
  2. How do you explain your observations?

Terminating a Job[edit]

While a job is executing, we may decide that we want to terminate it. Perhaps it is taking too long or we began executing it with the wrong arguments. We can often terminate the job with a special key sequence, CONTROL-C.

Let's try an example. We'll execute the cat command without any arguments, which will cause it to wait for something to be entered on the console.

jane-williams@codermerlin:~$ cat

The process is now waiting. If we realize that we made a mistake (perhaps we wanted to print the contents of a file and forgot to specify the file's name) we can terminate the job. Try it now by pressing CONTROL-C.

Foreground Jobs[edit]

Generally, when we execute a program we run it in the foreground. This means that our shell will execute the process and we won't be able to execute the next command until the currently running foreground process exits. This is ideal for short, interactive programs. Whenever we execute a program, running that program in the foreground is the default. For example, the find command can search for all files and directories within our home directory. Let's try it:

jane-williams@codermerlin:~/Merlin$ cd
jane-williams@codermerlin:~/Merlin$ find

We'll likely see a long list of files and directories quickly scroll by. But what if we want to generate a list of all of the files and directories on the entire server? This can take a while. Let's try it, just for fun:

jane-williams@codermerlin:~/Merlin$ find /

Eventually we'll encounter a permissions error and the process will halt. But if you don't have the patience to wait, remember that you can use the key sequence CONTROL-C to terminate the process immediately. Alternatively, we can use the key sequence CONTROL-Z to suspend the process. When we suspend a process we can resume it in the foreground with fg followed by the job number.

Background Jobs[edit]

When we have a complex process that will take a long time to execute we don't want to be blocked from executing other commands while we wait for the long-running process to finish. In these cases, we can execute the long-running process in the background. Running the program in the background means that the process will continue to execute but we'll be able to enter new commands immediately. We saw an example of this above where we executed multiple sleep jobs simultaneously. We were able to do so because the shell was ready to accept new input before the previous job completed.

We can begin a job in the background by using an ampersand (&) after typing the command. When we suspend a process we can resume it in the background with bg followed by the job number.

Change Your Password[edit]

When you first log on to the server you'll have been assigned a password. You should change this password to something that is easy for you to remember but difficult for others to guess (or derive). (A good summary article of how to select strong passwords may be found here.)

The program that you'll use to change your password is passwd. It will prompt you three times.

  • You'll first enter your current password
  • then the new password that you want to use
  • and then finally, to ensure that you're entering the new password correctly, you'll be prompted a second time for the new password.
HintIcon.png
Helpful Hint

Remember that when you enter the password, what you type will not be echoed to the terminal. You'll just need to pretend that the your keypresses are being received.

Change your default password to a strong password:

john-williams@codermerlin:~$ passwd
Changing password for john-williams.
(current) UNIX password:
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully

Display a Manual[edit]

Finally, let's try out one more very helpful command. If we want to learn more about a command we can read its manual. We do this with the command man. As an example, look at the man page for the emacs command. Type man emacs.

john-williams@codermerlin:~/Merlin$  man emacs

The screen will be filled with information about the emacs command. While a man page is displayed we can use the following keys to navigate:

  • Typing f will move us forward one page in the manual
  • Typing b will move us backward one page in the manual
  • Typing q will quit (exit the manual)

Exit the manual now.

Set Location[edit]

Some commands (such as helpme, which you'll learn about soon) rely on knowing your location. Often this is just a room number in a building or simply 'home'. It's important to set your location. If you're unsure of the value to use, ask your invigilator. As an example, to set your location to "Home":

john-williams@codermerlin:~/Merlin$  chfn --room Home

Note the option "--room" followed by the argument "Home". This will change your location to "Home". You can learn more about chfn by reading the manual.

You'll be prompted for your logon password for confirmation.

Exiting the Shell[edit]

In general, there's no need to exit the shell. It's almost always sufficient to close your terminal emulator. An advantage to this approach is that the next time you connect to the server you'll be in exactly the same place where you left off. However, on rare occasions the terminal emulator can become "confused" and display characters in odd places. In such a case, there's still no need to exit; instead, use the reset command.

👀 See Also[edit]

📺 Videos[edit]

Bash Terminal Beginner's Guide
Linus Torvalds

📖 Texts[edit]

W1002 The Client, the Server, and the Shell

📚 References[edit]


Key Concepts[edit]

KeyConceptsIcon.png
Key Concepts
  • A client-server model is a means of distributing resources between the provider of a service and a consumer of that service, the client. The advantages of this model include:
    • Users don't need to personally possess the hardware and software required to implement the service
    • Users don't need to worry about installation and configuration
    • Users are able to access their data from anywhere in the world
    • The state of a users data remains consistent, regardless of how and from where they access the server
  • A shell is a user interface which provides access to an operating system’s services. There are two basic types of shells:
    • A command-line interface (CLI)
    • A graphical user interface (GUI)
  • A prompt lets you know that the server is ready and waiting for your input
  • A cursor is a flashing block which indicates where new characters will appear as you type
  • A file is either a destination or source for a stream of data
  • A process is a program that is being executed by the operating system
  • All files are integrated into a single, hierarchical structure called a directory
    • A directory provides a logical grouping of related files (and potentially other directories)
    • The top of this tree is called the root and is referred to by a slash (/)
    • A subdirectory is a directory contained within another directory, called the parent directory
    • On Unix systems, the file structure is logical rather than physical
    • A path describes an ordered transition through one or more directories on the way to a destination (either a directory or a file)
      • Paths which begin at the root (symbolized by the initial slash) are called absolute paths
      • All other paths are relative paths
    • Your home directory is indicated by a special symbol, the tilde (~)
    • Your working directory can be thought of as your current directory
    • The command pwd prints the working directory
    • The command mkdir creates a new directory
    • The command ls lists the contents of a directory
    • The command cd changes your working directory
    • The command pushd changes our working directory temporarily
    • The command popd returns us to our previous directory after executing pushd
    • The command tree displays a directory hierarchy
    • The command rmdir removes (deletes) a directory
  • The echo command echoes the argument to the output
  • The touch command creates a new, empty file, if such a file does not already exist. If such a file does already exist, the access and modification times will be set to the current time.
  • The greater-than symbol (>) can be used to redirect output to a file
  • Two greater-than symbols (>>) can be used to append output to a file
  • The cat command allows us to quickly view the contents of one or more files
  • The rm command is used to remove (delete) a file
  • The mv command is used to either move or rename a file
  • The clear command clears the screen
  • A program is an executable file which contains a series of instructions which can be executed by the computer
  • A process is a program which is being executed on the computer
  • The shell manages processes using jobs
  • The jobs command will list the current jobs associated with your shell
    • A job can be terminated by using the key sequence CONTROL-C
    • A job can be suspended by using the key sequence CONTROL-Z
    • A job can be resumed in the foreground with fg
    • A job can be resumed in the background with bg
  • The man command displays the contents of a manual
  • The reset command resets your terminal

Exercises[edit]

ExercisesIcon.png
Exercises

Using pencil and paper, answer all questions in this experience.


After completing W1005:

  •  J1002  Create a journal and answer all questions in this experience. Be sure to include all sections of the journal, properly formatted.

After completing W1008:

  •  M1002-10  Complete  Merlin Mission Manager  Mission M1002-10.
  •  M1002-31  Verify your journal's conformance with  Merlin Mission Manager  Mission M1002-31.