Ch 6: Redirection and Pipelines in Linux

Why Learn Linux Redirection and Pipelines ?

Linux redirection is a powerful shell feature, letting you save command output to files, use files as input, and chain commands via Linux pipelines. Essential for analyzing logs, processing text, or automating tasks, it builds on skills from Chapter 3 (exploration: $ grep, $ cat) and Chapter 5 ($ echo).

Standard Input, Output, and Error

Understanding Data Streams

Every command has three streams:

• Standard Input (stdin): Input source (usually keyboard).
• Standard Output (stdout): Output destination (usually terminal).
• Standard Error (stderr): Error message destination (usually terminal).

Linux redirection controls these streams for efficient data flow.

Output Redirection

Managing Command Output

Redirect output to files instead of the terminal.

Overwriting a File

Use > to overwrite a file:
$ echo "Hello, Linux!" > output.txt
Output: Creates/overwrites output.txt with Hello, Linux!.

Appending to a File

Use >> to append:
$ echo "This is a new line." >> output.txt
Output: Adds This is a new line. to output.txt.

Input Redirection

Using Files as Input

Feed file contents to commands with <:
$ wc -l < output.txt
Output: Counts lines in output.txt (e.g., 2).

Error Redirection

Handling Error Messages

Redirect errors to files instead of the terminal.

Redirecting Errors

Use 2> for errors:
$ ls /nonexistent_directory 2> error.log
Output: Saves error to error.log (e.g., ls: cannot access '/nonexistent_directory': No such file or directory).

Combining Output and Errors

Use &> for both:
$ ls /nonexistent_directory &> output_and_error.log
Output: Saves output and errors to output_and_error.log.

Linux Pipelines

Chaining Commands with Pipes

Use | to send one command’s output as another’s input.

Basic Pipeline

$ ls -l | grep ".txt"
Output: Lists files with .txt (e.g., rw-r--r-- 1 user user 14 output.txt).

Chaining Commands

$ cat /var/log/syslog | grep "error" | wc -l
Output: Counts lines with “error” (e.g., 42).

Advanced Redirection

Discarding Output

Use /dev/null to discard output:
$ ls /nonexistent_directory 2> /dev/null
Output: Suppresses error messages.

Redirecting to Multiple Files

Use tee to split output:
$ ls -l | tee files.txt
Output: Lists files and saves to files.txt.

Advanced Redirection Techniques

Streamlining Workflows

Advanced techniques enhance Linux pipelines.

Process Substitution:
$ diff <(ls dir1) <(ls dir2)
Output: Compares dir1 and dir2 contents (e.g., 1c1 < file1.txt --- > file2.txt).

tee:
$ ls | tee output.txt
Output: Lists files and saves to output.txt.

Note: Compared to cat (displays file contents, see Chapter 3), tee splits output to files and the terminal, enabling versatile data flow for beginners.

Glossary of Commands and Tools

Reference: For detailed command documentation, visit Linux Manpages. For package installation, search on Debian APT.

Practice Linux Redirection and Pipelines

Test your skills:

1. $ echo "Hello, World!" > hello.txt: Create hello.txt.
2. $ echo "This is a new line." >> hello.txt: Append text.
3. $ wc -l < hello.txt: Count lines.
4. $ ls ~/*.txt | tee text_files.txt: List .txt files and save output.

Conclusion

You’ve mastered Linux redirection and Linux pipelines, from echo to tee. Practice these to manage data flow efficiently.