Bash Commands II #

This section delves deeper into bash commands, covering more advanced tools and techniques to help you search, filter, and manipulate text and files.

Advanced Text amd File Searching #

find #

Use find to search for files and directories from a starting point, based on various criteria. It is recursive, meaning it will search all subdirectories too.

find . -name "foo*"        # Search for files starting with "foo"
find . -iname "foo.txt"    # Search for files named "foo.txt" case-insensitive
find . -type d -name "foo" # Search for directories named "foo"
find . -mtime -7           # Search for files modified within the last 7 days
find . -size +1M           # Search for files larger than 1 MB

grep #

Use grep to search for text patterns within files.

grep "pattern" file.txt    # Search for "pattern" in file.txt
grep -i "pattern" file.txt # ... case-insensitive
grep -r "pattern" .        # ... recursively
grep -l "pattern" *.txt    # ... list only filenames
grep -n "pattern" file.txt # ... display matching lines numbers
grep -v "pattern" file.txt # Search for lines NOT containing "pattern"

sed #

sed (stream editor) is a powerful command-line tool used for text manipulation, including search and replace, insertion, deletion and more. It operates on a line-by-line basis and can be used with files, standard input, or as part of a pipeline.

Here are some examples of using sed:

sed 's/foo/bar/g' file.txt # Replace all occurrences of foo with bar
sed 's/foo/bar/' file.txt  # Replace first occurrence of foo with bar in each line
sed '3d' file.txt          # Delete the third line
sed '3,5d' file.txt        # Delete lines 3 to 5
sed 's/^/# /' file.txt     # Add a "#" symbol at the beginning of each line
sed 's/[[:space:]]*$//' file.txt # Remove trailing spaces from each line

Keep in mind that by default, sed outputs the modified text to the standard output. To save the changes to the original file, use the -i option followed by an optional file extension for a backup file:

# Replace all occurrences of "foo" with "bar" and save the changes to file.txt,
# creating a backup file called file.txt.bak
sed -i '.bak' 's/foo/bar/g' file.txt

Piping #

The pipe operator | is a powerful feature in Bash that allows you to chain multiple commands together. The output of one command becomes the input of the next command in the pipeline. This enables you to create complex operations by combining simple commands.

Here are some examples of using the pipe operator to combine commands:

# List all files in the current directory starting with a number
ls | grep "^[0-9]"

# List all processes containing the word "python"
ps aux | grep "python"

# Display the size of all files and directories in the current directory,
# sorted by size in descending order
du -sh * | sort -rh

# Count the number of files in the current directory
ls -1 | wc -l

# Search your command history for commands containing the word "ssh"
history | grep "ssh"

# Combine the contents of two files into one and remove duplicate lines
cat file1.txt file2.txt | uniq > merged.txt

Input/Output Redirection #

Bash provides features to redirect input and output to and from files, devices, and other processes. This enables you to read and write data from different sources and destinations using commands in the terminal.

< Input Redirection #

Input redirection is used to read data from a file instead of the keyboard (standard input). The < operator redirects the contents of a file to a command as input.

sort < unsorted.txt

> Output Redirection #

We’ve seen this already in the previous page. Output redirection is used to write the output of a command to a file instead of the screen (standard output). The > operator overwrites the contents of the file, while >> appends the output to the existing contents of the file.

find . -size +10M > large_files.txt
sed '/pattern/d' input.txt >> output_without_pattern.txt
sort < file.txt > sorted_file.txt

2> Standard Error Redirection #

By default, error messages are displayed on the screen (standard error). You can redirect error messages to a file using the 2> operator.

command 2> errors.txt      # Redirect error messages to errors.txt
command 2>> errors.txt     # Append error messages to errors.txt

&> Standard Output + Error Redirection #

To redirect both standard output and standard error to the same file, use the &> or &>> operators.

# Redirect both output and error messages to output_and_errors.txt
command &> output_and_errors.txt

# Append both output and error messages to output_and_errors.txt
command &>> output_and_errors.txt

2>&1 Standard Error to Standard Output Redirection #

Sometimes, you may want to redirect both the standard output and standard error of a command to the same file. To do this, you can use the 2>&1 operator.

Consider a situation where you want to run the grep command to search for a pattern in a file and store the results in results.txt. You also want to log any errors that occur during the search (such as the file not being found) in the same file.

Here’s how you would do that:

grep "pattern" input.txt > results.txt 2>&1

This command will redirect the standard output (the matching lines) to results.txt and then redirect the standard error (any errors that occur) to the same destination as the standard output (in this case, results.txt). Both the search results and any errors will be stored in results.txt.