Top Text Processing Commands in Linux Explained with Examples

If you work on a Linux system—whether as a developer, system administrator, or data analyst—you quickly learn that text processing is at the heart of almost everything you do. Linux is famous for its philosophy of “everything is a file,” which means configuration, logs, code, and even device output are all treated as text streams. Being able to manipulate, analyze, and transform text efficiently is an essential skill for getting the most out of your Linux experience.

You don’t need to be a command-line wizard to start using these tools. With a handful of powerful text processing commands, you can search, filter, edit, and extract data from any file or command output, no matter its size. This guide introduces you to the most essential text processing commands in Linux, explains how they work, and shows you practical examples for each—so you can apply them immediately in your own workflow.

Prerequisites

Before you start, make sure you have:

• Access to a Linux system or terminal emulator (local, remote, or in a virtual machine).
• Basic familiarity with the command line: navigating directories, running simple commands, using pipes (|), and redirecting output.
• Some sample text files to experiment with (or you can create them on the fly as you learn).

Why Text Processing Matters in Linux

Linux systems are text-centric. Log files, configuration files, and code are all plain text. The ability to quickly view, search, sort, and modify text lets you:

• Find errors in logs.
• Extract useful data from reports.
• Clean and reformat files for analysis.
• Write scripts that automate daily tasks.

Whether you’re analyzing logs, cleaning up data, or mass-editing configuration files, text processing commands are your Swiss Army knife.

Essential Text Processing Commands

1. cat: Concatenate and Display Files

The cat command is your entry point for reading files, joining them together, or sending text to another command.

Basic usage:

cat myfile.txt

Shows the entire content of myfile.txt.

Join two files:

cat file1.txt file2.txt > combined.txt

Creates a new file that combines the contents of both.

Number the lines:

cat -n myfile.txt

Each line is prepended with its line number.

Real-world tip: cat is often used in conjunction with other commands. For example, you can pass its output into a search or transformation command using pipes.

2. less and more: Paginate Large Files

When you’re dealing with big files, less and more let you scroll through content page by page.

View a file:

less /var/log/syslog

Press the spacebar to move forward.
Use b to go back a page.
Type /error to search for the word “error.”
Press q to quit.

more is similar but with fewer features. less is preferred for most tasks.

3. head and tail: View the Start or End of Files

Sometimes you only want to see the beginning or end of a file.

First 10 lines:

head myfile.txt

First 20 lines:

head -20 myfile.txt

Last 10 lines:

tail myfile.txt

Last 50 lines:

tail -50 myfile.txt

Follow live updates (great for logs):

tail -f /var/log/syslog

This command shows new lines as they’re added, which is invaluable for real-time monitoring.

4. grep: Search for Patterns in Text

grep is the go-to tool for searching within files. It looks for lines that match a specified pattern.

Find lines containing “error”:

grep "error" /var/log/syslog

Case-insensitive search:

grep -i "error" server.log

Search recursively in all .log files:

grep -r "timeout" /var/log/

Show line numbers:

grep -n "failed" auth.log

Use regular expressions for advanced searches:

grep "fail(ed|ure)" logs.txt

Real-world tip: Combine grep with tail -f to watch for events as they happen:

tail -f /var/log/syslog | grep "disk"

5. awk: Pattern Scanning and Processing

awk is a versatile language for scanning and processing text files. It can filter columns, do calculations, and format output.

Print the second column: Suppose data.txt contains:

Alice 10
Bob 15
Carol 20

Extract names:

awk '{print $1}' data.txt

Extract numbers:

awk '{print $2}' data.txt

Filter rows: Show only entries where the second column is greater than 12:

awk '$2 > 12' data.txt

Sum up a column:

awk '{sum += $2} END {print sum}' data.txt

Format output:

awk '{printf "Name: %s, Value: %d\n", $1, $2}' data.txt

Real-world tip: awk shines when dealing with structured data (like CSV, TSV, or whitespace-delimited files).

6. sed: Stream Editor for Transforming Text

sed is perfect for quick, powerful text substitutions, deletions, or insertions.

Replace “apple” with “orange” in a file:

sed 's/apple/orange/' fruits.txt

By default, this only affects the first match per line. To replace all matches:

sed 's/apple/orange/g' fruits.txt

Edit files in-place:

sed -i 's/2022/2023/g' report.txt

Delete blank lines:

sed '/^$/d' notes.txt

Real-world tip: Test your sed command without the -i flag first, to avoid accidental overwrites.

7. sort and uniq: Sorting and Removing Duplicates

Sort a file alphabetically:

sort names.txt

Numerical sort:

sort -n numbers.txt

Reverse order:

sort -r names.txt

Remove duplicate lines:

uniq names.txt

Sort and then remove duplicates (most reliable):

sort names.txt | uniq

Count occurrences:

sort items.txt | uniq -c

Real-world tip: uniq only removes adjacent duplicate lines, so always sort before using uniq for best results.

8. cut: Extract Columns from Text

cut is ideal for splitting out specific columns from data.

Extract usernames from /etc/passwd:

cut -d: -f1 /etc/passwd

Extract the first and third columns (tab-delimited):

cut -f1,3 data.tsv

Extract first five characters of each line:

cut -c1-5 codes.txt

9. tr: Translate or Delete Characters

Use tr to replace or remove characters.

Convert lowercase to uppercase:

cat names.txt | tr 'a-z' 'A-Z'

Remove all digits:

cat data.txt | tr -d '0-9'

Replace spaces with underscores:

cat phrases.txt | tr ' ' '_'

10. wc: Word, Line, and Character Count

Quickly count lines, words, and characters in a file.

Count lines, words, and characters:

wc myfile.txt

Count lines only:

wc -l myfile.txt

Count words across multiple files:

wc -w *.txt

11. diff and comm: Compare Files

Show the differences between two files:

diff file1.txt file2.txt

This outputs the lines that differ.

Side-by-side comparison:

diff -y file1.txt file2.txt

Compare sorted files and show common lines:

comm file1.txt file2.txt

12. paste: Merge Lines Side by Side

Combine lines from two files into columns.

paste names.txt scores.txt

If names.txt contains:

Alice
Bob
Carol

And scores.txt contains:

88
92
79

The output will be:

Alice   88
Bob     92
Carol   79

13. nl: Number Lines in a File

Add line numbers to each line of a file:

nl myfile.txt

14. rev: Reverse Lines Character by Character

Reverse every line in a file.

rev words.txt

If a line reads hello, the output will be olleh.

Combining Commands: The Power of Pipes

One of Linux’s superpowers is the ability to chain commands together using pipes (|). This lets you filter and process data in multiple steps without creating intermediate files.

Example: Find all unique error messages in a log:

grep "error" server.log | sort | uniq

Example: Count the number of failed login attempts:

grep "Failed password" /var/log/auth.log | wc -l

Example: Extract email addresses and save to a file:

grep -E -o "\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,6}\b" users.txt | sort | uniq > emails.txt

Practical Scenarios

Scenario 1: Cleaning Up a Large CSV File

Suppose you have a huge CSV with duplicate rows and messy whitespace.

cat large.csv | sort | uniq > cleaned.csv

This sorts the file and removes duplicate lines.

To trim extra spaces:

cat cleaned.csv | sed 's/^ *//;s/ *$//' > trimmed.csv

Scenario 2: Summarizing Web Server Logs

You want to find out which IP addresses visited your server most often.

awk '{print $1}' access.log | sort | uniq -c | sort -nr | head

This pipeline extracts the first column (IP addresses), counts occurrences, sorts by count, and shows the top visitors.

Scenario 3: Extracting and Formatting Data

Imagine you have a text file with tab-separated data and want only the second and fourth columns:

cut -f2,4 data.tsv > selected_columns.txt

Scenario 4: Monitoring Log Changes in Real Time

You’re troubleshooting an application and want to see new error messages as they appear.

tail -f /var/log/app.log | grep "ERROR"

Avoiding Common Pitfalls

• Remember that most text processing commands are case-sensitive by default.
• When using sed -i, always keep a backup of important files (cp file.txt file.txt.bak).
• Test your regular expressions on a small sample before running them on crucial data.
• Wildcards (*, ?) in shell commands can expand to unexpected files—use with care.
• When chaining commands, read each stage’s output to verify that it does what you expect.

Scripting for Automation

Once you’re familiar with these commands, you can write scripts to automate everyday tasks. For example, a script to extract and count all unique words in a text:

#!/bin/bash
tr -cs "[:alpha:]" "[\n*]" < "$1" | sort | uniq -c | sort -nr

Save as wordcount.sh, make it executable (chmod +x wordcount.sh), and run it on any file.

Conclusion

Text processing is a foundational skill in Linux. The commands discussed here—cat, grep, awk, sed, sort, uniq, cut, and others—form a powerful set of tools you can use for almost any task involving text. With these at your fingertips, you can search through logs, filter and reformat data, automate boring chores, and troubleshoot issues—all without leaving the command line.

The best way to master these commands is to practice. Try them on your own files, experiment with combining them, and write small scripts to make your daily work easier. Over time, you’ll find that text processing becomes second nature, and you’ll wonder how you ever managed without it.

Keep exploring, keep experimenting, and you’ll continue to unlock the immense power that Linux puts at your fingertips.