Text Transformation with sed

1. The Syntax of the sed Command

What is sed?

sed (short for Stream Editor) is a powerful text processing utility that lets you modify, add, delete, or transform lines in text files and data streams.

The main magic of sed lies in the fact that it edits text "on the fly" without altering the original file (unless you explicitly ask it to). This makes it a perfect tool for quick and automated data transformations.

Basic Command Syntax

Let's take a look at the main formula for working with sed:

sed [options] 'pattern/action' file

Where:

• pattern — the text or regular expression you're searching for.
• action — the operation you want to perform (e.g., replacing text).
• file — the text file containing the lines you want to modify.

To keep things simple at first, we'll focus on the most popular actions: text substitution (s — short for "substitute") and line deletion (d — short for "delete").

---

2. Text replacement s/old/new/

Example 1: Simple text replacement

Let's say we have a file example.txt with the following content:

Hello world!
Welcome to Linux.
Linux is awesome.

Let's try replacing the word "Linux" with sed:

sed 's/Linux/sed/' example.txt

The output will be as follows:

Hello world!
Welcome to sed.
sed is awesome.

That's it. The word "Linux" was replaced with sed only once per line. This is the default behavior.

---

Example 2: Global replacement

If you need to replace all occurrences of a word instead of just the first one on the line, add the g flag (globally):

sed 's/Linux/sed/g' example.txt

If there are multiple mentions of "Linux" on a line, they will all be replaced with sed.

---

Example 3: Case-insensitive replacement

To replace text without considering case, add the I flag (or i depending on your version):

sed 's/linux/sed/gi' example.txt

This will replace words like "Linux", "LINUX", "LiNuX", and so on.

---

Example 4: Replacement with result saved

By default, sed just outputs the result in the console without modifying the file. If you want to save the changes back into the file, add the -i (in-place) option:

sed -i 's/Linux/sed/g' example.txt

Now the file example.txt will be updated with the replacements.

---

3. Deleting Lines /pattern/d

Example 1: Deleting Lines by Pattern

If you wanna delete all lines containing the word "Linux":

sed '/Linux/d' example.txt

The output will look like this:

Hello world!

Here, the lines with "Linux" in them are removed.

---

Deleting Lines by Number

You can delete a line by its number. For example, to delete the second line, use:

sed '2d' example.txt

Output:

Hello world!
Linux is awesome.

To delete multiple lines: specify a range, for example, delete lines from the 2nd to the 3rd:

sed '2,3d' example.txt

---

4. Working with Regular Expressions

This is where sed shows its real power! In patterns, you can use regular expressions to find more complex matches.

Example 1: Replacing words with a specific pattern

Let’s say you have the following lines in your text:

error: something went wrong
warning: check your system
error: unable to connect

We want to replace all lines starting with "error" with "Issue Detected". Regular expressions to the rescue:

sed 's/^error:.*/Issue Detected/' logs.txt

Where:

• ^ — points to the beginning of a line.
• .* — means "anything after".

Result:

Issue Detected
warning: check your system
Issue Detected

---

5. Inserting and Adding Lines

Inserting a Line Before a Match

To insert a line before every line containing "Linux":

sed '/Linux/i\# Learning is fun' example.txt

Result:

Hello world!
Welcome to Linux.
# Learning is fun
Linux is awesome.

Example 2: Adding a Line After a Match

To add text after a line with a pattern:

sed '/Linux/a\# sed makes it easier' example.txt

Result:

Hello world!
Welcome to Linux.
# sed makes it easier
Linux is awesome.
# sed makes it easier

---

6. Practice: Processing Configuration Files

Let’s try solving a practical task. Imagine you have a configuration file config.txt:

host=localhost
port=8080
mode=production
logfile=/var/log/app.log

Task 1: Replace "localhost" with "127.0.0.1"

sed -i 's/localhost/127.0.0.1/' config.txt

Result:

host=127.0.0.1
port=8080
mode=production
logfile=/var/log/app.log

---

Task 2: Increment the port by 1

To do this, we use sed with a small trick:

sed -i 's/port=8080/port=8081/' config.txt

Result:

host=127.0.0.1
port=8081
mode=production
logfile=/var/log/app.log

---

Task 3: Remove lines with logfile

If the configuration no longer requires logging, let’s delete the line:

sed -i '/logfile/d' config.txt

Result:

host=127.0.0.1
port=8081
mode=production

---

7. Combining with Other Commands

sed works great when paired with other Linux tools. For instance:

grep "error" logs.txt | sed 's/error/ERROR/'

Here, we first search for lines with errors and then format them.

Common Errors and How to Avoid Them

• Error: forgot the -i option. Beginners often expect sed to modify the file but forget to add -i. Don't forget to explicitly specify this if you want to make changes to the file.
• Error: quotes. On some systems, especially when using single quotes, \ before special characters may not work correctly. Always escape characters properly.
• Error with line ranges. If you're using ranges (e.g., 2,3d), make sure they're specified correctly—the file shouldn't be empty.