How to Replace a String in a File Using Bash

Knowing how to replace a string in a file helps if you want to create Bash scripts to automate the configuration of systems or applications.

To replace a string in a file using a Bash script you can use the sed command. The sed command allows to perform text manipulation and it can be called as part of your scripts in the same way you can do with other Linux or Unix commands.

A practical example in which replacing strings in a file can be handy is when you want to generate files based on templates where certain strings are updated with user inputs.

First of all, we will look at a few examples on how to use sed in the terminal and then I will show you how to call sed in your Bash scripts to replace one or multiple strings in a file.

Time to start!

Using the Sed Command to Replace a String With a Given Value

The sed command stands for stream editor, it’s one of the most powerful Unix tools to filter and transform text.

I will start with a simple example that shows how to use the sed command directly in the Linux command line.

With the echo command I will print the message “Athens is the capital of Greece” in the terminal:

[myuser@host ~]$ echo "Athens is the capital of Greece"
Athens is the capital of Greece

Then using the pipe I will pass the output of the echo command to the sed command and replace the words Athens and Greece with Rome and Italy.

[myuser@host ~]$ echo "Athens is the capital of Greece" | sed 's/Athens/Rome/' | sed 's/Greece/Italy/'
Rome is the capital of Italy

I have used two pipes with two sed commands, one to replace the word Athens with Rome and the other one to replace Greece with Italy. They are both applied to the output of the echo command.

Let’s look at the syntax of the two sed commands:

sed 's/original_string/new_string/'

The letter s indicates the substitute command, it’s followed by three forward slashes.

Between the first and the second slash you can see original_string, that is the string we want to replace. Between the second and the third slash you specify new_string that is the string we want to replace the original_string with.

So, in our case when we write:

sed 's/Athens/Rome/'

We replace the string Athens with the string Rome in the message printed by the echo command. As we will see in the next section, we can also use a similar sed command to replace strings in a file.

Before moving to the next section I want to find out if it’s possible to use a single sed command to replace both strings. In this way we wouldn’t have to use two sed commands.

[myuser@host ~]$ echo "Athens is the capital of Greece" | sed 's/Athens/Rome/;s/Greece/Italy/'
Rome is the capital of Italy

As you can see we can use a single sed command and this time within the single quotes we specify two regular expressions separated by semicolon.

Using the Sed Command to Replace a String in a File

Now, instead of replacing strings in a message generated by the echo command, we will create a file that contains the same message.

To create the file we redirect the output of the echo command to the new file.

[myuser@host ~]$ echo "Athens is the capital of Greece" > message.txt

We can then use the same sed syntax we have seen in the previous section, this time we specify the sed command followed by the name of the file.

[myuser@host ~]$ sed 's/Athens/Rome/;s/Greece/Italy/' message.txt 
Rome is the capital of Italy

The output is correct.

Let’s check the content of the message.txt file using the cat command:

[myuser@host ~]$ cat message.txt 
Athens is the capital of Greece

Interesting…

…the file still contains the original message.

To update the content of the file with the sed command we need to pass an additional flag, the -i flag that edits files in place.

[myuser@host ~]$ sed -i 's/Athens/Rome/;s/Greece/Italy/' message.txt 
[myuser@host ~]$ cat message.txt 
Rome is the capital of Italy

This time the sed command has updated the content of our text file.

Replace All The Occurrences of a String in a File Using Bash

Usually we want to make sure that all the occurrences of a specific string are replaced by sed.

I will show you why this is not the case for the example we have seen so far.

I have updated the content of the message.txt file, here is the new message that contains the words Athens and Greece twice:

Athens is the capital of Greece. Athens is the largest city in Greece.

Here is the output of the sed command we have used so far:

[myuser@host ~]$ sed 's/Athens/Rome/;s/Greece/Italy/' message.txt 
Rome is the capital of Italy. Athens is the largest city in Greece.

Somehow only the first occurrence of the words Athens and Italy is replaced.

That’s because we are missing something at the end of each expression passed to the sed command. The letter g, that stands for global and replaces all the occurrences of a string in the file.

This is the command to replace all the occurrences of the string in our file:

[myuser@host ~]$ sed 's/Athens/Rome/g;s/Greece/Italy/g' message.txt 
Rome is the capital of Italy. Rome is the largest city in Italy.

Notice the letter g after the third forward slash of each regular expression.

A Simple Bash Script to Replace Strings with Sed

Now that we know how sed works, we can create a simple Bash script to modify our test message.

But, what would be the difference compared to what we have seen so far?

A Bash script allows you to customise the behaviour of sed using variables.

Let’s see how it works…

This time the message.txt file contains the following message:

Greece is a peninsular country.

Here is the first version of the script:

#!/bin/bash
  
FILENAME=$1
ORIGINAL_STRING=$2
NEW_STRING=$3

if [ $# -ne 3 ]; then
    echo "Please verify the number of arguments passed. Three arguments are required."
    exit 1
fi

sed -i 's/${ORIGINAL_STRING}/${NEW_STRING}/g' $FILENAME

The script requires three arguments:

• FILENAME: the path of the file in which we want to replace the string.
• ORIGINAL_STRING: the string we want to replace.
• NEW_STRING: the string we want to replace ORIGINAL_STRING with.

Before executing the sed command we verify that the number of arguments passed to the Bash script is correct.

We stop the execution of the script with the exit command and exit code 1 if the number of arguments is incorrect.

Finally, we run the sed command using the syntax explained in the previous section, the only difference is that we specify variable names in the sed command.

Let’s run the script…

[myuser@host ~]$ ./replace_string.sh message.txt Greece Italy
[myuser@host ~]$ cat message.txt 
Greece is a peninsular country.

For some reason, after running the script the content of the text file didn’t change.

Let’s troubleshoot this…

Troubleshoot Why the Bash String Replacement Doesn’t Work

How can you troubleshoot the execution of a Bash script?

To troubleshoot a Bash script you can add bash -x before the script name in the terminal.

[myuser@host ~]$ bash -x ./replace_string.sh message.txt Greece Italy
+ FILENAME=message.txt
+ ORIGINAL_STRING=Greece
+ NEW_STRING=Italy
+ '[' 3 -ne 3 ']'
+ sed -i 's/${ORIGINAL_STRING}/${NEW_STRING}/g' message.txt

The problem is that the variables in the sed command are not being replaced with the values we have passed to the script.

What’s missing?

The cause of the error is that we are using single quotes in the sed regular expression.

Single quotes in Bash don’t interpolate variables within them. To interpolate variables use double quotes instead.

Therefore, to interpolate the variables inside the sed expression we need to use double quotes. The last line of our script becomes:

sed -i "s/${ORIGINAL_STRING}/${NEW_STRING}/g" $FILENAME

And if I run the script with bash -x:

[myuser@host ~]$ bash -x ./replace_string.sh message.txt Greece Italy
+ FILENAME=message.txt
+ ORIGINAL_STRING=Greece
+ NEW_STRING=Italy
+ '[' 3 -ne 3 ']'
+ sed -i s/Greece/Italy/g message.txt

The sed command executed by the script is correct. And the content of the text file has become:

[myuser@host ~]$ cat message.txt 
Italy is a peninsular country.

All good!

This script is very simple on purpose, to give you an idea on how to use Bash scripting to call Linux commands or tools like sed.

And now, let’s deepen your sed knowledge.

Replace a String in All Files in a Directory Using Bash

In this paragraph, we will see how to replace a string in multiple files, in this example all the files are located in the current directory.

I have created a copy of our original message.txt file using the Linux cp command:

[myuser@host ~]$ cp message.txt message2.txt 
[myuser@host ~]$ cat message*
Greece is a peninsular country.
Greece is a peninsular country.

Now we have two text files with the same content.

How can we replace the string Greece in both of them with a single command?

The first thing I want to try is to apply a wildcard to the sed command and see what happens:

[myuser@host ~]$ sed -i 's/Greece/Italy/g' message*.txt 
[myuser@host ~]$ cat message.txt 
Italy is a peninsular country.
[myuser@host ~]$ cat message2.txt 
Italy is a peninsular country.

It worked!

As a result, both text files contain the string Italy.

The sed command accepts wildcards that we can use to replace strings in multiple files.

Now, let’s see an example where the files in which we want to replace a string are located in different directories.

Replace a String in All Files Recursively using Bash

Now, in our current directory let’s create a subdirectory called files. And inside this directory create two directories called dir1 and dir2.

The directory dir1 contains the file message1.txt and the dir2 contains message2.txt. Both text files contain the same message.

[myuser@host ~]$ mkdir -p files/dir1
[myuser@host ~]$ mkdir files/dir2
[myuser@host ~]$ echo "Greece is a peninsular country." > files/dir1/message1.txt
[myuser@host ~]$ echo "Greece is a peninsular country." > files/dir2/message2.txt

To apply the sed command to the files in each subdirectory we will have to use the find command together with the sed command.

The find command provides the -exec flag that allows to apply any commands to every file identified by find.

So, let’s:

• Use the find command to search all the files with .txt extension under the directory files.
• Apply the sed command to each file found.

Here is the full command:

[myuser@host ~]$ find ./files/ -name '*.txt' -exec sed -i 's/Greece/Italy/g' {} \;
[myuser@host ~]$ cat files/dir1/message1.txt 
Italy is a peninsular country.
[myuser@host ~]$ cat files/dir2/message2.txt 
Italy is a peninsular country.

It’s a long command! And it worked! 😀

I have also written an article about the find command if you want to get more familiar with it.

Case Insensitive Match to Replace a String in a File

All the examples we have seen so far to replace strings using sed are based on a case sensitive logic.

Here’s what I mean:

[myuser@host ~]$ echo "Greece is a peninsular country and several islands belong to greece." | sed 's/Greece/Italy/g'
Italy is a peninsular country and several islands belong to greece.

Can you see that the word “Greece” has been replaced by the sed command but the same hasn’t happened with the word “greece” (starting with lower case g).

That’s because by default the sed command uses a case sensitive match.

To apply a case insensitive match we need to add the the letter i to the last part of the regular expression we pass to the sed command:

[myuser@host ~]$ echo "Greece is a peninsular country and several islands belong to greece." | sed 's/Greece/Italy/gi'
Italy is a peninsular country and several islands belong to Italy.

Can you see ‘gi’ at the end of the regular expression?

So, the output this time is correct. Both instances of the word Greece (starting with lower and upper case g) have been replaced.

And this is it for this tutorial!

Conclusion

In conclusion, here is what we have learned in this tutorial:

• Using the sed command to replace a string in the output of the echo command.
• Replacing one string or multiple strings in a file with a single sed expression.
• Replacing all the occurrences of a string in a file.
• Applying sed to all the files in a directory and to all the files in a directory structure recursively.
• Doing a case insensitive match with sed.

In addition, we have also seen how to call the sed command within a Bash script. The script we have created is the starting point to create more complex scripts.

And you? How will you be using sed?