Anyone familiar with Bash scripting has probably needed to replace a configuration file value using sed as part of a pre-deployment process. If your variable (for example, one defined in your script) is not being replaced by sed, you are likely experiencing what is known as the "quoting trap", a problem that nearly every sed user encounters when first attempting to pass Bash variables into a sed expression.
I wasted a considerable amount of time trying to figure out why my sed command worked perfectly at the shell prompt but failed silently inside a Bash script. In the end, the fix came down to a single quote. That one character is critical because it determines how Bash passes arguments to sed, and understanding that relationship is the whole purpose of this article.
This article is intended for people who have some experience with Bash and want to use sed correctly within their scripts rather than only at the command line. By the end, you will understand how Bash interacts with sed, where things can go wrong, and how to build reliable automation scripts that use sed effectively.
Where This Gets Useful Fast:
The bash and sed combination shows up constantly in real sysadmin and DevOps work. Here is where you will use it:
- Updating config values using bash variables at deploy time
- Looping through a list of files and applying the same text change to each one
- Capturing sed output into a variable and using it in the next step of a script
- Wrapping sed in a reusable bash function that other scripts can call
- Conditionally applying text changes based on environment or flag values
None of these work the way you expect until you understand how bash handles quoting before sed ever sees your command.
If you want a broader look at how text processing tools fit together in bash, the text processing commands guide on LinuxTeck covers how sed, awk, grep, and cut each fill a different role.
How Bash Uses sed for Text Editing: Why Quoting Comes First
Before Bash can pass information to sed, it must first process the command itself. This means Bash handles any quotes, expands variables, and interprets special characters in your command line input before passing the final result to sed. The type of quotes you use determines how much of this processing takes place.
If you place double quotes around part of your command, Bash interprets the contents of those quotes, including variable expansion and escape sequences. Therefore, if the string contains a variable such as $MYVAR, Bash replaces it with its value before sed ever receives it.
If you place single quotes around the input, Bash leaves everything exactly as written—no variable expansion and no special character interpretation. What you type is exactly what sed receives.
LinuxTeck.com
NEW="bar"
# WRONG: single quotes — bash passes literal $OLD and $NEW to sed
# sed sees: s/$OLD/$NEW/g — matches nothing, no output change
echo "hello foo" | sed 's/$OLD/$NEW/g'
# CORRECT: double quotes — bash expands variables first
# sed sees: s/foo/bar/g — works as expected
echo "hello foo" | sed "s/$OLD/$NEW/g"
hello bar
The first line does nothing. The second line works. Same sed logic, different quotes. That is the entire issue that trips people up for weeks before they understand why.
Common Mistake:
Using single quotes in a bash script when the sed pattern contains a variable. The command runs without error, the file appears unchanged, and there is no indication of what went wrong. This is the most common reason people think "sed is not working in my script" when the real problem is bash quoting, not sed itself.
Fix: Switch single quotes to double quotes when your sed expression contains a $VARIABLE. Use sed "s/${OLD}/${NEW}/g" instead of sed 's/$OLD/$NEW/g'. The curly braces around the variable name are optional but recommended to prevent bash from misreading where the variable name ends.
Using Bash Variables Inside sed Expressions
Once you have the quoting right, using variables in sed opens up a lot of scripting possibilities. The pattern is always the same: declare your variable, then reference it inside double-quoted sed expressions using ${VARNAME}.
LinuxTeck.com
# Using variables in sed expressions
OLD_HOST="dev.server.local"
NEW_HOST="prod.server.internal"
# sed receives: s/dev.server.local/prod.server.internal/g
# Note: dots here are unescaped — they act as regex wildcards
# Use [.] or \. in the pattern if you need literal dot matching
sed -i.bak "s/${OLD_HOST}/${NEW_HOST}/g" app.conf
# Capture sed output into a bash variable
ORIGINAL="server_name=old.host.com"
UPDATED=$(echo "$ORIGINAL" | sed 's/old\.host\.com/new\.host\.com/')
echo "$UPDATED"
Notice the last sed uses single quotes because the literal strings are hardcoded in the expression, not bash variables. When the pattern has no variables, single quotes are safer because they prevent bash from misinterpreting any special characters in the sed expression itself.
Note:
If your bash variable contains characters that are special in sed, like /, &, or \, double-quoting alone will not protect you. A / in a variable breaks the delimiter. An unescaped & in the replacement means "insert the matched text here." If your variables may contain these, either sanitise them first or switch to a pipe delimiter: sed "s|${OLD}|${NEW}|g". The pipe delimiter eliminates the slash conflict entirely.
sed Inside Loops: Processing Multiple Files and Lines
One of the most common patterns in bash scripting is applying a sed transformation across multiple files or through a list of values. A for loop feeding into sed is something you will write regularly once you start doing any volume of file management.
LinuxTeck.com
# Apply the same sed change to multiple config files
OLD_ENV="staging"
NEW_ENV="production"
for config_file in /etc/myapp/*.conf; do
echo "Updating: $config_file"
sed -i.bak "s/${OLD_ENV}/${NEW_ENV}/g" "$config_file"
done
echo "All config files updated."
Updating: /etc/myapp/db.conf
Updating: /etc/myapp/cache.conf
All config files updated.
Always quote "$config_file" inside the loop. If a filename contains a space, an unquoted variable will split the path and sed will fail with a confusing "no such file" error.
The while read loop is better when you are processing lines from a file or command output rather than a glob of files:
LinuxTeck.com
# Read a list of hostnames from a file, update each one in inventory.conf
while IFS= read -r OLD_HOST; do
NEW_HOST="${OLD_HOST}.internal"
echo "Replacing $OLD_HOST with $NEW_HOST"
sed -i "s|${OLD_HOST}|${NEW_HOST}|g" inventory.conf
done < hosts_to_migrate.txt
Replacing web02 with web02.internal
Replacing db01 with db01.internal
The pipe delimiter | is used here to prevent syntax errors in case your bash variables contain forward slashes, like directory paths or URLs. Keep in mind that a dot . is a regex wildcard in sed regardless of the delimiter you choose; if you need to match a literal dot in the pattern, escape it as \. inside the source pattern. Using bash for loops with sed is one of the most productive combinations in automation scripting.
Common Mistake:
Running sed -i inside a loop against the same target file causes heavy disk I/O and a real risk of data corruption. If hosts_to_migrate.txt contains 50 entries, sed opens, modifies, writes, and closes inventory.conf 50 separate times. Worse, if an earlier iteration transforms web01 to web01.internal and a later iteration also matches web01, you can end up with web01.internal.internal, a silent double-replacement that is very hard to spot and even harder to roll back.
Fix: For bulk replacements against a single file, compile all your substitutions into one sed call using multiple -e expressions or a -f script file so the file is written exactly once. Only use the loop pattern when each iteration targets a different file.
Conditionals and sed: Applying Changes Only When Needed
Bash conditionals and sed work together naturally. You check a condition in bash, then call sed only if that condition is true. This is how deploy scripts stay safe and idempotent; they do not blindly overwrite things that are already correct.
LinuxTeck.com
# Only update the config if it still points to the old host
CONFIG="/etc/myapp/db.conf"
OLD="db.dev.local"
NEW="db.prod.internal"
if grep -q "${OLD}" "${CONFIG}"; then
sed -i.bak "s|${OLD}|${NEW}|g" "${CONFIG}"
echo "Config updated: ${OLD} replaced with ${NEW}"
else
echo "Config already up to date. No changes made."
fi
The grep -q check before the sed call is worth building into every deploy script that edits a file. It gives you a clean log message either way, and it prevents sed from writing a new backup file when nothing actually needed to change. You can also toggle a specific config line on or off using this pattern:
LinuxTeck.com
# Comment out or uncomment a config line based on an argument
# Usage: ./toggle_config.sh enable|disable
ACTION="$1"
CONFIG="/etc/myapp/app.conf"
if [ "$ACTION" = "disable" ]; then
# Match lines starting with max_connections, prepend #
sed -i "/^max_connections/s/^/#/" "$CONFIG"
echo "max_connections commented out"
elif [ "$ACTION" = "enable" ]; then
# Match commented lines starting with #max_connections, remove #
sed -i "/^#max_connections/s/^#//" "$CONFIG"
echo "max_connections enabled"
else
echo "Usage: $0 enable|disable"
exit 1
fi
Wrapping sed in Reusable Bash Functions
Once you find yourself writing the same sed pattern in three different scripts, that is the moment to wrap it in a function. A bash function that handles the sed call centrally also gives you one place to update if the logic needs to change, and one place to add error checking.
LinuxTeck.com
# Reusable sed wrapper functions for config management
# Replace a value in a file, with backup
function replace_in_file() {
local pattern="$1"
local replacement="$2"
local file="$3"
if [ ! -f "$file" ]; then
echo "ERROR: File not found: $file"
return 1
fi
sed -i.bak "s|${pattern}|${replacement}|g" "$file"
echo "Updated $file: [$pattern] replaced with [$replacement]"
}
# Comment out a key in a config file
function comment_out_key() {
local key="$1"
local file="$2"
sed -i "/^${key}/s/^/#/" "$file"
}
# Usage examples
replace_in_file "db_host=localhost" "db_host=db.prod.internal" "/etc/myapp/db.conf"
comment_out_key "debug_mode" "/etc/myapp/app.conf"
Using local for all variables inside bash functions is important. Without it, the variable names pattern, file, and replacement are global in the shell, which means calling this function could silently overwrite a variable of the same name you set earlier in the script. For more on writing reusable bash code, the bash functions guide on LinuxTeck goes deeper on this.
What Breaks in Real Scripts and How to Fix It
There are a handful of situations where bash and sed together produce silent failures or unexpected output. These are the ones that get real scripts wrong on production systems.
The ampersand in the replacement string
In sed's replacement string, & is a special character that means "insert the entire matched text here." If your bash variable or hardcoded replacement contains a literal &, sed will replace it with the matched text instead.
LinuxTeck.com
# This wraps the match in square brackets
echo "version=2" | sed "s/version=2/[&]/"
# Output: [version=2]
# Escape & with \& to use it as a literal ampersand
echo "status=ok" | sed 's/ok/ok \& confirmed/'
# Output: status=ok & confirmed
# If a bash variable may contain &, escape it before passing to sed
RAW="done & verified"
SAFE=$(echo "$RAW" | sed 's/&/\\&/g')
echo "status=pending" | sed "s/pending/${SAFE}/"
status=ok & confirmed
status=done & verified
Running sed from a script file with -f
When the sed expressions get long or you need to apply many of them at once, move them into a separate .sed file and call it with the -f flag. This is cleaner than a chain of -e arguments and easier to maintain:
LinuxTeck.com
# s/env=dev/env=prod/g
# s/log_level=debug/log_level=info/g
# s/port=3000/port=8080/g
# /^#/d
# Apply the sed script file to a config
sed -f promote.sed app.conf
# Apply in place with backup
sed -i.bak -f promote.sed app.conf
log_level=info
port=8080
Note:
The -f script file does not expand bash variables. Expressions inside a .sed file are treated as literal sed syntax, not bash. If you need variable expansion in your expressions, you must generate the sed script file dynamically from your bash script using echo or printf before calling sed -f on it, then delete the temporary file afterward.
Questions I Get Asked About This All the Time
My sed command works at the terminal but does nothing inside my bash script. Why?
Almost always a quoting issue. If your sed expression contains a bash variable, you must use double quotes so bash expands the variable before sed runs. Single quotes pass the literal text $VARNAME to sed, which does not match anything in your file. Switch from sed 's/$OLD/$NEW/g' to sed "s/$OLD/$NEW/g" and it will work. If the expression has no variables and still does nothing, check that the pattern actually matches the content in the file by running grep "pattern" file first.
How do I use a bash variable that contains a forward slash in a sed pattern?
Switch to a different delimiter. The / in s/old/new/ is just a convention. You can use any character that does not appear in your pattern. With file paths in a variable, | is the cleanest choice: sed "s|${OLD_PATH}|${NEW_PATH}|g". This eliminates the need to escape every forward slash in the path. The pipe delimiter works identically to the slash in all GNU sed substitution commands.
Can I capture the output of sed into a bash variable instead of writing to a file?
Yes, using command substitution. Wrap the sed command in $() and assign it: RESULT=$(echo "$INPUT" | sed 's/old/new/g'). Or read from a file without modifying it: RESULT=$(sed 's/old/new/g' file.conf). The variable will contain the full sed output. Just do not use -i when capturing to a variable, because -i writes to the file in place and produces no stdout output, so the variable will be empty.
Why does my replacement text get replaced by the matched pattern when I use & in the replacement?
Because & is a special character in sed's replacement string. It means "insert the entire matched text here." So sed 's/hello/[&]/' produces [hello], not [&]. To use a literal ampersand in the replacement, escape it with a backslash: sed 's/hello/hello \& goodbye/'. If the ampersand is coming from a bash variable, sanitise the variable first by running VAR=$(echo "$VAR" | sed 's/&/\\&/g') before using it in your main sed expression.
Does sed -f work with bash variables inside the .sed file?
No. A .sed script file is read as pure sed syntax. Bash does not process it, so $VARNAME inside the file is passed to sed literally and matches nothing. If you need variable expansion, either generate the sed script file dynamically at runtime using echo "s/${OLD}/${NEW}/g" > /tmp/runtime.sed then call sed -f /tmp/runtime.sed file.conf and clean up the temp file, or use -e expressions in the bash script directly where variable expansion does work.
Is there a way to test what sed will do before actually changing my file?
Yes. Leave off the -i flag entirely. Without it, sed prints to stdout and your original file is untouched. Run sed 's/old/new/g' file.conf and review the output before committing. For a side-by-side comparison, use diff file.conf <(sed 's/old/new/g' file.conf). Lines marked with < are the current file content and lines marked with > are what sed would produce. Make this a habit before any sed -i on important files.
Summary
Now that you have this working, the part worth keeping in your muscle memory is the quoting rule: single quotes when the expression is literal, double quotes when bash variables are involved. Everything else, the loops, the conditionals, the functions, follows naturally once that foundation is solid.
The GNU sed manual is the authoritative reference if you need to go deeper on addressing, branching, or the full set of sed commands beyond substitution. For the next step in bash text processing, how bash uses sed pairs naturally with learning awk for when you need column-level logic rather than line-level text replacement.
Related Articles
- sed Commands in Linux: Full Reference with Examples
- Bash Functions: Writing Reusable Code (Part 21 / 34)
- Bash for Loop in Linux with Examples (Part 18 / 34)
- Bash Script Exit Codes and Error Handling (Part 5 of 34)
- Text Processing Commands in Linux
Learn step-by-step how to automate Linux tasks with real-world scripts and practical examples.
