When your deployment script runs at the last minute on a Friday, everything appears fine except that the old database hostname is still present in the configuration file. After a few minutes of searching through files with grep, staring blankly into space, and attempting to run the script again, someone finally asks, "Did you sed that file?" That is the moment sed transitions from a tool you had read about or possibly heard mentioned in passing to a tool you are actually going to use.
This document is intended for Linux users who have been exposed to sed in some form (either as part of a larger script or in a one-line shell command) but have never truly understood how to write sed commands by hand. Once you complete this tutorial, you will be able to perform real substitutions, make true in-place edits to files, replace multiple lines of content, and chain together different sed commands so they work effectively in a real-world production environment.
Real Scenario:
Picture this: you have a hundred config files spread across a server. Someone changed the app version string, the old API endpoint is deprecated, and you have to update every single reference by end of shift.
- Opening each file manually is not happening
- A simple find-and-replace in a text editor does not scale across directories
- One sed command with the right flags handles all of it in seconds
That gap between knowing sed exists and knowing how to actually reach for it — that is what this article closes.
If you want to go deeper on how sed fits into your larger scripting toolkit, the guide on using sed inside bash scripts on LinuxTeck covers the integration side in detail.
How sed Substitution Works and Why It Behaves That Way
Sed stands for stream editor. It reads text line by line, applies whatever instructions you give it, and sends the result to standard output. It does not open a file in memory the way a text editor does. It processes one line at a time, which is why it stays fast even on large files.
The basic structure of any sed command is this:
LinuxTeck.com
sed OPTIONS 'COMMAND' filename
# The substitution command — the one you will use 90% of the time
sed 's/FIND/REPLACE/FLAGS'
# Example: replace 'hello' with 'world' in a file
sed 's/hello/world/' greetings.txt
# The parts of the s command
# s = substitute
# / = delimiter (can be any character)
# FIND = pattern to search for
# REPLACE = what to put in its place
# FLAGS = modifiers that control behavior (like g, I, p)
The delimiter does not have to be a forward slash. That trips a lot of people up. If your pattern contains slashes (like file paths), you can use any character as the delimiter. sed 's|/old/path|/new/path|' works just as well and is a lot easier to read.
Note:
By default, sed writes to standard output only. The original file is not touched unless you use the -i flag. This is actually useful because you can test your command first, see the output, and only run with -i when you are confident it is right.
Substitution Flags That Change Everything
Without flags, sed replaces only the first match on each line. That sounds obvious until you spend ten minutes wondering why only half your replacements went through. The flags after the final delimiter control exactly how sed behaves.
LinuxTeck.com
sed 's/cat/dog/g' animals.txt
# Replace only the 2nd occurrence on each line
sed 's/cat/dog/2' animals.txt
# Replace the 3rd and all after it
sed 's/cat/dog/3g' animals.txt
# Case-insensitive match with I flag (GNU sed only — not available on BSD/macOS sed)
sed 's/error/WARNING/gI' app.log
# Print only lines where a substitution happened (-n suppresses default print)
sed -n 's/error/WARNING/gp' app.log
the cat sat on the dog mat (2 flag — only second replaced)
WARNING: disk threshold exceeded
WARNING: connection timeout
The -n flag combined with p is particularly useful when you are auditing a large log file. Instead of printing every line, you get only the ones that matched and changed. Think of it as a grep that also rewrites on the fly.
In-Place File Editing with -i (and the Backup You Should Always Make)
The -i flag is where sed becomes genuinely powerful for sysadmins and DevOps work. It edits the file directly. No pipes, no redirect to a temp file. The change happens in place.
LinuxTeck.com
sed -i 's/old_hostname/new_hostname/g' /etc/app/config.conf
# Edit in place WITH a backup (recommended approach)
sed -i.bak 's/old_hostname/new_hostname/g' /etc/app/config.conf
# Verify the change worked
grep new_hostname /etc/app/config.conf
# Roll back to original if something went wrong
cp /etc/app/config.conf.bak /etc/app/config.conf
db_host=new_hostname
Common Mistake:
On macOS, sed -i requires an extension argument even if you do not want a backup. Running sed -i 's/old/new/g' file on macOS throws an error. Fix: use sed -i '' 's/old/new/g' file on macOS, or sed -i 's/old/new/g' file on Linux/GNU sed. This catches people constantly when scripts written on a Mac fail on a Linux server.
Address Ranges, Line Targeting, and Pattern-Based Scope
Most sed guides stop at global substitution and call it done. That misses half the tool. Sed lets you target specific lines, ranges of lines, and blocks of text between patterns. That is where it gets genuinely useful for config file work.
LinuxTeck.com
sed '5s/old/new/' config.txt
# Substitute on lines 3 through 8
sed '3,8s/debug/info/g' app.conf
# Substitute from line 10 to end of file
sed '10,$s/STAGING/PRODUCTION/g' deploy.conf
# Substitute only between pattern markers
sed '/START/,/END/s/foo/bar/g' config.txt
# Substitute only on lines matching a pattern
sed '/^database/s/localhost/db.prod.internal/g' app.conf
# Negate: substitute on lines that do NOT match a pattern
sed '/^#/!s/old/new/g' config.txt
database_port=5432
database_name=appdb
That last example, the negation with !, is particularly valuable for config files where comment lines start with #. You can make substitutions that skip all comments and only touch live config values. That is the kind of surgical editing that normally takes ten minutes of manual work.
Backreferences, Captured Groups, and the & Shortcut
This is where sed gets genuinely powerful and where most intermediate users stop short. Captured groups let you reference parts of the matched text in your replacement. You are not just swapping a word, you are reshaping the structure of the text itself.
LinuxTeck.com
# Wrap every IP address found in square brackets
sed 's|[0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+|[\&]|g' access.log
# Captured groups — swap first and last name
# "John Smith" becomes "Smith, John"
sed 's/\([A-Za-z]\+\) \([A-Za-z]\+\)/\2, \1/' names.txt
# Add quotes around config values key=value -> key="value"
sed 's/=\(.*\)/="\1"/' settings.conf
# Extended regex with -E — no backslash before ( ) { }
# Reformat date: 2024-06-15 becomes 15-06-2024
sed -E 's/([0-9]{4})-([0-9]{2})-([0-9]{2})/\3-\2-\1/' dates.txt
Smith, John
port="8080"
15-06-2024
Note:
In basic regex mode (default), group parentheses must be escaped: \( and \). With -E (extended), you write them without the backslash: ( and ). Most people find -E far cleaner for complex patterns. You can also check the sed commands reference on LinuxTeck for a full breakdown of regex modes.
Chaining Commands and Real-World Automation Scripts
Running a single sed substitution is one thing. Where it becomes a real sysadmin tool is when you chain commands, apply them across multiple files, or wire them into a deployment pipeline. Here are the patterns you will actually reach for in production.
Chaining multiple substitutions without repeating the command:
LinuxTeck.com
# Deploy config update script
# Updates staging config to production values
CONFIG="/etc/myapp/config.conf"
# Chain multiple substitutions — single quoted multi-line form (safer than backslash continuation)
sed -i.bak '
s/STAGING/PRODUCTION/g;
s/db.staging.internal/db.prod.internal/g;
s/log_level=debug/log_level=warn/g;
/^#/!s/port=8080/port=443/g
' "$CONFIG"
if [ $? -eq 0 ]; then
echo "Config updated successfully"
echo "Backup saved as ${CONFIG}.bak"
else
echo "sed failed — restoring backup if available"
if [ -f "${CONFIG}.bak" ]; then
cp "${CONFIG}.bak" "$CONFIG"
fi
exit 1
fi
Backup saved as /etc/myapp/config.conf.bak
And here is the pattern for running sed across every config file in a directory tree. This is what makes it a proper automation tool, not just a convenience:
LinuxTeck.com
# Bulk config update across all .conf files
# Useful after server migrations or domain renames
OLD_DOMAIN="app.staging.example.com"
NEW_DOMAIN="app.prod.example.com"
CONFIG_DIR="/etc/myapp"
# Escape dots so they match literally in regex (. means "any char" in regex)
OLD_ESC="${OLD_DOMAIN//./\\.}"
NEW_ESC="${NEW_DOMAIN//./\\.}"
find "$CONFIG_DIR" -name "*.conf" -type f | while read -r file; do
echo "Processing: $file"
sed -i.bak "s/$OLD_ESC/$NEW_ESC/g" "$file"
done
echo "Done. Backup files created with .bak extension."
Processing: /etc/myapp/server.conf
Processing: /etc/myapp/cache.conf
Done. Backup files created with .bak extension.
This pattern is what you wire into a deployment script, a CI/CD pipeline, or a server provisioning playbook. Combine it with the bash scripting automation guide for a full picture of how these pieces connect.
Delete Lines, Insert Text, and Work Beyond Just Substitution
Sed does more than find and replace. The d command deletes lines. The a and i commands append or insert lines. These round out the toolkit for config manipulation and log cleanup.
LinuxTeck.com
sed '/^$/d' config.txt
# Delete all comment lines (lines starting with #)
sed '/^#/d' config.txt
# Delete lines containing a specific string
sed '/deprecated/d' config.txt
# Delete a specific line by number
sed '4d' config.txt
# Append a new line AFTER the [database] section header
sed '/^\[database\]/a port=5432' config.ini
# Insert a comment line BEFORE the server_name directive
sed '/^server_name/i # Updated by deploy script' nginx.conf
# Replace the entire log_level line regardless of its current value
sed '/^log_level/c log_level=error' app.conf
port=5432
host=localhost
name=appdb
The c command is particularly useful when you want to set a config value to a known state without caring what it was before. You are not searching for a specific existing value, you are just replacing the whole line. That is safer for idempotent scripts.
Questions I Get Asked About This All the Time
Why does my sed substitution work on the command line but not in my script?
Usually it is a quoting problem. In a script, you often have variables inside the sed expression. If you use single quotes, the variable is not expanded. Switch to double quotes for the sed command when you need variable expansion: sed "s/$OLD/$NEW/g" file. If the variable contains slashes, use a different delimiter like | or @.
What does the /g flag actually do — why is it not on by default?
Without /g, sed replaces only the first match on each line. That is the default POSIX behavior, kept for historical reasons. It is actually useful sometimes, like when you only want to change the first occurrence of something on a line. Add /g whenever you want all occurrences replaced.
How do I make sed changes permanent without creating a temp file?
Use sed -i to edit in place. On GNU/Linux (the default on most servers), sed -i 's/old/new/g' file is enough. Always consider sed -i.bak instead so you have a backup. The risk with -i is that there is no undo if your regex is wrong.
Can sed handle multi-line patterns or do I need awk for that?
Sed can handle multi-line work but it gets complicated fast. The hold space commands h, H, g, and G let you buffer lines and process them together. For most multi-line text manipulation though, awk or perl are genuinely easier. Use sed for line-level operations and reach for awk when you need to reason across multiple lines at once.
My pattern has a forward slash in it. Do I have to escape it?
You can either escape the slash with a backslash \/, or change the delimiter entirely. Using a different delimiter is usually cleaner. sed 's|/old/path|/new/path|g' reads much better than sed 's/\/old\/path/\/new\/path/g'. Any character that does not appear in your pattern works as a delimiter.
Is there a safe way to test a sed command before running -i on a real file?
Yes, just run without -i first. Sed prints to stdout by default, so you see exactly what the output would look like without touching the file. Once the output looks right, add -i or -i.bak to apply it. Also useful: diff <(sed 's/old/new/g' file) file shows you precisely what would change.
Summary
Now that you have these patterns working, you can handle most text manipulation tasks that come up in real sysadmin and DevOps work. The gap between running a quick substitution and writing a proper deployment script is smaller than it looks once you have sed substitution properly mapped out. Start with the in-place edits and chained -e expressions and those will cover the majority of what you will actually need.
The next step worth learning is combining sed with grep and awk in pipeline form for log analysis and data extraction. The GNU sed manual is the definitive reference when you run into edge cases with regex or need to verify specific behavior across sed versions.
Related Articles
- sed Commands in Linux with Examples
- How Bash Uses sed for Text Editing (Part 24 / 34 )
- How Bash Uses grep for Text Processing (Part 23 / 34 )
- Linux Text Processing Commands
- Linux Bash Scripting Automation 2026
- Linux Shell Scripting Command Cheat Sheet
Learn step-by-step how to automate Linux tasks with real-world scripts and practical examples.
