Modern Linux Tools Part 3: Search and Replace With ripgrep and sd
This is Part 3 of the Modern Linux Tools series. Part 2 covered the file tools. This part covers search and replace: ripgrep against grep, and sd against sed.
Same rules as before. A clean Debian 13 VM with 2 vCPUs and 2 GB of RAM, both tools installed from the Debian repositories, and the same data for both. I report the cases where the old tool wins.
sudo apt install ripgrep sd
The packages are ripgrep (binary rg, version 14.1.1) and sd (version 1.0.0). I compared them with GNU grep 3.11 and the GNU sed that ships with Debian 13.
The test data:
- The 105,003-file tree from Part 2, in a git repository whose
.gitignorelistsbuild/and*.log. - A 105 MB text file of 1.5 million lines,
interface port1 set ip 10.0.0.1 255.255.255.0 allowaccess ping https ssh, repeated. Every line matches the pattern I use on it. - A second copy of that file with one extra line at the end that matches a unique pattern.
/usr/liband/usr/share, about 550 MB of real files on the VM.
A benchmarking trap with grep
Before the numbers, a warning that applies to any grep benchmark you run yourself. GNU grep checks where its output is going. When stdout is /dev/null, it stops at the first match, because nobody can see the output. hyperfine sends output to /dev/null by default, so my first run showed grep searching a 105 MB file in 1.9 ms. Run by hand, the same command took 0.116 s.
All the timings below send the output into a pipe (| cat), which forces both tools to do the full work.
ripgrep against grep
Different defaults, different results
rg skips three things by default: files matched by .gitignore, hidden files and directories, and binary files. grep -r skips none of them. Searching the test tree for line 7:
| Command | Matching lines |
|---|---|
grep -r "line 7" . | 112,356 |
rg "line 7" . | 99,281 |
rg -uuu "line 7" . | 112,356 |
grep -r --exclude-dir=.git --exclude="*.log" --exclude-dir=build "line 7" . | 99,281 |
The 13,075 missing lines in the rg result are all in *.log files, which my .gitignore excludes. rg -uuu turns off all filtering and matches grep -r exactly. With the same excludes written out by hand, grep and rg agree on every line. So the results are the same when the file set is the same, and the file set is where they differ.
The flags to remember are -u (stop honouring ignore files), -uu (also search hidden files) and -uuu (also search binary files). If rg returns nothing and grep finds something, start there.
Directory trees: ripgrep wins
| Search | grep | rg |
|---|---|---|
| Test tree, same file set | 955 ms | 518 ms |
Test tree, no filtering (rg -uuu) | 1.209 s | 692 ms |
/usr/lib and /usr/share, files containing systemd | 288 ms (grep -rIl) | 172 ms (rg -l) |
On a tree of many files rg is about 1.7 to 1.9 times faster, with and without filtering. It searches files in parallel, and on a 2-vCPU VM that is all the advantage it can have. A machine with more cores should widen the gap.
Single large files: it depends on the search
| Search on a 105 MB file | grep | rg |
|---|---|---|
| Literal, every line matches | 113 ms | 124 ms |
| Regex with alternation, every line matches | 203 ms | 205 ms |
| Case-insensitive, every line matches | 138 ms | 221 ms |
| Literal, one match in the file | 62 ms | 26 ms |
| Case-insensitive, one match in the file | 199 ms | 284 ms |
Neither tool wins everywhere. rg is clearly faster at finding a rare literal, at 2.4 times. grep is faster for case-insensitive searches, at 1.4 to 1.6 times, and it is level on the others. These timings apply to this VM and to one file shape. Do not take them as a general ranking.
Output and syntax
When you pipe rg, its output looks like grep, with one difference. grep -rn prints line numbers. rg does not, unless you add -n.
$ rg "line 39" site05/dev05 -g "file039*" | head -2
site05/dev05/file039.yaml:line 39
site05/dev05/file039.yaml:line 39
$ grep -rn "line 39" site05/dev05 --include="file039*" | head -2
site05/dev05/file039.yaml:1:line 39
site05/dev05/file039.yaml:2:line 39
On a terminal, rg switches to a grouped layout with the file name as a heading, and with line numbers on. --heading -n shows what a terminal sees:
site05/dev05/file039.yaml
1:line 39
2:line 39
3:line 39
A few features have no short grep equivalent:
- File types.
rg -t pysearches Python files, andrg --type-listlists 206 built-in types.-g '*.yaml'filters by glob.rg --fileslists the filesrgwould search, which makes a good replacement for a quickfind. --stats. It prints the number of matches, files searched, bytes searched and the time:
5012 matches
5012 matched lines
242 files contained matches
4411 files searched
769402 bytes searched
0.020042 seconds spent searching
--json. It prints one JSON object per event, which is easier to feed to a script than parsingfile:line:text.- Multiline search with replacement.
rg -Usearches across lines and-rrewrites the output. For a FortiGate-style policy block:
$ rg -U 'edit 1\n\s+set action (\w+)' -r '$1' /tmp/pol.txt
accept
The grep way needs PCRE, NUL-delimited input and a tr to clean up:
$ grep -Pzo 'edit 1\n\s+set action \K\w+' /tmp/pol.txt | tr '\0' '\n'
accept
The regex engine
rg uses Rust’s regex engine, which has no lookahead or backreferences. A lookahead fails with a parse error unless you add -P for PCRE2:
$ rg 'line (?=7)' site03/dev00
rg: regex parse error: ...
$ rg -P 'line (?=7)' site03/dev00
site03/dev00/file076.cfg:line 76
That is a trade. The default engine gives predictable run time, and -P is there when you need the features.
My verdict: rg is the better default for searching code and configuration trees, because of the speed on many files and the sensible filtering. Use grep in scripts that must run anywhere, and remember grep is still the faster choice for case-insensitive searches in a single large file.
sd against sed
sd does one thing, substitution, and it does it with a simpler syntax. sed also deletes lines, selects line ranges, prints selectively and edits streams in other ways. sd does none of that. Judge it on substitution only.
Escaping
The strongest case for sd is a literal string full of characters that mean something in a regular expression. Replacing an address like 10.0.0.1/24:
$ sd -F '10.0.0.1/24' '10.0.0.2/24' a.txt
$ sed -i 's/10\.0\.0\.2\/24/10.0.0.3\/24/' b.txt
$ sed -i 's|10.0.0.3/24|10.0.0.4/24|' b.txt
sd -F treats the pattern as a fixed string, so there is nothing to escape. In sed you escape the dots and the slash, or you switch the delimiter to | and leave the dots as unescaped wildcards.
Capture groups
Both tools handle captures. sd uses the Rust regex syntax, so \d works without -E or -P:
$ echo 'host=web01 port=8080' | sed -E 's/host=(\w+) port=([0-9]+)/\2:\1/'
8080:web01
$ echo 'host=web01 port=8080' | sd 'host=(\w+) port=(\d+)' '$2:$1'
8080:web01
sd also supports lazy quantifiers, which POSIX sed -E does not:
$ echo 'a1 b22 c333' | sed -E 's/[0-9]+?/N/'
Na1 b22 c333
$ echo 'a1 b22 c333' | sd '\d+?' 'N'
aN bNN cNNN
In the sed line the ? made the quantifier optional, so the pattern matched an empty string at the start of the line. sd -f i turns on a case-insensitive flag, and -f takes the usual regex flags.
A capture group followed by letters is ambiguous. $1abc could mean group 1 followed by abc, or a group named 1abc. Version 1.0.0 of sd stops and tells you:
$ echo 'web01' | sd '(web)01' '$1abc'
error: The numbered capture group `$1` in the replacement text is ambiguous.
hint: Use curly braces to disambiguate it `${1}abc`.
$ echo 'web01' | sd '(web)01' '${1}abc'
webabc
I like that behaviour. Many tools silently replace the unknown group with an empty string and give you a wrong result.
The dangerous default
sd edits files in place by default. sed prints to the screen unless you add -i. If you run sd on a file the way you run sed on a file, the file changes and there is no prompt.
-p (--preview) shows the result without writing:
$ sd -p '10.0.0.1/24' '10.0.0.2/24' a.txt
ip address 10.0.0.2/24
host=web01 port=8080
line one
$ head -1 a.txt
ip address 10.0.0.1/24
The preview prints the whole file with the changed text highlighted, not only the changed lines. On a large file, pipe it through less. A habit that costs nothing: put -p on every new sd command, check the result, then delete the flag.
Speed on many files
I copied 5,000 small files to a scratch directory before each run, replaced line 7 with LINE 7 in all of them, and compared the output of the two tools. The files were identical afterwards, with 5,784 replacements in each.
| Command | Mean time |
|---|---|
find /tmp/w -type f -exec sed -i 's/line 7/LINE 7/g' {} + | 214.9 ms |
find /tmp/w -type f -exec sd 'line 7' 'LINE 7' {} + | 201.4 ms |
fdfind -t f . /tmp/w -X sd 'line 7' 'LINE 7' | 205.9 ms |
There is no meaningful speed difference. The cost is the disk, and both tools spend it the same way. I also checked what happens to a file with no match. Both tools rewrite it anyway, and its modification time changes from 2020 to the current time. If you rely on timestamps, or a backup tool that watches them, neither tool is safe to run across a whole tree.
My verdict: sd is nicer to write, especially for literal strings and captures, and its ambiguity error is a real safety feature. It is not faster. The in-place default means you should preview first, and sed stays the tool for anything beyond substitution.
Summary
| Tool | Replaces | Result | Verdict |
|---|---|---|---|
| ripgrep | grep | 1.7 to 1.9x faster on trees, mixed on single files | Default for searching code and config trees |
| sd | sed | Same speed | Nicer syntax, preview before every run |
Two things to carry forward. First, rg and sd both change behaviour in ways that matter: rg filters files by default and sd writes by default. Second, the speed gain from rg comes from searching many files, so a single big log file is the case where you will see the least benefit.
The next part covers navigation and history: zoxide, fzf and atuin.