Modern Linux Tools Part 3: Search and Replace With ripgrep and sd

This is Part 3 of the Modern Linux Tools series. Part 2 covered the file tools. This part covers search and replace: ripgrep against grep, and sd against sed.

Same rules as before. A clean Debian 13 VM with 2 vCPUs and 2 GB of RAM, both tools installed from the Debian repositories, and the same data for both. I report the cases where the old tool wins.

sudo apt install ripgrep sd

The packages are ripgrep (binary rg, version 14.1.1) and sd (version 1.0.0). I compared them with GNU grep 3.11 and the GNU sed that ships with Debian 13.

The test data:

  • The 105,003-file tree from Part 2, in a git repository whose .gitignore lists build/ and *.log.
  • A 105 MB text file of 1.5 million lines, interface port1 set ip 10.0.0.1 255.255.255.0 allowaccess ping https ssh, repeated. Every line matches the pattern I use on it.
  • A second copy of that file with one extra line at the end that matches a unique pattern.
  • /usr/lib and /usr/share, about 550 MB of real files on the VM.

A benchmarking trap with grep

Before the numbers, a warning that applies to any grep benchmark you run yourself. GNU grep checks where its output is going. When stdout is /dev/null, it stops at the first match, because nobody can see the output. hyperfine sends output to /dev/null by default, so my first run showed grep searching a 105 MB file in 1.9 ms. Run by hand, the same command took 0.116 s.

All the timings below send the output into a pipe (| cat), which forces both tools to do the full work.

ripgrep against grep

Different defaults, different results

rg skips three things by default: files matched by .gitignore, hidden files and directories, and binary files. grep -r skips none of them. Searching the test tree for line 7:

CommandMatching lines
grep -r "line 7" .112,356
rg "line 7" .99,281
rg -uuu "line 7" .112,356
grep -r --exclude-dir=.git --exclude="*.log" --exclude-dir=build "line 7" .99,281

The 13,075 missing lines in the rg result are all in *.log files, which my .gitignore excludes. rg -uuu turns off all filtering and matches grep -r exactly. With the same excludes written out by hand, grep and rg agree on every line. So the results are the same when the file set is the same, and the file set is where they differ.

The flags to remember are -u (stop honouring ignore files), -uu (also search hidden files) and -uuu (also search binary files). If rg returns nothing and grep finds something, start there.

Directory trees: ripgrep wins

Searchgreprg
Test tree, same file set955 ms518 ms
Test tree, no filtering (rg -uuu)1.209 s692 ms
/usr/lib and /usr/share, files containing systemd288 ms (grep -rIl)172 ms (rg -l)

On a tree of many files rg is about 1.7 to 1.9 times faster, with and without filtering. It searches files in parallel, and on a 2-vCPU VM that is all the advantage it can have. A machine with more cores should widen the gap.

Search on a 105 MB filegreprg
Literal, every line matches113 ms124 ms
Regex with alternation, every line matches203 ms205 ms
Case-insensitive, every line matches138 ms221 ms
Literal, one match in the file62 ms26 ms
Case-insensitive, one match in the file199 ms284 ms

Neither tool wins everywhere. rg is clearly faster at finding a rare literal, at 2.4 times. grep is faster for case-insensitive searches, at 1.4 to 1.6 times, and it is level on the others. These timings apply to this VM and to one file shape. Do not take them as a general ranking.

Output and syntax

When you pipe rg, its output looks like grep, with one difference. grep -rn prints line numbers. rg does not, unless you add -n.

$ rg "line 39" site05/dev05 -g "file039*" | head -2
site05/dev05/file039.yaml:line 39
site05/dev05/file039.yaml:line 39

$ grep -rn "line 39" site05/dev05 --include="file039*" | head -2
site05/dev05/file039.yaml:1:line 39
site05/dev05/file039.yaml:2:line 39

On a terminal, rg switches to a grouped layout with the file name as a heading, and with line numbers on. --heading -n shows what a terminal sees:

site05/dev05/file039.yaml
1:line 39
2:line 39
3:line 39

A few features have no short grep equivalent:

  • File types. rg -t py searches Python files, and rg --type-list lists 206 built-in types. -g '*.yaml' filters by glob. rg --files lists the files rg would search, which makes a good replacement for a quick find.
  • --stats. It prints the number of matches, files searched, bytes searched and the time:
5012 matches
5012 matched lines
242 files contained matches
4411 files searched
769402 bytes searched
0.020042 seconds spent searching
  • --json. It prints one JSON object per event, which is easier to feed to a script than parsing file:line:text.
  • Multiline search with replacement. rg -U searches across lines and -r rewrites the output. For a FortiGate-style policy block:
$ rg -U 'edit 1\n\s+set action (\w+)' -r '$1' /tmp/pol.txt
    accept

The grep way needs PCRE, NUL-delimited input and a tr to clean up:

$ grep -Pzo 'edit 1\n\s+set action \K\w+' /tmp/pol.txt | tr '\0' '\n'
accept

The regex engine

rg uses Rust’s regex engine, which has no lookahead or backreferences. A lookahead fails with a parse error unless you add -P for PCRE2:

$ rg 'line (?=7)' site03/dev00
rg: regex parse error: ...
$ rg -P 'line (?=7)' site03/dev00
site03/dev00/file076.cfg:line 76

That is a trade. The default engine gives predictable run time, and -P is there when you need the features.

My verdict: rg is the better default for searching code and configuration trees, because of the speed on many files and the sensible filtering. Use grep in scripts that must run anywhere, and remember grep is still the faster choice for case-insensitive searches in a single large file.

sd against sed

sd does one thing, substitution, and it does it with a simpler syntax. sed also deletes lines, selects line ranges, prints selectively and edits streams in other ways. sd does none of that. Judge it on substitution only.

Escaping

The strongest case for sd is a literal string full of characters that mean something in a regular expression. Replacing an address like 10.0.0.1/24:

$ sd -F '10.0.0.1/24' '10.0.0.2/24' a.txt

$ sed -i 's/10\.0\.0\.2\/24/10.0.0.3\/24/' b.txt
$ sed -i 's|10.0.0.3/24|10.0.0.4/24|' b.txt

sd -F treats the pattern as a fixed string, so there is nothing to escape. In sed you escape the dots and the slash, or you switch the delimiter to | and leave the dots as unescaped wildcards.

Capture groups

Both tools handle captures. sd uses the Rust regex syntax, so \d works without -E or -P:

$ echo 'host=web01 port=8080' | sed -E 's/host=(\w+) port=([0-9]+)/\2:\1/'
8080:web01
$ echo 'host=web01 port=8080' | sd 'host=(\w+) port=(\d+)' '$2:$1'
8080:web01

sd also supports lazy quantifiers, which POSIX sed -E does not:

$ echo 'a1 b22 c333' | sed -E 's/[0-9]+?/N/'
Na1 b22 c333
$ echo 'a1 b22 c333' | sd '\d+?' 'N'
aN bNN cNNN

In the sed line the ? made the quantifier optional, so the pattern matched an empty string at the start of the line. sd -f i turns on a case-insensitive flag, and -f takes the usual regex flags.

A capture group followed by letters is ambiguous. $1abc could mean group 1 followed by abc, or a group named 1abc. Version 1.0.0 of sd stops and tells you:

$ echo 'web01' | sd '(web)01' '$1abc'
error: The numbered capture group `$1` in the replacement text is ambiguous.
hint: Use curly braces to disambiguate it `${1}abc`.

$ echo 'web01' | sd '(web)01' '${1}abc'
webabc

I like that behaviour. Many tools silently replace the unknown group with an empty string and give you a wrong result.

The dangerous default

sd edits files in place by default. sed prints to the screen unless you add -i. If you run sd on a file the way you run sed on a file, the file changes and there is no prompt.

-p (--preview) shows the result without writing:

$ sd -p '10.0.0.1/24' '10.0.0.2/24' a.txt
ip address 10.0.0.2/24
host=web01 port=8080
line one

$ head -1 a.txt
ip address 10.0.0.1/24

The preview prints the whole file with the changed text highlighted, not only the changed lines. On a large file, pipe it through less. A habit that costs nothing: put -p on every new sd command, check the result, then delete the flag.

Speed on many files

I copied 5,000 small files to a scratch directory before each run, replaced line 7 with LINE 7 in all of them, and compared the output of the two tools. The files were identical afterwards, with 5,784 replacements in each.

CommandMean time
find /tmp/w -type f -exec sed -i 's/line 7/LINE 7/g' {} +214.9 ms
find /tmp/w -type f -exec sd 'line 7' 'LINE 7' {} +201.4 ms
fdfind -t f . /tmp/w -X sd 'line 7' 'LINE 7'205.9 ms

There is no meaningful speed difference. The cost is the disk, and both tools spend it the same way. I also checked what happens to a file with no match. Both tools rewrite it anyway, and its modification time changes from 2020 to the current time. If you rely on timestamps, or a backup tool that watches them, neither tool is safe to run across a whole tree.

My verdict: sd is nicer to write, especially for literal strings and captures, and its ambiguity error is a real safety feature. It is not faster. The in-place default means you should preview first, and sed stays the tool for anything beyond substitution.

Summary

ToolReplacesResultVerdict
ripgrepgrep1.7 to 1.9x faster on trees, mixed on single filesDefault for searching code and config trees
sdsedSame speedNicer syntax, preview before every run

Two things to carry forward. First, rg and sd both change behaviour in ways that matter: rg filters files by default and sd writes by default. Second, the speed gain from rg comes from searching many files, so a single big log file is the case where you will see the least benefit.

The next part covers navigation and history: zoxide, fzf and atuin.