Tor Deep Dive Part 6: Ebury, and How to Check Your Own Debian Hosts for It

In Part 1 the relays were taken over with Ebury, and in Part 5 I showed what the interception side looks like. This part is about the malware on the host. It matters beyond Tor, because Ebury has nothing to do with relays. It is a backdoor for any Linux server that runs OpenSSH, and the way it hides is the useful lesson: it lives inside a library that sshd loads anyway, so most of the usual integrity checks say nothing.

I want to be clear about what I did and did not test. I did not run Ebury. I do not have a sample and I would not run one on a machine I care about. What I did do is check a clean Debian 13 host with the detection commands, build a harmless imitation of the infected file layout on the same host, and run the same commands against it. I also ran ESET’s own detection script against the clean host, against a decoy socket, and against the imitation layout. That shows which checks fire and which stay silent. It says nothing about the malware’s behaviour, so the behaviour below comes from ESET’s published research and is attributed as such.

What Ebury is

Ebury is an OpenSSH backdoor and credential stealer. ESET first documented it in February 2014, and their 2024 white paper, “Ebury is alive but unseen”, puts the total at around 400,000 Linux servers compromised since at least 2009. That figure is cumulative over almost 15 years, as of mid-2023. ESET says more than 100,000 were still compromised at the end of 2023. It is not a snapshot of today.

The 2024 paper dates version 1.8 to November 2023 and 1.8.2 to January 2024. The relay incident used 1.8.3.

Two things Ebury does matter here:

  • It steals credentials. Passwords from logins to the server, and passwords, key passphrases and keys used by the ssh client on the infected host. This is why the relay operators were told to rotate everything, and why a compromised jump host is a very bad day.
  • It gives the attacker a backdoor login that stays out of the normal logs. ESET’s 2014 analysis describes the original trigger as a specially crafted SSH version string. Newer versions have changed the encoding, so I would not rely on any specific string today.

The paper also lists how hosts are first compromised: credential stuffing, reusing credentials and keys harvested from known_hosts and wtmp on hosts it already owns, hypervisors and hosting infrastructure, a Control Web Panel zero-day, and SSH man-in-the-middle on the same network. In the relay case the entry point was a reused password with SSH password authentication on, which is the boring end of that list.

Where it hides: a library sshd already loads

Early Ebury variants replaced the ssh, sshd and ssh-add binaries. A package check catches that. Later versions moved into a shared library. ESET’s 2024 paper says Ebury hijacks libkeyutils.so.1, a dependency of libkrb5 that both the OpenSSH client and server load.

On Debian 13 I can confirm that chain on the clean host:

sudo ldd /usr/lib/openssh/sshd-session | grep -iE 'keyutils|gssapi|krb5|pam|crypto'
	libpam.so.0 => /lib/x86_64-linux-gnu/libpam.so.0
	libgssapi_krb5.so.2 => /lib/x86_64-linux-gnu/libgssapi_krb5.so.2
	libkrb5.so.3 => /lib/x86_64-linux-gnu/libkrb5.so.3
	libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3
	libk5crypto.so.3 => /lib/x86_64-linux-gnu/libk5crypto.so.3
	libkrb5support.so.0 => /lib/x86_64-linux-gnu/libkrb5support.so.0
	libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1

Two Debian 13 details. OpenSSH 10 splits the daemon into sshd and sshd-session, and it is sshd-session that carries the Kerberos libraries and so libkeyutils. Running ldd /usr/sbin/sshd shows only libcrypto, which misleads you if you check the wrong binary. And apt-cache rdepends --installed libkeyutils1 lists only libkrb5-3 and mokutil on this host, so nothing you would deliberately change touches that library.

ESET’s indicator list gives several ways Ebury gets in. The ones that matter for Debian are:

  • Replace libkeyutils.so and repoint the symlink to a malicious copy with a .2 suffix.
  • Place a trojanized libkeyutils.so in a lib/tls/ directory, where it is loaded instead of the real one.
  • Patch libkeyutils.so to load a three-character library such as libns2.so or libpw3.so in place of libc.so.6.
  • Load a side library through dlopen, with names like libkeystats.so, libkeyctl.so and librwctl.so.
  • On rare occasions, mostly on non-Linux systems, patch the OpenSSH executables themselves.

ESET gives the payload size as roughly 28 kB (version 1.2.1) to 64 kB (version 1.8). The .2 suffix is what Zimmer saw on the relays: a setuid libkeyutils.so.1.10.2 that no package owned.

Checks on a clean Debian 13 host

These are the commands from Zimmer’s tor-relays post, plus some I added. The host is a stock Debian 13 install with OpenSSH 10.0p2, sshd running key-only.

find / -xdev -name 'libkeyutils*' -perm -4000 2>/dev/null

No output. Nothing named libkeyutils* carries the setuid bit.

ls -l /usr/lib/x86_64-linux-gnu/libkeyutils*
dpkg -S "$(readlink -f /usr/lib/x86_64-linux-gnu/libkeyutils.so.1)"
lrwxrwxrwx 1 root root    19 Apr 13  2025 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1 -> libkeyutils.so.1.10
-rw-r--r-- 1 root root 22448 Apr 13  2025 /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.10
libkeyutils1:amd64: /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.10

The symlink points to the file the package shipped, libkeyutils1 owns it, and it is 22,448 bytes. Record the size and the SHA-256 for your own hosts so a later change has something to be compared against. On this host sha256sum gives e5d5a7450d08eff7d4bbcaac75ef2b94d3447c81a1b2ddf3ab85d2de4709a9a8, and the package’s own md5sums list agrees with the file.

grep -E '@event-|@/dev/(event|stats)-' /proc/net/unix || echo "(none)"

(none). This looks for abstract UNIX sockets named like the ones in Zimmer’s indicator list. ESET’s paper says versions after 1.5 spawn a process that listens on an abstract socket, and the compromised SSH client passes credentials to it. The relay example was @event-0YsOG9WSOE.

systemctl status ssh | sed -n '/CGroup/,$p'

The CGroup section shows one process, the sshd listener. On a compromised relay Zimmer found a fake /sbin/rsyslogd in that cgroup. A clean host should not have one. ESET’s README lists the legitimate programs Ebury has been loaded into with LD_PRELOAD to keep that process running, including /sbin/rsyslogd, /sbin/auditd, /usr/sbin/crond and /bin/sync, so a process with one of those names that is running somewhere odd deserves a look.

Two more I added, because the implant changes what sshd logs and how it behaves:

sudo sshd -T | grep -E '^(loglevel|usepam|authorizedkeysfile|permittunnel|passwordauthentication) '
usepam yes
passwordauthentication no
loglevel INFO
authorizedkeysfile .ssh/authorized_keys .ssh/authorized_keys2
permittunnel no

Zimmer’s relays had sshd’s log level lowered so logins stopped being logged. Write these values down for your hosts and alert when loglevel changes. A baseline only helps if you recorded it before the incident.

And package verification:

sudo dpkg -V libkeyutils1 openssh-server; echo "exit=$?"

Silent, exit 0, so no packaged file has changed. The next section shows why that is weaker than it sounds.

Simulating the infected layout

To see which checks fire, I imitated the persistence layout on the lab VM. I copied the real, clean library to libkeyutils.so.1.10.2, set the setuid bit on the copy, repointed the libkeyutils.so.1 symlink to it, and started a harmless Python listener on an abstract socket named @event-TESTONLY. Nothing in that file is malicious. It is byte-for-byte the genuine library. The point is the layout, not the payload.

L=/usr/lib/x86_64-linux-gnu
sudo cp -p $L/libkeyutils.so.1.10 $L/libkeyutils.so.1.10.2
sudo chmod u+s $L/libkeyutils.so.1.10.2
sudo ln -sfn libkeyutils.so.1.10.2 $L/libkeyutils.so.1

Then the same checks:

CheckResult on the simulated layout
find / -xdev -name 'libkeyutils*' -perm -4000Fires. Prints /usr/lib/x86_64-linux-gnu/libkeyutils.so.1.10.2
dpkg -S "$(readlink -f ...libkeyutils.so.1)"Fires. dpkg-query: no path found matching pattern ...libkeyutils.so.1.10.2
grep -E '@event-...' /proc/net/unixFires. Shows the @event-TESTONLY socket
Unowned-file sweep of the lib directoryFires. unowned: .../libkeyutils.so.1.10.2
dpkg -V libkeyutils1Silent. Exit 0, no output
ldd on sshd-sessionResolves libkeyutils.so.1 as normal, nothing flags the change

The last two rows are the lesson. dpkg -V compares the files listed in the package’s md5sums against the disk. The malicious file is not in that list, because no package ships it. The symlink is not in it either, because md5sums records regular files only. The original libkeyutils.so.1.10 is still on disk and still matches, so dpkg -V reports nothing even though sshd would now load a different file. debsums reads the same md5sums data, so I expect the same blind spot, but I did not install it to prove that.

Two checks cover what dpkg -V cannot see:

# files in the library directory that no package owns
for f in /usr/lib/x86_64-linux-gnu/*; do
  dpkg -S "$f" >/dev/null 2>&1 || echo "unowned: $f"
done

# where the symlink points versus what the package shipped
ls -l /usr/lib/x86_64-linux-gnu/libkeyutils.so.1
dpkg -L libkeyutils1 | grep 'so.1$'

On the clean host the unowned sweep prints nothing, which makes it a good candidate for a scheduled check. A real server with software installed outside apt will be noisier, so run it once and keep the baseline. A file integrity tool catches the same layout without relying on package lists, and Part 8 shows AIDE doing exactly that.

I restored the host afterwards: symlink back to libkeyutils.so.1.10, extra file removed, dpkg -V silent, no setuid library files, listener gone.

ESET’s own detection script

ESET publishes the indicators and the tooling in two repositories, and the names are not obvious. Ebury is filed under its 2014 operation name, Windigo, in eset/malware-ioc (the windigo folder holds the README, YARA rules, Snort rules and a MISP export). The detection script is in a separate repository, eset/malware-research, in the ebury folder. On a Debian 13 VM:

sudo apt install -y git
git clone --depth 1 https://github.com/eset/malware-ioc.git ~/malware-ioc
git clone --depth 1 https://github.com/eset/malware-research.git ~/malware-research
ls -l ~/malware-research/ebury
-rw-rw-r-- 1 anon anon  718 README.adoc
-rwxrwxr-x 1 anon anon 4839 detect_ebury.sh

I read the script before running it. It is about 100 lines of bash, and it does one thing: it holds a table of 25 abstract UNIX socket names, each mapped to the Ebury version known to use it, and tries to connect to each one with Perl or Python 3. If a connect succeeds it reports the version, looks up the socket’s inode in /proc/net/unix, and searches /proc for the process that owns it. It does not read or modify any file. If it finds a socket but cannot find the inode or the PID, it assumes the userland rootkit is hiding them and re-runs itself with H=1 I=1 LD_PRELOAD=":" LD_DEBUG=":".

The repository history shows the table is maintained. The script was added on 14 May 2024, and later commits add socket names for 1.8.2 variants (September 2024 to November 2025), then 1.8.2.e7 and 1.8.3 on 22 June 2026. The I variable was added in a commit dated 8 October 2026. Zimmer’s post says ESET’s public script predates 1.8.3. The version of the script I cloned today does list 1.8.3, with the socket name event-0YsOG9WSOE. The IoC README in the other repository still says it covers versions up to 1.8.2, so I suspect that is the document he meant, but I cannot tell from here.

I ran it three ways on the lab VM.

CaseResult
Clean hostEbury UNIX socket was not found. This is not a guarantee that the system is free from Ebury. Exit 0
Harmless Python listener bound to the abstract socket event-0YsOG9WSOEFound abstract UNIX socket "event-0YsOG9WSOE" for Ebury version 1.8.3, then the socket inode and the PID of my listener. Exit 1
The simulated library layout from the previous section (setuid copy, repointed symlink), no socketSame output as the clean host. Exit 0

So the script does what it says and detects a running Ebury payload by its socket. The third row is the useful one: it says nothing about the library layout. If the payload is on disk but the process is not running, or the socket name is one ESET has not seen yet, it reports clean. That is not a criticism, since the script’s own output says a clean result is not a guarantee. It is a reason to run it alongside the file checks above and not instead of them. In my testing the socket check and the file checks caught different halves of the problem, and dpkg -V caught neither.

The decoy listener only proves the detection logic works. It is not Ebury, and a real infection could use a name the script does not know.

Why a clean result does not prove much

Every command above runs on the host you are checking, and a userland rootkit can lie to them.

  • Ebury runs inside the processes it hooks. ESET’s paper says that since version 1.6 it is injected into shells spawned through the compromised sshd, so the shell you log in with may be one it controls. That affects find, ls and grep run from it.
  • ESET’s IoC README gives a command to start a shell free of the rootkit at the time of writing: H=1 LD_DEBUG="" LD_PRELOAD="" "$SHELL". It also suggests systemd-run -S, which starts a shell that is not a child of the OpenSSH server, and warns that its advice may already be out of date. I have not tested either against real Ebury, so I am relaying ESET’s guidance.
  • For anything that matters, boot a live image or mount the disk read-only on another machine and run the checks against the mounted filesystem.

Old advice that no longer holds

Older Ebury material has two detection tricks worth knowing about and not trusting.

ESET’s 2014 article describes ssh -G as a check: on a clean system the option was rejected with an “illegal option” error, and on an infected one only the usage message appeared. That behaviour is gone. Modern OpenSSH, including the OpenSSH_10.0p2 on my lab host, supports -G and prints the effective client configuration, so the test no longer tells you anything.

The same article finds the malware’s POSIX shared memory segment with ipcs -m, looking for segments over 3 MB with 666 permissions. The 2024 paper says versions before 1.5 used a fixed shared memory segment and later versions use an abstract socket. So the shared memory check is for old variants, and a clean result says nothing about 1.8.

Hardening so the next one costs more

Most of this is what the relay incident already taught, applied to ordinary hosts.

  1. Key-only SSH, and no shared passwords. The initial compromise was a reused password. PasswordAuthentication no removes the password for the server-side stealer to take. It does not protect the client side, because the ssh client on an infected host also sees your key passphrases, which is why management hosts need to stay clean.
  2. Keep management SSH off the public internet. A bastion, a VPN or Tailscale in front means a stolen password is not enough to log in.
  3. Ship logs off the host and alert on a log-level drop. If sshd goes quiet, that is a finding. Wazuh does this well, and the Host-Based Detection series covers the file integrity side.
  4. Watch setuid files and library directories with a file integrity tool such as AIDE or Wazuh syscheck. A new setuid file in /usr/lib/x86_64-linux-gnu/ is one of the clearest signals in this whole story.
  5. Run ESET’s script from time to time, from a trusted shell, and treat it as a cheap socket check on top of the file checks, not a verdict.
  6. Rebuild instead of cleaning. ESET’s remediation advice is to reinstall the OS and treat every credential used on the machine as compromised. That matches the Tor Project’s advice for the relays.

Where this leaves the series

Part 5 showed the interception and Part 6 showed the host. Neither needed Tor itself to be broken. The relay software did what it was told, and an attacker who controlled the host told it to do something else. The next part looks at the other side of the problem: finding a bad exit from the outside, with a small Python scanner that pins each exit and compares the SSH and TLS fingerprints it sees against the real ones.

Sources: