rarias/jungle
rarias/jungle
2
0
Fork 3
Code Issues 53 Pull Requests 10 Actions Packages Projects Releases Wiki Activity

Hardening SSH login in apex #217

Open
rarias wants to merge 3 commits from disable-password-login into master
pull from: disable-password-login
merge into: rarias:master
Conversation 4 Commits 3 Files Changed 1 +12
rarias commented 2025-11-20 15:27:16 +01:00
Owner
Copy Link

Bots didn't took long to find the SSH port. Enable fail2ban to take down repeated attempts from the same IP and disable password login to cover for potentially weak passwords.

Bots didn't took long to find the SSH port. Enable fail2ban to take down repeated attempts from the same IP and disable password login to cover for potentially weak passwords.
rarias added 2 commits 2025-11-20 15:27:17 +01:00 
We are seeing a lot of failed attempts from the same IPs:

    apex% sudo journalctl -u sshd -b0 | grep 'Failed password' | wc -l
    2441
Disable password login via SSH in apex
All checks were successful
CI / build:cross (pull_request) Successful in 12s
Details
CI / build:all (pull_request) Successful in 15s
Details
341551d5ba
rarias requested review from abonerib 2025-11-20 15:29:53 +01:00
abonerib approved these changes 2025-11-20 15:37:39 +01:00
Dismissed
abonerib left a comment
Collaborator
Copy Link

LGTM

LGTM
rarias commented 2025-11-21 12:00:13 +01:00
Author
Owner
Copy Link

Default fail2ban config is not particularly clever:

apex% sudo journalctl -u fail2ban | tail -30
Nov 21 10:39:05 apex fail2ban.actions[1289560]: NOTICE [sshd] Unban 134.209.159.145
Nov 21 10:39:24 apex fail2ban.filter[1289560]: INFO [sshd] Found 134.209.159.145 - 2025-11-21 10:39:24
Nov 21 10:40:06 apex fail2ban.filter[1289560]: INFO [sshd] Found 134.209.159.145 - 2025-11-21 10:40:05
Nov 21 10:40:47 apex fail2ban.filter[1289560]: INFO [sshd] Found 134.209.159.145 - 2025-11-21 10:40:47
Nov 21 10:40:47 apex fail2ban.actions[1289560]: NOTICE [sshd] Ban 134.209.159.145
Nov 21 10:50:47 apex fail2ban.actions[1289560]: NOTICE [sshd] Unban 134.209.159.145
Nov 21 10:51:05 apex fail2ban.filter[1289560]: INFO [sshd] Found 134.209.159.145 - 2025-11-21 10:51:05
Nov 21 10:51:46 apex fail2ban.filter[1289560]: INFO [sshd] Found 134.209.159.145 - 2025-11-21 10:51:45
Nov 21 10:52:26 apex fail2ban.filter[1289560]: INFO [sshd] Found 134.209.159.145 - 2025-11-21 10:52:26
Nov 21 10:52:27 apex fail2ban.actions[1289560]: NOTICE [sshd] Ban 134.209.159.145
Nov 21 11:02:26 apex fail2ban.actions[1289560]: NOTICE [sshd] Unban 134.209.159.145
Nov 21 11:02:45 apex fail2ban.filter[1289560]: INFO [sshd] Found 134.209.159.145 - 2025-11-21 11:02:45
Nov 21 11:03:26 apex fail2ban.filter[1289560]: INFO [sshd] Found 134.209.159.145 - 2025-11-21 11:03:26
Nov 21 11:04:05 apex fail2ban.filter[1289560]: INFO [sshd] Found 134.209.159.145 - 2025-11-21 11:04:05
Nov 21 11:04:05 apex fail2ban.actions[1289560]: NOTICE [sshd] Ban 134.209.159.145
Nov 21 11:14:05 apex fail2ban.actions[1289560]: NOTICE [sshd] Unban 134.209.159.145
Nov 21 11:25:36 apex fail2ban.filter[1289560]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 11:25:36
Nov 21 11:26:22 apex fail2ban.filter[1289560]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 11:26:22
Nov 21 11:27:08 apex fail2ban.filter[1289560]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 11:27:08
Nov 21 11:27:08 apex fail2ban.actions[1289560]: NOTICE [sshd] Ban 139.59.24.81
Nov 21 11:37:08 apex fail2ban.actions[1289560]: NOTICE [sshd] Unban 139.59.24.81
Nov 21 11:37:43 apex fail2ban.filter[1289560]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 11:37:42
Nov 21 11:38:29 apex fail2ban.filter[1289560]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 11:38:28
Nov 21 11:39:16 apex fail2ban.filter[1289560]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 11:39:16
Nov 21 11:39:17 apex fail2ban.actions[1289560]: NOTICE [sshd] Ban 139.59.24.81
Nov 21 11:49:17 apex fail2ban.actions[1289560]: NOTICE [sshd] Unban 139.59.24.81
Nov 21 11:49:41 apex fail2ban.filter[1289560]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 11:49:40
Nov 21 11:50:26 apex fail2ban.filter[1289560]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 11:50:26
Nov 21 11:51:13 apex fail2ban.filter[1289560]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 11:51:12
Nov 21 11:51:13 apex fail2ban.actions[1289560]: NOTICE [sshd] Ban 139.59.24.81

I will also enable the bantime-increment.enable option so the ban time increases exponentially.

I think I will also increase the maxretry to 5, so when it accidentally assigns 3 bad attemps to a single attempt it doesn't lock the user.

Default fail2ban config is not particularly clever: ``` apex% sudo journalctl -u fail2ban | tail -30 Nov 21 10:39:05 apex fail2ban.actions[1289560]: NOTICE [sshd] Unban 134.209.159.145 Nov 21 10:39:24 apex fail2ban.filter[1289560]: INFO [sshd] Found 134.209.159.145 - 2025-11-21 10:39:24 Nov 21 10:40:06 apex fail2ban.filter[1289560]: INFO [sshd] Found 134.209.159.145 - 2025-11-21 10:40:05 Nov 21 10:40:47 apex fail2ban.filter[1289560]: INFO [sshd] Found 134.209.159.145 - 2025-11-21 10:40:47 Nov 21 10:40:47 apex fail2ban.actions[1289560]: NOTICE [sshd] Ban 134.209.159.145 Nov 21 10:50:47 apex fail2ban.actions[1289560]: NOTICE [sshd] Unban 134.209.159.145 Nov 21 10:51:05 apex fail2ban.filter[1289560]: INFO [sshd] Found 134.209.159.145 - 2025-11-21 10:51:05 Nov 21 10:51:46 apex fail2ban.filter[1289560]: INFO [sshd] Found 134.209.159.145 - 2025-11-21 10:51:45 Nov 21 10:52:26 apex fail2ban.filter[1289560]: INFO [sshd] Found 134.209.159.145 - 2025-11-21 10:52:26 Nov 21 10:52:27 apex fail2ban.actions[1289560]: NOTICE [sshd] Ban 134.209.159.145 Nov 21 11:02:26 apex fail2ban.actions[1289560]: NOTICE [sshd] Unban 134.209.159.145 Nov 21 11:02:45 apex fail2ban.filter[1289560]: INFO [sshd] Found 134.209.159.145 - 2025-11-21 11:02:45 Nov 21 11:03:26 apex fail2ban.filter[1289560]: INFO [sshd] Found 134.209.159.145 - 2025-11-21 11:03:26 Nov 21 11:04:05 apex fail2ban.filter[1289560]: INFO [sshd] Found 134.209.159.145 - 2025-11-21 11:04:05 Nov 21 11:04:05 apex fail2ban.actions[1289560]: NOTICE [sshd] Ban 134.209.159.145 Nov 21 11:14:05 apex fail2ban.actions[1289560]: NOTICE [sshd] Unban 134.209.159.145 Nov 21 11:25:36 apex fail2ban.filter[1289560]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 11:25:36 Nov 21 11:26:22 apex fail2ban.filter[1289560]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 11:26:22 Nov 21 11:27:08 apex fail2ban.filter[1289560]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 11:27:08 Nov 21 11:27:08 apex fail2ban.actions[1289560]: NOTICE [sshd] Ban 139.59.24.81 Nov 21 11:37:08 apex fail2ban.actions[1289560]: NOTICE [sshd] Unban 139.59.24.81 Nov 21 11:37:43 apex fail2ban.filter[1289560]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 11:37:42 Nov 21 11:38:29 apex fail2ban.filter[1289560]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 11:38:28 Nov 21 11:39:16 apex fail2ban.filter[1289560]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 11:39:16 Nov 21 11:39:17 apex fail2ban.actions[1289560]: NOTICE [sshd] Ban 139.59.24.81 Nov 21 11:49:17 apex fail2ban.actions[1289560]: NOTICE [sshd] Unban 139.59.24.81 Nov 21 11:49:41 apex fail2ban.filter[1289560]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 11:49:40 Nov 21 11:50:26 apex fail2ban.filter[1289560]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 11:50:26 Nov 21 11:51:13 apex fail2ban.filter[1289560]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 11:51:12 Nov 21 11:51:13 apex fail2ban.actions[1289560]: NOTICE [sshd] Ban 139.59.24.81 ``` I will also enable the [bantime-increment.enable](https://search.nixos.org/options?channel=25.05&show=services.fail2ban.bantime-increment.enable&query=fail2ban) option so the ban time increases exponentially. I think I will also increase the maxretry to 5, so when it accidentally assigns 3 bad attemps to a single attempt it doesn't lock the user.
👍 1
rarias added 1 commit 2025-11-21 12:21:24 +01:00
Increase fail2ban ban time on each attempt
All checks were successful
CI / build:cross (pull_request) Successful in 12s
Details
CI / build:all (pull_request) Successful in 15s
Details
15a26570be
rarias commented 2025-11-21 15:10:40 +01:00
Author
Owner
Copy Link

It seems now it bans bots for quite a while as expected (10 min * 2^(5 - 1) = 160 min = 2h 40min):

apex% sudo journalctl -u fail2ban -f -n 30
Nov 21 14:30:02 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 14:30:02
Nov 21 14:30:44 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 14:30:44
Nov 21 14:31:27 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 14:31:27
Nov 21 14:32:08 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 14:32:08
Nov 21 14:32:49 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 14:32:49
Nov 21 14:32:50 apex fail2ban.actions[1460258]: NOTICE [sshd] Ban 209.38.29.100
Nov 21 14:36:46 apex fail2ban.actions[1460258]: NOTICE [sshd] Unban 139.59.24.81
Nov 21 14:37:14 apex fail2ban.filter[1460258]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 14:37:14
Nov 21 14:37:14 apex fail2ban.observer[1460258]: INFO [sshd] Found 139.59.24.81, bad - 2025-11-21 14:37:14, 4 # -> 5, Ban
Nov 21 14:37:14 apex fail2ban.actions[1460258]: NOTICE [sshd] Ban 139.59.24.81
Nov 21 14:37:14 apex fail2ban.observer[1460258]: INFO [sshd] IP 139.59.24.81 is bad: 4 # last 2025-11-21 13:16:46 - incr 10m to 2h 40m
Nov 21 14:37:14 apex fail2ban.observer[1460258]: NOTICE [sshd] Increase Ban 139.59.24.81 (5 # 2h 40m -> 2025-11-21 17:17:14)
Nov 21 14:42:49 apex fail2ban.actions[1460258]: NOTICE [sshd] Unban 209.38.29.100
Nov 21 14:43:13 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 14:43:13
Nov 21 14:43:13 apex fail2ban.observer[1460258]: INFO [sshd] Found 209.38.29.100, bad - 2025-11-21 14:43:13, 1 # -> 2.0
Nov 21 14:43:51 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 14:43:50
Nov 21 14:43:51 apex fail2ban.observer[1460258]: INFO [sshd] Found 209.38.29.100, bad - 2025-11-21 14:43:50, 1 # -> 2.0
Nov 21 14:44:29 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 14:44:29
Nov 21 14:44:29 apex fail2ban.observer[1460258]: INFO [sshd] Found 209.38.29.100, bad - 2025-11-21 14:44:29, 1 # -> 2.0
Nov 21 14:44:29 apex fail2ban.actions[1460258]: NOTICE [sshd] Ban 209.38.29.100
Nov 21 14:44:29 apex fail2ban.observer[1460258]: INFO [sshd] IP 209.38.29.100 is bad: 1 # last 2025-11-21 14:32:50 - incr 10m to 20m
Nov 21 14:44:29 apex fail2ban.observer[1460258]: NOTICE [sshd] Increase Ban 209.38.29.100 (2 # 20m -> 2025-11-21 15:04:29)
Nov 21 15:04:30 apex fail2ban.actions[1460258]: NOTICE [sshd] Unban 209.38.29.100
Nov 21 15:05:00 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 15:04:59
Nov 21 15:05:00 apex fail2ban.observer[1460258]: INFO [sshd] Found 209.38.29.100, bad - 2025-11-21 15:04:59, 2 # -> 3.0
Nov 21 15:05:35 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 15:05:35
Nov 21 15:05:35 apex fail2ban.observer[1460258]: INFO [sshd] Found 209.38.29.100, bad - 2025-11-21 15:05:35, 2 # -> 3.0
Nov 21 15:05:36 apex fail2ban.actions[1460258]: NOTICE [sshd] Ban 209.38.29.100
Nov 21 15:05:36 apex fail2ban.observer[1460258]: INFO [sshd] IP 209.38.29.100 is bad: 2 # last 2025-11-21 14:44:30 - incr 10m to 40m
Nov 21 15:05:36 apex fail2ban.observer[1460258]: NOTICE [sshd] Increase Ban 209.38.29.100 (3 # 40m -> 2025-11-21 15:45:35)
It seems now it bans bots for quite a while as expected (10 min * 2^(5 - 1) = 160 min = 2h 40min): ``` apex% sudo journalctl -u fail2ban -f -n 30 Nov 21 14:30:02 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 14:30:02 Nov 21 14:30:44 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 14:30:44 Nov 21 14:31:27 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 14:31:27 Nov 21 14:32:08 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 14:32:08 Nov 21 14:32:49 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 14:32:49 Nov 21 14:32:50 apex fail2ban.actions[1460258]: NOTICE [sshd] Ban 209.38.29.100 Nov 21 14:36:46 apex fail2ban.actions[1460258]: NOTICE [sshd] Unban 139.59.24.81 Nov 21 14:37:14 apex fail2ban.filter[1460258]: INFO [sshd] Found 139.59.24.81 - 2025-11-21 14:37:14 Nov 21 14:37:14 apex fail2ban.observer[1460258]: INFO [sshd] Found 139.59.24.81, bad - 2025-11-21 14:37:14, 4 # -> 5, Ban Nov 21 14:37:14 apex fail2ban.actions[1460258]: NOTICE [sshd] Ban 139.59.24.81 Nov 21 14:37:14 apex fail2ban.observer[1460258]: INFO [sshd] IP 139.59.24.81 is bad: 4 # last 2025-11-21 13:16:46 - incr 10m to 2h 40m Nov 21 14:37:14 apex fail2ban.observer[1460258]: NOTICE [sshd] Increase Ban 139.59.24.81 (5 # 2h 40m -> 2025-11-21 17:17:14) Nov 21 14:42:49 apex fail2ban.actions[1460258]: NOTICE [sshd] Unban 209.38.29.100 Nov 21 14:43:13 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 14:43:13 Nov 21 14:43:13 apex fail2ban.observer[1460258]: INFO [sshd] Found 209.38.29.100, bad - 2025-11-21 14:43:13, 1 # -> 2.0 Nov 21 14:43:51 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 14:43:50 Nov 21 14:43:51 apex fail2ban.observer[1460258]: INFO [sshd] Found 209.38.29.100, bad - 2025-11-21 14:43:50, 1 # -> 2.0 Nov 21 14:44:29 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 14:44:29 Nov 21 14:44:29 apex fail2ban.observer[1460258]: INFO [sshd] Found 209.38.29.100, bad - 2025-11-21 14:44:29, 1 # -> 2.0 Nov 21 14:44:29 apex fail2ban.actions[1460258]: NOTICE [sshd] Ban 209.38.29.100 Nov 21 14:44:29 apex fail2ban.observer[1460258]: INFO [sshd] IP 209.38.29.100 is bad: 1 # last 2025-11-21 14:32:50 - incr 10m to 20m Nov 21 14:44:29 apex fail2ban.observer[1460258]: NOTICE [sshd] Increase Ban 209.38.29.100 (2 # 20m -> 2025-11-21 15:04:29) Nov 21 15:04:30 apex fail2ban.actions[1460258]: NOTICE [sshd] Unban 209.38.29.100 Nov 21 15:05:00 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 15:04:59 Nov 21 15:05:00 apex fail2ban.observer[1460258]: INFO [sshd] Found 209.38.29.100, bad - 2025-11-21 15:04:59, 2 # -> 3.0 Nov 21 15:05:35 apex fail2ban.filter[1460258]: INFO [sshd] Found 209.38.29.100 - 2025-11-21 15:05:35 Nov 21 15:05:35 apex fail2ban.observer[1460258]: INFO [sshd] Found 209.38.29.100, bad - 2025-11-21 15:05:35, 2 # -> 3.0 Nov 21 15:05:36 apex fail2ban.actions[1460258]: NOTICE [sshd] Ban 209.38.29.100 Nov 21 15:05:36 apex fail2ban.observer[1460258]: INFO [sshd] IP 209.38.29.100 is bad: 2 # last 2025-11-21 14:44:30 - incr 10m to 40m Nov 21 15:05:36 apex fail2ban.observer[1460258]: NOTICE [sshd] Increase Ban 209.38.29.100 (3 # 40m -> 2025-11-21 15:45:35) ```
rarias requested review from abonerib 2025-11-21 15:10:46 +01:00
abonerib approved these changes 2025-11-21 17:40:36 +01:00
All checks were successful
CI / build:cross (pull_request) Successful in 12s
Details
CI / build:all (pull_request) Successful in 15s
Details
This pull request can be merged automatically.
You are not authorized to merge this pull request.
View command line instructions.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin disable-password-login:disable-password-login
git checkout disable-password-login
Sign in to join this conversation.
Reviewers
No Reviewers
abonerib
Labels
Clear labels
bug
ceph
ci
config
cross
hw
io
kernel
mpi
net
nix
ops
repair
slurm
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
abonerib arocanon varcila rarias
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: rarias/jungle#217
No description provided.
Delete Branch "disable-password-login"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?