Trace and Mitigate SYN-Flood DDoS on Linux Servers

This guide is for Australian small-business owners, site operators, and sysadmins who need to trace SYN-flood DDoS pressure, clear half-open TCP handshake backlog, and lock down Linux network sockets from the terminal—without guessing.

When legitimate users cannot connect, SSH feels sluggish, or the kernel quietly fills SYN_RECV slots, you are often looking at a classic SYN flood or a mis-tuned TCP backlog—not a random “internet blip.” Below you will SSH in, run real diagnostics, edit /etc/sysctl.conf, verify syncookies, and know when remote help from Fixwebnode Linux server support is the safer path for production hosts in Australia.

Why SYN-flood tracing and socket hardening matter

A SYN flood abuses the three-way handshake: the attacker sends many SYN packets, your server replies SYN-ACK, and the final ACK never arrives. Connection slots stay in SYN_RECV, the backlog fills, and real clients time out. On busy Ubuntu or Debian VPS hosts this shows up as refused connections, stalled HTTPS, mail delays, and control-panel lockouts—even when CPU looks “fine.”

Fixwebnode provides direct remote server support for this exact failure mode: live socket inspection, sysctl hardening, backlog tuning, and firewall rate limits so your data path stays open. We work across Australian time zones as a specialist provider—not a freelance marketplace—so you speak with the same technical team that applies the fixes.

How do I know if a SYN flood is trapping my Linux server in SYN_RECV?

If new connections hang while existing sessions sometimes work, run socket-state checks over SSH. A sudden wall of SYN_RECV entries—especially from many source IPs on the same listening port—is the classic signal that half-open handshakes are consuming backlog slots and that you should enable tcp_syncookies and tighten backlog and firewall limits immediately.

Common issues Australian operators hit during SYN floods

• Backlog exhaustion in SYN_RECV — Botnet or spoofed SYNs pin allocation slots half-open; netstat/ss shows a long SYN_RECV list and new clients fail.
• tcp_syncookies disabled or never loaded — Default or drifted sysctl leaves the stack vulnerable; cookies are off in /proc even after a “security checklist.”
• Handshake errors from bad retries and queue sizes — tcp_max_syn_backlog, somaxconn, and SYN-ACK retries fight each other, so legitimate mobile users on Australian ISPs drop while attackers still burn slots.
• No edge rate limit on NEW TCP — Host-only tuning helps, but unrestricted NEW SYNs still hammer the NIC and conntrack until a firewall rule or upstream filter is applied.

Issue 1 — Discover botnet-style half-open states with netstat and ss

Goal: prove whether allocation slots are trapped in SYN_RECV before you change kernel parameters.

Step 1 — SSH to the host as a sudo-capable user

ssh -l youradmin your.server.example
sudo -s
cd /root

Use key-based auth if possible. Work from a stable out-of-band path (console/IPMI/provider VNC) if SSH itself is already flaky.

Step 2 — List half-open sockets the brief way

netstat -an | grep SYN_RECV
netstat -an | grep SYN_RECV | wc -l

A quiet server may show zero or a handful of lines. Hundreds or thousands on the same local port (for example :443) under load is a red flag.

Step 3 — Prefer ss for faster, clearer counts on modern kernels

ss -ant state syn-recv
ss -ant state syn-recv | head -n 50
ss -ant state syn-recv '( sport = :443 or sport = :80 )' | wc -l

Step 4 — Group by local port and remote noise

ss -ant state syn-recv | awk 'NR>1 {print $4}' | sed 's/.*://' | sort | uniq -c | sort -nr
ss -ant state syn-recv | awk 'NR>1 {print $5}' | sed 's/:[0-9]*$//' | sort | uniq -c | sort -nr | head

Many sources hitting one service port points to distributed flood behaviour. One loud IP may be a misbehaving scanner you can drop at the firewall.

Step 5 — Capture a short packet sample if safe

timeout 15 tcpdump -ni any 'tcp[tcpflags] & tcp-syn != 0 and tcp[tcpflags] & tcp-ack == 0' -c 200 -w /root/syn-sample.pcap
tcpdump -nn -r /root/syn-sample.pcap | head

You are confirming inbound SYN volume—not crafting attack traffic. Delete captures when finished if they contain customer IPs.

When to call Fixwebnode: SSH drops mid-diagnosis, counts climb while you watch, or you manage multiple production nodes and need a coordinated remote session.

Issue 2 — Enable clean tcp_syncookies via /etc/sysctl.conf

SYN cookies let the kernel complete valid handshakes without holding a full queue entry for every SYN. This is a standard defensive control on Linux servers facing public HTTP/S, mail, or API ports.

Step 1 — Read current runtime values

sysctl net.ipv4.tcp_syncookies
cat /proc/sys/net/ipv4/tcp_syncookies
sysctl net.ipv4.tcp_max_syn_backlog net.core.somaxconn net.ipv4.tcp_synack_retries

Expect tcp_syncookies = 1 on a hardened host. 0 means cookies are off.

Step 2 — Apply an immediate runtime mitigation

sysctl -w net.ipv4.tcp_syncookies=1
sysctl -w net.ipv4.tcp_max_syn_backlog=4096
sysctl -w net.core.somaxconn=4096
sysctl -w net.ipv4.tcp_synack_retries=3

These values are a sensible starting point for many small VPS workloads; tune upward only if you understand memory and legitimate burst traffic.

Step 3 — Persist secure parameters in /etc/sysctl.conf

cp -a /etc/sysctl.conf /etc/sysctl.conf.bak.$(date +%F)
grep -E 'tcp_syncookies|tcp_max_syn_backlog|somaxconn|tcp_synack_retries' /etc/sysctl.conf || true

Edit the file with a real editor:

nano /etc/sysctl.conf

Inject (or replace) a clean block similar to:

# SYN flood / half-open hardening
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 4096
net.core.somaxconn = 4096
net.ipv4.tcp_synack_retries = 3
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_rfc1337 = 1

Step 4 — Load and verify

sysctl -p
sysctl net.ipv4.tcp_syncookies net.ipv4.tcp_max_syn_backlog net.core.somaxconn
ss -ant state syn-recv | wc -l

After cookies and backlog are correct, SYN_RECV pressure should fall for the same attack intensity, and legitimate handshakes should complete more reliably.

Step 5 — Align listening application backlog if needed

Nginx, HAProxy, and some app servers have their own backlog settings. After raising somaxconn, restart the listener so it inherits the new queue ceiling:

systemctl restart nginx
# or: systemctl restart haproxy
ss -ltnp | egrep ':80|:443'

When to call Fixwebnode: sysctl -p errors, conflicting drop-in files under /etc/sysctl.d/, containers/hosts fighting over netns values, or you need cookie settings reviewed against a full hardening baseline.

Issue 3 — Resolve complex TCP handshake errors under connection floods

Symptoms: browsers spin, curl hangs at “Trying …”, mail submission stalls, and logs show repeated upstream timeouts even though disk and CPU are not pegged.

Step 1 — Separate flood from app failure

ss -s
ss -ant | awk 'NR>1 {print $1}' | sort | uniq -c | sort -nr
journalctl -u nginx -n 100 --no-pager
tail -n 100 /var/log/nginx/error.log

If ESTAB is low but SYN-RECV (or SYN-SENT on reverse proxies) is high, fix the network path first—not PHP workers.

Step 2 — Check conntrack saturation (common on NAT/firewall hosts)

sysctl net.netfilter.nf_conntrack_count net.netfilter.nf_conntrack_max 2>/dev/null || true
dmesg | tail -n 50 | grep -i conntrack || true

If count rides near max, raise carefully and reduce junk NEW flows at the firewall:

sysctl -w net.netfilter.nf_conntrack_max=262144
echo 'net.netfilter.nf_conntrack_max = 262144' >> /etc/sysctl.conf

Step 3 — Rate-limit new SYNs at the host firewall (nftables example)

nft list ruleset
# Example defensive limit on HTTPS (adjust interface/set policy to your host)
nft add table inet filter 2>/dev/null || true
nft 'add chain inet filter input { type filter hook input priority 0 ; }' 2>/dev/null || true
nft add rule inet filter input tcp dport 443 ct state new limit rate 50/second burst 100 accept

On iptables-legacy hosts, equivalent thinking uses the limit match on NEW SYNs. Always keep a working SSH accept rule above drop/limit policies, and test from a second session.

Step 4 — Re-test the handshake path

curl -I --max-time 10 https://127.0.0.1/
ss -ant state syn-recv | wc -l
sysctl net.ipv4.tcp_syncookies

Local success plus falling SYN_RECV counts means the stack is accepting clean handshakes again. If only off-box clients fail, involve upstream DDoS scrubbing or provider null-routing next.

When to call Fixwebnode: You are unsure which firewall stack is authoritative, risk locking yourself out, or the flood exceeds what host sysctl can absorb.

Issue 4 — Secure lingering sockets and listener exposure

After the flood eases, reduce permanent attack surface so the next wave has fewer open doors.

Step 1 — Inventory public listeners

ss -ltnp
ss -lunp

Step 2 — Close or bind unused services

systemctl disable --now useless-service
# Prefer bind to 127.0.0.1 for admin UIs when possible

Step 3 — Confirm SSH is not starved

ss -ant '( sport = :22 )'
grep -E '^(Port|MaxStartups|LoginGraceTime)' /etc/ssh/sshd_config

Reasonable MaxStartups protects sshd from connection storms without blocking your admin keys. Reload sshd only from a second live session after syntax check:

sshd -t && systemctl reload sshd

When DIY is enough vs when to book Fixwebnode

DIY is enough when you still have stable SSH, SYN_RECV counts are explainable, sysctl -p applies cleanly, and enabling tcp_syncookies plus modest backlog and firewall limits restores handshakes within minutes.

Book Fixwebnode when the host is already half-unreachable, changes must be made across several production servers, conntrack/firewall interactions are unclear, or you need a remote specialist to harden sockets while keeping customer traffic online. Fixwebnode is a direct Linux server support provider for operators across Australia—including remote incident work coordinated from our service areas—with hands-on sysctl, socket, and service recovery rather than generic advice.

Book remote help and keep traffic flowing

If a botnet is pinning your allocation slots in half-open states, do not wait for a full outage. Open a conversation with Fixwebnode for remote tracing, tcp_syncookies hardening, and socket recovery on Ubuntu/Debian and related Linux servers.

Start here: Linux server support and bug fixing with Fixwebnode. Bring your ss/netstat counts and a short timeline of symptoms—we will take it from the terminal with you.

Conclusion

SYN floods fail the TCP handshake on purpose: they leave sockets in SYN_RECV, exhaust backlog, and starve real users. You can prove it with netstat -an | grep SYN_RECV and ss, inject durable tcp_syncookies settings through /etc/sysctl.conf, align backlog and firewall limits, then re-verify handshake health. When pressure outruns safe DIY, Fixwebnode’s remote specialists keep Australian production hosts reachable without marketplace runaround.