Linux Disk Full? Safely Clean System Logs Without Downtime

This guide is for site owners and small-business operators whose Linux VPS or dedicated server reports “No space left on device,” 5xx errors, or failed deploys—and who need to free disk safely without crashing the live site.

When root or /var fills up, Linux often stops writing sessions, caches, and application data. Websites stall even though CPU looks fine. Below is a practical remote runbook: diagnose usage, clean system and web logs the right way, verify services stay up, and know when to hand off. If you need hands-on server support from Fixwebnode, we work remotely with Geelong and wider Australian teams on this exact failure mode.

Why a full disk from logs matters for your live site

Log growth is quiet until it is not. A single verbose access log, an unvacuumed systemd journal, or unrotated application logs can eat tens of gigabytes overnight after a bot wave or a misconfigured debug flag. Once the filesystem hits 100%, package updates fail, SSL renewals fail, PHP sessions stop writing, and databases may refuse to start. Cleaning logs is normal maintenance—but truncating the wrong file handle or deleting an active binary can restart services hard or leave processes writing to deleted inodes until reboot.

Fixwebnode treats this as production server support: free space first, keep nginx/Apache and PHP-FPM running, then fix rotation so the outage does not repeat. Coverage and remote help are organised across our service areas; the steps below are what we run on typical Ubuntu/Debian and RHEL-family web hosts.

Why is my Linux server disk full from logs, and how do I clean them safely?

Most “disk full” web outages come from oversized systemd journals, web-server access/error logs, or application/database logs that never rotated—not from your HTML files. Free space by measuring /var first, vacuuming or truncating active logs (never blindly rm open files), confirming nginx/PHP-FPM still respond, then fixing logrotate so growth stops. Call a specialist if root is 100% full, the host will not boot cleanly, or you cannot risk touching production logs yourself.

Common issues when Linux server disk is full from logs

These are distinct root causes we see on production web servers. Match your symptoms before you delete anything.

• Systemd journal unbounded growth — /var/log/journal or persistent journal uses most of the disk; journalctl is slow; df shows /var full while website files are small.
• Web server access/error logs never rotated — single access.log or error.log under /var/log/nginx or /var/log/apache2 is many GB; deploys and Let’s Encrypt renewals fail with “No space left on device.”
• Application, PHP, or database logs without retention — Laravel/PHP debug logs, MySQL/MariaDB binary logs, or container json-logs under /var/lib/docker grow until the site cannot write sessions or the DB cannot extend files.
• Deleted-but-still-open log files — someone ran rm on a live log; df still shows full space until the process is restarted; lsof shows (deleted) entries.

Before you change anything: diagnose safely

Work over SSH as a sudo-capable user. Do not reboot first—reboots with 0% free space can fail mid-fsck or block services from starting.

Step 1 — Confirm which mount is full

df -hT
df -hi

Note whether the problem is block space (Use% near 100%) or inodes (IUse%). Log explosions are usually block space on / or /var.

Step 2 — Find the largest directories

sudo du -xhd1 /var 2>/dev/null | sort -h
sudo du -xhd1 /var/log 2>/dev/null | sort -h
sudo du -xhd1 /var/lib 2>/dev/null | sort -h

Step 3 — List the biggest files under /var/log

sudo find /var/log -type f -size +100M -exec ls -lh {} \; 2>/dev/null | sort -k5 -h

Step 4 — Check space held by deleted open files

sudo lsof +L1 2>/dev/null | grep -E 'log|deleted' | head -n 40

If large (deleted) files appear, truncating or restarting the holding process (carefully) is required—plain rm already happened and did not free blocks.

Issue 1 — Systemd journal filled the disk

Symptoms: /var/log/journal or journal disk usage is multiple GB; journalctl crawls; little else under the docroot is large.

Step 1 — Measure journal size

sudo journalctl --disk-usage

Step 2 — Vacuum to a safe cap (frees space immediately)

sudo journalctl --vacuum-size=200M
# or keep roughly two days:
sudo journalctl --vacuum-time=2d

Step 3 — Make the limit permanent

sudo mkdir -p /etc/systemd/journald.conf.d
sudo tee /etc/systemd/journald.conf.d/size.conf <<'EOF'
[Journal]
SystemMaxUse=200M
RuntimeMaxUse=100M
MaxRetentionSec=7day
EOF
sudo systemctl restart systemd-journald

Step 4 — Verify

sudo journalctl --disk-usage
df -h /var

Restarting systemd-journald does not restart nginx or PHP-FPM. Your site should keep serving. If vacuum errors or the journal directory is on a separate broken mount, stop and get remote help rather than deleting /var/log/journal by hand.

Issue 2 — Nginx or Apache access/error logs ballooned

Symptoms: Multi‑GB files in /var/log/nginx or /var/log/apache2; bot traffic or a missing logrotate stanza; panel “disk full” alerts while the site still partially loads static files.

Step 1 — Identify the hot files

sudo ls -lhS /var/log/nginx/ 2>/dev/null
sudo ls -lhS /var/log/apache2/ 2>/dev/null
sudo ls -lhS /var/log/httpd/ 2>/dev/null

Step 2 — Truncate active logs in place (preferred over rm)

# Nginx example — keeps the same inode so workers keep writing safely
sudo truncate -s 0 /var/log/nginx/access.log
sudo truncate -s 0 /var/log/nginx/error.log

# Apache example
sudo truncate -s 0 /var/log/apache2/access.log
sudo truncate -s 0 /var/log/apache2/error.log

Truncation frees space without removing the path the daemon has open. Avoid rm on live logs unless you will reload/restart the service immediately afterward.

Step 3 — Clear old rotated archives you no longer need

sudo find /var/log/nginx -type f \( -name '*.gz' -o -name '*.log.[0-9]*' \) -mtime +7 -print
# After reviewing the list:
sudo find /var/log/nginx -type f \( -name '*.gz' -o -name '*.log.[0-9]*' \) -mtime +14 -delete

Step 4 — Force a healthy rotate and confirm the site

sudo logrotate -f /etc/logrotate.d/nginx 2>/dev/null || sudo logrotate -f /etc/logrotate.d/apache2 2>/dev/null
sudo nginx -t && sudo systemctl reload nginx
# or:
# sudo apachectl configtest && sudo systemctl reload apache2
curl -sI https://127.0.0.1/ | head -n 5
df -h /var

Step 5 — Ensure logrotate exists and compresses

sudo cat /etc/logrotate.d/nginx 2>/dev/null || sudo cat /etc/logrotate.d/apache2 2>/dev/null

You want daily or weekly rotation, missingok, compress, delaycompress, and a postrotate that reloads the web server. If the file is missing, recreate it before the next traffic spike.

Issue 3 — Application, PHP, or database logs without retention

Symptoms: /var/www, /home, /var/lib/mysql, or /var/lib/docker dominates du; site throws 500s when writing logs or sessions; MySQL error log or binlog.* files are huge; Docker’s *-json.log files grow without cap.

Step 1 — Pinpoint the consumer

sudo du -xhd1 /var/www /home /var/lib/mysql /var/lib/docker 2>/dev/null | sort -h
sudo find /var/www /home -type f \( -name '*.log' -o -name 'laravel.log' -o -name 'debug.log' \) -size +50M -exec ls -lh {} \; 2>/dev/null

Step 2 — Application logs (example: Laravel / generic app log)

# Inspect then truncate a known app log only — adjust path to yours
sudo ls -lh /var/www/html/storage/logs/ 2>/dev/null
sudo truncate -s 0 /var/www/html/storage/logs/laravel.log
# Fix ownership if the app user must append again
sudo chown www-data:www-data /var/www/html/storage/logs/laravel.log 2>/dev/null

Turn off APP_DEBUG=true in production after cleanup so the file does not explode again.

Step 3 — PHP-FPM / PHP slow and error logs

sudo ls -lhS /var/log/php* /var/log/php-fpm* 2>/dev/null
sudo truncate -s 0 /var/log/php8.2-fpm.log 2>/dev/null
sudo systemctl status php*-fpm --no-pager
# Reload only if needed after config changes:
# sudo systemctl reload php8.2-fpm

Step 4 — MySQL/MariaDB binary logs (only if binlogs are the culprit)

sudo du -sh /var/lib/mysql/*bin* 2>/dev/null
sudo mysql -e "SHOW BINARY LOGS;" 2>/dev/null

# Purge binary logs older than 3 days — requires DB access; do not delete files with rm
sudo mysql -e "PURGE BINARY LOGS BEFORE DATE(NOW() - INTERVAL 3 DAY);"

If you are unsure whether replicas depend on those binlogs, skip purge and escalate.

Step 5 — Docker container json-logs (if Docker is installed)

sudo du -sh /var/lib/docker/containers/*/*-json.log 2>/dev/null | sort -h | tail
# Truncate a specific huge json log (replace CONTAINER_ID)
# sudo truncate -s 0 /var/lib/docker/containers/CONTAINER_ID/CONTAINER_ID-json.log

sudo tee /etc/docker/daemon.json <<'EOF'
{
 "log-driver": "json-file",
 "log-opts": {
 "max-size": "50m",
 "max-file": "3"
 }
}
EOF
# Applying daemon.json needs a Docker restart — schedule a short window:
# sudo systemctl restart docker

Step 6 — Verify the site stack still answers

df -h /
sudo systemctl is-active nginx apache2 php*-fpm mysql mariadb 2>/dev/null
curl -sI --max-time 10 http://127.0.0.1/ | head -n 8
sudo tail -n 20 /var/log/nginx/error.log 2>/dev/null

Issue 4 — Space not freed after deleting logs (open deleted files)

Symptoms: You already deleted logs; du looks fine but df still shows 100% full.

Step 1 — Find the process holding deleted files

sudo lsof +L1 2>/dev/null | sort -k7 -n | tail -n 30

Step 2 — Prefer truncate via proc fd when safe

# Example pattern — replace PID and fd from lsof output
# sudo : > /proc/PID/fd/FD_NUMBER

Step 3 — Or recycle the service cleanly (short, planned reload)

sudo systemctl reload nginx
# If reload is not enough for that worker:
# sudo systemctl reload php8.2-fpm
df -h /

Reload is gentler than restart and usually drops deleted file handles without a hard outage. Full restarts belong in a maintenance window if the process will not release space.

When DIY is enough vs when to book Fixwebnode

DIY is reasonable when: you still have SSH, a few hundred MB free (or can free journal/web logs quickly), you know which paths are logs, and you can truncate and reload without changing database retention you do not understand.

Book Fixwebnode remote server support when:

• Root is completely full and even journalctl --vacuum or package tools fail.
• The site is already down, PHP-FPM/nginx will not start, or MySQL will not open tables.
• Growth returns within hours (active abuse, broken vhost, runaway debug, or missing logrotate across many sites).
• You host multiple client sites on one box and cannot risk truncating the wrong tree.
• Deleted open files, container layers, or binlogs need coordinated restarts.

Fixwebnode is a direct specialist provider for Linux server administration—not a freelance marketplace. The same remote runbooks we use for production hosts are reflected in regional Linux admin offerings such as Providence Linux administration & support and Austin Linux administration & support, and we apply the same discipline for Geelong-based businesses that need disk recovery without guesswork.

Keep logs from filling the disk again

After emergency cleanup, lock in retention:

1. Cap journald with SystemMaxUse as shown above.
2. Confirm every vhost and app writes under a path covered by logrotate.
3. Disable production debug flags and noisy plugin logging.
4. Add a simple weekly check:

df -h / /var | awk 'NR==1 || /\//'
sudo du -xhd1 /var/log | sort -h | tail -n 15

Optional alert when usage exceeds 85% via your existing monitoring agent or a cron that emails df output. Prevention is cheaper than emergency truncation during peak traffic.

Get remote help before the next outage

If your Linux server in the Geelong area—or any remote VPS your team manages—is stuck at 100% disk from journals, nginx logs, or application noise, do not keep deleting files at random. Fixwebnode can step in remotely, free space safely, verify nginx/PHP-FPM/database health, and leave rotation in a known-good state.

Start a conversation or book practical server support when you want a specialist on the session with you. Bring SSH access and a short note of what df -h and du -xhd1 /var/log already showed—we will take it from the live symptoms, not a generic script.