How to Detect Dead Daemons Before Your Uptime Check Notices

Your uptime monitor shows green right up until the endpoint stops answering. That is the problem with outside-in checks: they only see the symptom. The daemon that serves your API can die, get OOM-killed, or enter a systemd restart loop minutes — sometimes hours — before the last retry finally fails and your phone goes off.

This guide covers how host-level monitoring closes that gap: watching the processes, units and kernel events that die before the endpoint does.

Why external probes miss dead daemons

An HTTP check answers one question: "does the service respond?" It says nothing about why it responds slowly, flaps, or is about to stop. In practice, outages announce themselves at the host level first:

• systemd restart loops — the unit keeps crashing and restarting; the port still answers during uptime windows, so the probe sees a green check while the service is already half dead.
• OOM kills — the kernel reaper picks your worker; memory pressure usually builds for a while before that.
• Zombie/stuck processes — the process exists but stopped doing work. A TCP connect still succeeds; the response just never arrives.
• Cascading dependencies — your app is fine, but the local Redis or sidecar it needs is not.

If your monitoring only does external probes, you learn about all of these at the worst possible moment: when the public endpoint fails and customers are already affected.

What a host agent can actually see

An agent running on the host has a fundamentally different vantage point. Instead of inferring health from responses, it reads the signals directly:

• Process checks — is postgres, nginx, node server.js or your own binary actually running? Not "does the port answer" — does the process exist.
• systemd unit state — active, failed, activating, restart counts. A unit in a crash loop is a detection you get minutes before the outage.
• Kernel events — OOM kills surface in the kernel log; an agent watching it tells you which process the kernel sacrificed and when.
• Listening ports — expected listeners that disappear, unexpected ones that appear. The second category doubles as a security signal.
• Log anomalies — error patterns in journald or files: exception spikes, connection refusals, disk-full warnings.
• Resource pressure — memory headroom shrinking toward the OOM threshold, disk filling toward the point where your database stops writing.

The difference in practice: your external monitor tells you "the site is down". The agent tells you "the payments service restarted 14 times in the last hour and memory is at 94%" — while the site still looks fine.

Setting it up with Uptriva

Uptriva's agent is a single self-updating binary. The short version:

1. Install the agent on the host (one command, systemd service). It registers against your account and starts reporting heartbeats.
2. Create a service monitor for the daemon you care about — by process name or systemd unit. The agent reports state continuously; a dead process is a down monitor.
3. Add event rules for the leading indicators: OOM kills, port changes, log error patterns, SSH login anomalies.
4. Route it into your escalation chain — agent events go through the same contacts, channels and escalation levels as external monitors.

Because agent and external checks live in the same system, a single status page can show both: public endpoints your customers care about and the internal services that feed them.

The honest caveat

Agents see inside, but only on hosts where they run. For managed services you cannot touch (hosted databases, third-party APIs), external probes remain your only option — which is exactly why the two belong together rather than one replacing the other.

The agent is open source, self-updates, and runs on Linux (amd64/arm64). Setup takes about two minutes: Uptriva agent docs.