Port Monitoring: When the Service Is Running but Nobody Can Connect

Two AM on a Wednesday. I get paged. "Users cannot log in." I check the web server. Nginx is running. I check the app. PHP-FPM is running. Health check endpoint returns 200. I check the database. PostgreSQL process is active. PID file is there. Logs look clean. Everything is "up." But nobody can log in.

Twenty minutes of debugging later, I find it. Someone pushed a firewall change at 11 PM. The new iptables rules dropped incoming connections on port 6379. Redis was running perfectly. It just could not receive any traffic. Every session lookup failed. Every login attempt got a connection refused from Redis, fell through to the error handler, and showed users a generic "something went wrong" page. The web server was fine. The app was fine. The database was fine. Redis was fine. The firewall between them was not.

This is the problem that port monitoring solves. Not "is the server alive." Not "does the homepage load." The specific question: can a client actually establish a TCP connection to this service on this port right now?

How a Port Check Actually Works

A port check is straightforward at the protocol level. Here is what happens:

1. The monitoring system sends a TCP SYN packet to the target host and port
2. If the port is open and a service is listening, the target responds with SYN-ACK
3. The monitoring system completes the three-way handshake with ACK
4. The connection time (latency of the handshake) is recorded
5. The connection is closed cleanly

If the connection times out, the port is filtered (firewall dropping packets). If the target responds with RST, the port is closed (no service listening). If the handshake completes but takes too long, the service is under stress. Each outcome tells you something different about what is happening.

The entire process takes milliseconds and puts almost zero load on the target. You can run port checks every 30 seconds on a production database server and it will never notice. This is not a load test. It is a handshake.

For a more detailed comparison with HTTP-level checks, read our article on HTTP vs TCP monitoring.

Why Your Ping Check and HTTP Check Are Not Enough

I run into this constantly. Teams have monitoring. Good monitoring, even. Ping checks on every server. HTTP checks on every web endpoint. Dashboard is green. And then a backend service dies and they find out from user complaints thirty minutes later.

The reason is simple. Ping checks test the network path. They tell you the machine is reachable at the IP level. They say nothing about what is running on that machine. A server can respond to ICMP echo requests while every service on it is dead. The kernel handles ping. Services are a different layer entirely.

HTTP checks test web endpoints. They verify that your nginx, Apache, or application server is responding on port 80 or 443. Great. But your infrastructure is more than a web server. Here is what a typical stack looks like:

Look at that list. The majority of services that your application depends on are invisible to both ping and HTTP monitoring. If PostgreSQL crashes, your web server keeps returning 200 on the health check until a request actually needs the database. By then, users are seeing errors.

Port monitoring fills this gap directly. It tests each service individually on its specific port. For more on the TCP monitoring approach, see our guide on TCP port monitoring.

The Firewall Stories

I want to spend time on this because it is the single most common cause of "service is running but unreachable" incidents in my experience. Firewall misconfigurations. And they happen in patterns.

The "Security Hardening" Incident

A security team reviews firewall rules. They see port 6379 open. "Redis should not be exposed." They close it. They are right that Redis should not be exposed to the internet. But they also closed it to the application servers on the internal network. The web app loses its session store, cache, and rate limiter all at once. The security team's change was pushed at 6 PM. Nobody noticed until 8 AM the next morning because the HTTP health check still returned 200.

Port monitoring on Redis from the application server's network would have caught this in under a minute.

The iptables Flush

An admin debugging a network issue runs iptables -F to flush all rules. Problem solved, connectivity restored. But the flush also removed the rules that allowed traffic to the database, the message queue, and the SMTP server. The default policy was DROP. Everything except SSH (which the admin was connected through) became unreachable.

I have seen this exact scenario at least four times in my career. The fix is always the same: restore the iptables rules from the saved configuration. The time to detection is the variable. Without port monitoring, it can be hours.

The Cloud Security Group Change

In AWS, Azure, or GCP, security groups are the firewall. Someone modifies a security group to tighten rules. They accidentally remove the inbound rule for port 5432 on the database instance. The database is running. It is healthy. It is accepting connections from localhost. But no application server in the VPC can reach it because the security group is blocking traffic before it even gets to the instance.

This is especially insidious because the database's own monitoring (if it has any) shows it as healthy. The process is running, queries are not timing out (because there are no queries), and memory usage is low. Everything looks perfect from the database's perspective. Only an external port check reveals the problem.

The UFW Reboot

Server runs Ubuntu with UFW (Uncomplicated Firewall). Admin adds a firewall rule with ufw allow but forgets to make it persistent. Or the rule was added to iptables directly, bypassing UFW. Server reboots for a kernel update. UFW reloads its saved rules. The non-persistent rule is gone. A port that was open for six months is suddenly closed.

Port monitoring catches this the moment the server comes back up after reboot.

Beyond Firewalls: Other Reasons Ports Stop Accepting Connections

Connection Limit Exhaustion

PostgreSQL has a max_connections setting. Default is 100. A traffic spike or a connection leak in your application exhausts all available connections. The port is open. The handshake completes. But the service immediately rejects the connection because it has no slots left. From a simple TCP check, this looks like the port is up. A more aggressive port check that monitors response time will catch the degradation.

Service Crash Without Process Manager

Elasticsearch throws an OutOfMemoryError. The JVM dies. The process is gone. If there is no systemd service configured for restart (or the restart limit has been hit), the port stays closed until someone manually starts the service. The host is reachable. Ping works. HTTP works (because nginx is fine). But port 9200 is dead.

SSL/TLS Certificate Expiry on Non-HTTP Ports

Your automated certificate renewal updates the cert for nginx on port 443. But the same certificate is also used by your SMTP server on port 587. The renewal process did not restart the SMTP service. Or the cert path changed and the SMTP config still points to the old location. Web works. Email does not. Users stop receiving password resets, order confirmations, and invoices. You find out days later when the support queue fills up.

Resource Exhaustion on the Host

The host runs out of file descriptors. Or the ephemeral port range is exhausted. Or the TCP backlog queue is full. The service process is running but the kernel cannot allocate new connections to it. These are OS-level issues that the application itself may not be aware of. Port monitoring detects them because the TCP handshake fails.

Which Ports You Should Be Monitoring

Every team's list is different, but here is a starting checklist based on what I have seen matter most in production incidents.

Always Monitor

• Port 443 (HTTPS) -- your public-facing web traffic
• Port 80 (HTTP) -- even if it only redirects, it should be responding
• Port 22 (SSH) -- if you lose SSH during an incident, you cannot fix anything remotely
• Your database port (5432, 3306, 27017) -- the most common cause of "app is up but broken" incidents
• Your cache port (6379, 11211) -- cache failures degrade everything

Monitor If You Run Them

• Port 587 or 465 (SMTP) -- outgoing email is often the last thing monitored and the first thing users notice when broken
• Port 5672 (RabbitMQ / AMQP) -- message queue failure creates silent backlogs
• Port 9200 (Elasticsearch) -- search breaks silently when Elasticsearch dies
• Port 53 (DNS) -- if you run your own nameservers
• Custom service ports (3000, 4000, 8080) -- microservices, internal APIs, admin panels
• Port 2049 (NFS) -- shared storage dependency

Setting Up Port Monitoring with UptyBots

Here is how I set up port monitoring for a new infrastructure. The process is the same whether you have five services or fifty.

Step 1: Build Your Port Inventory

Log into each server. Run ss -tlnp or netstat -tlnp. List every listening port and the service behind it. Note which ports are exposed externally and which are internal only. This inventory becomes your monitoring checklist. You will be surprised how many services you have running that nobody explicitly decided to monitor.

Step 2: Create Monitors with Clear Names

Name each monitor so that a 2 AM alert is immediately understandable. "Production DB - PostgreSQL 5432" is good. "db1-port" is not. You are writing these names for the worst version of yourself: tired, groggy, looking at your phone in the dark. Be explicit.

Step 3: Set Intervals by Criticality

• Critical (database, cache, main API): every 30 seconds to 1 minute
• Important (email, search, message queue): every 1 to 2 minutes
• Non-critical (FTP, internal tools): every 5 minutes

Step 4: Enable Multi-Location Checks

A port can be reachable from one network and blocked from another. Routing issues, regional firewall rules, ISP-level filtering. UptyBots checks from multiple geographic locations to catch these discrepancies. Read our multi-location monitoring case study to see this in action.

Step 5: Route Alerts to the Right People

Database port alerts go to the DBA. SMTP alerts go to the email team. SSH alerts go to the ops team. Generic "everything goes to everyone" alerting creates noise and fatigue. Be specific about routing. UptyBots supports email, Telegram, and webhook notifications. Read more about preventing alert fatigue.

Port Monitoring vs Ping Monitoring

These get confused often enough that it is worth spelling out the difference.

Use both. Ping tells you the machine is alive. Port checks tell you the services on it are alive. HTTP checks tell you the application is alive. Three layers, three different kinds of failures caught. See our article on HTTP vs TCP monitoring differences for the full breakdown.

Reading Port Monitoring Alerts Like an Ops Person

When a port alert fires, the details tell you more than "something is down." Here is how to read them.

1. Single port down, host up. The specific service crashed or was blocked. Check the process status and firewall rules for that port. Not a server-wide issue.
2. All ports down on one host. The machine itself is unreachable. Could be a crash, a network failure, or a hosting provider issue. Check ping monitoring for correlation.
3. One location reports down, others report up. Routing issue. Traffic from that geographic region cannot reach your server. Not a service problem. It is a network path problem.
4. Port up but response time spiking. Service is under stress. Connection pool exhaustion, high CPU, memory pressure. The crash is coming. Restart or scale before it arrives.
5. Port was up, went down, came back, went down again. The service is crash-looping. It starts, fails, restarts, fails again. Check application logs for the startup error.

Use the HTTP Status Explainer if you see HTTP error codes alongside your port alerts.

The Real Cost of Not Monitoring Ports

I will tell you exactly what happens. Your database port goes down at 2 PM. Your HTTP health check still passes because it checks a static endpoint that does not query the database. Your users start seeing errors on pages that need data. They try refreshing. They try clearing cache. Some of them email support. Support files a ticket. The ticket gets triaged. An engineer picks it up at 2:45 PM. They check the web server. Fine. They check the app logs. Connection refused to PostgreSQL. They check the database. Process is running. They check the firewall. Port 5432 is blocked.

Forty-five minutes from failure to diagnosis. If you had a port monitor on 5432, the alert would have fired at 2:01 PM. You would have seen "PostgreSQL 5432 - DOWN" on your phone. You would have checked the firewall first because the alert told you exactly which service was affected. Five minutes from alert to fix.

Use our Downtime Cost Calculator to put a dollar figure on that 40-minute gap. The numbers are usually enough to justify monitoring every port in your infrastructure. Read more about these hidden impacts in our article on the hidden costs of slow websites.

Scenarios from My Ops Career

The Redis firewall incident. Already told this one. Firewall change at 11 PM, sessions broke for everyone, found at 2 AM. Port monitoring would have cut detection from 3 hours to 1 minute.

The Elasticsearch OOM. Elasticsearch ran out of heap space indexing a large batch job. JVM crashed. Port 9200 stopped responding. The web app's search bar started returning errors, but everything else worked. Users reported "search is broken" through support chat. It took the team 25 minutes to trace it to Elasticsearch because they checked the web server and app first. A port monitor on 9200 would have pointed directly to the problem.

The SSL cert that only broke SMTP. Let's Encrypt renewed the web certificate. Postfix was using the same cert for STARTTLS on port 587. The renewal process reloaded nginx but did not reload Postfix. Web worked. Email failed. Password reset emails stopped going out. Nobody noticed for 18 hours because the web monitoring was all green. A port check on 587 would have detected the TLS handshake failure immediately.

The connection leak. A code change introduced a connection leak in the database pool. Under normal load, connections were acquired and never released. Over four hours, the 100 available PostgreSQL connections were exhausted. New connection attempts to port 5432 were refused. Port monitoring caught the degradation pattern: response time on port 5432 climbed steadily, then the port started refusing connections entirely.

Conclusion

Your HTTP monitoring checks the front door. Ping monitoring checks if the building exists. Port monitoring checks every room inside. Without all three, you have blind spots that will produce incidents you discover from user complaints instead of from alerts.

UptyBots makes port monitoring straightforward. Multi-location checks, configurable intervals, alert routing to the right people, and response time tracking that catches degradation before it becomes a crash. The setup takes five minutes per service. The payoff is measured in hours of downtime prevented and incidents caught before users notice.

See setup tutorials or get started with UptyBots monitoring today.