Monitoring Setup with Grafana and Prometheus

This is the definitive “Remote Monitoring” guide. You will have two separate machines communicating with each other.

The Architecture

• Server A (Target): Runs your apps + lightweight “Exporters” (agents) that expose metrics.

• Server B (Monitoring): Runs Prometheus (database) & Grafana (dashboard). It reaches out to Server A to “scrape” data.

Step 1: Prepare the Target Server (Your App Server)

Do this on the server running your actual applications.

1. Create a folder: mkdir -p ~/monitoring-agent

2. Create a docker-compose.yml file inside it:

YAML

version: '3.8'

services:
  node_exporter:
    image: quay.io/prometheus/node-exporter:latest
    container_name: node_exporter
    command:
      - '--path.rootfs=/host'
    pid: host
    restart: unless-stopped
    volumes:
      - '/:/host:ro,rslave'
    ports:
      - "9100:9100" # Important: Expose this so Server B can reach it

  cadvisor:
    image: gcr.io/cadvisor/cadvisor:latest
    container_name: cadvisor
    restart: unless-stopped
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro
      - /dev/disk/:/dev/disk:ro
    ports:
      - "8080:8080" # Important: Expose this so Server B can reach it
    devices:
      - /dev/kmsg

3. Run it:

   Bash

   docker compose up -d
   

4. Firewall Check: Ensure your cloud provider (AWS/DigitalOcean/Hetzner) firewall allows Incoming Traffic on ports 9100 and 8080 ONLY from the IP address of Server B (your Monitoring server). Do not open these to the whole world.

Step 2: Prepare the Monitoring Server (Your 4GB VPS)

Do this on your separate monitoring machine.

1. Create a folder structure:

   Bash

   mkdir -p ~/monitoring/prometheus
   cd ~/monitoring
   

2. Create the prometheus/prometheus.yml configuration file.

   Replace TARGET_SERVER_IP with the actual IP address of Server A.

YAML

global:
  scrape_interval: 15s # How often to check for data

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']

  - job_name: 'remote_server_A'
    static_configs:
      - targets: ['TARGET_SERVER_IP:9100'] # Node Exporter (Host stats)
        labels:
          alias: 'App-Server-Main'

  - job_name: 'remote_server_A_containers'
    static_configs:
      - targets: ['TARGET_SERVER_IP:8080'] # cAdvisor (Container stats)
        labels:
          alias: 'App-Server-Containers'

3. Create the docker-compose.yml file in ~/monitoring/:

YAML

version: '3.8'

services:
  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    restart: unless-stopped
    volumes:
      - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus_data:/prometheus
    ports:
      - "9090:9090"

  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    restart: unless-stopped
    volumes:
      - grafana_data:/var/lib/grafana
    ports:
      - "3000:3000"

volumes:
  prometheus_data:
  grafana_data:

4. Run it:

   Bash

   docker compose up -d
   

Step 3: Setup the Dashboards

Now the data is flowing, you need to visualize it.

1. Login: Open http://YOUR_MONITORING_SERVER_IP:3000.

   • Default login: admin / admin.

2. Connect Data Source:

   • Go to Connections (or Configuration) -> Data Sources -> Add data source.

   • Select Prometheus.

   • URL: http://prometheus:9090 (Keep exactly this; Docker handles the internal DNS).

   • Click “Save & Test”. You should see “Successfully queried the Prometheus API”.

3. Import Dashboards:

   • Go to Dashboards -> New -> Import.

   • We will use standard community IDs (no need to build your own):

     • Host Stats (Node Exporter): Enter ID 1860 and click Load. Select your Prometheus data source and click Import.

     • Container Stats (cAdvisor): Enter ID 14282 and click Load.

       • Note: If 14282 doesn’t work well for you, try ID 193.

Verification

You should now see beautiful graphs on your Monitoring Server showing the CPU/RAM usage of your Target Server.

Important Security Note:

Since you are communicating over the public internet, your metrics are technically visible to anyone who guesses the IP and port if you didn’t set up the firewall in Step 1.4.

• Best Practice: Use a VPN (like Tailscale or WireGuard) to connect the two servers privately.

• Minimum Requirement: Configure ufw or AWS Security Groups to allow traffic on ports 9100/8080 only from your Monitoring Server’s specific IP.