Amazon Cloudwatch only receiving mem_used_percent and nothing else, despite numerous other metrics specified in config

Question

I am trying to get CloudWatch running properly on my Lightsail instance, which I appear to achieved with only partial success.

I have ran the Wizard using sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard which has produced a config file outlining numerous metrics including cpu, memory and disk usage as outlined here. The service loads and starts the config file, and doesn't complain about invalid json (this did happen a few times, but I fixed it).

I can stop the service with sudo amazon-cloudwatch-agent-ctl -a stop

I then reload the config with sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -s -m ec2 -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json

Verify the service is running: sudo amazon-cloudwatch-agent-ctl -a status

Which outputs this:

 {
  "status": "running",
  "starttime": "2022-01-10T21:53:12+00:00",
  "configstatus": "configured",
  "cwoc_status": "stopped",
  "cwoc_starttime": "",
  "cwoc_configstatus": "not configured",
  "version": "1.247349.0b251399"
}

Logging into my CloudWatch console, I can see the data being received, and the single line appearing on the graph there corresponds to the times that I started and stopped the service-- so it's definitely doing something. And yet... the only metric that appears on that graph is mem_used_percent... why? Why only this one metric? Where is the rest of my data pertaining to cpu, etc? What am I doing wrong?

Here is my config.json, which as I said, is being loaded by the service without issue.

{
    "agent": {
        "metrics_collection_interval": 60,
        "run_as_user": "root"
    },
    "metrics": {
        "append_dimensions": {
            "ImageID": "${aws:ImageId}",
            "InstanceId":"${aws:InstanceId}",
            "InstanceType":"${aws:InstanceType}"
        },
        "metrics_collected": {
            "cpu": {
                "resources": [
                   "*"
                ],
                "measurement": [
                    "cpu_usage_active"
                ],
                "metrics_collection_interval": 60,
                "totalcpu": false
            },
            "disk": {
                "measurement": [
                        "free",
                        "total",
                        "used",
                        "used_percent"
                ],
                "metrics_collection_interval": 60,
                "resources": [
                        "*"
                ]
            },
            "mem": {
                "measurement": [
                        "mem_active",
                        "mem_available",
                        "mem_available_percent",
                        "mem_free",
                        "mem_total",
                        "mem_used",
                        "mem_used_percent"
                ],
                "metrics_collection_interval": 60
            },
            "netstat": {
                "measurement": [
                        "tcp_established",
                        "udp_socket"
                ]
            }
        }
    }
}

Any help greatly appreciated here. TIA.

Answer 1

You likely haven't fetched the configuration yet. Check the logfile, i.e. /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log, to see which inputs are loaded:

2022-05-18T10:18:57Z I! Loaded inputs: mem disk

To fetch the configuration, do as follows (you'll need to adapt this to your environment - this is for systemd, on-premise, without SSM):

sudo amazon-cloudwatch-agent-ctl -a fetch-config -m onPremise -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json
sudo systemctl status amazon-cloudwatch-agent.service restart

After:

2022-05-18T11:45:05Z I! Loaded inputs: mem net netstat swap cpu disk diskio