Export and visualize Netdata metrics in Graphite

Collecting metrics is an essential part of monitoring any application, service, or infrastructure, but it's not the final step for any developer, sysadmin, SRE, or DevOps engineer who's keeping an eye on things. To take meaningful action on these metrics, you may need to develop a stack of monitoring tools that work in parallel to help you diagnose anomalies and discover root causes faster.

We designed Netdata with interoperability in mind. The Agent collects thousands of metrics every second, and then what you do with them is up to you. You can store metrics in the database engine, or send them to another time series database for long-term storage or further analysis using Netdata's exporting engine.

In this guide, we'll show you how to export Netdata metrics to Graphite for long-term storage and further analysis. Graphite is a free open-source software (FOSS) tool that collects graphs numeric time-series data, such as all the metrics collected by the Netdata Agent itself. Using Netdata and Graphite together, you get more visibility into the health and performance of your entire infrastructure.

Let's get started.

Install the Netdata Agent

If you don't have the Netdata Agent installed already, visit the installation guide for the recommended instructions for your system. In most cases, you can use the one-line installation script:

wget -O /tmp/netdata-kickstart.sh https://my-netdata.io/kickstart.sh && sh /tmp/netdata-kickstart.sh

Do you want automatic updates? default: enabled

Do you want nightly or stable releases? default: nightly

Do you want to contribute anonymous statistics? default: enabled

Once installation finishes, open your browser and navigate to http://NODE:19999, replacing NODE with the IP address or hostname of your system, to find the Agent dashboard.

Install Graphite via Docker

For this guide, we'll install Graphite using Docker. See the Docker documentation for details if you don't yet have it installed on your system.

If you already have Graphite installed, skip this step. If you want to install via a different method, see the Graphite installation docs, with the caveat that some configuration settings may be different.

Start up the Graphite image with docker run.

docker run -d \
  --name graphite \
  --restart=always \
  -p 80:80 \
  -p 2003-2004:2003-2004 \
  -p 2023-2024:2023-2024 \
  -p 8125:8125/udp \
  -p 8126:8126 \
  graphiteapp/graphite-statsd

Open your browser and navigate to http://NODE, to see the Graphite interface. Nothing yet, but we'll fix that soon enough.

Enable the Graphite exporting connector

You're now ready to begin exporting Netdata metrics to Graphite.

Begin by using edit-config to open the exporting.conf file.

cd /etc/netdata   # Replace this path with your Netdata config directory
sudo ./edit-config exporting.conf

If you haven't already, enable the exporting engine by setting enabled to yes in the [exporting:global] section.

[exporting:global]
    enabled = yes

Next, configure the connector. Find the [graphite:my_graphite_instance] example section and uncomment the line. Replace my_graphite_instance with a name of your choice. Let's go with [graphite:netdata]. Set enabled to yes and uncomment the line. Your configuration should now look like this:

[graphite:netdata]
    enabled = yes
    # destination = localhost
    # data source = average
    # prefix = netdata
    # hostname = my_hostname
    # update every = 10
    # buffer on failures = 10
    # timeout ms = 20000
    # send names instead of ids = yes
    # send charts matching = *
    # send hosts matching = localhost *

Set the destination setting to localhost:2003. By default, the Docker image for Graphite listens on port 2003 for incoming metrics. If you installed Graphite a different way, or tweaked the docker run command, you may need to change the port accordingly.

[graphite:netdata]
    enabled = yes
    destination = localhost:2003
    ...

We'll not worry about the rest of the settings for now. Restart the Agent using sudo systemctl restart netdata, or the appropriate method for your system, to spin up the exporting engine.

See and organize Netdata metrics in Graphite

Head back to the Graphite interface again, then click on the Dashboard link to get started with Netdata's exported metrics. You can also navigate directly to http://NODE/dashboard.

Let's switch the interface to help you understand which metrics Netdata is exporting to Graphite. Click on Dashboard and Configure UI, then choose the Tree option. Refresh your browser to change the UI.

You should now see a tree of available contexts, including one that matches the hostname of the Agent exporting metrics. In this example, the Agent's hostname is arcturus.

Let's add some system CPU charts so you can monitor the long-term health of your system. Click through the tree to find hostname → system → cpu metrics, then click on the user context. A chart with metrics from that context appears in the dashboard. Add a few other system CPU charts to flesh things out.

Next, let's combine one or two of these charts. Click and drag one chart onto the other, and wait until the green Drop to merge dialog appears. Release to merge the charts.

Finally, save your dashboard. Click Dashboard, then Save As, then choose a name. Your dashboard is now saved.

Of course, this is just the beginning of the customization you can do with Graphite. You can change the time range, share your dashboard with others, or use the composer to customize the size and appearance of specific charts. Learn more about adding, modifying, and combining graphs in the Graphite docs.

Monitor the exporting engine

As soon as the exporting engine begins, Netdata begins reporting metrics about the system's health and performance.

You can use these charts to verify that Netdata is properly exporting metrics to Graphite. You can even add these exporting charts to your Graphite dashboard!

Add exporting charts to Netdata Cloud

You can also show these exporting engine metrics on Netdata Cloud. If you don't have an account already, go sign in and get started for free. If you need some help along the way, read the get started with Cloud guide.

Add more metrics to a War Room's Nodes view by clicking on the Add metric button, then typing exporting into the context field. Choose the exporting contexts you want to add, then click Add. You'll see these charts alongside any others you've customized in Netdata Cloud.

What's next?

What you do with your exported metrics is entirely up to you, but as you might have seen in the Graphite connector configuration block, there are many other ways to tweak and customize which metrics you export to Graphite and how often.

For full details about each configuration option and what it does, see the exporting reference guide.