2

I am unable to populate substrate node metrics data in Grafana dashboard. I was following the instructions as shared in substrate doc Visualise Node Metrics I am able to establish the connection between Grafana and Prometheus after this fix I am importing the basic json template for metrics in grafana which is shared as part of the documentation above. I am reporting this issue to understand if everything provided in substrate doc is correct.enter image description here

4
  • too little detail, normally you may check it step by step. e.g: 1.query prometheus endpoint to see if data is ready, 2.check Grafana template config to see if metrics defined consistent with what in prometheus. 3... and so on.
    – Ron
    May 25, 2022 at 7:20
  • The last time I checked the grafana dashboard config was outdated. It was updated two years ago. It that the one you are using? May 25, 2022 at 9:55
  • @ron I know it's little detail, hard to detail being new to this. I have followed all the steps from doc. Also there are hardly any resource for new dev to refer on this topic. I have captured the screen presenting what all I am doing, I hope this helps you to understand my problem better. link . May 25, 2022 at 11:44
  • @oliver not sure how old is this stuff. I am using the Visualise Node Metrics doc for achieving this. But it does not work with said steps. I am not sure if all steps are complete there in doc and it's a challenge to troubleshoot as there is no other reference material to refer May 25, 2022 at 11:46

1 Answer 1

1

Use this one instead:

It works for me.

Here is the JSON for convenience:

{
  "annotations": {
    "list": [
      {
        "builtIn": 1,
        "datasource": {
          "type": "datasource",
          "uid": "grafana"
        },
        "enable": true,
        "hide": true,
        "iconColor": "rgba(0, 211, 255, 1)",
        "name": "Annotations & Alerts",
        "target": {
          "limit": 100,
          "matchAny": false,
          "tags": [],
          "type": "dashboard"
        },
        "type": "dashboard"
      }
    ]
  },
  "description": "Grafana dashboard for substrate template metrics. Instructions on use: https://substrate.dev/docs/en/tutorials/visualize-node-metrics/",
  "editable": true,
  "fiscalYearStartMonth": 0,
  "gnetId": 13759,
  "graphTooltip": 0,
  "id": 6,
  "links": [],
  "liveNow": false,
  "panels": [
    {
      "datasource": {
        "type": "prometheus",
        "uid": "tSoTyKbnk"
      },
      "description": "What your node sees as the highest block (NOT finalized)",
      "fieldConfig": {
        "defaults": {
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "super-light-yellow",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "id": 12,
      "options": {
        "colorMode": "value",
        "graphMode": "none",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "last"
          ],
          "fields": "/^Height\\(best\\)$/",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "8.5.3",
      "targets": [
        {
          "expr": "substrate_block_height{status=\"best\"}",
          "interval": "",
          "legendFormat": "Height(best)",
          "refId": "A"
        }
      ],
      "title": "Block Height (best)",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "tSoTyKbnk"
      },
      "description": "What your node sees as the highest finalized block ( less than or equal to best)",
      "fieldConfig": {
        "defaults": {
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 3,
        "w": 6,
        "x": 6,
        "y": 0
      },
      "id": 13,
      "options": {
        "colorMode": "value",
        "graphMode": "none",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "last"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "value"
      },
      "pluginVersion": "8.5.3",
      "targets": [
        {
          "expr": "substrate_block_height{status=\"finalized\"}",
          "interval": "",
          "legendFormat": "Height(finalized)",
          "refId": "A"
        }
      ],
      "title": "Block Height (finalized)",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "tSoTyKbnk"
      },
      "description": "Total transactions in best block",
      "fieldConfig": {
        "defaults": {
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "yellow",
                "value": 100
              }
            ]
          }
        },
        "overrides": []
      },
      "gridPos": {
        "h": 3,
        "w": 6,
        "x": 0,
        "y": 3
      },
      "id": 16,
      "options": {
        "colorMode": "value",
        "graphMode": "area",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "last"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "auto"
      },
      "pluginVersion": "8.5.3",
      "targets": [
        {
          "expr": "substrate_proposer_number_of_transactions",
          "interval": "",
          "legendFormat": "Height(best)",
          "refId": "A"
        }
      ],
      "title": "Block Total Tx",
      "type": "stat"
    },
    {
      "datasource": {
        "type": "prometheus",
        "uid": "tSoTyKbnk"
      },
      "description": "What your node sees as the highest finalized block ( less than or equal to best)",
      "fieldConfig": {
        "defaults": {
          "mappings": [],
          "thresholds": {
            "mode": "absolute",
            "steps": [
              {
                "color": "green",
                "value": null
              },
              {
                "color": "yellow",
                "value": 500000
              },
              {
                "color": "red",
                "value": 1000000
              }
            ]
          },
          "unit": "decbytes"
        },
        "overrides": []
      },
      "gridPos": {
        "h": 3,
        "w": 6,
        "x": 6,
        "y": 3
      },
      "id": 17,
      "options": {
        "colorMode": "value",
        "graphMode": "area",
        "justifyMode": "auto",
        "orientation": "auto",
        "reduceOptions": {
          "calcs": [
            "last"
          ],
          "fields": "",
          "values": false
        },
        "textMode": "value"
      },
      "pluginVersion": "8.5.3",
      "targets": [
        {
          "expr": "substrate_state_cache_bytes",
          "format": "time_series",
          "interval": "",
          "legendFormat": "Size (bytes)",
          "refId": "A"
        }
      ],
      "title": "State Cashe (bytes)",
      "type": "stat"
    },
    {
      "aliasColors": {},
      "bars": true,
      "dashLength": 10,
      "dashes": false,
      "datasource": {
        "type": "prometheus",
        "uid": "tSoTyKbnk"
      },
      "description": "Number of connected peers",
      "fieldConfig": {
        "defaults": {
          "links": []
        },
        "overrides": []
      },
      "fill": 1,
      "fillGradient": 0,
      "gridPos": {
        "h": 8,
        "w": 12,
        "x": 0,
        "y": 6
      },
      "hiddenSeries": false,
      "id": 10,
      "legend": {
        "alignAsTable": true,
        "avg": true,
        "current": true,
        "max": true,
        "min": true,
        "show": true,
        "total": false,
        "values": true
      },
      "lines": true,
      "linewidth": 1,
      "nullPointMode": "null",
      "options": {
        "alertThreshold": true
      },
      "percentage": false,
      "pluginVersion": "8.5.3",
      "pointradius": 2,
      "points": true,
      "renderer": "flot",
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": false,
      "steppedLine": false,
      "targets": [
        {
          "expr": "substrate_sub_libp2p_peers_count",
          "instant": false,
          "interval": "",
          "legendFormat": "Number of network gossip peers",
          "refId": "A"
        }
      ],
      "thresholds": [],
      "timeRegions": [],
      "title": "LibP2P peers",
      "tooltip": {
        "shared": true,
        "sort": 0,
        "value_type": "individual"
      },
      "type": "graph",
      "xaxis": {
        "mode": "time",
        "show": true,
        "values": []
      },
      "yaxes": [
        {
          "$$hashKey": "object:673",
          "decimals": 0,
          "format": "none",
          "label": "Peers",
          "logBase": 1,
          "min": "0",
          "show": true
        },
        {
          "$$hashKey": "object:674",
          "format": "short",
          "logBase": 1,
          "show": true
        }
      ],
      "yaxis": {
        "align": false
      }
    },
    {
      "aliasColors": {},
      "bars": false,
      "dashLength": 10,
      "dashes": false,
      "datasource": {
        "type": "prometheus",
        "uid": "tSoTyKbnk"
      },
      "description": "histogram 90th quantiles for time to construct a block (seconds) More info: https://prometheus.io/docs/prometheus/latest/querying/functions/#histogram_quantile",
      "fill": 1,
      "fillGradient": 8,
      "gridPos": {
        "h": 8,
        "w": 12,
        "x": 0,
        "y": 14
      },
      "hiddenSeries": false,
      "id": 15,
      "legend": {
        "avg": false,
        "current": false,
        "max": false,
        "min": false,
        "show": true,
        "total": false,
        "values": false
      },
      "lines": true,
      "linewidth": 1,
      "nullPointMode": "null",
      "options": {
        "alertThreshold": true
      },
      "percentage": false,
      "pluginVersion": "8.5.3",
      "pointradius": 2,
      "points": false,
      "renderer": "flot",
      "seriesOverrides": [],
      "spaceLength": 10,
      "stack": false,
      "steppedLine": false,
      "targets": [
        {
          "expr": "histogram_quantile(0.90, sum(rate(substrate_proposer_block_constructed_bucket[5m])) by (le))",
          "format": "time_series",
          "instant": false,
          "interval": "",
          "legendFormat": "90th quantile for block construction time (s)",
          "refId": "A"
        }
      ],
      "thresholds": [],
      "timeRegions": [],
      "title": "Proposer Block Construction Time",
      "tooltip": {
        "shared": true,
        "sort": 0,
        "value_type": "individual"
      },
      "type": "graph",
      "xaxis": {
        "mode": "time",
        "show": true,
        "values": []
      },
      "yaxes": [
        {
          "$$hashKey": "object:517",
          "format": "short",
          "label": "Time (s)",
          "logBase": 1,
          "show": true
        },
        {
          "$$hashKey": "object:518",
          "format": "short",
          "logBase": 1,
          "show": true
        }
      ],
      "yaxis": {
        "align": false
      }
    }
  ],
  "refresh": "",
  "schemaVersion": 36,
  "style": "dark",
  "tags": [
    "substrate",
    "blockchain",
    "node",
    "template",
    "polkadot",
    "kusama"
  ],
  "templating": {
    "list": []
  },
  "time": {
    "from": "now-5m",
    "to": "now"
  },
  "timepicker": {
    "refresh_intervals": [
      "5s",
      "10s",
      "30s",
      "1m",
      "5m",
      "15m",
      "30m",
      "1h",
      "2h",
      "1d"
    ]
  },
  "timezone": "",
  "title": "Substrate Node Template Metrics",
  "uid": "PUYzGbwWa",
  "version": 4,
  "weekStart": ""
}
1
  • 1
    Hi thanks , I tried this json template it did work for me. May 31, 2022 at 15:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.