Deploy Grafana Dashboard via ArgoCD

In the DevOps era, automation and version control are becoming key elements of infrastructure management. Effective deployment of monitoring solutions such as Grafana and Prometheus is vital to respond to system changes in a timely manner. In this article, we will take a closer look at the Monitoring as Code (MaC) approach, which allows monitoring to be managed using configuration files, providing flexibility and reliability.

How Monitoring as Code Works with Grafana

Dashboard JSON files: In Grafana, dashboards can be exported in JSON format. This JSON code describes the dashboard structure, panels, metrics, data sources, and other settings. On Mac, this JSON code is stored in a Git repository.

Configuration files: In addition to dashboards, you can also describe data source configurations, alerts, and other aspects of monitoring as code. This can be done using YAML or JSON files that describe how Grafana and other monitoring tools should be configured.

GitOps Workflow: GitOps is a method for managing infrastructure and applications using Git as the source of truth. In GitOps, you store all configurations and manifests in a Git repository and use automated tools (like ArgoCD) to synchronize changes across a cluster or environment.

Deploy via GitOps: When you add or update dashboard and monitoring configuration files in your Git repository, ArgoCD or another GitOps tool automatically applies these changes to your environment. This could include updating ConfigMap in Kubernetes, deploying new dashboards in Grafana, configuring data sources, and so on.

Why is it best to deploy dashboards via GitOps

Sources of Truth: By storing monitoring configurations in Git, you create a single source of truth. This allows you to track changes, see the history of changes, and understand how and why configurations changed.

Versioning: Git automatically versions all configuration changes. This makes it easy to roll back to previous versions in case of problems or need to analyze changes.

Automated deployment: With GitOps tools like ArgoCD, you can automate the process of deploying and updating dashboards. This reduces the chance of human error and speeds up the process of applying changes.

Consistency: GitOps ensures that changes made to the configuration are automatically applied to all environments, be it testing, staging, or production. This helps maintain consistency across environments.

Feedback: Integration with CI/CD systems allows for integrated testing and validation of monitoring configurations. This provides feedback on how changes impact monitoring and dashboards before they are deployed.

Access control: Git provides access control and auditing mechanisms. You can use Git permissions to control who can change configurations, as well as track who made changes and when.

Configuration Management: Managing monitoring as code simplifies the management of complex configurations. This allows you to centrally manage and maintain monitoring at scale.

Scalability: As your infrastructure grows, you can easily scale your monitoring configurations using the same GitOps principles. Updates can be automatically applied to all new components or services.

Let me tell you how I did it, because under the NDA I can't show the configs and screenshots themselves, I'll tell you how I did it without these details

Let's start with the diagram

Creating a repository

• I have created or selected a Git repository to store the configurations in. Typically this is a separate repository for configuration management or an existing repository for infrastructure.

2. Adding a JSON file to the repository

   • I saved the dashboard JSON code to a file, let's say the modified name is just grafana-dashboard.json, and added it to my Git repository. This is done so that all changes can be tracked and versioned.

Creating a YAML Manifest for a ConfigMap in Kubernetes

1. Creating a ConfigMap Manifest

   • I created a YAML file for the ConfigMap that will contain the dashboard JSON code. In this file, I described the ConfigMap with its name and JSON content. Here is an example of the manifest:

apiVersion: v1
kind: ConfigMap
metadata:
  name: grafana-dashboard
  namespace: <YOUR-NAMESPACE>
  labels:
    grafana_dashboard: "1"
data:
  dashboard.json: |
    { "dashboard": { ... } }

2. Syntax check

   • I checked the YAML file for syntax errors and made sure the JSON code was correct and matched the structure required by Grafana.

3. Why is that: ConfigMap in Kubernetes allows you to store configuration data, such as dashboard JSON, separately from the applications themselves. This simplifies configuration and update management.

Adding a Manifest to a Git Repository

1. Adding a manifest to a repository

   • I added a YAML file with the ConfigMap to the Git repository, in a separate folder, kubernetes/configs/. This allows ArgoCD to track and manage the deployment of this resource.

2. Commit and push changes

Setting up ArgoCD to track and sync changes

1. Creating an application in ArgoCD

   • I created a new application in ArgoCD. In the ArgoCD interface, I specified the Git repository, branch, and manifest path. This allows ArgoCD to track changes and automatically sync them with the Kubernetes cluster.

2. Setting up a synchronization policy

   • I selected “Automatic” sync to have ArgoCD automatically apply changes from the Git repository without requiring manual intervention.

ArgoCD automatically deploys ConfigMaps and creates dashboards in Grafana

1. Automatic deployment

   • Once ArgoCD was set up, it automatically deployed a ConfigMap containing the dashboard's JSON code to the Kubernetes cluster. Grafana discovers this ConfigMap and uses it to create or update the dashboard.

2. Deployment Monitoring

   • I checked in the ArgoCD interface that the deployment was successful and the ConfigMap was applied. I also checked the Grafana logs to make sure the dashboards were created without errors.

Checking and monitoring dashboards in Grafana

1. Checking dashboards

   • I opened Grafana and checked that the dashboards were displayed correctly. I made sure that all the panels and visualizations worked as expected.

Quite simple, and most importantly, it simplified the monitoring operations in our project.

Best Practice for monitoring infrastructure and its individual components: applications, databases, etc. can be studied within the framework online course “Observability: monitoring, logging, tracing” under the guidance of experts in the field.