MegaLinter Grafana Dashboards
Thoughts for the week
As Practicalli projects are being updated to Megalinter version 8 it seemed a good time to try the Megalinter API reporter to publish data to use with Grafana. A chance to learn more about creating Dashboards and if Grafana is any easier than DataDog.
Megalinter not publishing logs on Grafana yet
Unfortunately I havent been able to make the Megalinter API Reporter work with this journal project, Clojure CLI Config or a new megalinter-test project I created.
Health is a little better after resting over the weekend, although still have lots of symptoms.
Megalinter Grafana dashboard example project
NOTE: I am trying to diagnose why this project does not seem to be sending logs to Grafana.
GitHub Profileλ︎
Simple Icons CDN is used to include the icons for discussion and social media services such as GitHub Discussions, YouTube, Zulip, Slack and LinkedIn.
I recently replaced the Twitter link & icon with that for my clj.social account on Mastodon.
<a href="https://youtube.com/practicalli" target="blank">
<img height="64" width="64" src="https://cdn.simpleicons.org/youtube/crimson" />
</a>
<a href="https://github.com/orgs/practicalli/discussions" target="blank">
<img height="64" width="64" src="https://cdn.simpleicons.org/github/white" />
</a>
<a href="https://clojurians.zulipchat.com/#narrow/stream/250781-practicalli" target="blank">
<img height="64" width="64" src="https://cdn.simpleicons.org/zulip/green" />
</a>
<a href="https://clojurians.slack.com/" target="blank">
<img height="64" width="64" src="https://cdn.simpleicons.org/slack/firebrick" />
</a>
<a href="https://clj.social/@practicalli" target="blank">
<img height="64" width="64" src="https://cdn.simpleicons.org/mastodon/skyblue" />
Debianλ︎
The date command on Linux systems is used to show the system date or set a specific date and time. The system date can only be changed by the root account or accounts in the sudo group.
When using the Gnome desktop the system date is automatically managed, keeping the date and time current.
The timedatectl command is used to control automatic updating of the system time. This must be disabled to set the date and time to something other than the current.
When the timedateclt is disabled, then the date command can be used to set a specific date and or time.
date command will show the current date, confirming that the OS system date was changed.
Linux used to use the ntp service which is available via the Debian
ntppackage, but not used by Gnome desktop
Regolithλ︎
Unless a monitor has a KVM switch (or using KVM hardware) then Regolith will automatically change workspace arrangements when an external monitor is disconnected.
The swap focus i3 package seems to override the Super+. key binding that should otherwise be used to restore a window manager session.
Output - i3-swap-focus package should also be removed
The following package was automatically installed and is no longer required:
i3-swap-focus
Use 'apt autoremove' to remove it.
REMOVING:
regolith-wm-swap-focus*
Summary:
Upgrading: 0, Installing: 0, Removing: 1, Not Upgrading: 301
Freed space: 17.4 kB
Continue? [Y/n]
(Reading database ... 198912 files and directories currently installed.)
Removing regolith-wm-swap-focus (4.11.6-1regolith-testing) ...
root@gkar:~# apt autoremove
REMOVING:
i3-swap-focus
Summary:
Upgrading: 0, Installing: 0, Removing: 1, Not Upgrading: 301
Freed space: 27.6 kB
Continue? [Y/n]
(Reading database ... 198908 files and directories currently installed.)
Removing i3-swap-focus (0.4.3-1regolith-testing) ...
Grafana dashboard for megalinterλ︎
Megalinter can sent logs from linter runs to Grafana to visualise the result of runs.
Grafana Cloud Free Tier provides 14 days of logs & metrics retention for 3 users. No credit card is required to sign up and the service is free forever.
I signed up using my GitHub account and created the practicalli.grafana.net stack URL, using the EU Germany deployment region as its the closest to my geographical location.
Now to add configuration to each of the Practicalli Projects
Get the Loki logs service connection URL and user name
API_REPORTER_URL=https://logs-prod-012.grafana.net/loki/api/v1/push
API_REPORTER_BASIC_AUTH_USERNAME=xxxxxx
Get the Prometheus metrics service connection URL and user name
API_REPORTER_METRICS_URL=https://influx-prod-24-prod-eu-west-2.grafana.net/api/v1/push/influx/write
API_REPORTER_METRICS_BASIC_AUTH_USERNAME=xxxxxxx
Create Service accountλ︎
Administration -> Users and Access -> Cloud Access Policies, then click on Create Access Policy
Add token to the newly created policy and use the generated token value as the password value for the Loki logs and Prometheus metrics services password
Passwords
API_REPORTER_BASIC_AUTH_PASSWORD=glc_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=
API_REPORTER_METRICS_BASIC_AUTH_PASSWORD=glc_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx=
Add all environment variables to the GitHub repository that will publish megalinter statistics.
Thought: can these be added as an Org secrets rather than Repository secrets (I dont see why not, its worth a try). It will save adding these environment variables to the 22 Practicalli projects that use MegaLinter
Add these tokens to the Practicalli GitHub Org as Org secrets: organizations/practicalli/settings/secrets/actions
Create New Organization secret, providing the environment variable as the name and url, username and token as the value respectively.
The repository access could be all public repositories (no sensitive information in them or their logs) or to be suitably security paranoid use selected repositories and select each specific repository that should have access.
Add secrets to GitHub workflowλ︎
MegaLinter GitHub workflow snippet
jobs:
megalinter:
name: MegaLinter
runs-on: ubuntu-latest
env:
# Grafana Dashboard Connections - GitHub Organization secrets
API_REPORTER: true
API_REPORTER_URL: ${{ secrets.API_REPORTER_URL }}
API_REPORTER_BASIC_AUTH_USERNAME: ${{ secrets.API_REPORTER_BASIC_AUTH_USERNAME }}
API_REPORTER_BASIC_AUTH_PASSWORD: ${{ secrets.API_REPORTER_BASIC_AUTH_PASSWORD }}
API_REPORTER_METRICS_URL: ${{ secrets.API_REPORTER_METRICS_URL }}
API_REPORTER_METRICS_BASIC_AUTH_USERNAME: ${{ secrets.API_REPORTER_METRICS_BASIC_AUTH_USERNAME }}
API_REPORTER_METRICS_BASIC_AUTH_PASSWORD: ${{ secrets.API_REPORTER_METRICS_BASIC_AUTH_PASSWORD }}
API_REPORTER_DEBUG: false
steps:
- run: echo "🚀 Job automatically triggered by ${{ github.event_name }}"
- run: echo "🐧 Job running on ${{ runner.os }} server"
- run: echo "🐙 Using ${{ github.ref }} branch from ${{ github.repository }} repository"
After pushing changes to the Megalinter config, I do not see any data in the Grafana dashboards.
TODO: how to tell if MegaLinter has actually pushed logs to Grafana? Or if there is an error with the configuration?
See the content of the API notifications in execution logs, you can define API_REPORTER_DEBUG=true
Megalinter Grafana dashboard example project
NOTE: I am trying to diagnose why this project does not seem to be sending logs to Grafana.
Thank you.
Practical.li Website