Like Prometheus, but for logs!
Grafana Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus. It is designed to be very cost effective and easy to operate. It does not index the contents of the logs, but rather a set of labels for each log stream.
Loki is a central piece of what’s known as the PLG Stack for application monitoring - Promtail, Loki and Grafana. I’m sure a lot of you are already familiar with Grafana data analysis and visualization tools either from the great community add-on or use in some other aspect of your life.
But Grafana is also central to system monitoring. The same company also owns Loki and Promtail which are used to collect and aggregate logs and other metrics from your systems. Then Grafana can pull in this information from Loki so you can explore, analyze, and create metrics and alerts. Grafana isn’t the only tool that can read from Loki but it is usually used in this stack since its all designed to work well together.
Essentially the process you probably want to set up is this:
- Promtail scrapes your logs and feeds them to Loki
- Loki aggregates and indexes and makes its API available
- Add Loki as a data source to Grafana and explore
Also in this repository! You can find it here.
I found the Loki best practices guide to be very helpful. Has some good advice on how to set the stack up and how to analyze it if you start to run into issues.