Questions tagged [grafana-loki]
Like Prometheus, but for logs.
558
questions
-1
votes
0
answers
39
views
What is the correct procedure for upgrading Loki S3 storage when expanding Minio storage through the addition of a server pool?
I recently configured Loki Distributed with Minio S3 with a retention period of 30 days.
The minio cluster has a storage of 160 GB.
The log is getting filled up rapidly 15GB/day and we will probably ...
0
votes
0
answers
17
views
Grafana Loki escape character
I send docker logs from fluent bit to grafana
tcpdump shows that fluent bit send log with escape character:
i need to escape double quotes by \
example of tcpdump output:
[["1701677792000000000&...
0
votes
1
answer
19
views
How to exclude messages by masked string
I need to remove the number which is changing in a string when running != in a query. Example:
Message = maintainInventoryDetail ERROR: Open task exists: 9315433 for this Reference. CAN'T BE UPDATED. ...
-1
votes
0
answers
21
views
How can I read/parse/query a haproxy log file?
I have been given a haproxy log file (not access to the haproxy server), about 0.6GB, with rows like:
Nov 17 11:53:03 localhost haproxy[48735]: : [17/Nov/2023:11:53:03.186] gaee-https~ npgis-servers/ ...
0
votes
0
answers
16
views
Count in log file plot in grafana
I have a log file which have total number of request count in last field. How to plot in Grafana after every 6 minutes?
log file in which total count in last field
0
votes
0
answers
25
views
Best/easiest way to either import data contained in a JSON file on disk or from a CSV generated by a web server over HTTP into Grafana/Prometheus
We have a bunch of custom servers running running a custom service that outputs stats about its operation in a CSV format of about 12 columns and a variable number of lines with a header row and ...
-1
votes
0
answers
39
views
grafana not able to find old logs [closed]
Configured grafana as ui and data source as loki, in grafana dashboard i`m able to check logs when i triggered request but after some time, logs were vanishing and giving below error,using azure ...
0
votes
0
answers
30
views
Logs from Vector not getting to Loki
I have three docker containers all running locally, vector, loki, and grafana, all on the same network. I am sending syslog messages to vector and transforming them. I have two sinks, one "...
0
votes
0
answers
19
views
How to send openetelemetry data from instrumented libraries to OTLP
I have troubles with gathering the output from openetelemetry in nodejs app.
Here is my setup:
tracing.js:
const { logs, SeverityNumber } = require('@opentelemetry/api-logs');
let otelLogs = logs....
0
votes
0
answers
43
views
How to sum/count the total number of occurrence of particular string or pattern in Grafana-Loki
In Grafana,
I am searching for total no of occurance of particualr string / pattern like below
{k8s_cluster="mycluster", namespace="mynamespace", app="myappname"} |= ...
0
votes
0
answers
30
views
Setting up Loki on EKS Cluster
We have encountered an issue with setting up Loki, which has brought to light the need to consider updating our Helm version from 3.5.2 to 3.12.3 or the latest. The current version is causing a syntax ...
0
votes
0
answers
28
views
Connect Amazon Managed Grafana to Loki in EKS cluster
I have Loki running in my EKS cluster. I want to add a Loki datasource in Amazon Managed Grafana.
How can I connect AMG to Loki instance inside the EKS cluster?
0
votes
0
answers
21
views
Change value from a line in table view
I'm developing dashboards on grafana using Loki.
Question : How do I change this value so that one becomes FR (originally AuthStormShield-ChildStormShield ) & the other US (originally ...
0
votes
0
answers
24
views
log4net:ERROR Appender named [loki] not found
I'm currently working on an ASP.NET MVC (4.7.2) application and have integrated log4net for logging purposes. I am trying to configure a custom appender called "loki" using the log4net....
1
vote
1
answer
82
views
How can I configure sending pipelines to Loki using Promtail
I want to add a logLevel pipeline to Loki so that I can filter logs by their level (INFO, WARN, DEBUG...). I try to do everything according to the documentation, but for some reason the pipeline is ...
0
votes
0
answers
64
views
Extracting and Filtering JSON Array Entries in Grafana using Promtail and Loki
I am using Promtail to ship data to Loki, which I then visualize in Grafana. I have a JSON file that is recreated every minute with a format like:
{
"Date": "2023-10-23T11:43:57....
0
votes
1
answer
49
views
Loki-distributed TSDB Indexes
I've got loki deployed in kubernetes in microservices mode and store chunks in GCS while using TSDB indexes - my questions are: do these indexes end up stored in GCS as well or do I need to setup ...
0
votes
0
answers
39
views
Grafana flow mode service_graphs
I am using flow mode to configure my grafana agent.
In the traditional modo I can see here: https://grafana.com/docs/tempo/latest/configuration/grafana-agent/service-graphs/, how can enable service ...
0
votes
0
answers
63
views
Promtail pipeline labels not displayed in Grafana when using JSON parsing stage
I'm trying to extract values from JSON formatted log lines using Promtail's pipeline to create labels for visualization in Grafana. However, upon querying in Grafana, the expected labels are not ...
0
votes
1
answer
81
views
Special Characters in Grafana Variable Causing "no data" Output in Query
I've encountered an odd interaction in my Grafana dashboard where I'm storing a specific string in a variable called errorMsg. The string is: ERROR: DoorBoard: SetErrorClass[Uart]: Open /dev/ttyUSB2 ...
2
votes
1
answer
63
views
Handling 'no data' in Grafana subtraction query using LogQL with Loki
I'm using Promtail to forward logs to Loki and visualize the data with Grafana. I have a query that calculates the number of devices online without errors. The query works as follows:
(
(count(count ...
0
votes
0
answers
42
views
Fluent-bit - Splitting json
I'm sending logs to ES with fluentd. App logs are in JSON format. Here is one of them.
I am using ECS and Grafana.
{
"@timestamp": "2020-12-24T12:16:43.2784Z",
"level"...
0
votes
0
answers
44
views
Why is helm chart not working- loki retention log?
I'm trying to deploy a loki helm chart, using Argocd, in the loki helm configuration values I'm using the 24h log retention setting. I am using Ceph RBD as persistent storage. The helm chart values ...
1
vote
1
answer
31
views
How to display systems that are offline based on Loki log queries in Grafana?
"I'm using Loki to store logs and Grafana for visualization. I want to create a Grafana table that lists all systems that are considered offline. A system is considered offline if it has sent a &...
0
votes
1
answer
123
views
How to install Loki on minikube [closed]
I'm installing Loki on my Kubernetes minikube instance.
When I look at my pods I get one running out of three:
default loki-write-0 1/1 Running 0 ...
0
votes
0
answers
54
views
Promtail: combine several files in one label
On my instance (nginx reverse-proxy) I have several log files, two for each site.
For example:
$ ls /var/logs/nginx
site-1.access.log
site-1.error.log
site-2.access.log
site-2.error.log
site-3.access....
0
votes
0
answers
125
views
OTEL Collector not receiving the logs with loki when exporting from .net 6 application
OTLP Endpoint not receiving the logs when we export it from .NET 6 application. But it is exporting to console.
Here is the GitHub link for sample .NET application that we cloned and tried locally by ...
1
vote
1
answer
77
views
Display simple scraped values from Loki logs using Grafana
I have some log lines in Loki,
[10-Oct-2023 08:22:19 UTC] [1270628]: INFO SEND: Sending 235 emails…
[10-Oct-2023 08:23:19 UTC] [1270628]: INFO SEND: Sending 220 emails…
[10-Oct-2023 08:24:19 UTC] [...
0
votes
2
answers
238
views
Grafana Loki previous logs lost on docker container remove and then run
I am losing my logs everytime I remove and then start Grafana Loki and promtail containers (i.e. all the logs till the time of container removal). I want to know if its possible to retain logs on ...
0
votes
0
answers
72
views
Loki parse JSON and sum by field
I'm logging a JSON which is not showing up in detected fields or labels.
I'd like to parse the stderr json in my log picture, specifically capturing the field "dml_affected_rows/row_count", ...
0
votes
1
answer
58
views
Unable to ship logs to Grafana Loki with FluentD & Fluent-bit
I am trying to integrate application logs running on EC2 instances with Grafana Loki using FluentD & Fluent-bit.
I am facing the following:
Fluent-bit service is running normally
The application ...
0
votes
0
answers
42
views
Json Logql query for getting multiple line graphs grouping by a single json key
new to logql here.
This is a sample json log line I have:
{
"accessedts":1696477777255,
"command":"get_user_details",
"country":"US",
"referrer&...
0
votes
0
answers
185
views
Promtail to Loki with Labels
I am using a pattern to add tags to different log fields of my nginx ingress. Here is the query I use in Loki + referer field to look only the domian request.
{app="nginx-ingress-microk8s-...
0
votes
1
answer
252
views
How would i delete all logs in loki via its api?
I struggle to implement a clean solution for an "delete all" button for our application while using the loki api.
Setup: Loki in monolithic mode in a cluster.
Goal: Loki accepts one api ...
0
votes
0
answers
49
views
"Unable to retrieve logs in Grafana-Loki in the expected format."
I configured Loki with Grafana to monitor logs and I get logs successfully.
Here the issue is in one log I have multiple lines like the below:
But in Grafana I get each line in log as an individual ...
1
vote
1
answer
237
views
Use nlog to push logs directly to Grafana Loki Cloud
I have an old WPF app that I want to add some alerting to, so I created a free account in Grafana Cloud and I want to be able to see logs there.
The app is using nlog, so I added a sink in nlog.config ...
0
votes
1
answer
73
views
Grafana shows inconsistent behavior when changing time interval
I have these two queries, fetching Loki logs
1: sum(count_over_time({container="pippo", namespace="family"} |= `fancy logs` [5m]))
2: sum(count_over_time({container="pippo&...
0
votes
0
answers
118
views
Loki Helm Chart: Logs Are Only Visible for the Last Hour Despite 2 Days of Installation
I have installed loki using this official helm chart, For some reason it doesn't show logs before 1h I come back after 1h and I only see 1h of logs, it's been installed since 2 days, and this is the ...
0
votes
0
answers
27
views
In a table chart : stuck on how to sum a specific value by a chosen category
First of all, I'd like to make it clear that I'm a rookie and that I'm taking things one step at a time. Hope you'll help me cause I'm stuck for couple of days ;)
Here my question based on the screen
...
0
votes
0
answers
31
views
Chunks & indexes not getting deleted from cassandra storage even after setting retention period
Below is my schema config for loki
schemaConfig:
configs:
- from: 2023-09-24
index:
period: 24h
prefix: loki_index_
chunks:
prefix: chunk
...
1
vote
1
answer
421
views
Grafana cannot retrieve logs (and labels) from Loki
I have Docker Swarm environment with several nodes, one of them called gateway. On all nodes installed Docker Plugin grafana/loki-docker-driver:2.9.1 instead of using Promtail. I used Portainer ...
0
votes
1
answer
132
views
How to extract systemd unit/service name from journal logs in Grafana? [closed]
I'm visualizing my systemd journal logs in Grafana, which I send using Promtail and store in Loki. I'd like to be able to see the systemd unit or service name that has generated each log entry in ...
0
votes
1
answer
192
views
Combining results from two InfluxDB Flux queries in a single Grafana table
I'm using Grafana with InfluxDB and I've encountered a problem when trying to visualize the results of two different Flux queries in a single table.
I have two Flux queries: one fetches the used ...
0
votes
1
answer
221
views
Promtail with Cloud Grafana Loki and winston JS log fields not detected
I have setup promtail (docker image) reading my .log files. Those logs are sent to Grafana Loki cloud.
I receive those logs succesfully but fields are not detected. Exemple of a log: {"env":&...
0
votes
1
answer
67
views
Loki docker client is not sending logs to Grafana cloud's preconfigured loki datasource
I have followed the steps mentioned here https://grafana.com/docs/loki/latest/send-data/docker-driver/configuration/ for sending logs from docker compose containers to grafana cloud's preconfigured ...
0
votes
0
answers
56
views
Caculate latest value for Grafana dashboard
I have the following Loki query that shows me the error rate for my containers:
(sum(rate({namespace="namespace", cluster="cluster", level="error"}[1m])) by (cluster, ...
0
votes
0
answers
147
views
Handling special characters in Grafana variable used in regex filter
I am using Grafana for visualizing my logs and came across a specific challenge related to handling special characters, specifically the square brackets ([...]).
Situation:
In one of my dashboards, I ...
0
votes
1
answer
575
views
How to convert a timestamp in Grafana from a formatted string to epoch milliseconds and create a time range URL parameter?
I am currently working with Grafana and trying to create a link that redirects to another dashboard with a specified time range, which is 5 minutes before and after a certain timestamp. I have the ...
0
votes
0
answers
187
views
Use Loki and Grafana to implement Full Text Search?
I have multiple log files, thus my loki stream looks like:
{job="my_job", filename="20230920_1.log"}
{job="my_job", filename="20230920_2.log"}
I generate these ...
0
votes
1
answer
198
views
Calculating Unique System Counts per Error Message with Loki Query in Grafana Dashboard
I am currently working on a Grafana dashboard where I visualize various errors logged in different systems and count the number of times these errors occur. I am using Loki to send journal logs to ...