Questions tagged [prometheus]
The Prometheus monitoring system, including the server, alertmanager, push gateway, exporters, client libraries and other components.
6,746
questions
0
votes
0
answers
20
views
Issues with multiple keep blocks in Prometheus relabel Config
I am trying to use relabel blocks to select metrics based on labels, I want to select all the metrics that have any one one below labels with the given values
- source_labels: [label1]
action: keep
...
1
vote
0
answers
8
views
Prometheus export stats from within container
We are running our prometheus exporters in the lxc containers we want to monitor.
Unfortunately we can not run the exporter on the host and export from there as recommended with node_exporter or ...
0
votes
0
answers
12
views
Is it possible to directly access values stored in Prometheus from React?
I'm implementing a statistics chart in React. Data used for statistics is stored in Prometheus.
I want to collect the number of http request successes/errors during a specific period and display them ...
0
votes
0
answers
20
views
Monitoring JWT secured APIs using Prometheus
I have been using Prometheus and exporter for monitoring purpose. I am stuck on one point to monitor status code and response time of API which required JWT authentication. Is there any way to monitor ...
0
votes
1
answer
21
views
Number of HTTP requests in absolute time window (pie chart)
We are using Prometheus 2.46.0 to scrape metrics every 30 seconds from our .NET Core application which exposes metrics using prometheus-net NuGet package. One of the exposed metrics is a counter ...
0
votes
0
answers
30
views
How to plot max latency in grafana
I'm using Quarkus framework with Micrometer as the metrics library. I have configured reporting timers with percentiles histogram for certain endpoints. In Grafana, I've created a dashboard to monitor ...
0
votes
1
answer
63
views
Calculate delta on filter metric
I would like to calculate the daily increase of a metric via prometheus.
Currently I am using: delta(metric_name[1d]).
Unfortunately there can be an instant were the value is 0, therefore it messes ...
0
votes
1
answer
39
views
Disable exemplars support in Spring Boot 3.2 to avoid Prometheus problem with scrapping metrics
New Spring Boot added exemplars support:
https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-3.2-Release-Notes#broader-exemplar-support-in-micrometer-112
After upgrading to Spring Boot 3.2....
-2
votes
1
answer
35
views
How can I add a custom static label to a Prometheus query\metric?
Ok, so I have this query right here container_cpu_reserved which gives me the following metric:
container_cpu_reserved{datacenter="dev-01", instance="worker-02", job="nomad&...
0
votes
1
answer
16
views
How can I add metric labels to Prometheus alert manager description?
I want to access lables of the metric which matches the expression to inform the team about which exact queue exceeds the limit:
An example of the metric
job_duration_bucket{resource_server="rs1&...
0
votes
0
answers
25
views
Trouble with Prometheus relabel_configs Regex for Matching Specific URL Patterns
I'm configuring Prometheus Operator and encountering an issue with relabel_configs in my setup. I need to match URLs based on specific conditions and relabel them accordingly, but I'm running into a ...
0
votes
0
answers
20
views
Kubernetes: Support for Prometheus Native Histograms
I see Prometheus has support for a new type of Histogram called Native Histograms.
I have been trying to find out if there is support for these in Kubernetes. i.e. https://kubernetes.io/docs/reference/...
0
votes
1
answer
48
views
Is there complete list of spring boot metrics with short description?
I briefly looked at actuator documentation but I was not able to find a list of metrics.
The most detailed information provided here but I want to have more details:
JVM Metrics
Auto-configuration ...
0
votes
0
answers
27
views
Prometheus Grafana - duplicate metrics returned
Whenever I'm querying metrics, I am getting duplicate results - two versions of the same metric.
Thq query: {exported_job="@abc/mailer", job="otel-collector"}
The two duplicate ...
0
votes
0
answers
44
views
Unable to get kafka_connect_connector_status on Promethues
I am using Confluent for Kubernetes to deploy Kafka Connect. For metrics monitoring I am using promethues operator and have added a job in Secrets name additional-scrape-configs. The configurations ...
0
votes
0
answers
31
views
What is the consumer group used to consume from a kafka changelog topic?
I have this kafka-streams application which is returning Prometheus metrics. If I were to believe these metrics, the application has a lag on most partitions of the changelog topic.
I wanted to verify ...
1
vote
2
answers
36
views
Use 2 label values for 1 template variable
On my dashboard in Grafana, I wan't to have a template variable that select id from a metrics labels:
Here is the definition in my application using prometheus client in python:
py_status = Enum("...
1
vote
0
answers
21
views
Visualizing a Prometheus type summary in Grafana using InfluxDB
Using Grafana backed by an InfluxDB data source, how can I visualize the average response times observed by a HTTP client that exposes metrics in a Prometheus style endpoint with a Summary type metric?...
-1
votes
0
answers
33
views
How to design a framework using micrometer observability for multiple Spring Boot applications in Java? [closed]
I'm planning to create a library using Spring Boot and Java. The main purpose of this library will be to collect the photon metrics using micrometer & Prometheus.
This library can be used as a ...
1
vote
1
answer
29
views
"Where" and "Join" in PomQL?
I have two Prometheus metrics that report samples every day:
metric A: product_bugs, it's a gauge, it has a label for type of product (X, Y, Z). It reports the number of bugs for a product type.
...
0
votes
0
answers
34
views
Custom metrics not found in prometheus when setting up a simple python app with prometheus + kubernetes + minikube + docker
I'm currently learning the prometheus + docker + kubernetes stack and use minikube for kubernetes simulation (instead of the cloud). In my setup, I create a simple flask app which should log metrics ...
1
vote
0
answers
18
views
prometheus is not available in actuator endpoints
I've tried to configure prometheus for metrics in grafana,
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-actuator</artifactId> &...
1
vote
0
answers
54
views
Alert not triggered in prometheus
I have this alert in prometheus:
- alert: Error_pods
expr: sum by (namespace) (kube_pod_status_ready{namespace="gradl-enterprise", condition="false"}) > 0
for: 5m
And I can see ...
2
votes
0
answers
41
views
spring boot observability new property missing tags
After upgrading my spring applications I can see that the metrics exposed through actuator to Prometheus has changed its tags location in the properties from metrics.tags to observations.key-values as ...
0
votes
1
answer
31
views
multiple hosts and multiple metric_path in prometheus using files
i have 100's of servers and every server has 10's of metric_path. i am able to scrape all hosts but its automatically adding /metric after every host.
`global:
scrape_interval: 30s
scrape_configs:
...
0
votes
1
answer
48
views
Promql for getting percentage stats for http status codes
I would like to plot a graph and add it dashboard for displaying percentage of 2xx, 4xx and 5xx (and even grouped by specific http status codes like 401, 503, 502, 500 etc.) based on http status codes....
0
votes
0
answers
19
views
I am looking for an alert configuration for locked tables in waiting state in mysql on grafana and with prometheus data source
I have a panel with these requests, how to configure an alert that is triggered every time there are 10 table locks waited:
rate(mysql_global_status_table_locks_immediate{instance="$host"}[$...
0
votes
0
answers
29
views
How to report one time metric using OpenTelemetry
I want to instrument my application to report the execution time of my parser function as a metric. The metric attributes will include an ID for each request, and I aim to create a bar graph where ...
0
votes
1
answer
27
views
How to turn on or off Grafana alert indicators on panel?
I realize that on some of my Grafana panels, I can't see vertical lines indicating the start and end of alerts. Some of my panels are working fine, I can see alert indicators (see screenshot). The ...
0
votes
0
answers
13
views
Prometheus Alerting when Kubernetes service are down
I have few services in my Kubernetes cluster. I need to get alert when any of the service is down.
Right now, I'm getting alerts for other scenarios like pod not ready, Crashloopbackoff etc.
I'm using ...
0
votes
0
answers
11
views
Label not applied in job after adding another service covery
I used to have a prometheus config to scrape nodes for node-exporter using a simple file which was working fine
- job_name: 'node-exporter'
honor_labels: true ...
-1
votes
1
answer
54
views
How to measure database latency for MongoDB using Spring Boot and Micrometer?
I would like to measure database latency for my Spring Boot microservice, which is using MongoDB as a database. I'm using Micrometer, metrics are scraped by Prometheus and visualized by Grafana. Do I ...
0
votes
0
answers
14
views
Prometheus/Alertmanager: routing alerts to two different emails
I have following Alertmanager configuration:
global:
resolve_timeout: 1m
route:
# A default receiver
receiver: "EmailNotifications-EPG"
routes:
- receiver: "...
0
votes
0
answers
33
views
How can i use vmrange label for calculate Percentage of requests less equal 500ms
I'm testing switching from prometheus to using victoria metrics.
In prometheus, my promQL:
(sum(rate(demo_service_request_duration_seconds_bucket{le=\"0.5\"}[5m])) by (exported_job) / sum(...
0
votes
0
answers
33
views
Checking if all expected results of a Prometheus query are present
Let us assume we have a group of two endpoints which have three services for which the same metrics should be sent to Prometheus. In order to give an example, let us assume in a healthy situation the ...
0
votes
0
answers
11
views
Issue with Proxy Configuration in Blackbox Exporter
Hello StackOverFlow Community,
I’m facing an issue with configuring a proxy in the Blackbox Exporter.
Despite setting the proxy_url parameter and enabling proxy_from_environment in the blackbox.yml ...
0
votes
0
answers
10
views
Trying to get Prometheus metrics from a file
I'm currently attempting to set up GPU metric monitoring using Prometheus.
I have 2 servers, one running Debian, the other running Ubuntu. On the Debian server, I'm fetching data from nvidia-smi and ...
0
votes
0
answers
22
views
Is there a way to find max cpu size from a Prometheus query.?
enter image description here
Below is the query to get cpu usage(blue color)
sum(
1 - sum without (mode) (rate(node_cpu_seconds_total{mode=~"idle|iowait|steal"}[2m]))
*
on(...
1
vote
2
answers
35
views
Get the increase for a prometheus metric that has been recorded as a rate
Let's say I have a metric foo_count that get's rolled up before being written to remote storage to reduce cardinality and uses this rule:
- record: rollup:foo_count:1m
expr: sum(rate(foo_count[1m])) ...
0
votes
0
answers
31
views
Can I preserve labels from promql query with aggregations?
I have a query like:
sum by (city) (probe_success{job=~"dns.*"})
which returns the values of successful samples grouped by city. It`s OK, but I need to create vmalert rules with the next ...
1
vote
0
answers
15
views
Kafka producer metrics are not presented
I'd like to add metrics to my application and count how many messages are sent and consumed to/from my kafka. So I tried to add MicrometerConsumerListener and MicrometerProducerListener to Consumer ...
0
votes
0
answers
54
views
"text format parsing error in line 1: unexpected end of input stream" error while pushing metrics
I am trying to push the metrics from the value I derived from the zuul logs in ansible playbook. Below is the code:
name: Push AF Upload metrics
win_shell: >
curl -X POST --netrc -H "Content-...
0
votes
0
answers
26
views
Best/easiest way to either import data contained in a JSON file on disk or from a CSV generated by a web server over HTTP into Grafana/Prometheus
We have a bunch of custom servers running running a custom service that outputs stats about its operation in a CSV format of about 12 columns and a variable number of lines with a header row and ...
2
votes
0
answers
40
views
Spring Boot app /prometheus endpoint taking a long time without a custom controller
My application is using:
spring-boot: 1.5.22.RELEASE
spring-boot-starter-actuator: 1.5.22.RELEASE
micrometer-spring-legacy: 1.3.5
micrometer-registry-prometheus: 1.3.5
application.properties:
...
0
votes
0
answers
20
views
Jsonpath for prometheus json exporter
I need to generate metrics for Prometheus based on the service's json response. I installed prometheus-json-exporter, I want to do something like in the example https://github.com/prometheus-community/...
0
votes
1
answer
25
views
Limit email count which is getting fired by alertmanager
Can someone provide some suggestions on limiting the email alerting count to send only one email instead of 100 duplicate emails when a particular metric condition is triggered and alert starts to ...
0
votes
0
answers
21
views
Can we get information of pod deleted In Kubernetes Environment using Prometheus
As we have setup Prometheus and Grafana using helm charts and we have configured email alerts for:
If a pod is created send me email alert
If a pod if in pending, error, unknown state send me email ...
0
votes
0
answers
19
views
Prometheus monitoring webservers
I want to receive and analyze metrics using Prometheus from the endpoint provided by Spring Boot app.
Both of them run in docker using docker compose.
I want Prometheus to go to backend endpoint {ip}:{...
0
votes
0
answers
15
views
RabbitMQ Prometheus Metric "/api/metrics" returns "Empty reply from server"
Running RMQ on a GKE pod.
Version: 3.7.14
The prom target shows that the endpoint is down. While checking the endpoint response manually the metrics is returning an "Empty Reply from server&...
-2
votes
0
answers
54
views
How to track PersistentVolume usage with prometheus for non gp2 storage class?
I am running a kubenetes cluster in an ubuntu node. I have created a persistent volume (storage class: openebs-hostpath) and corresponding claim. I want to track how much of the claim is being used by ...