Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [prometheus]

For questions about Prometheus, an open-source systems monitoring and alerting toolkit.

Filter by
Sorted by
Tagged with
0 votes
0 answers
11 views

How to get a label value from another metric and add it on prometheus

Suppose a metric is like the following, weblogic_id(domain="name", instance="instance"} Is there any way I can get the domain="name" from that metric and add it to all ...
Nirzak's user avatar
  • 1
0 votes
0 answers
14 views

Troubleshooting the 'View in Alertmanager' Button in MultiClusterObservability Email Notifications

This question concerns Openshift, but perhaps someone who simply deals with Kubernetes might be able to help. I have 2 clusters and on one of them I deployed Prometheus Alertmanager on Redhat ACM ...
Daveron's user avatar
1 vote
2 answers
131 views

Prometheus not Starting (Exec Format Error)

I am trying to run Prometheus on my Ubuntu Linux VM on a Mac M1 (if those details are necessary). OS: Ubuntu 23.10 Prometheus: 2.51.2 I make the config files, and as I run the following lines, sudo ...
drew's user avatar
  • 11
0 votes
2 answers
68 views

How collect these metrics in Prometheus for an AWS ECS container?

We have a container running in ECS, and we want to collect the following metrics (truncated list) in Prometheus and eventually view them in Grafana. How do I setup the Prometheus agent on the ...
Chris F's user avatar
  • 423
1 vote
1 answer
20 views

Dynamic service discovery for Docker containers in Prometheus configuration

I have an application running in a Docker stack, and it dynamically scales its instances based on demand. This application exports its own metrics, and I want Prometheus to scrape these metrics in ...
user3270532's user avatar
0 votes
0 answers
18 views

AWS EKS Fargate pod scheduling issue with Prometheus deployment

I'm encountering a problem with scheduling Prometheus pods on AWS EKS Fargate within my Kubernetes cluster. Here are the details of my setup: I have configured an AWS EKS Fargate profile named ...
Typewriter's user avatar
0 votes
1 answer
31 views

Concat multiple meta labels on prometheus for relabling

How can I change the value of the label __param_target to resemble how I would interact with a Kubernetes service in-cluster, following the format: http://serviceName.namespace.svc:servicePort/path ? ...
Elad Avikzer's user avatar
0 votes
0 answers
6 views

Remove trailing space in Mac OS hostname?

I'm trying to run Prometheus on my Mac, but I get the following MyHostName $ prometheus --config.file=prometheus.yaml parse external URL "": parse "http://MyHostName :9090/": ...
Chris F's user avatar
  • 423
0 votes
0 answers
12 views

Question about AWS Prometheus tutorial

I'm following this cool example on how to run a container and monitor it CloudWatch with Prometheus - Monitor and scale your Amazon ECS on AWS Fargate application using Prometheus metrics. All my ...
Chris F's user avatar
  • 423
0 votes
1 answer
305 views

How to configure MySQL exporter Helm chart for multiple targets (remote & local)?

I have been trying for a few days to set up a multi-target MySQL exporter (MSE) deployed with Helm inside a k8s cluster. The chart that I am using is the https://artifacthub.io/packages/helm/...
aboitier's user avatar
  • 199
0 votes
1 answer
46 views

No targets in Prometheus at Kubernetes

Why are there no targets at my Prometheus? I did everything like in the Prometheus documentation. Also, I have pods with the label: "name=cadvisor". I have the following configuration file: #...
Denis Bibishkin's user avatar
0 votes
1 answer
265 views

Prometheus/BlackBox Exporter probe_success 0 but service still running

I'm currently utilizing Prometheus with Blackbox Exporter for monitoring our services. However, I've encountered an issue where one of our external services is showing probe_success as 0, despite ...
AwsmOfficial's user avatar
1 vote
1 answer
392 views

Add prometheus config for all jobs

In my prometheus.yml I have several jobs defined like: - job_name: alpha scheme: 'https' metrics_path: '/metrics' static_configs: - targets: - one.example - two....
Vietna's user avatar
  • 25
0 votes
0 answers
14 views

Write_Prometheus plugin with basic auth

0 I am testing the below Write_Prometheus plugin to send the cassandra metrics to prometheus server which is external machine but i see that all metrics are sending without any basic auth happening. ...
rohit singhal's user avatar
0 votes
1 answer
145 views

How to scrape only some targets of EC2 based on tag

I have a requirement, where I have enabled NGINX metrics and I am trying to plot metrics with Prometheus. I am trying to plot metrics for NGINX EC2 server which has a tag named test=nginx or by ...
Siva Malla's user avatar
0 votes
0 answers
13 views

Can I use statsD to scrape Kubernetes metric endpoint or from Kubernetes metric exporters?

Can I use statsD to scrape Kubernetes metric endpoint or from Kubernetes metric exporters(for kubernetes infrastructure metrics like nodes, pods, control plane)? I see people are using Prometheus more....
bunny's user avatar
  • 101
0 votes
0 answers
45 views

Is there a way to send alert emails if Kubernetes detects an authentication attempt, even if I don't have access to the master node?

I have a Kubernetes cluster in OVH, and I need to send alerts if the API detects an authentication attempt. I know I could use a script to monitor Kubernetes audit log files and send emails if there ...
M.JOUAHER's user avatar
1 vote
1 answer
51 views

get node_exporter metrics for a single system user

I have installed the node_exporter within the userspace of a linux system user. Everything works and I get the metrics in my grafana. But what I get is apperently the metrics for the whole host system ...
Vietna's user avatar
  • 25
3 votes
1 answer
141 views

Prometheus and Azure WebApps: How to monitor multiple instances with prometheus

I have an application that has two instances running and each request runs on a different machine. I am monitoring an event with a counter in my application and each instance runs it's own counter ...
Menelaos Vergis's user avatar
1 vote
1 answer
1k views

Prometheus scrape from service vs pods

Let's say I have multiple pods (replicas) of application under a service (with targetPort pointing to port where application keeps its metrics for Prometheus) named "app-monitor-service". ...
user avatar
0 votes
2 answers
124 views

Aws EKS Prometheus helm failed pod CrashLoopBackOff with log "trigger reload: reload request failed: Post \"http://127.0.0.1:9090/-/reload\"

I followed the steps here https://docs.aws.amazon.com/prometheus/latest/userguide/AMP-onboard-ingest-metrics-new-Prometheus.html In a nutshell, I installed the Prometheus community helm chart on my ...
Harry's user avatar
  • 101
0 votes
1 answer
409 views

Metrics Server and Prometheus have overwriting resources

I am trying to install Metrics Server, but it needs to create a resource that already exists from an installation of Prometheus. They both want to make v1beta1.metrics.k8s.io. I went to see if the ...
Vaccano's user avatar
  • 206
0 votes
0 answers
17 views

How do I visualize multiple static Prometheus TSDB files from a single Grafana instance?

I have a requirement in my project where I receive multiple static TSDB files which are essentially Prometheus backups and then visualize these on Grafana using dashboards. I am looking to understand ...
PunitK's user avatar
  • 1
0 votes
0 answers
24 views

How to configure a prometheus installation of openshift to include the labels of node role in the exporter?

We want to alert based on cpu requests only on worker nodes namespace_cpu:kube_pod_container_resource_requests:sum{kubernetes_node_role=~"worker"} gives no datapoints and namespace_cpu:...
Adrián Blázquez's user avatar
3 votes
1 answer
116 views

Measuring image pull latency in k8s cluster

I maintain a Kubernetes clusters that pulls a number of images (mostly from Docker Hub, but also some from GCR and GHCR). I am interested in building a dashboard to measure the amount of time it takes ...
Derek Brown's user avatar
0 votes
2 answers
680 views

How to defragment etcd database in Kubernetes cluster

I am trying to resolve a recurring Prometheus alert I'm getting that states that my Kubernetes cluster etcd database is fragmented and should be defragmented. I have never used the etcdctl command ...
Joseph Gagnon's user avatar
0 votes
1 answer
197 views

List all active promtail targets

I have spend a significant time reading the manual, but I still cannot figure out how to actually get a list of the targets that promtail is currently tailing. The metrics endpoint only exposes ...
MrMeeSeeks's user avatar
0 votes
1 answer
1k views

Add more static instance labels in prometheus config

I'm adding serves to my prometheus config like this: - job_name: cluster1 static_configs: - targets: - server1.example.com - server2.example.com - server3.example....
Vietna's user avatar
  • 25
-1 votes
1 answer
37 views

Enterprise Ready Prometheus

I am new to Prometheus and have been reading about its popularity as a tool for monitoring applications, containers and K8s objects. However, most of the blogs that I have come across have at best ...
Gaurav Parashar's user avatar
1 vote
1 answer
461 views

Prometheus scrape config for kubelet node role not auth'ing

I have a kubernetes (not minikube) cluster (1.25) installed on a dedicated machine (1-node cluster). I installed Prometheus. I am using a ServiceAccount, Role, ClusterRoleBinding like this: apiVersion:...
codedread's user avatar
  • 111
1 vote
1 answer
532 views

How to add an aditional ServiceMonitor for prometheus-operator?

I installed prometheus-operator using kube-prometheus-stack and so far everything has worked fine, but I need to add one more service to monitoring. This is a Windows Server that is not running on the ...
timeuz's user avatar
  • 13
2 votes
0 answers
41 views

Why is an 'average' as a metric favored (over 'latency') in this case?

In a talk on RED method, the presenter defends his use of 'average' by the following comparison with latencies. The linked-to part of the talk written down (non-verbatim and with my formatting): ...
VanillaDonuts's user avatar
3 votes
0 answers
198 views

How to collect statistics from CI over time?

CI environments like GitLab (self-hostet and Cloud), GitHub, CodeCov, Codacy, ... collect statistics over time, so a developer or team lead can see the evolution of the project over time: number of ...
Paebbels's user avatar
  • 131
0 votes
1 answer
708 views

Kubernetes Job Metrics in Prometheus

I want to create a Kubernetes Job object like the example: https://kubernetes.io/docs/concepts/workloads/controllers/job/ Now imagine I want to have the result (in this case the value of Pi: 3.14159......
Ezariah A's user avatar
0 votes
1 answer
785 views

How to purge / prevent using the prometheus local storage when using remote_write

I configured Prometheus to push metrics using remote_write but the local storage is still in use and keeps growing How could Prometheus be configured to stop storing locally and only push metrics or ...
nbari's user avatar
  • 101
0 votes
1 answer
266 views

Snapshot or Saving Prometheus Data for Performance testing and comparison

I have an application and system that exposes its resource metrics to Prometheus and Grafana that I want to do performance testing on. I want to be able to conduct a test, capture the metrics on CPU, ...
DogEatDog's user avatar
  • 191
3 votes
1 answer
3k views

What is the difference between Fluent Bit and Prometheus?

From what I've seen, Prometheus scrapes data and metrics and displays them and Fluent Bit collects logs and metrics and can output them. So, are they doing the same thing, i.e. should I choose one ...
Grafana's user avatar
  • 33
0 votes
1 answer
93 views

Apache Camel-k metrics in rancher embedded prometheus

I have a Kubernetes running Rancher 2.4.3. I have the cluster monitoring enabled in rancher, so that exists a Prometheus instance running, so as a Prometheus Operator. Recently I deployed a Apache ...
Beto Neto's user avatar
  • 101
2 votes
1 answer
2k views

Does Prometheus log scrape-duration?

I'd like to graph scrape duration for a particular target. The Prometheus web interface has a Status of Targets page, which tabulates the (time elapsed since) "last scrape" and corresponding ...
benjimin's user avatar
  • 279
0 votes
1 answer
637 views

Scraping different Prometheus metrics at different rates

I am running a Tendermint network. I am using Prometheus to track various metrics, and to track node health. For performance metrics such as tendermint_consensus_total_txs, I want updates every second....
Ethan Reesor's user avatar
3 votes
0 answers
928 views

Create a Kubernetes Service that loads an external host url AND port

I have a url like this: http://myserver.mydomain.com:98765 I need to have this url be a Kubernetes service so I can address it in Kubernetes like this: my-service.mynamespace.svc.cluster.local. (So, ...
Vaccano's user avatar
  • 206
2 votes
2 answers
3k views

Monitoring Kubernetes CronJob & Job in Grafana for today day only

I'm trying to create a Dashboard in Grafana that will show the status of Cronjob & Job success & fail. Currently i'm using kube_job_status_succeeded to get a status job, but the issue here ...
DagaReiN's user avatar
2 votes
1 answer
192 views

How prometheus scrapes individual pods actuator endpoints when they are exposed via a single service in kubernetes?

I have created a deployment with 2 replica sets. But the Service exposes the 8080 and the management endpoint using NodePort. How should I set up the pods deployment if Prometheus deployment should ...
GauravEdekar's user avatar
0 votes
1 answer
658 views

How to setup basic auth for Prometheus deployed on K8s cluster using yamls?

How to setup basic auth for Prometheus deployed on K8s cluster using yamls?? I was able to achieve this easily when Prometheus was deployed on a host locally using tar file. But when it is deployed as ...
Saurabh's user avatar
1 vote
2 answers
5k views

Enable ingress-nginx Prometheus monitoring without Helm

A few weeks ago, I installed ingress-nginx in my Kubernetes cluster. Now, I realize that was a mistake because that did not come with Prometheus monitoring enabled automatically, and even worse, it ...
chrispytoes's user avatar
2 votes
0 answers
1k views

Unable to fetch metrics from custom metrics API: Internal error occurred

I am running a local Kubernetes cluster with Minikube on a Windows machine using --driver=hyperv. I am trying to set up an HPA (HorizontalPodAutoscaler) based on Pods custom metrics. Using Helm, I ...
soders's user avatar
  • 21
0 votes
2 answers
860 views

Getting error while trying to communicate with apiserver in kube-state-metrics pod

I am getting error: # kubectl logs kube-state-metrics-fdc49658-g98sl -n monitoring I0523 11:30:25.206556 1 main.go:86] Using default collectors I0523 11:30:25.206766 1 main.go:98] Using ...
Chang Zhao's user avatar
1 vote
1 answer
473 views

Where to install Prometheus for Kubernetes monitoring

I'm considering using Prometheus to monitor and analyze different metrics from my AWS EKS Kubernetes cluster running in Fargate, as it seems to be the standard tool during these times. However, ...
Nicolás García's user avatar
2 votes
1 answer
3k views

How to filter label_values in Loki?

I want to create a query variable in grafana using Loki that returns all the possible label values of the label baz such that they only include label values that exist where the label foo equals "...
user2324712's user avatar
0 votes
1 answer
546 views

Node-centric network metrics are very different from the sum of metrics of the Pods running on that node

Context I am trying to chart the network bandwidth usage of a node in 2 different manners: By looking at global metrics for that node By summing up the corresponding metric for each Pod To achieve ...
Antoine Cotten's user avatar