System monitoring and reporting with Jenkins

Question

Are there any plugins that would enable me to distribute system monitoring and reporting via Jenkins, if I provided it with data in a required format. E.g. I would like to write a simple script that would display the status of every server and maybe some metrics. If that runs, say every hour, and dumps data in an XML or CSV or whatever conventional format of the plugin, can I distribute reports with that data using Jenkins using a special plugin?

I understand there are separate apps that do system monitoring but the fact of the matter I am trying to make Jenkins my one-stop-shop for what my less technical usership needs so, because I enabled them to run builds and deployments using Jenkins as they wish, I would prefer not to give them a different interface with a different login and other overhead.

Answer 1

having "ab"used Jenkins to be many things, the most sensible (IMHO) you can do is expose monitoring dashboards via build report page. I think you should not even put Jenkins and monitoring under the same url, due to the "who will guard the guard" problem.

Also, nowadays monitoring tools are too versatile and have too many "faces" (a face for each characteristic user, and some specific users prefer to have their own personal "face" to monitoring) About "reporting" I don't know what do you mean. if its reporting of builds statuses - READ ON. Also there are other things to let user go through (logs, alerts, test reports)

So what I would suggest to address monitoring visibility is:

1. monitor your Jenkins in your monitoring :) (builds, statuses, etc.)
2. define monitoring url inside Jenkins (e.g. as Global Env Variable)
3. let's assume you have a job that deploys service $svc on environment $env. using Groovy you can generate:
   1. generate a url to point at that monitoring service "dashboard" for that service in that env. let's call it serviceMonitorUrl
   2. environmentMonitorUrl, and others
   3. using addSummary and related stuff you can add to the build report a list of links to monitoring
   4. you can decorate your build history column with "labels" pointing at branch, env, hash, whatever
   5. your userdom will get build page with those links, and will be able to "click" them.

Having said that, there are Jenkins pipelines visualizations you can also use to visualize Jenkins jobs statuses, etc. e.g. this one.

Generally: Jenkins is good for CI related scheduling. But the truth is - CI should not be the focal point of users. CI should be "transparent" until something is broken. Thus making users "aware" of Jenkins (CI), and doing everything through it, unless business requirement is met by it (like company's product is being controlled/operated though Jenkins) - you should not do this. This kind of "knowledge" is not really serving the users. They need to be able to see where to look for information easily, not be specific application's advanced users.

Think what happens when you decide to migrate from Jenkins to something else. you are hooked now.

(it isn't necessarily bad, just be aware of it)

Answer 2

If I understand the question correctly, you want a way to distribute status information of other servers, not specifically the Jenkins server, to less-technical users. Is that correct? I guess the follow-up question would be, do the servers in question already have some sort of API or other interface to query the status?

If so, then your script could do something as simple as using sh to call curl or wget to query the information from those servers. Since you seem to be ok with using whatever output format that a plugin would need, perhaps you can save the output as HTML and then use the Jenkins HTML Publisher plugin to generate reports. These reports would be viewable on the Jenkins build page.

Alternatively, you could output the status information as text and include it in the body of an e-mail sent to the less technical users via the Jenkins Email-ext plugin

Answer 3

SSO

Centralizing user logins is certainly a reasonable goal. There are various "single sign-on" solutions out there. But even something hacky like copying a htpasswd file around can work. But squeezing everything into the jenkins UI just because that is the only login people have seems like asking for trouble. I'd let everyone see my dashboards without logins before I tried to make it all work through jenkins. You'll want to ease logins to other systems like email or PagerDuty eventually so finding a nice general solution to this will be handy for a long time.

Dashboards

There are lots of nice ways to do dashboards. If you want to do more yourself and not pay monthly charges I'd look at Prometheus. If you want to do less yourself and monthly charges are survivable I'd look at datadog. There are are other choices available of course, but these are some of the better result-for-the-effort-put-in choices out there right now. Either way it will be a much more natural process than reinventing the wheel to make it work through jenkins.

Any of these solutions will take data from a variety of inputs and let you have a coherent dashboard. Most of them will also let you interact with the graphs to zoom in on certain ranges of time or subsets of servers. You can usually let your users create their own dashboards as well which will reduce the number of innane tweaks that you're asked to make over time.