Observability (sh.itjust.works)
submitted 2 weeks ago* (last edited 2 weeks ago) by to c/selfhosted@lemmy.world
 

I run a quite large vps. now and i want to monitor it ive been using beszel an uptime kuma, does anyone know a better solution ive tried grafana an prometheous. but it was not as good as i was hoping. i saw netdata on selfh.st does anyone have expierance with that.

you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 2 weeks ago (6 children)

What would you recommend for historic metrics? Specifically for podman containers.

  • source
  • parent
  • hideshow 6 child comments
  • [–] 1 point 2 weeks ago (4 children)

    Prometheus and Grafana are my typical recommendation there, with the podman exporter. Prometheus for time series data, Grafana to visualize and alert.

    One thing I think many folks get wrong is thinking about Grafana (or any other dashboard) as the way to watch for problems, but that requires you to look at them. They should be thought of as a way to evaluate for conditions that there was a problem, then create an alert based on the indicators. This way you can address it before it becomes a problem, or at a minimum be aware that conditions are leading toward a problematic state.

    If its repeatable and managed with a container restart or something, you can automate that from the alert, and never have to look again.

    If its something you can't control (for example, availability of a 3rd party service), you can use it as a way to be notified of the status change. I have a custom exporter I wrote for work to keep track of some services that result in a bunch of posts to Teams by people asking if its working for them. So I have an automation that posts to Teams when a service has a problem so my IT guy doesn't need to respond to all of them individually.

    So I'd recommend focusing on good alerts and automations over beautiful dashboards first (though dashboards can be a lot of fun to make!)

  • source
  • parent
  • hideshow 4 child comments
  • [–] 2 points 6 days ago

    Hmm, I wonder if I even need a dashboard (other than for fun). Perhaps I should do as you say and first look at what I really need to make sure is working (backups) and then monitor that and set up alerts for it to begin with. Thanks for the insight!

  • source
  • parent
  • [–] 1 point 2 weeks ago (2 children)

    I'd suggest VictoriaMetrics over pure Prometheus - better ram footprint is pretty useful nowadays.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 2 weeks ago (1 child)

    They played a good bit fast and loose with their claims some time back (even pulling a few things after an "oopsie - decimal point in the wrong spot!" type issue), and I kind of wrote them off after that. I'll have to check in on them again.

  • source
  • parent
  • hideshow 1 child comment