Hey, I'm trying to get this project off the ground and looking for feedback, users, and maybe even some contributors.
Idea is simple way to just get decent anomaly detection on metrics. Dagster does the orchestration on top of whatever database you use. You just define batches of metrics in SQL and Anomstack will then ingest, train, score and alert on your metrics for anomaly detection.
Bit more work to do but just wanted to put it out there.
Basically I've built versions of this in almost all my last jobs and it's silly and kills me each time. Main use case in my head is like for daily, hourly, business metrics etc. But it can be any metrics really, just not anything too near real-time. Also not sure about how well it would scale but I figure for a lot of use cases being able to just stand up a decent anomaly detection system on your metrics could actually be very useful.
Anyway would love to hear any feedback, ideas, or suggestions.
We also have some specific logs collectors too - i think in here might be best place to look around at the moment, should take you to the logs part of the integrations section in our demo space (no login needed, sorry for the long horrible url, we adding this section to our docs soon but at moment only lives in the app)
Nice to see that the log analysis is being worked on.
I'll see if I can figure out the integrations you pointed out. They look more like they are aimed at monitoring the metrics of the tools, not using the tools to aggregate logs. Right?
The way most ops systems treat logs and metrics as completely separate areas has always struck me as odd. Both are related to each other, and having them in the same system should be default. That's why I've put as much effort into the ELK Stack as I have. They've seemed to be the only ones who have really grasped that idea. (Though it's been a year or two since I've really surveyed the space...)
One question not log related, is it required to sign up for a cloud account to get multiple nodes displaying in the same screen? From the docs on streaming, I think you can configure nodes to send data to a parent node without a cloud account, but I either haven't configured it properly yet, or something else is in the way, since the node I'm trying to set up as a parent isn't showing anything from the child node.
FYI, you need to add the api-key config section to the stream.conf file on the parent node in order to enable the api key and allow child nodes to send data to the parent node. I thought it went into the netdata.conf file... I also kinda wonder why it matters what file has what config since the different config sections all have section headings like `[stream]` or `[web]`.
So, the answer to my question is that you can get multiple nodes showing up without a cloud account. Just have to configure it correctly.
It uses the same actual dashboard as cloud so that we only have one dashboard to maintain so you get the cloud dashboard locally basically and the parent can then kind of act like its own little Netdata Cloud.
A handful of features not available this way since they depend on the metadata being stored in cloud as opposed to on a parent node but we are trying to bridge that gap where possible such that the metadata could actually live on a parent.
> So I turned to Netdata. A one liner on each server and we had super sexy and fast dashboard for each server. No birds eye view, but fine. I then spent maybe 3-4 days trying to figure out how to get alerting to work (just email, but fine) and get temperature readings (or something like that).
I work in Netdata on ML. Just wanted to mention that as of last release a parent node will show all children in the agent dashboard so if doing again as of today a parent netdata might have got you the birds eye view as a starting point https://github.com/netdata/netdata/releases/tag/v1.41.0#v141...
(of course we also have Netdata Cloud which would have probably worked too but maybe was not as built out 3 years ago as is now - but don't want to go into sales mode and get blasted :) )
Cool! - we always looking for feedback, feel free to hop into our discord, forum, or GH discussions (links here: https://www.netdata.cloud/community/) to leave any feedback or ask any questions if you run into any issues.
(cheers for the mention here too - always nice to try get some feedback and discussion going on HN as its so candid :0 )
(Not trying to be smart or anything, genuinely trying to get feedback as feedback from random users on HN is actually a really good and different audience than maybe we usually hear from so just trying to get a couple of bullet points maybe if you have a few minutes).
Idea is simple way to just get decent anomaly detection on metrics. Dagster does the orchestration on top of whatever database you use. You just define batches of metrics in SQL and Anomstack will then ingest, train, score and alert on your metrics for anomaly detection.
Bit more work to do but just wanted to put it out there.
Basically I've built versions of this in almost all my last jobs and it's silly and kills me each time. Main use case in my head is like for daily, hourly, business metrics etc. But it can be any metrics really, just not anything too near real-time. Also not sure about how well it would scale but I figure for a lot of use cases being able to just stand up a decent anomaly detection system on your metrics could actually be very useful.
Anyway would love to hear any feedback, ideas, or suggestions.