Usually this is best done by in-house engineers, I think. At least, that's what the founders at my startups did. You want to really be intimate with your metrics.

I suppose it could be productized, I am pretty surprised that there isn't a super-easy plug-and-play solution (a la Dropbox, Slack, etc). Probably because coming up with something that's customizable for different startups' needs must be really challenging.