With tools like Riemann, New Relic, etc., is there any benefit to building custom app metrics tools for monitoring application health?
A third party app metrics tools is great if it already covers all (or most of) the parameters you want to monitor and it presents the information in a convenient enough manner for your use case - you just need to (find it first and then) learn how to install/use/maintain it and then do that.
- avoids the software development and maintenance costs, which can be quite high depending on the capabilities
- can be immediately used, a custom tool needs to be developed first
- technical support (not always great, tho)
- standardisation: if the tool is a popular one - expertise on it can be requested on a job description instead of having to train every user on it
However there are many cases in which custom development is needed, some already mentioned by Michael: reliability, security/confidentiality, lack of integrations for your custom applications.
Depending on each particular case, custom development can go from tiny scripts extending the coverage of a third party tool to complex wrappers assembling multiple such tools to achieve the necessary monitoring capabilities
Such development is not always lightweight:
- it may require in-depth tool knowledge and/or language expertise outside of your team
- it may need management for heavy/complex or even conflicting dependencies
- the adaptation layers for the tool's actual usage environment may be very heavy
Sometimes the cost of such development is higher overall than that of developing a custom system.
Othertimes the functionality actually needed from the third party tools is a relatively small percent of its capability set and the overhead for learning/installing/operating the tool is simply not worthy.
You guessed it - I wrote my own monitoring tool :)