Prometheus is the de-facto standard for gathering metrics on Kubernetes and making sure that you discover problems early - before they impact users. But setting up a well-oiled Prometheus stack is surprisingly hard! Many companies have such a high volume of daily Prometheus alerts that they end up missing critical issues among the noise. In this talk, we will cover the architectural details you must understand about Kube-Prometheus-Stack to be successful. Then we will dive into best practices and techniques that you can use to make sure that no critical issue goes undetected, and that your team is not overwhelmed with alerts no matter how big your environment and how many clusters you have.