Automated testing in a DevOps environment, particularly in validating the application security, can quickly shift from an enabler to an impediment, particularly when replicated across seemingly similar projects. It is tempting to rubber stamp successful DevSecOps tests and systems across each new product, but one size does not fit all. We will examine real world examples to show how to turn that big hammer into a Swiss Army knife to helps implement the right information security risk strategy.