Thanks Ryan! We primarily used Jenkins for our CICD pipelines, we didn't get a chance to explore other tools. However, I presume, the challenges of CICD in k8s, especially git flows and release tracking would exist in other tools too- mostly due to the lack of consensus in standards around this area. Please do share your insights if you've worked on any of those tools.
As for reasons to absolutely consider using Kubernetes, I would say - scaling & distributing load. That is, your use case is combating highly variable load and needs a mechanism to scale workloads and distribute traffic across several instances. Of course, the architecture for this, must be built from ground up to have decoupled services(microservices) to be effective. For instance, the use case must not rely too much on state / ACID properties of transactions- this is something that has certainly affected us in the past; a multi master DB across data-centres maintaining ACID properties can be a real challenge.