I was planning to create a github issue in the drone-vault repository but it appears issues are disabled.
We had this plugin running fine until we updated images over the weekend. In the recent update, it appears the Drone-vault plugin is falling into an infinite loop and constantly requesting auth tokens from Vault. Here is a visual of logs from our Vault cluster:
Those big spikes are hundreds of thousands of Auth Update requests coming from the drone-vault plugin. The gap in the middle is when it actually took down both Vault and Consul.
Our solution was to revert to a previous drone-vault image, but it appears the drone/vault Dockerhub doesn’t have any images previous to the last update. We actually had to salvage the old image from one of our Kubernetes nodes and save it off in our own ECR repository.
I haven’t dug into the code at all, but I’m wondering if it was something to do with this: drone-vault/kube.go at master · drone/drone-vault · GitHub
Which was pushed 2 weeks ago as a part of this commit: refresh kubernetes token · drone/[email protected] · GitHub
I don’t see protection around an infinite loop there, but I also don’t know the code at all.
If nothing else, I have to suggest you keep old images around in Dockerhub. If we weren’t able to salvage the old image from weeks ago then we’d be forced to build our own image from a previous state in the repository.