Kubernetes

💡 Want to know how it really works?

Check out the full technical case study here →

Anyone here using Datadog?

It’s a wonderful tool, that quickly can become expensive. My client approached me to reduce our logging costs by removing unnecessary sources. The company was feeding logs to datadog from almost every source.

Costs were horrible. Consider 5 TBs of logs. No, not monthly. Daily. That gives roughly 150 TB / month, 1.8 PB a year. Petabytes.

Background

During one of my assignments, we were evaluating Karpenter. While being a rather cool piece of software, it caused us some pain. The general idea was to deploy Grafana Loki and use an autoscaling tool to maintain node pools automatically.

Karpenter seemed to be the perfect tool for this use case. However, it had (maybe due to my misconfiguration) some undesired side effects. It was quite aggressive in adding and removing nodes, which caused disruptions.