Hi everyone! Here goes our latest bunch of interesting Kubernetes-related articles recently spotted online:

1. “Kubernetes: containers, and the “lost” SIGTERM signals” by Arseny Zinchenko.

“We have an API service with Gunicorn in Kubernetes that periodically returns 502, 503, 504 errors. I started debugging it, and found a weird thing: there were no messages in the logs about the received SIGTERM, so I first went to deal with Kubernetes - why doesn't it send it?”

2. “Stateful apps in Kubernetes. From history and fundamentals to operators” by Palark.

“In this article, we will explore how stateful apps work in Kubernetes and what you should consider before and while running your stateful components in K8s. To make it even more practical, we will cover several well-known K8s operators to tackle your ClickHouse, Redis, Kafka, PostgreSQL, and MySQL instances.”

3. “Understanding DNS in Kubernetes” by Povilas Versockas.

“In this post, we will cover the following: Overview of DNS Resolution and CoreDNS, the default DNS provider in Kubernetes; Kubernetes DNS policies, such as ClusterFirst, Default, and None, and their effects on pod DNS configurations. Differences between The GNU C Library (glibc) and musl libraries.”

4. ArgoCD Series by Maryam Tavakkoli, a CNCF Ambassador. “Part 1: Terminologies and Architecture" and “Part 2: (Basic) Core Concepts”.

“In this ArgoCD series, I aim to explain its concepts and terminologies from the beginning and provide a detailed technical guide on using it, all with declarative approaches.”

#articles

Our selection of the latest prominent software updates from the cloud native ecosystem:

1. Traefik 3.0 was released two months ago, but it’s an essential update we missed in our digests before. New features for this Cloud Native application proxy include support for WebAssembly, OpenTelemetry, Kubernetes Gateway API, SPIFFE, gRPC-Web, and production-ready HTTP/3.

2. Vitess, a Cloud Native database solution for horizontal scaling of MySQL, was updated to version 20. It brought automated and scheduled backups, enhanced DML support, and experimental multi-tenant imports in VReplication.

3. Coroot v1.3.0 was released. This Open Source APM & observability tool got support for monitoring MySQL and memcached, an automated discovery for database monitoring, an AWS integration, external calls tracing, and more.

4. Podman Desktop 1.11 has got an experimental light mode (which is called the most-requested feature), upgraded UI, node and volume listings in the Kubernetes functionality, and macOS Rosetta support.

5. Kubewarden, a policy engine for Kubernetes, was updated to 1.14. It comes with a new host capability that allows policies to fetch the container image configuration, a CEL policy capable of running Kubernetes VAP policies without any modifications, and a new CEL Policy on Artifact Hub.

6. KCL v0.9.0 brought several new features to this constraint-based record and functional language for configuration and policy scenarios. They include numerous new language and toolchain features (such as TOML format in kcl run and kcl import, adding dependencies from private third-party OCI Registries and Git repositories in kcl mod add), a new fast runtime mode, optimised performance for KCL IDE, new standard libraries, such as file for file input/output operations and template for writing template configurations, and much, much more.

#news #releases

Looking for a practical way to learn Kubernetes security? You might be interested in this project!

Kubernetes Goat provides you with a cluster that is “vulnerable by design". After deploying it, you’re getting easy-to-use access to 20+ scenarios covering various security aspects. Accompanied by guides, you can follow these scenarios to validate your knowledge and get new practical skills. Here are some of the techniques and technologies they cover:

- DIND exploitation and container escape;
- getting access to internal and non-exposed services;
- exploiting the misconfigured/overly permissive permissions;
- Docker & Kubernetes CIS benchmarks;
- kubeaudit for auditing Kubernetes clusters;
- Falco for detecting security issues;
- Cilium Tetragon for performing runtime security monitoring;
- Kyverno policy engine.

▶️ GitHub repo

#tools #security

Our selection of the latest prominent software updates from the cloud native ecosystem:

1. k8sgpt is a tool bringing AI power to simplify troubleshooting and scanning your Kubernetes clusters. With its most recent 0.3.38 release, it got support for two new AI providers, Ollama and IBM watsonx.ai.

2. Percona Operator for PostgreSQL was updated to v2.4.0, and several new features were introduced. They include fully automated upgrades of PgSQL major versions, support for PgSQL tablespaces, and using AWS IAM roles to access S3 buckets for backups.

3. KWOK (Kubernetes Without Kubelet) is a toolkit for simulating a K8s cluster with thousands of nodes. Its latest release, v0.6.0, brings numerous changes, such as sidecar container support in stage policy, Helm charts support, numerous improvements in Stage API and kwokctl (--all and --force flags for kwokctl delete cluster, nerdctl support, etc.).

4. Harvester, the hyperconverged infrastructure (HCI) solution from SUSE built on Kubernetes, was updated to v1.3.1. New features include support for NVIDIA vGPU, ARM64, HA two-node clusters, devices with frequent power interruptions or relocations (such as edge), managed DHCP, and Fleet integration.

5. KubeBlocks, “a control plane software that runs and manages databases, message queues and other stateful applications on K8s”, has released v0.9.0. Some of its highlights include support for topologies in ClusterDefinition API, managing horizontal scaling of distributed databases, using InstanceSets instead of StatefulSets to manage Pods, PostgreSQL PITR, MySQL Replication mode, Redis Cluster mode support.

#news #releases

Need to validate your Kubernetes configuration and do it fast? Try this tool in your CI or locally.

Kubeconform is a K8s manifest validator inspired by kubeval (which hasn’t been developed for years). It validates your manifests against official Kubernetes OpenAPI specifications and focuses on being highly performant. Here are some of its features:

- Adjustable strictness considering missing schemas, additional properties (not in the schema), and duplicated keys;
- A configurable set of kinds (or GVKs) to ignore or reject, file paths to ignore;
- Schemas caching and adjustable number of concurrent goroutines;
- Various output formats (including text, JSON, and JUnit);
- Support for multiple schema locations to validate CRDs (CustomResourceDefinitions) and OpenShift schemas;
- Ready-to-use CI integrations for GitHub Actions and GitLab;
- Installable via Golang package manager, Homebrew, and winget (Windows). Helm charts are also available.

▶️ GitHub repo

➕ A few related repos from various contributors:
- kubeconform-helm is a set of tools to test Helm chats with kubeconform
- helm-kubeconform is a kubeconform Helm plugin
- helm-kubeconform-action is a GitHub Action to validate Helm charts with kubeconform
- kustomize-plugin-kubeconform is a kubeconform plugin to validate manifests schema within Kusomize

#tools

Do you love Vim and kubectl? There’s something exceptional for you!

kubectl.nvim is a Neovim plugin providing Vim-like navigation for your Kubernetes cluster and familiar key bindings. Here’s what it offers today:

- Navigation through your K8s resources via Vim buffer, with hierarchy awareness (e.g. going through Deployment to a Pod and its container).
- Other UI features include coloured output, sorting by headers, and floating windows for additional data (descriptions, logs).
- Changing contexts and namespaces.
- Running custom kubectl commands.
- Executing into containers.
- Displaying a diff for configurations.

▶️ GitHub repo
📣 Reddit announcement

#tools #CLI

Good Monday, everyone! Here's the latest bunch of interesting Kubernetes-related articles we've seen online:

1. “Kubernetes Storage Performance Comparison Rook Ceph and Piraeus Datastore (LINSTOR)” by Gareth Anderson.

“Understanding Kubernetes storage is crucial for deployments that rely on persistent volumes within K8s. In this article, we’ll explore various software options for K8s storage based on online research [LongHorn, OpenEBS, Vitastor, Rook, Piraeus]. Additionally, we’ll delve into two specific choices that offer replicated block storage: Piraeus Datastore (LINSTOR) and Rook Ceph.”

2. “The Engines that run our Kubernetes Workloads” by Henrik Gerdes.

“Do container engines impact start-up time and memory consumption? Since I didn't find any real up-to-date comparisons, I took a look for myself and ended up comparing these implementations: runc, crun, gvisor/runsc, and youki.”

3. “Self-signed Root CA in Kubernetes with k3s, cert-manager and traefik. Bonus howto on regular certificates” by Remy van Elst.

“In this episode of 'Remy discovers Kubernetes', I'm setting up cert-manager, not with Lets Encrypt, but with a self-signed certificate authority. I'll also show you how to set up a regular certificate, one you've for example bought somewhere. I'll also cover nameConstraints to make the risk of compromise of your trusted root ca lower.”

4. “Istio from A to Y” by Quentin Joly, SRE at French government.

“Istio is an incredibly powerful and complete product, but it’s not without its flaws. It’s very easy to get lost in Istio’s configuration and end up with a mesh that doesn’t work as expected (plus, the logs are not always very explicit). That’s why it’s important to understand Istio’s concepts well before diving into the configuration of your mesh.”

#articles

Open Policy Agent tops the list of the most used Open Source tools for Kubernetes security, according to the 2024 edition of “The state of Kubernetes security report” unveiled by Red Hat last month (source). These results are based on a survey of 600 DevOps, engineering, and security professionals worldwide.

#tools #reports #security

Top 10 CNCF projects by their velocity during the last 12 months:

1. Kubernetes
2. OpenTelemetry
3. Argo
4. Backstage
5. Prometheus
6. gRPC
7. Cilium
8. Envoy
9. Istio
10. Keycloak

Overall, the KCL programming language demonstrated the most impressive growth by moving from 105th place to 67th.

Source: CNCF blog post; full data in Google Sheets.

#news #reports #cncfprojects

Our selection of the latest prominent software updates from the cloud native ecosystem:

1. Testkube, “the Kubernetes-native testing framework made for testers and developers,” announced its v2.0 release. It comes with the general availability of Test Workflows, a new architecture for executing tests that enables the parallelisation and scaling of hundreds to thousands of tests.

2. Dagger 0.12 brings an interactive debugging shell, interactive and much faster TUI, heavily improved Web UI (including local view, CI view, a new flame chart view, etc.), corporate network support, and compatibility mode among numerous new features.

3. Red Hat released OpenShift 4.16. It comes with Kubernetes v1.29, CRI-O v1.29, Admin Network Policy, Cluster Observability Operator v0.3.0, the new oc adm upgrade status command, user-managed load balancers, and many other new features.

4. Traefik v3.1.0 got production-ready Gateway API and improved support for WASM plugins.

5. Seabird, a Kubernetes IDE for the GNOME desktop, reached v0.5. Now, it has easily configurable port forwarding, new properties and columns displayed for various objects, resource navigation status, and a new object dialogue widget (optimised for touch device users).

6. Envoy Gateway has got lots of new features with v1.1. They include support for Zipkin tracing and Wasm extension, mTLS for external clients, numerous improvements in BackendTrafficPolicy, HTTP/2 settings in ClientTrafficPolicy, new Backend and EnvoyExtensionPolicy CRDs, support for Gateway API 1.1.0, and much more.

7. kube-scheduler-wasm-extension is a new project allowing you to extend the kube-scheduler with a custom scheduler plugin compiled to a Wasm binary. Its first version, 0.1.0, was released just yesterday.

P.S. The first beta version of Kubernetes v1.31 was also released last week.

#news #releases

Did you know that Kubernetes v1.31 is around the corner? It will be released in two weeks, and if you also think it’s time to learn about this update, here are helpful resources.

1. The “Kubernetes 1.31 – What’s new?” article from Sysdig lists some of 34(!) new alpha features and 11 enhancements graduating to stable. Particularly, they highlight HonorPVReclaimPolicy being enabled by default and a new custom profile option for kubectl debug.

2. The “Kubernetes Removals and Major Changes In v1.31” post in the Kubernetes blog lists the deprecations and removals in this release. E.g., they include the removal of all in-tree integrations with cloud providers as well as CephFS and Ceph RBD volume plugins.

2. The 1.31 Enhancements Tracking board on GitHub shows all release enhancements with their SIGs, status, and current stage (alpha/beta/stable).

4. “Kubernetes 1.31 Release Information” has a planned release timeline and various related links. According to it, v1.31.0-rc0 will be released tomorrow (July 30th), and v1.31.0 is planned for August 13th.

UPDATE (added on August 14th):
5. “Kubernetes 1.31: a security perspective” from ARMO covers v1.31 enhancements improving security.

6. “What To Expect From Kubernetes 1.31” on Cloud Native Now comes with some comments from the Kubernetes v1.31 release team lead, Sysdig developer leader, and OpenUK's CEO.

#news #releases

Good Monday! Here goes our latest bunch of interesting Kubernetes-related articles recently spotted online:

1. “The hater’s guide to Kubernetes” by Paul Butler, Jamsocket.

“We’ve been running Kubernetes in production for a few years now at Jamsocket, and I’ve found a good flow with it. Kubernetes serenity has been achieved internally. A big key to this has been carving out a small chunk of Kubernetes’ features and pretending the rest don’t exist. This post started as an internal guide to the way we use Kubernetes, so it’s not meant to apply prescriptively to every startup; nonetheless I think it’s a good starting place for avoiding many of the sandbars in the vast seas of Kubernetes.”

2. “The Kubernetes Troubleshooting Handbook” by Piotr Zaniewski.

“In this blog we will explore various techniques and tools to help with troubleshooting and debugging Kubernetes. Whether you’re an experienced Kubernetes user or just getting started, this guide will provide valuable insights into efficient debugging practices.”

3. “A Chaos Engineering Experiment”. “Part 1: Deploy on Friday? How About Destroy on Friday!” and “Part 2: Destroy on Friday: The Big Day” by Lex Neva, Honeycomb.

“We recently took a daring step to test and improve the reliability of the Honeycomb service: we abruptly destroyed one third of the infrastructure in our production environment using AWS’s Fault Injection Service. You might be wondering why the heck we did something so drastic. In this post, we’ll go over why we did it and how we made sure that it wouldn’t impact our service.”

4. “A skeptic's first contact with Kubernetes” by David Ventura.

“While there are a lot of tutorials covering the usage and operation of a Kubernetes cluster, along with basic descriptions of its components, they did not quite work for me. Many of these resources lightly cover what the components do, but often miss the underlying reason or the tradeoffs. This post aims to cover these concepts from a perspective I would've found useful, and it does not aim to be exhaustive.”

5. “Automated container CVE and vulnerability patching using Trivy and Copacetic” by Edgaras Apsega, a CNCF Ambassador.

“This blog post will guide you through setting up an automated system on MacOS for container vulnerability scanning and patching using copa (container patching automation) and trivy (a vulnerability scanner) tools that are under Cloud Native Computing Foundation governance.”

Happy reading & sharing! 🙌

#articles

Struggling with network policies in Kubernetes? This tool helps manage them using CLI or Web UI.

netfetch aims to “demystify network policies” in Kubernetes. To do so, it scans your cluster, visualises the network, and helps you improve the network policies. Here are its main features:

- Command-line (CLI) and dashboard (Web UI) interfaces, which vary a bit in their functionality.
- Scan a cluster, detect the Pods without network policies, and evaluate a security score.
- Build an interactive network map or save scan output as a text file.
- Improve network policies by creating default deny rules and suggesting new network policies for existing workloads.
- Network policy editing and previewing in Web UI.
- Written in Go and Vue. Installable via Helm, Homebrew (for Mac), or binaries.

▶️ GitHub repo

#tools #networking #security

Our selection of the latest prominent software updates from the cloud native ecosystem:

1. Inspektor Gadget, a collection of eBPF-based tools to debug and inspect Kubernetes resources and apps, released its v0.30.0. It introduced an initial support for WebAssembly (Wasm), printing kernel stack map, and generating deterministic tarballs for image export.

2. Cilium 1.16 was released with lots of new features resulting from 2969(!) commits. They include BGPv2 APIs, BGP ClusterIP advertisement, the new virtual network device called NetKit, port range support in Network Policies, L7 Envoy Proxy as a dedicated DaemonSet, Gateway API 1.1 and Gateway API GAMMA support, new ELF loader logic decreasing Cilium’s memory usage, CEL filters support in Hubble, and much, much more.

3. StackRox, the Kubernetes security platform from Red Hat, was updated to v4.5.0 with its Scanner V4 becoming GA (Generally Available), a new option to filter scanned images by vulnerability severities, and a few other new features.

4. Flagger, the progressive delivery tool from Flux, was updated to v1.38. It comes with a new Keptn metrics provider, ServiceAccount annotations support in the loadtester chart, and honorLabels for PodMonitor in the Flagger chart.

5. KEDA, a Kubernetes-based Event Driven Autoscaling component, got its v2.15.0 with two new scalers, for Dynatrace and Splunk.

#news #releases

Ever thought of running a Kubernetes cluster for your home needs? Perhaps even implemented it already? Well, in both cases, this “wife-approved HomeOps” project might come in handy…

The home-ops repo provides all the configurations you need to deploy a GitOps-controlled, Talos-based Kubernetes cluster with distributed block storage, backups, and other services. Here’s what this setup covers:

- IaC- and GitOps-driven configuration management based on Terraform, Flux, and Renovate;
- Networking powered by Cilium, ExternalDNS, and Cloudflare;
- Persistent storage based on Rook/Ceph;
- Backups with VolSync;
- SSL certificates management via cert-manager;
- ingress-nginx as its Kubernetes ingress controller;
- SOPS and External Secrets Operator for secrets management;
- Observability stack with kube-prometheus-stack, Prometheus operator, Blackbox exporter, and Loki.

▶️ Main GitHub repo
▶️ Repo with a template used as the foundation for home-ops (it comes with getting started instructions)

#tools #gitops

Our selection of the latest prominent software updates from the cloud native ecosystem:

1. Kueue, a job queueing controller developed under the Kubernetes SIG, has recently seen its v0.8.0 with numerous new features. They include new commands (such as stop|resume localqueue and create|list resourceflavor), CLI autocompletion support, experimental support for Helm charts, more granular preemption condition reasons, and more.

2. KubeEdge, a CNCF incubating project, was updated to v1.18. It got high availability support for RouterManager, node authorization mode in CloudCore, device status reporting, and on-the-fly configurations in the keadm tool.

3. Microcks, a CNCF sandbox project for API mocking and testing, released 1.10.0 with stateful mocks. It also migrated from JUnit 4 to JUnit 5, brought MQTT and RabbitMQ support to the Uber distribution, and several other improvements.

4. Rancher v2.9.0 was released with Kubernetes v1.29 and v1.30 support, RKE deprecation (re-platforming to RKE2 or K3s is required), improved UI, authentication support for generic OpenID Connect providers, and other new features.

5. Trivy, a security scanner from Aqua Security, was updated to 0.54. It introduced VEX (Vulnerability Exploitability eXchange) repository integration, vulnerability support for SPDX formats, Azure Linux 3.0 support, and openSUSE Tumbleweed detection and scanning.

6. Dex, an OIDC identity and OAuth 2.0 provider, got its v2.41.0 with gRPC Connectors API support, enriched logging, and several other improvements.

#news #releases

"Keep the space shuttle flying" (c)

Source on GitHub; Reddit discussion.

#fun

Here goes our latest bunch of interesting Kubernetes-related articles recently spotted online:

1. “How LinkedIn moved its Kubernetes APIs to a different API group” by Ahmet Alp Balkan & Ronak Nathani Butler, LinkedIn.

“We recently migrated one of LinkedIn’s major internal custom Kubernetes APIs to a new API group, while also introducing major changes to the API. [..] This article will explain why we moved this API between Kubernetes API groups, the limitations of the API versioning machinery in Kubernetes, and how we created our own solution – a “mirror controller” – to seamlessly migrate to a new API while the old API was actively being used.”

2. “Argo Rollouts – What Is It, How It Works & Tutorial” by James Walker, Spacelift.

“This article will explain more about Argo Rollouts, how it works, and how to get started using it in your own cluster. We’ll finish by sharing a simple example of how to launch a canary rollout for a Kubernetes deployment.”

3. “Kubernetes security fundamentals: Authorization” by Rory McCune, Datadog.

“In this post, we'll focus on another key aspect of Kubernetes security: authorization, which allows a cluster to know if the requester is allowed to take a specific action. Authorization is the second step (after authentication) in the process that requests to the main API server go through before they’re applied to the cluster.”

4. “Full-Guide: How to Easily Publish Helm Charts on GitHub with GitHub Pages” by Artem Lajko.

“In this short blog, I am going to show you how you can easily publish your locally written Helm chart on GitHub using GitHub Pages. We will cover the following steps: Create a Helm Chart; Create auto releases; Use GitHub workflows to generate Helm docs on push; Automated test the Helm chart on a Kind cluster after push; Publish it on GitHub using GitHub Pages; Use the published Helm Chart; Add the Helm Chart to artifacthub.io.”

5. “ArgoCD/Flux vs Kluctl” by Alexander Block.

“Kluctl is very flexible when it comes to deployment strategies. All features implemented by Kluctl can be used via the CLI or via the Kluctl Controller. This makes Kluctl comparable to ArgoCD and Flux, as these projects also implement the GitOps strategy. This comparison assumes that you already know Flux and/or ArgoCD to some degree, or at least have heard of them.”

Enjoy reading & sharing! 🙌

#articles

Can the automatic shutdown of your Kubernetes workloads be beneficial? It could cut your cloud costs and reduce your carbon footprint. The new sleepcycles tool aims to do just that, allowing you to:

- Define the working hours (shutdown & wake up schedule) for your K8s resources using the cron expressions;
- Specify these schedules for Deployments, CronJobs, StatefulSets, and HPAs (HorizontalPodAutoscalers);
- Specify the needed time zones;
- Use these features for applications provisioned with Argo CD.

The project is implemented as a Kubernetes controller following the K8s operator pattern. It’s written in Go and can be installed via a Helm chart.

▶️ GitHub repo
📢 Project announcement

#tools

How about starting this week with favourite Kubernetes interview questions?

Here are the top interview questions about K8s, based on a recent discussion on Reddit and its users’ feedback:

- What’s the difference between a Pod, a Service, and a Deployment?
- Let's say you are joining a company and you are working with several product teams that have already undergone the process of containerizing their applications. How would you go about deploying and operating these applications?
- Can you explain to me how applications in Kubernetes accept traffic from clients?
- Can you describe the specific steps that are happening when a client hits a load balancer that's pointing to your Kubernetes nodes?
- How do you troubleshoot a CrashLoopBackOff?
- My application in a Pod is unable to reach the API server via the kubernetes endpoint. Can you debug this issue for me?
- If I want to ensure some data survives a Pod restarting how would you do it?

… and perhaps the funniest of all those questions:

- If you were a Kubernetes resource, what would you be and why? 🤣

#career #fun