📎📎 Cozystack v0.38: VPC networking, VM console & faster API

This release introduces Virtual Private Cloud (VPC) support, enabling advanced networking capabilities for tenant applications. We've also added VNC console support in the dashboard, made Kubernetes worker versions configurable, and delivered numerous improvements and fixes across the platform.

🗜 Virtual Private Cloud (VPC) Networking
Cozystack v0.38.0 introduces Virtual Private Cloud (VPC) support, enabling platform administrators to create isolated network segments for tenant applications. VPCs provide network isolation and allow fine-grained control over network topology, subnets, and routing. Each VPC can contain multiple subnets, and administrators can configure subnet details including IP ranges, gateway settings, and DNS configuration.

The VPC feature integrates seamlessly with the Cozystack dashboard, allowing users to view and manage VPCs and their subnets through an intuitive interface. Subnet details are exposed in the dashboard as tables, making it easy to understand network configuration at a glance. VPC configuration is stored in ConfigMaps with predictable naming, ensuring reliable access to subnet information.

This feature is particularly valuable for multi-tenant environments where network isolation is critical, and for applications that require specific network configurations or routing rules.

🥁 VNC Console for Virtual Machines
The Cozystack dashboard now includes a built-in VNC console for virtual machines, enabling users to access VM console directly from the web interface without requiring external tools. This feature provides immediate access to virtual machine consoles for troubleshooting, configuration, and maintenance tasks. The VNC console integration streamlines VM management workflows and improves the user experience by keeping all VM operations within the Cozystack dashboard.

🪑Additional Repositories
- Introduce boot/install mode: Introduced boot/install mode in boot-to-talos tool.
Handle valuesFiles from cozypkg.cozystack.io/values-files annotation: Added support for handling valuesFiles from annotation in cozypkg.

📚 Docs & ecosystem updates
- New and updated docs for VPC networking and its configuration.
- System resource planning recommendations and storage updates.
- Improved OpenAPI UI docs, updated managed apps reference, naming conventions, LINSTOR and golden image guides, and other quality-of-life documentation improvements.

All changes & improvements: v0.38.0, v0.38.1, v0.38.2, v0.38.3, v0.38.4

Huge thanks to everyone who contributed to the 0.38 line:
@IvanHunters, @insignia96, @kvaps, @lllamnyp, @nbykov0, @scooby87

Special shout-out to our first-time contributor:
@tabu-a — welcome aboard! 🚀

Join the community
Telegram group
Slack group (Get invite at https://slack.kubernetes.io)

Recordings from CozySummit'25 Virtual

🇨🇭 How we build a multi-AZ cloud in Switzerland - Matthieu Robin, Hidora

A real production story from Hidora on designing and operating a resilient multi-availability-zone cloud — architecture, challenges, and lessons learned.

00:00:00 Introduction: Building a multi-AZ cloud provider in Switzerland using CozyStack.
00:01:00 Hidora's history: Transition from consulting to a PaaS offering based on Jelastic.
00:02:30 Problems with proprietary solutions: Vendor lock-in and limitations in answering customer needs.
00:04:00 The shift to Enterprise: Requirements for data sovereignty, high availability, S3, and VPC.
00:05:30 Choosing the right foundation: Selecting Kubernetes as the orchestrator and rejecting OpenShift due to slow development.
00:07:00 Core philosophy: Full Open Source by Design to ensure independence.
00:08:00 Discovering CozyStack: Aligning with the project's vision and choosing to sponsor its development.
00:09:40 Roadmap requirements: Building critical features like Backup as a Service, PVC Encryption, and S3 support.
00:10:30 Journey to Production: Spending months on operation testing (HA, RPO/RTO measurements).
00:11:45 Security Validation: External and internal penetration testing results.
00:12:30 Launching I Cube: Achieving multi-AZ replication in Switzerland with high availability.
00:14:00 Future plans: Developing an API to manage CozyStack services and integrate third-party services.
00:15:00 Q&A: Infrastructure technology used (Talos) and team learning curve.
00:16:30 Q&A: Control Plane resource consumption and comparing LinStor performance against Ceph.
00:18:00 Q&A: Comparison with Harvester and the need for VPC/VLAN support in CozyStack.

▶️ Watch the talk: https://www.youtube.com/watch?v=AYrr7pBIAe8

Recordings from CozySummit'25 Virtual

🧪 Home Lab to the Moon and Back - Kingdon Barrett, Navteca, LLC

How small experiments and home labs can grow into serious cloud platforms — a fun but deeply practical talk on learning by doing.

00:00:00 Introduction: The problem of home lab heat and high energy consumption.
00:01:30 Speaker background: Flux maintainer and Home Lab Gremlin.
00:03:00 Addressing the heat problem: Shutting off unnecessary test clusters.
00:04:30 CozyStack operations in Home Lab: Challenges with physically rebooting nodes and service recovery time.
00:05:40 Solution for quick recovery: Running high-availability loads in Wasm (Wom bundles).
00:06:45 CozyStack's Managed Kubernetes structure (Cluster API, Kamaji, ephemeral worker nodes).
00:08:00 Strategy: The difficult goal of migrating the Home Lab to efficient ARM 64 architecture.
00:09:30 Alternative: Migrating workloads to cheaper and faster AWS Graviton (ARM 64) instances.
00:10:45 Preparation for Multi-Arch: Building multi-arch container images.
00:11:30 The ARM 64 Installation Barrier: Hitting a snag with a required leader election component.
00:12:30 Challenges of cloud migration: No proper Layer 2 network; using Cloud Init instead of Matchbox.
00:14:00 Infrastructure Architecture: Terraform for AWS setup (VPC, Bastion Host, private network).
00:15:30 Optimization Techniques: Using a Registry Cache to save egress costs.
00:16:30 Technical difficulties: Issues making Tailscale work due to IPv6/IPv4 conflicts.
00:17:30 Workflow: Building a custom Talos OCI image for CozyStack extensions.
00:18:30 Q&A: Possibility of a hybrid approach (Self-hosting primary + Cloud secondary) and data locality.

▶️ Watch here: https://www.youtube.com/watch?v=DnnX0pedWwg

Recordings from CozySummit'25 Virtual

🧩 Extensibility without chaos: lessons from building Cozystack - Timofei Larkin, Ænix

Lessons learned from building Cozystack: how to design extensible systems without turning them into unmaintainable monsters.

00:00:00 Introduction: Role of the maintainer in keeping the CozyStack architecture simple and clear.
00:01:30 The challenge: CozyStack must serve diverse users (Home Lab to Enterprise) requiring extreme reliability.
00:03:00 Pitfalls of Overengineering: Lessons from the highly customizable ETC Operator (unbounded customization leads to complexity).
00:05:00 Philosophy of Restraint: Examples from the Go language (single for loop).
00:06:00 Lessons from the Monotonic Clock incident: Taking time for small, surgical changes rather than rushing a big new feature.
00:08:45 Applying the lessons: Building an AWS Security Groups tool for CozyStack.
00:10:30 Solution: Mapping Pod ownership uniformly via Helm labels for a completely transparent extension.
00:12:00 Small, well-placed changes lead to large benefits (Example: Open API Dashboard).
00:13:00 Extensibility when simple changes are insufficient: Learning from Kubernetes' move from in-tree volume plugins to CSI.
00:15:00 Designing the Backup API: Difficulty due to diverse workloads (VMs, cluster databases, stateless apps).
00:16:30 Solution: Implementing the Strategy Pattern (inspired by Cluster API).
00:17:30 Conclusion: Key takeaways (Keep the Core Clean, Give users a way to add behavior, Discover the Patterns).
00:18:20 Q&A: Building and documenting the open-source management interface dashboard.
00:19:30 Q&A: Strategy for keeping up with Kubernetes versions.

▶️ Watch the session: https://www.youtube.com/watch?v=SIWvlme58Bk

Recordings from CozySummit'25 Virtual

☁️ From AWS EC2 to Cozystack: A Beginner’s Roadmap to Cloud Independence - by Kirti Goyal

A practical guide for teams thinking about moving away from hyperscalers toward self-hosted and independent cloud platforms.

00:00:00 Introduction: Seeking cloud independence by owning infrastructure instead of renting it.
00:01:00 CozyStack flips the model: Building your own cloud with control over resources and architecture using Kubernetes API.
00:02:00 CozyStack solves cost control, cloud independence, and simplified deployment.
00:02:30 CozyStack Architecture Overview (OS, Talos, Kubernetes, and Services).
00:03:00 AWS to CozyStack: Mapping managed services (EC2 to VMs, EKS to Managed K8s, S3 to Volumes).
00:04:00 Multi-tenancy: Supporting hard multi-tenancy to solve the "noisy neighbor" problem.
00:05:00 Target Audience: Startups, AI/Edge workloads, Government sectors, and students.
00:06:00 Q&A: Missing features compared to traditional cloud providers.
00:07:00 Q&A: Cost comparison and community engagement.

▶️ Watch here:
https://www.youtube.com/watch?v=mMDZzwHI2mI

🎥 Cozystack community meeting 2026-01-08: Backup system, Release 1.0, Packeges API, Linstor updates

https://youtu.be/LcNBH8eBvnA

Cozystack is a free PaaS and framework for building clouds | CNCF Sandbox Project

Agenda and notes

- CozySummit 2025 recordings playlist https://www.youtube.com/watch?v=Y6JscJoK5JA&list=PLj6h78yzYM2NZ1_Y9LVDhjtlcl3g5tKiY
- Backups API demonstration
- Version 1.0: What is left for Cozystack v1.0 release
- Packages API: Cozystack-operator and new packages approach
- Talm updates: encryption, project root detection, talosctl embedding, license update, talos 1.12 support
- boot-to-talos: automatic bond and vlan configuration
- cozypkg to cozyhr: tool for managing helm releases
- local-ccm: introducing new CCM for local nodes management
- LINSTOR enhancements: smart linstor-scheduler, LUKS patches and guard for dual-primary mode

Open Floor:
- OIDC support for grafana and Kubernetes
- Does Cozystack provide a built-in workflow engine similar to Argo Workflows (with triggers, DAGs, and event-based execution), or should external tools like Argo Workflows / Argo Events be deployed inside tenant Kubernetes clusters?

Join the community:
Telegram group t.me/cozystack
Slack group (Get invite at https://slack.kubernetes.io)

Cozystack resources:
https://cozystack.io
https://cozystack.io/docs/get-started
https://cozystack.io/blog
https://github.com/cozystack/cozystack

Ænix resources
https://aenix.io
https://xn--r1a.website/aenix_io
https://xn--r1a.website/aenix_community

💪💪 Cozystack v0.39: Streamlined Management & Enhanced Telemetry

Release v0.39 focuses on consolidating platform management and boosting observability with unified monitoring dashboards. It also introduces more robust handling of storage and network resources. Major highlights include the shift to Grafana Alloy for metrics collection, improved stability, and a focus on the overall reliability of all platform components.

We’ve overhauled our monitoring stack: Grafana Alloy now replaces the previous Prometheus agent and node-exporter setup. As a more modern and versatile tool for metrics, logs, and traces, Alloy provides a more flexible way to handle telemetry. The new telemetry is built directly into the Cozystack dashboard, giving you full visibility into cluster components from the moment you install it.

All changes and improvements: v0.39.5, v0.39.4, v0.39.3, v0.39.2, v0.39.1

Huge thanks to everyone who contributed to the 0.39 line!

📎 Join the community
Telegram group
Slack group (Get invite at https://slack.kubernetes.io)

Recordings from CozySummit'25 Virtual

🔐 SeaweedFS S3 API in 2025: Enterprise‑grade security and control - Chris Lu, SeaweedFS

Enterprise-grade security, access control, and S3 compatibility — what modern object storage looks like today.

00:00:00 Introduction: Overview of SeaweedFS history and the focus on new Enterprise S3 features for 2025.
00:01:40 Server-Side Encryption (SSE-S3): Server-managed keys for automatic encryption at rest (ideal for logs/backups).
00:02:45 Server-Side Encryption (SSE-KMS): Using external Key Management Systems for compliance and audit trails.
00:03:45 Server-Side Encryption (SSE-C): Customer-provided keys for full control (suited for financial/governmental use cases).
00:04:45 Technical details: Support for FIPS 140-3 and efficient range request handling.
00:05:40 Object Versioning: Protecting against accidental deletions using deletion markers.
00:07:00 Object Lock (WORM): Retention modes (Governance and stricter Compliance).
00:08:00 Object Lock: Legal Hold mechanism for litigation and auditing purposes.
00:08:45 Access Control: Enhanced IAM, AWS-style Bucket Policies, and conditional operations support.
00:09:30 Architectural Change: S3 API servers bypass the Filer for data transfer (improving latency and throughput).
00:11:00 Future Direction: Supporting structured data via a new Message Queue component.
00:12:00 Message Q: Schema-based messaging, conversion to Parquet, and basic SQL queryability.
00:13:00 Kafka Integration: SeaweedFS acts as a Kafka Gateway with native protocol support.
00:14:00 Summary of new features: Encryption, Governance, Performance, and Message Q.
00:15:00 Q&A: Comparison with Ceph (better at handling many small files and horizontal scaling).

▶️ Watch the talk: https://www.youtube.com/watch?v=n0hRleSLAcc

😍😍 Cozystack v0.40 — Enhanced Storage & Platform Architecture

Spotlight: Optimized Pod Placement with LINSTOR Schedule
Cozystack now includes a custom Kubernetes scheduler extender to help Kubernetes make better placement decisions for pods that use LINSTOR storage. When a pod requests LINSTOR-backed storage, the scheduler communicates with the LINSTOR controller to find nodes that have local replicas of the requested volumes. This way, pods are scheduled on nodes with existing data to minimize network traffic and improve I/O performance.

The scheduler features an admission webhook that automatically routes pods that require LINSTOR CSI volumes to the custom scheduler, ensuring seamless integration with no manual config required. This feature significantly improves performance for workloads using LINSTOR storage by reducing network latency and improving data locality.

Learn more about LINSTOR in the documentation.

SeaweedFS Traffic Locality
SeaweedFS has been upgraded to version 4.05 with new traffic locality capabilities that optimize S3 service traffic distribution. The update includes a new admin component with a web-based UI and authentication support, as well as a worker component for distributed operations. These enhancements improve S3 service performance and provide better visibility through enhanced Grafana dashboard panels for buckets, API calls, costs, and performance metrics.

The traffic locality feature ensures that S3 requests are routed to the nearest available volume servers, cutting down latency and improving overall performance for distributed storage operations. We’ve also added TLS support for management components to keep your storage operations secure.

ValuesFrom Configuration Mechanism
Cozystack now uses FluxCD's valuesFrom mechanism. By moving away from Helm lookup functions, we’ve made config propagation much cleaner and eliminated the need for force reconcile controllers. Configuration from ConfigMaps (cozystack, cozystack-branding, cozystack-scheduling) and namespace service references (etcd, host, ingress, monitoring, seaweedfs) is now centrally managed through a cozystack-values Secret in each namespace.

For users, this means simpler Helm templates and faster reconciliation. Configuration is now more transparent, as HelmReleases automatically pull exactly what they need from the centralized secret.

LINSTOR Auto-diskful
The LINSTOR integration now includes automatic diskful functionality that converts diskless nodes to diskful when they hold DRBD resources in Primary state for an extended period (30 minutes). This feature addresses scenarios where workloads are scheduled on nodes without local storage replicas by automatically creating local disk replicas when needed, improving I/O performance for long-running workloads.

When enabled with cleanup options, the system can automatically remove disk replicas that are no longer needed, preventing storage waste from temporary replicas. This intelligent storage management reduces network traffic for frequently accessed data while maintaining efficient storage utilization.

Automated Version Management Systems
Cozystack now features automated version management for PostgreSQL, Kubernetes, MariaDB, and Redis. It tracks upstream versions and provides means for automated version updates, ensuring that platform users always have access to the latest stable versions while maintaining compatibility with existing deployments.

Integrated with the Cozystack API and dashboard, these systems provide administrators with full visibility into available versions and upgrade paths. This infrastructure establishes the foundation for future automated upgrade workflows and comprehensive version compatibility management across the platform.

All changes and improvements: v0.40.2, v0.40.1, v0.40.0

Huge thanks to everyone who contributed to the 0.40 line: @IvanHunters, @insignia96, @kvaps, @lllamnyp, @nbykov0, @scooby87

📎 Join the community
Telegram group
Slack group (Get invite at https://slack.kubernetes.io)

🤨 When it comes to running virtual machines in Kubernetes via KubeVirt, the first question engineers ask is: “What is the overhead?”

Let’s dive into the details and break it down by three key areas: compute, storage, and network.

P.S. This article is based on a discussion in the professional community.

https://blog.aenix.io/kubevirt-the-truth-about-virtualization-overhead-in-kubernetes-ba1a5ec21a79

🎥 Cozystack community meeting 2026-01-22: Release 1.0 demo, MongoD, Kilo, Local-CCM, v0.41

https://www.youtube.com/watch?v=hUuEFcj2fVM

Cozystack is a free PaaS and framework for building clouds | CNCF Sandbox Project

Agenda and notes

- Release v0.41
- Release v1.0-alpha
- Migration and changes
- MongoDB support
- Introducing Kilo package
- Introducing Local-CCM

Join the community:
Telegram group t.me/cozystack
Slack group (Get invite at https://slack.kubernetes.io)

Cozystack resources:
https://cozystack.io
https://cozystack.io/docs/get-started
https://cozystack.io/blog
https://github.com/cozystack/cozystack

Ænix resources
https://aenix.io
https://xn--r1a.website/aenix_io
https://xn--r1a.website/aenix_community

😜 Cozystack v0.41.0 — managed MongoDB
This version features MongoDB as a new managed application, significantly expanding our database offerings alongside existing PostgreSQL, MySQL, and Redis services. This release also brings crucial stability enhancements for core Kubernetes components, storage system improvements, and updated documentation.

Spotlight: MongoDB Managed Application.
Cozystack now offers MongoDB as a robust, fully managed database service. Users can now deploy production-ready MongoDB instances through the application catalog. The service provides enterprise-grade capabilities such as automated ReplicaSet configuration for high availability, seamless integration with Cozystack's storage backends to ensure data persistence, full control over resource allocation (configurable CPU, memory, and storage), and native monitoring integration.

You can deploy MongoDB through the Cozystack dashboard or using the standard application deployment workflow.

💪 Improvements
[linstor] Update piraeus-server patches with critical fixes: Backported critical patches to piraeus-server that address storage stability issues and improve DRBD resource handling. These patches fix edge cases in device management and ensure more reliable storage operations.

[linstor] Refactor node-level RWX validation: Refactored the node-level ReadWriteMany (RWX) validation logic in LINSTOR CSI. The validation has been moved to the CSI driver level with a custom linstor-csi image build, providing more reliable RWX volume handling and clearer error messages when RWX requirements cannot be satisfied.

[kubernetes] Increase default apiServer resourcesPreset to large: Increased the default resource preset for kube-apiserver to large to ensure more reliable operation under higher workloads and prevent resource constraints.

[kubernetes] Increase kube-apiserver startup probe threshold: Increased the startup probe threshold for kube-apiserver to allow more time for the API server to become ready, especially in scenarios with slow storage or high load.

[etcd] Increase probe thresholds for better recovery: Increased etcd probe thresholds to provide more time for recovery operations, improving cluster resilience during network issues or temporary slowdowns.

🫡 Dependencies
Cilium CNI was upgraded to v1.18.6 to incorporate the latest security and performance improvements, and Talos Linux was updated to v1.11.6 with the newest security patches.

All changes and improvements: https://github.com/cozystack/cozystack/releases/tag/v0.41.0

Recordings from CozySummit'25 Virtual

💾 Cozystack Storage Deep Dive - Moritz Wanzenböck, LINBIT

A deep technical look at how storage is designed and implemented inside Cozystack, including architectural decisions and trade-offs.

00:00:00 Introduction: The Storage Layer is a founding component of CozyStack architecture.
00:00:45 CozyStack's storage requirements (Open Source, no external dependencies, scalability, performance).
00:02:40 Perios Data Store: An open-source CNCF sandbox project leveraging existing Linux storage technologies.
00:04:00 Bottom of the stack: Using LVM or ZFS for volume management and block-level features like encryption.
00:05:00 DRBD: Linux kernel module for block-level replication and protection against node failure.
00:06:00 LinStor: The management interface (Control Plane) for the Linux storage stack. 00:07:00 Live Demo: LinStor cluster resource overview and storage pool configuration (using ZFS).
00:08:30 Storage Classes: Local Storage Class for fault-tolerant workloads (e.g., etcd).
00:09:45 Storage Classes: Replicated Storage Class using DRBD for data that needs copies (e.g., aggregated logs).
00:10:45 Monitoring and Snapshots: Integrating with Prometheus/Grafana and the Kubernetes Snapshots API (Valero support).
00:12:15 Storage for Managed Kubernetes: Managed clusters automatically inherit a storage provider from the host cluster.
00:13:20 New Feature: Natively supporting RWX (Read Write Many) volumes.
00:14:40 Q&A: Snapshot creation on LVM2 and ZFS, and synchronization on replicated volumes.
00:16:00 Q&A: Data integrity checks (ZFS scrubs) and relying on underlying technologies (ZFS/RAID) for disk failures.
00:17:40 Q&A: LinStor's agnostic approach to disk types (NVMe drives) and performance.
00:18:50 Q&A: Configuring custom disk nodes and supporting NFS as an RWX storage option.

▶️ Watch the session: https://www.youtube.com/watch?v=hbIbx_5WWX4