​​concise commandline monitoring for containers.

Top-like interface for container metrics. ctop comes with built-in support for Docker and runC. As well as a single container view for inspecting a specific container.

Connectors for other container and cluster systems are planned for future releases.

https://github.com/bcicen/ctop

#go #devops #docker

​​A powerful little #go TUI framework

The fun, functional and stateful way to build terminal apps. A Go framework based on The #elm Architecture. Bubble Tea is well-suited for simple and complex terminal applications, either inline, full-window, or a mix of both.

Bubble Tea is in use in production and includes a number of features and performance optimizations we’ve added along the way. Among those is a standard framerate-based renderer, a renderer for high-performance scrollable regions which works alongside the main renderer, and mouse support.

https://github.com/charmbracelet/bubbletea

​​Boundary enables identity-based access management for dynamic infrastructure.

Boundary provides simple and secure access to hosts and services.

Traditional approaches like SSH bastion hosts or VPNs require distributing and managing credentials, configuring network controls like firewalls, and exposing the private network. Boundary provides a secure way to access hosts and critical systems without having to manage credentials or expose your network, and is entirely open source.

Boundary is designed to be straightforward to understand, highly scalable, and resilient. It can run in clouds, on-prem, secure enclaves and more, and does not require an agent to be installed on every end host.

Unlike firewalls, Boundary performs per-access authentication and authorization checks, allowing for much higher level mappings of users to services or hosts than at network layers. Although complementary to secrets managers (like HashiCorp's own Vault), Boundary fills a different niche, allowing the credential that is eventually used to be hidden entirely from the user.

- Authenticate with any trusted identity provider you are already using. No need to distribute new credentials and manage them.
- Authorize access based on logical roles and services, instead of physical IP addresses. Manage dynamic infrastructure and integrate service registries so hosts and service catalogs are kept up-to-date.
- Automate credential injection to securely access services and hosts with HashiCorp Vault. Reduce risk of leaking credentials with dynamic secrets and just-in-time credentials.

https://github.com/hashicorp/boundary

#go

​​An event-driven, portable runtime for building microservices on cloud and edge. Any language, any framework, anywhere.

Dapr is a portable, serverless, event-driven runtime that makes it easy for developers to build resilient, stateless and stateful microservices that run on the cloud and edge and embraces the diversity of languages and developer frameworks.

Dapr codifies the best practices for building microservice applications into open, independent, building blocks that enable you to build portable applications with the language and framework of your choice. Each building block is independent and you can use one, some, or all of them in your application.

Dapr injects a side-car (container or process) to each compute unit. The side-car interacts with event triggers and communicates with the compute unit via standard HTTP or gRPC protocols. This enables Dapr to support all existing and future programming languages without requiring you to import frameworks or libraries.

Dapr offers built-in state management, reliable messaging (at least once delivery), triggers and bindings through standard HTTP verbs or gRPC interfaces. This allows you to write stateless, stateful and actor-like services following the same programming paradigm. You can freely choose consistency model, threading model and message delivery patterns.

Dapr runs natively on Kubernetes, as a self hosted binary on your machine, on an IoT device, or as a container that can be injected into any system, in the cloud or on-premises.

Dapr uses pluggable component state stores and message buses such as Redis as well as gRPC to offer a wide range of communication methods, including direct dapr-to-dapr using gRPC and async Pub-Sub with guaranteed delivery and at-least-once semantics.

Features:
- Event-driven Pub-Sub system with pluggable providers and at-least-once semantics
- Input and output bindings with pluggable providers
- State management with pluggable data stores
- Consistent service-to-service discovery and invocation
- Opt-in stateful models: Strong/Eventual consistency, First-write/Last-write wins
- Cross platform virtual actors
- Secrets management to retrieve secrets from secure key vaults
- Rate limiting
- Built-in Observability support
- Runs natively on Kubernetes using a dedicated Operator and CRDs
- Supports all programming languages via HTTP and gRPC
- Multi-Cloud, open components (bindings, pub-sub, state) from Azure, AWS, GCP
- Runs anywhere, as a process or containerized
- Lightweight (58MB binary, 4MB physical memory)
- Runs as a sidecar - removes the need for special SDKs or libraries
- Dedicated CLI - developer friendly experience with easy debugging
- Clients for Java, .NET Core, Go, Javascript, Python, Rust and C++

https://dapr.io/

#devops #go #k8s

​​Validate, define, and use dynamic and text-based data.

CUE is an open source language, with a rich set of APIs and tooling, for defining, generating, and validating all kinds of data: configuration, APIs, database schemas, code, … you name it.

Features:
- Data Validation. Validate text-based data files or programmatic data such as incoming RPCs or database documents.
- Configuration. Just add validation to existing data (CUE, YAML, JSON), reduce boilerplate in large-scale configurations, or both.
- Schema Definition. Define schema to communicate an API or standard and validate backwards compatibility.
- Generate Code and Schema. Keep validation code in sync across code bases, Protobuf definitions, and OpenAPI definitions.
- Scripting. Automate the use of your data without writing yet another tool.
- Querying. Find the locations of instances of CUE types and values in data. Coming soon.

https://cuelang.org/

#devops #go #noyaml

​​Gitkube is a tool for building and deploying Docker images on Kubernetes using git push.

After a simple initial setup, users can simply keep git push-ing their repos to build and deploy to Kubernetes automatically.

Features:
- No dependencies except native tooling (git, kubectl)
- Plug and play installation
- Simple public key based authentication
- RBAC ready - Control access to git remotes using RBAC
- Support for namespace based multi-tenancy - Remotes can only deploy to their own namespace
- No assumptions about repository structure

https://github.com/hasura/gitkube

#go #devops #k8s

​​Keel is a tool for automating Kubernetes deployment updates.

kubectl is the new SSH. If you are using it to update production workloads, you are doing it wrong. See examples on how to automate application updates.

Keel runs as a single container, scanning Kubernetes and Helm releases for outdated images. Policies and trigger types are specified in your application deployment files or Helm charts. Single command, no dependencies. No lock-in, no custom configuration files.

Comes with a web interface.

https://github.com/keel-hq/keel

#go #vue #devops #k8s

​​k0s - Zero Friction Kubernetes

k0s is an all-inclusive Kubernetes distribution with all the required bells and whistles preconfigured to make building a Kubernetes clusters a matter of just copying an executable to every host and running it.

Key Features:
- Packaged as a single static binary
- Self-hosted, isolated control plane
- Variety of storage backends: etcd, SQLite, MySQL (or any compatible), PostgreSQL
- Elastic control-plane
- Vanilla upstream Kubernetes
- Supports custom container runtimes (containerd is the default)
- Supports custom Container Network Interface (CNI) plugins (calico is the default)
- Supports x86-64 and arm64

Quickstart:

curl -sSLf get.k0s.sh | sh
k0s server --enable-worker

https://github.com/k0sproject/k0s

#k8s #devops #go

​​Flix: Next-generation reliable, safe, concise, and functional-first programming language.

Flix is inspired by OCaml and Haskell with ideas from Rust and Scala. Flix looks like Scala, but its type system is based on Hindley-Milner. Two unique features of Flix are its polymorphic effect system and its support for first-class Datalog constraints.

Flix aims to offer a unique combination of features that no other programming language offers, including: algebraic data types and pattern matching (like Haskell, OCaml), extensible records (like Elm), type classes (like #haskell, #rust), higher-kinded types (like Haskell), local type inference (like Haskell, OCaml), channel and process-based concurrency (like #go), a polymorphic effect system (unique feature), first-class Datalog constraints (unique feature), and compilation to JVM bytecode (like #scala).

https://flix.dev/

Personal opinion: it has all the features new modern programming language should have. It has nice syntax, powerful ideas, and (hopefully) good JVM integration and code-reuse. That's a recipe for a success. I will keep an eye on this amazing language.

It also has a playground on its website, check it out!
I would love to highlight this example of polymorphic effects:

​​Bit is a modern Git CLI.

bit is an experimental modernized git CLI built on top of git that provides happy defaults and other niceties:
- command and flag suggestions to help you navigate the plethora of options git provides you
- autocompletion for files and branch names when using bit add or bit checkout
- automatic fetch and branch fast-forwarding reducing the likelihood of merge conflicts
- suggestions work with git aliases
- new commands like bit sync that vastly simplify your workflow
- commands from git-extras such as bit release & bit info
- fully compatible with git allowing you to fallback to git if need be
- get insight into how bit works using bit --debug

https://github.com/chriswalz/bit

#git #go

​​A static analysis tool for securing #go code

GoKart is a static analysis tool for Go that finds vulnerabilities using the SSA (single static assignment) form of Go source code. It is capable of tracing the source of variables and function arguments to determine whether input sources are safe, which reduces the number of false positives compared to other Go security scanners. For instance, a SQL query that is concatenated with a variable might traditionally be flagged as SQL injection; however, GoKart can figure out if the variable is actually a constant or constant equivalent, in which case there is no vulnerability.

The motivation for GoKart was to address this: could we create a scanner with significantly lower false positive rates than existing tools? Based on our experimentation the answer is yes. By leveraging source-to-sink tracing and SSA, GoKart is capable of tracking variable taint between variable assignments, significantly improving the accuracy of findings. Our focus is on usability: pragmatically, that means we have optimized our approaches to reduce false alarms.

https://github.com/praetorian-inc/gokart

​​Ain is a terminal HTTP API client. It's an alternative to postman, paw or insomnia.

Features:
- Flexible organization of API:s using files and folders.
- Use shell-scripts and executables for common tasks.
- Put things that change in environment variables or .env-files.
- Share the resulting curl, wget or httpie command-line.
- Pipe the API output for further processing.
- Tries hard to be helpful when there are errors.

Ain was built to enable scripting of input and further processing of output via pipes. It targets users who work with many API:s using a simple file format. It uses curl, wget or httpie to make the actual calls.

https://github.com/jonaslu/ain

Personal opinion: I don't like GUIs, they are harder to work with. So, any tools that can be replaced should be replaced.

#go

​​Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON.

With Miller, you get to use named fields without needing to count positional indices, using familiar formats such as CSV, TSV, JSON, and positionally-indexed. Then, on the fly, you can add new fields which are functions of existing fields, drop fields, sort, aggregate statistically, pretty-print, and more.

1. Miller operates on key-value-pair data while the familiar Unix tools operate on integer-indexed fields: if the natural data structure for the latter is the array, then Miller's natural data structure is the insertion-ordered hash map.
2. Miller handles a variety of data formats, including but not limited to the familiar CSV, TSV, and JSON. (Miller can handle positionally-indexed data too!)

https://github.com/johnkerl/miller

#c #go #shell

​​cheat allows you to create and view interactive cheatsheets on the command-line. It was designed to help remind unix system administrators of options for commands that they use frequently, but not frequently enough to remember.

https://github.com/cheat/cheat

#go #shell

​​Prevent Kubernetes misconfigurations from reaching production (again 😤 )! Datree is a CLI tool to ensure K8s manifests and Helm charts follow best practices as well as your organization’s policies.

It’s far more effective than manual processes, such as sending an email to a slew of developers, begging them to set various limits, which likely falls on deaf ears because developers are already overwhelmed.

The CLI integration provides a policy enforcement solution for Kubernetes to run automatic checks on every code change for rule violations and misconfigurations. When rule violations are found, Datree produces an alert which guides the developer to fix the issue inside the CI process — or even earlier as a pre-commit hook — while explaining the reason behind the rule.

Right now, there are 30 battle-tested rules for you to choose from.

https://github.com/datreeio/datree

#k8s #devops #go

​​Kubegres is a #k8s operator allowing to deploy one or many clusters of postgresql instances and manage databases replication, failover and backup.

Features:
- It can manage one or many clusters of Postgres instances. Each cluster of Postgres instances is created using a YAML of "kind: Kubegres". Each cluster is self-contained and is identified by its unique name and namespace.
- It creates a cluster of PostgreSql servers with Streaming Replication enabled: it creates a Primary PostgreSql pod and a number of Replica PostgreSql pods and replicates primary's database in real-time to Replica pods.
- It manages fail-over: if a Primary PostgreSql crashes, it automatically promotes a Replica PostgreSql as a Primary.
- It has a data backup option allowing to dump PostgreSql data regularly in a given volume.
- It provides a very simple YAML with properties specialised for PostgreSql.

https://github.com/reactive-tech/kubegres

#go #devops

​​KubeLinter is a static analysis tool that checks Kubernetes YAML files and Helm charts to ensure the applications represented in them adhere to best practices.

KubeLinter runs sensible default checks, designed to give you useful information about your Kubernetes YAML files and Helm charts. This is to help teams check early and often for security misconfigurations and DevOps best practices. Some common examples of these include running containers as a non-root user, enforcing least privilege, and storing sensitive information only in secrets.

KubeLinter is configurable, so you can enable and disable checks, as well as create your own custom checks, depending on the policies you want to follow within your organization.

When a lint check fails, KubeLinter reports recommendations for how to resolve any potential issues and returns a non-zero exit code.

https://github.com/stackrox/kube-linter

#go #k8s #devops

​​High-performance load testing tool, written in #go.

Features:
- Protocol Agnostic - Currently supporting HTTP, HTTPS, HTTP/2. Other protocols are on the way.
- Scenario-Based - Create your flow in a JSON file. Without a line of code!
- Different Load Types - Test your system's limits across different load types.

https://github.com/ddosify/ddosify

Статический анализ GitHub Actions

Сразу после релиза новой версии линтера, я задался вопросом обновления своего шаблона для создания новых питоновских библиотек: https://github.com/wemake-services/wemake-python-package

И я понял, что я несколько отстал в вопросе стат анализа GitHub Actions и прочей инфраструктуры.
Расскажу о своих находках.

pre-commit ci

Все знают про пакет pre-commit? Несколько лет назад он получил еще и свой собственный CI, который умеет запускаться без дополнительного конфига. И автоматически пушить вам в ветку любые изменения. Что супер удобно для всяких ruff / black / isort и прочего. У нас такое стоит в большом количестве проектов. Вот пример из typeshed. Вот что поменялось автоматически.

Строить CI на базе pre-commit очень удобно, потому что тебе просто нужно скопировать пару строк в конфиг. А плюсов много:
- Автоматически исправляются многие проблемы
- Автоматически запускается CI, 0 настроек
- Локально все тоже работает одной командой: pre-commit run TASK_ID -a

actionlint

Первый раз я увидел actionlint внутри CPython и затащил его в mypy. Actionlint на #go, он предлагает набор проверок для ваших GitHub Actions от безопасности до валидации спеки вашего yml. Довольно полезно, позволяет найти много мест для улучшений.

test.yaml:3:5: unexpected key "branch" for "push" section. expected one of "branches", ..., "workflows" [syntax-check]
  |
3 |     branch: main
  |     ^~~~~~~
test.yaml:10:28: label "linux-latest" is unknown. available labels are "macos-latest", ..., "windows". if it is a custom label for self-hosted runner, set list of labels in actionlint.yaml config file [runner-label]
   |
10 |         os: [macos-latest, linux-latest]
   |                            ^~~~~~~~~~~~~
test.yaml:13:41: "github.event.head_commit.message" is potentially untrusted. avoid using it directly in inline scripts. instead, pass it through an environment variable. see https://docs.github.com/en/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions for more details [expression]
   |
13 |       - run: echo "Checking commit '${{ github.event.head_commit.message }}'"
   |                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Даже умеет автоматом shellcheck запускать на ваши run: скрипты!

zizmor

Исходники. Уже на #rust, он более злой. Делает похожие вещи: находит проблемы безопасности. Находит много проблем.

Вот пример, сколько всего он нашел в mypy.

warning[artipacked]: credential persistence through GitHub Actions artifacts
  --> mypy/.github/workflows/mypy_primer.yml:37:9
   |
37 |         - uses: actions/checkout@v4
   |  _________-
38 | |         with:
39 | |           path: mypy_to_test
40 | |           fetch-depth: 0
   | |________________________- does not set persist-credentials: false
   |
   = note: audit confidence → Low

error[dangerous-triggers]: use of fundamentally insecure workflow trigger
 --> mypy/.github/workflows/mypy_primer_comment.yml:3:1
  |
3 | / on:
4 | |   workflow_run:
... |
7 | |     types:
8 | |       - completed
  | |_________________^ workflow_run is almost always used insecurely
  |
  = note: audit confidence → Medium

check-jsonschema

Еще есть вот такой проект, он в основном полезен за счет доп интеграций: можно проверять dependabot.yml, renovate.yml, readthedocs.yml и многое другое.

Ставится просто как:

  - repo: https://github.com/python-jsonschema/check-jsonschema
    rev: 0.30.0
    hooks:
      - id: check-dependabot
      - id: check-github-workflows

Выводы

Как всегда – статический анализ многому меня научил. Я узнал много нового про безопасность GitHub Actions, про вектора атаки, про лучшие практики. А сколько проблем в ваших actions?

Скоро ждите весь новый тулинг в python шаблоне v2025 😎

Находки в опенсорсе: EasyP – тулбокс для ProtoBuf файлов

https://www.youtube.com/watch?v=XI-dNpM77iM

easyp – пакетный менеджер, билд-система и линтер для .proto файлов.
Хоть easyp и написан на #go 😱, одна из его фишек в том – что вы можете использовать любые плагины для генерации финального кода: он может быть хоть на #python, хоть на #rust.

Если много используете ProtoBuf – обязательно для ознакомления!

Как оно работает?

# Секция для правил линтера:
lint:
  use:
    - DEFAULT

# Секция с зависимостями:
deps:
  - github.com/googleapis/googleapis
  - github.com/grpc-ecosystem/grpc-gateway@v2.20.0

# Секция для правил сборки и генерации итоговых файлов:
generate:
  plugins:
    - name: go
      out: .
      opts:
        paths: source_relative
    - name: go-grpc
      out: .
      opts:
        paths: source_relative
        require_unimplemented_servers: false

Прощайте огромные Makefile с кучей скриптов для сборки.

Что будет в видео?

В видео:
- спросим зачем ребята его сделали
- узнаем как оно работает
- покажем, откуда можно устанавливать зависимости
- потыкаем разные юзкейсы
- поговорим про безопасность решения
- обсудим планы на следующие релизы
- расскажем как находить ломающие изменения в вашей ProtoBuf спецификации

А еще в видео кот на гитаре играет!

Репозиторий: https://github.com/easyp-tech/easyp
Документация: https://easyp.tech
Чатик сообщества: @easyptech

Поставьте пацанам звездочек, если проект понравился!

Обсуждение: Какие инструменты для ProtoBuf вы используете? Какие есть пролемы? Что можно было бы добавить в качестве новой фичи в easyp?

| Поддержать | YouTube | GitHub | Чат |