​​Statistical Data Validation for Pandas.

A data validation library for scientists, engineers, and analysts seeking correctness.

pandera provides a flexible and expressive API for performing data validation on tidy (long-form) and wide data to make data processing pipelines more readable and robust.

pandas data structures contain information that pandera explicitly validates at runtime. This is useful in production-critical data pipelines or reproducible research settings. With pandera, you can:

- Check the types and properties of columns in a pd.DataFrame or values in a pd.Series.
- Perform more complex statistical validation like hypothesis testing.
- Seamlessly integrate with existing data analysis/processing pipelines via function decorators.
- Define schema models with a class-based API with pydantic-style syntax and validate dataframes using the typing syntax.

https://pandera.readthedocs.io/en/stable/index.html

#python #ds

The smallest, fastest Kubernetes.

MicroK8s is a small, fast, single-package Kubernetes for developers, IoT and edge.

Single-package fully conformant lightweight Kubernetes that works on 42 flavours of Linux. Perfect for:
- Developer workstations
- IoT
- Edge
- CI/CD

Why MicroK8s?
- Small. Developers want the smallest K8s for laptop and workstation development. MicroK8s provides a standalone K8s compatible with Azure AKS, Amazon EKS, Google GKE when you run it on Ubuntu.
- Simple. Minimize administration and operations with a single-package install that has no moving parts for simplicity and certainty. All dependencies and batteries included.
- Secure. Updates are available for all security issues and can be applied immediately or scheduled to suit your maintenance cycle.
- Current. MicroK8s tracks upstream and releases beta, RC and final bits the same day as upstream K8s. You can track latest K8s or stick to any release version from 1.10 onwards.
- Comprehensive. MicroK8s includes a curated collection of manifests for common K8s capabilities and services.

https://github.com/ubuntu/microk8s

#python #devops #k8s

​​Cache the dependencies of your #rust project and speed up your Docker builds up to 5 times.

https://github.com/LukeMathWalker/cargo-chef

Announcement: https://www.lpalmieri.com/posts/fast-rust-docker-builds/

A collection of extra types and features for mypy.

I wrote this plugin on Saturday, so it is quite new!
Currently it features only AttrOf type and ensure_attr helper function.

In the next releases we will work on KeyOf and AllKeys types that will help with TypedDict and other objects with getitem magic methods. Stay tuned!

Do you have any ideas for new types we can add? Submit them to our task tracker!

https://github.com/wemake-services/mypy-extras/

#python

A #clojure library designed to generate cryptographically strong random numbers suitable for managing data such as passwords, account authentication, security tokens, and related secrets.

The secrets is an implementation of the secrets module from Python's standard library for Clojure.

https://github.com/lk-geimfari/secrets.clj

Grafana Tempo is an open source, easy-to-use and high-scale distributed tracing backend. Tempo is cost-efficient, requiring only object storage to operate, and is deeply integrated with Grafana, Prometheus, and Loki. Tempo can be used with any of the open source tracing protocols, including Jaeger, Zipkin, and OpenTelemetry.

https://grafana.com/oss/tempo/

​​Stories for VSCode.

This is absolutely cursed.

https://github.com/benawad/vscode-stories

​​An independent #rust text editor that runs in your terminal!

Ox is a code editor. It was written in Rust using ANSI escape sequences. It assists developers with programming by providing several tools to speed up and make programming easier and a refreshing alternative to heavily bloated and resource hungry editors such as VS Code and JetBrains. Ox is lightweight so it can be used on older computers.

Bear in mind, this is a personal project and is nowhere near ready to replace your existing tools just yet.

It runs in the terminal and runs on platforms like Linux and macOS but doesn't work on Windows directly (it works if you use WSL) due to a lack of a good command line. There are many text editors out there and each one of them has their flaws and I hope to have a text editor that overcomes many of the burdens and issues.

Ox is not based on any other editor and has been built from the ground up without any base at all.

https://github.com/curlpipe/ox

​​A type-safe query builder for TypeScript and Postgres.

Features:
- Type-safe query builder - sticking as close to SQL as possible
- BYOCP (bring your own connection pool)
- Supports Postgres only
- Excellent autocomplete
- Train case to camelCase and vice versa conversion
- Auto-migration generation based on your schema changes
- Awesome CLI to run your migrations

https://github.com/Ff00ff/mammoth

#ts

​​Fast, consistent builds for Python and more. Pants v2 is a fast, scalable build system for growing codebases. It's currently focused on Python, with support for other languages coming soon.

> Installing, configuring and orchestrating the invocation of Python build tools like flake8, mypy, or tox—all while not re-executing work unnecessarily—is a hard problem, especially as your codebase grows. The lack of a robust, scalable build system for Python has been a problem for a long time, and this has become even more acute in recent years, with Python codebases increasing in size and complexity.

Some noteworthy features include:
- Minimal metadata and boilerplate
- Fine-grained workflow
- Shared result caching
- Concurrent execution
- A responsive, scalable UI
- Unified interface for multiple tools and languages
- Extensibility and customizability via a plugin API

https://www.pantsbuild.org/
Announcement: https://blog.pantsbuild.org/introducing-pants-v2/

Written in #python and #rust

Improve your code review with conventional comments!

Adhering to a consistent format improves reader's expectations and machine readability. Here's the format we propose:

<label> [decorations]: <subject>

[discussion]

Where:
- label - This is a single label that signifies what kind of comment is being left.
- subject - This is the main message of the comment.
- decorations (optional) - These are extra decorating labels for the comment. They are surrounded by parentheses and comma-separated.
- discussion (optional) - This contains supporting statements, context, reasoning, and anything else to help communicate the “why” and “next steps” for resolving the comment.

These comments can be parsed into JSON easily for further processing.

https://conventionalcomments.org/

​​SQLDelight - Generates typesafe Kotlin APIs from SQL.

SQLDelight generates typesafe #kotlin APIs from your SQL statements. It verifies your schema, statements, and migrations at compile-time and provides IDE features like autocomplete and refactoring which make writing and maintaining SQL simple.

https://github.com/cashapp/sqldelight

I was recognised as a Github Star! https://stars.github.com/profiles/sobolevn/

This is very important to me because I spend like 50% of my overall time on Github.
It also brings a lot of extra motivation to continue my free community work I do (like this telegram channel).

If you want to share this moment with me and say "thank you" - click "Follow" button here: https://github.com/sobolevn

Let's build more awesome tools together!

​​Flow-based visual scripting for #python.

Ryven is a flow-based visual scripting environment for Python for creating diagram-like representations of your Python scripts. It provides you with a simple system to create new nodes executing any Python code. It might come in handy as a tool for exploration, too, once users created a lot more nodes. Convenient use cases include IoT, data processing and data manipulation, generally visualizing algorithms, Raspberry Pi, and basically any web API.

Features:
- Simple concept, intuitive UI
- Create clear and good looking flows
- Any Python code can be put into a node, no limits!
- Easy process of creating new nodes
- There are data and execution connections but pure dataflows are possible
- Stylus support
- Live source code manipulation
- It is free

https://ryven.org/

​​Fuzzy Finder in #rust!

Half of our life is spent on navigation: files, lines, commands, etc. You need skim! It is a general fuzzy finder that saves you time. skim provides a single executable: sk. Basically anywhere you would want to use grep, try sk instead.

https://github.com/lotabout/skim

The Open Architecture Playbook. Use it to create better and faster (IT)Architectures. OSS Tools, templates and more for solving IT problems using real open architecture tools that work!

Smart people have been thinking on how to create IT architectures as long as there has been computers. Ideas come and go, however creating a good architectures can still be complex and time consuming. Especially when you try to invent the wheel for yourself. With this interactive playbook you can create your IT architecture better and faster. The focus of this architecture playbook is in on:

1. Knowledge reuse. Why reinvent the wheel again? It is far and more fun to create a better wheel for your organisation or IT project instead! Focus on the hard complex context specific issues. Use good open tools and knowledge for the easy 80%!
2. Easier creation of architecture documents and deliverables. This playbook has an extensive list of all open tools available for creating your IT architecture or design. Using these open tools will speed up the process of creating your architecture deliverables and reduce your risks.
3. Quality improvement. By making use of content parts provided for various architecture deliverables you will lower your business risks. Complex business IT projects will fail. Now and in future. But if you make use of proven methods and tools developed from decades of IT architecture scientific research you will lower the risk for your project. Architecture will help to make your projects more successful in terms of costs, speed and profitability.

https://nocomplexity.com/documents/arplaybook/index.html

​​Erlang/Elixir NIFs in Nim

Nimler is a library for authoring #erlang and #elixir NIFs in the #nim programming language. It has mostly complete bindings for the Erlang NIF API and some accessories for making writing NIFs easier, including idiomatic functions for converting between Erlang terms and Nim types, and simplifications for using resource objects.

Mostly, Nimler is a minimal, zero-dependency wrapper for Erlang NIF API.

https://github.com/wltsmrz/nimler

​​Prevent cloud misconfigurations during build-time for Terraform, Cloudformation, Kubernetes, Serverless framework and other infrastructure-as-code-languages with Checkov (Чехов).

Checkov is a static code analysis tool for infrastructure-as-code.

It scans cloud infrastructure provisioned using Terraform, Cloudformation, Kubernetes, Serverless or ARM Templates and detects security and compliance misconfigurations.

Features:
- Over 400 built-in policies cover security and compliance best practices for AWS, Azure and Google Cloud.
- Scans Terraform, CloudFormation and Kubernetes, Serverless framework and ARM template files.
- Detects AWS credentials in EC2 Userdata, Lambda environment variables and Terraform providers.
- Evaluates Terraform Provider settings to regulate the creation, management, and updates of IaaS, PaaS or SaaS managed through Terraform.
- Policies support evaluation of variables to their optional default value.
- Supports in-line suppression of accepted risks or false-positives to reduce recurring scan failures. Also supports global skip from using CLI.
- Output currently available as CLI, JSON, JUnit XML and github markdown and link to remediation guides.

https://github.com/bridgecrewio/checkov

#python #devops #k8s

​​Command-line DNS #rust client

Dogs can look up!

dog is a command-line DNS client, like dig. It has colourful output, understands normal command-line argument syntax, supports the DNS-over-TLS and DNS-over-HTTPS protocols, and can emit JSON.

https://github.com/ogham/dog

​​Free/Libre Open Source Software Binaries of VSCode

Microsoft’s vscode source code is open source (MIT-licensed), but the product available for download (Visual Studio Code) is licensed under this not-FLOSS license and contains telemetry/tracking. According to this comment from a Visual Studio Code maintainer:

> When we Microsoft build Visual Studio Code, we do exactly this. We clone the vscode repository, we lay down a customized product.json that has Microsoft specific functionality (telemetry, gallery, logo, etc.), and then produce a build that we release under our license.

> When you clone and build from the vscode repo, none of these endpoints are configured in the default product.json. Therefore, you generate a “clean” build, without the Microsoft customizations, which is by default licensed under the MIT license

The VSCodium project exists so that you don’t have to download+build from source. This project includes special build scripts that clone Microsoft’s vscode repo, run the build commands, and upload the resulting binaries for you to GitHub releases. These binaries are licensed under the MIT license. Telemetry is disabled.

https://vscodium.com/

​​Keep an eye on your #docker image size and prevent it from growing too big!

Limit your docker image size with a simple CLI command. Perfect to be used inside your CI process.

Also available as a Github Action.

https://github.com/wemake-services/docker-image-size-limit

#python