Oban is a aobust job processing library written in Elixir.

Oban's primary goals are reliability, consistency and observability.

It is fundamentally different from other background job processing tools because it retains job data for historic metrics and inspection. You can leave your application running indefinitely without worrying about jobs being lost or orphaned due to crashes.

#elixir, #erlangvm, #beam, #job, #processing, #library

Faust is a stream processing library, porting the ideas from Kafka Streams to Python.

It is used at Robinhood to build high performance distributed systems and real-time data pipelines that process billions of events every day.

Faust provides both stream processing and event processing, sharing similarity with tools such as Kafka Streams, Apache Spark/Storm/Samza/Flink.

#python, #distributed, #stream, #kafka, #robinhood, #stream, #processing, #asyncio, #queue, #state, #management

Debezium is „… an open source distributed platform for change data capture. Start it up, point it at your databases, and your apps can start responding to all of the inserts, updates, and deletes that other apps commit to your databases. Debezium is durable and fast, so your apps can respond quickly and never miss an event, even when things go wrong.“

Similar tools:
- lawrencejones / pgsink (postgres only)

#opensource #database #eventstream #kafka #processing

Pathom is a logic engine for attribute processing for Clojure and Clojurescript.

#clojure #graphql #eql #clojurescript #attribute #processing

Tenacity is an easy-to-use multi-track audio editor and recorder for Windows, MacOS, GNU/Linux and other operating systems. It is a fork of the widely popular Audacity and is being developed by a wide, diverse group of volunteers.

- Recording from audio devices (real or virtual)
- Export / Import a wide range of audio formats (extensible with FFmpeg)
- High quality including up to 32-bit float audio support
- Plug-ins providing support for VST, LV2, and AU plugins
- Scripting in the built-in scripting language Nyquist, or in Python, Perl and other languages with named pipes
- Editing arbitrary sampling and multi-track timeline
- Accessibility including editing via keyboard, screen reader support and narration support
- Tools useful in the analysis of signals, including audio

#cpp #audio #floss #recorder #processing

Asynq is a Go library for queueing tasks and processing them asynchronously with workers. It's backed by Redis and is designed to be scalable yet easy to get started.

Highlevel overview of how Asynq works:

- Client puts tasks on a queue
- Server pulls tasks off queues and starts a worker goroutine for each task
- Tasks are processed concurrently by multiple workers

#golang #go #async #queueing #tasks #processing #redis

Albumentations is a Python library for image augmentation. Image augmentation is used in deep learning and computer vision tasks to increase the quality of trained models. The purpose of image augmentation is to create new training samples from the existing data.

#python #ml #dl #detection #image #processing #classification

Velox is a C++ database acceleration library which provides reusable, extensible, and high-performance data processing components. These components can be reused to build compute engines focused on different analytical workloads, including batch, interactive, stream processing, and AI/ML. Velox was created by Facebook and it is currently developed in partnership with Intel, ByteDance, and Ahana.

#cpp #vectorized #database #db #acceleration #processing #system