​​Quill provides a Quoted Domain Specific Language (QDSL) to express queries in #scala and execute them in a target language. The library's core is designed to support multiple target languages, currently featuring specializations for Structured Query Language (SQL) and Cassandra Query Language (CQL).

Features:

- Boilerplate-free mapping: The database schema is mapped using simple case classes.
- Quoted DSL: Queries are defined inside a quote block. Quill parses each quoted block of code (quotation) at compile time and translates them to an internal Abstract Syntax Tree (AST)
- Compile-time query generation: The ctx.run call reads the quotation's AST and translates it to the target language at compile time, emitting the query string as a compilation message. As the query string is known at compile time, the runtime overhead is very low and similar to using the database driver directly.
- Compile-time query validation: If configured, the query is verified against the database at compile time and the compilation fails if it is not valid. The query validation does not alter the database state.

https://github.com/getquill/quill

​​Despite all the recent hype, setting up a new #ts library can be tough. Between Rollup, Jest, tsconfig, Yarn resolutions, ESLint, and getting VSCode to play nicely....there is just a whole lot of stuff to do (and things to screw up). TSDX is a zero-config CLI that helps you develop, test, and publish modern TypeScript packages with ease - so you can focus on your awesome new library and not waste another afternoon on the configuration.

Features: linting, testing, building, and local development.

https://github.com/jaredpalmer/tsdx

​​Cloister: The helper application to manage cluster of #elixir nodes.

> Cloister is being developed with the drop-in support of Docker. Distributed Erlang is a charm to work with unless your DevOps team is engaged in containerizing the whole universe. They usually have many cryptic arguments full of words you as an old-school software engineer would barely understand, and sooner or later you are faced with a fact that now we run everything in a dockerized environment, which means no one can guarantee your application has an IP/DNS, or would not have automatically restarted depending on the current moon phase.

> That is where cloister might step into to help. It takes care about the cluster handling, based on either the list of node names (IP/DNS, old school,) or a service name exposed by docker.

https://github.com/am-kantox/cloister

​​Recoil is an experimental state management library for #react apps. It provides several capabilities that are difficult to achieve with React alone, while being compatible with the newest features of React.

https://github.com/facebookexperimental/Recoil

#js

P.S. One more?!

​​A lightweight text editor written in #lua

It aims to provide something practical, pretty, small and fast, implemented as simply as possible; easy to modify and extend, or to use without doing either.

Additional functionality can be added through plugins which are available from the plugins repository; additional color themes can be found in the colors repository. The editor can be customized by making changes to the user module.

https://github.com/rxi/lite

P.S. One more?!

​​Use esbuild with webpack to transform #js and #ts code.

esbuild is by far one of the fastest TS/ESNext to ES6 compilers, so it makes sense to use it over Babel/TSC with webpack to take advantage of both worlds (Speed and the webpack ecosytem).

Why is it fast? Several reasons:

- It's written in #go, a language that compiles to native code
- Parsing, printing, and source map generation are all fully parallelized
- Everything is done in very few passes without expensive data transformations
- Code is written with speed in mind, and tries to avoid unnecessary allocations

https://github.com/egoist/esbuild-loader

​​Introducing @nuxt/components: module to scan and auto import components for Nuxt.js 2.10+

No need to import your components anymore.

- Scan and auto import components
- Multiple paths with customisable prefixes and patterns
- Lazy loading support
- Hot replacement
- Support for library authors

https://github.com/nuxt/components

#nuxt #vue #ts

​​A #css framework for building faithful recreations of operating system GUIs. XP.css is an extension of 98.css. A CSS file that takes semantic HTML and makes it look pretty. It does not ship with any JavaScript, so it is compatible with your frontend framework of choice.

https://botoxparty.github.io/XP.css/

​​telefork: like fork() but teleports the forked process to a different computer!

It does this using a bunch of ptrace magic to serialize the memory mappings of the process, stream them over a pipe and recreate them on the other end along with the registers and some other process state.

https://github.com/trishume/telefork

#rust

n8n: Free and Open Workflow Automation Tool.

n8n (pronounced nodemation) helps you to interconnect every app with an API in the world with each other to share and manipulate its data without a single line of code. It is an easy to use, user-friendly and highly customizable service, which uses an intuitive user interface for you to design your unique workflows very fast. Hosted on your server and not based in the cloud, it keeps your sensible data very secure in your own trusted database. n8n has 100+ different nodes to automate workflows.

https://github.com/n8n-io/n8n

#ts #nocode

​​Mypy stubs, i.e., type information, for numpy, pandas and matplotlib for your #ds #python projects.

Lots of functions are already typed, but a lot is still missing (numpy and pandas are huge libraries).

https://github.com/predictive-analytics-lab/data-science-types

​​Great Expectations: Always know what to expect from your data.

Great Expectations helps data teams eliminate pipeline debt, through data testing, documentation, and profiling.

Software developers have long known that testing and documentation are essential for managing complex codebases. Great Expectations brings the same confidence, integrity, and acceleration to data science and data engineering teams.

See Down with Pipeline Debt! for an introduction to the philosophy of pipeline testing: https://medium.com/@expectgreatdata/down-with-pipeline-debt-introducing-great-expectations-862ddc46782a

Key features:
- Expectations or assertions for data. They are the workhorse abstraction in Great Expectations, covering all kinds of common data issues
- Batteries-included data validation
- Tests are docs and docs are tests: many data teams struggle to maintain up-to-date data documentation. Great Expectations solves this problem by rendering Expectations directly into clean, human-readable documentation
- Automated data profiling: wouldn't it be great if your tests could write themselves? Run your data through one of Great Expectations' data profilers and it will automatically generate Expectations and data documentation
- Pluggable and extensible

https://github.com/great-expectations/great_expectations

#python #ds #docops

​​Materialize is a streaming database for real-time applications. Written in #rust

Materialize lets you ask questions of your live data, which it answers and then maintains for you as your data continue to change. The moment you need a refreshed answer, you can get it in milliseconds. Materialize is designed to help you interactively explore your streaming data, perform data warehousing analytics against live relational data, or just increase the freshness and reduce the load of your dashboard and monitoring tasks.

Materialize focuses on providing correct and consistent answers with minimal latency. It does not ask you to accept either approximate answers or eventual consistency. Whenever Materialize answers a query, that answer is the correct result on some specific (and recent) version of your data. Materialize does all of this by recasting your SQL92 queries as dataflows, which can react efficiently to changes in your data as they happen. Materialize is powered by timely dataflow, which connects the times at which your inputs change with the times of answers reported back to you.

We support a large fraction of PostgreSQL, and are actively working on supporting more builtin PostgreSQL functions.

Example:

 Views define commonly reused subqueries.
CREATE VIEW revenue (supplier_no, total_revenue) AS
    SELECT
        l_suppkey,
        SUM(l_extendedprice * (1 - l_discount))
    FROM
        lineitem
    WHERE
        l_shipdate >= DATE '1996-01-01'
        AND l_shipdate < DATE '1996-01-01' + INTERVAL '3' month
    GROUP BY
        l_suppkey;

-- Materialized views are maintained automatically, and can depend on non-materialized views.
CREATE MATERIALIZED VIEW tpch_q15 AS
  SELECT
    s_suppkey,
    s_name,
    s_address,
    s_phone,
    total_revenue
FROM
    supplier,
    revenue
WHERE
    s_suppkey = supplier_no
    AND total_revenue = (
        SELECT
            max(total_revenue)
        FROM
            revenue
    )
ORDER BY
    s_suppkey

https://github.com/MaterializeInc/materialize

​​A modified browser built using Electron that helps in responsive web development.

Features
- Mirrored User-interactions across all devices.
- Customizable preview layout to suit all your needs.
- One handy elements inspector for all devices in preview.
- 30+ built-in device profiles with option to add custom devices.
- One-click screenshot all your devices.
- Hot reloading supported for developers.

https://github.com/manojVivek/responsively-app

#js

Ormolu is a formatter for #haskell source code. The project was created with the following goals in mind:

- Using GHC's own parser to avoid parsing problems caused by haskell-src-exts.
- Let some whitespace be programmable. The layout of the input influences the layout choices in the output. This means that the choices between single-line/multi-line layouts in each particular situation are made by the user, not by an algorithm. This makes the implementation simpler and leaves some control to the user while still guaranteeing that the formatted code is stylistically consistent.
- Writing code in such a way so it's easy to modify and maintain.
- Implementing one “true” formatting style which admits no configuration.
- That formatting style aims to result in minimal diffs while still remaining very close to “conventional” Haskell formatting people use.
- Choose a style compatible with modern dialects of Haskell. As new Haskell extensions enter broad use, we may change the style to accomodate them.
- Idempotence: formatting already formatted code doesn't change it.
- Be well-tested and robust to the point that it can be used in large projects without exposing unfortunate, disappointing bugs here and there.

https://github.com/tweag/ormolu

​​Introducing @nuxt/content

The content/ directory for your #nuxt app, acting as a git-based headless CMS.

- #vue components in Markdown
- Powerful QueryBuilder API
- Handles MD, CSV, YAML, JSON
- Blazing fast hot reload in dev
- Syntax highlighting

https://content.nuxtjs.org/

#js #ts

​​sql-lint is a linter for SQL, it brings back any error from the supported servers as well as custom errors written for sql-lint.

sql-lint will show errors about the following things (and more)

- DELETE statements missing WHERE clauses
- DROP/TRUNCATE/ALTER/CREATE statements with invalid options specified
- Odd code points in queries
- Invalid arguments to the LIMIT clause
- Any SQL server errors

sql-lint brings back errors from the supported servers too. It will catch any error from the server. these include but are not limited to:

- Unknown columns on a table
- A non existent database
- A non existent table
- Syntax errors

https://github.com/joereynolds/sql-lint

#ts #sql

Modern Database Access for TypeScript & Node.js
Prisma is an open-source database toolkit. It replaces traditional ORMs and makes database access easy with an auto-generated query builder for TypeScript & Node.js.

Prisma makes database access easy with an auto-generated and type-safe query builder that's tailored to your database schema. It provides an alternative to traditional ORMs and SQL query builders. Start with a new database or introspect an existing one to get started.

Also supports schema migrations.

https://www.prisma.io/

Personal opinion: it looks amazing! We need something like this in Python!

#ts #js

​​A crawler for automated functional testing of #python web applications.

Crawling a server-side-rendered web application is a low cost way to get low quality test coverage of your JavaScript-light web application.

If you have only partial test coverage of your routes, but still want to protect against silly mistakes, then this is for you.

Why should I use this? Here's an example: Flaskr, the Flask tutorial application has 166 lines of test code to achieve 100% test coverage.

Using Python Testing Crawler in a similar way to the Usage example below, we can hit 73% with very little effort. Disclaimer: Of course! It's not the same quality or utility of testing! But it is better than no tests, a complement to hand-written unit or functional tests and a useful stopgap.

https://github.com/python-testing-crawler/python-testing-crawler

Your desktop is a mess? You cannot find anything in your downloads and documents? Sorting and renaming all these files by hand is too tedious? Time to automate it once and benefit from it forever with organize!

Config example:

  - folders: ~/Downloads
    subfolders: true
    filters:
      - extension: pdf
    actions:
      - echo: "Found PDF!"
      - move: "~/Documents/pdfs"

https://github.com/tfeldmann/organize

#python

​​A next-generation curated knowledge sharing platform for data scientists and other technical professions.

The Knowledge Repo project is focused on facilitating the sharing of knowledge between data scientists and other technical roles using data formats and tools that make sense in these professions. It provides various data stores (and utilities to manage them) for "knowledge posts", with a particular focus on notebooks (R Markdown and Jupyter / IPython Notebook) to better promote reproducible research.

For more information about the motivation and inspiration behind this project, we encourage you to read our Medium Post: https://medium.com/airbnb-engineering/scaling-knowledge-at-airbnb-875d73eff091

https://github.com/airbnb/knowledge-repo

#python #ds #docops