Forwarded from Находки в опенсорсе
Great Expectations: Always know what to expect from your data.
Great Expectations helps data teams eliminate pipeline debt, through data testing, documentation, and profiling.
Software developers have long known that testing and documentation are essential for managing complex codebases. Great Expectations brings the same confidence, integrity, and acceleration to data science and data engineering teams.
See Down with Pipeline Debt! for an introduction to the philosophy of pipeline testing: https://medium.com/@expectgreatdata/down-with-pipeline-debt-introducing-great-expectations-862ddc46782a
Key features:
- Expectations or assertions for data. They are the workhorse abstraction in Great Expectations, covering all kinds of common data issues
- Batteries-included data validation
- Tests are docs and docs are tests: many data teams struggle to maintain up-to-date data documentation. Great Expectations solves this problem by rendering Expectations directly into clean, human-readable documentation
- Automated data profiling: wouldn't it be great if your tests could write themselves? Run your data through one of Great Expectations' data profilers and it will automatically generate Expectations and data documentation
- Pluggable and extensible
https://github.com/great-expectations/great_expectations
#python #ds #docops
Great Expectations helps data teams eliminate pipeline debt, through data testing, documentation, and profiling.
Software developers have long known that testing and documentation are essential for managing complex codebases. Great Expectations brings the same confidence, integrity, and acceleration to data science and data engineering teams.
See Down with Pipeline Debt! for an introduction to the philosophy of pipeline testing: https://medium.com/@expectgreatdata/down-with-pipeline-debt-introducing-great-expectations-862ddc46782a
Key features:
- Expectations or assertions for data. They are the workhorse abstraction in Great Expectations, covering all kinds of common data issues
- Batteries-included data validation
- Tests are docs and docs are tests: many data teams struggle to maintain up-to-date data documentation. Great Expectations solves this problem by rendering Expectations directly into clean, human-readable documentation
- Automated data profiling: wouldn't it be great if your tests could write themselves? Run your data through one of Great Expectations' data profilers and it will automatically generate Expectations and data documentation
- Pluggable and extensible
https://github.com/great-expectations/great_expectations
#python #ds #docops
Forwarded from Полезняшки от "Разбора Полетов"
Apache Arrow and Java: Lightning Speed Big Data Transfer
https://xn--r1a.website/iv?url=https://www.infoq.com/articles/apache-arrow-java/&rhash=6030af2cd8fcfe
https://xn--r1a.website/iv?url=https://www.infoq.com/articles/apache-arrow-java/&rhash=6030af2cd8fcfe
InfoQ
Apache Arrow and Java: Lightning Speed Big Data Transfer
Apache Arrow puts forward a cross-language, cross-platform, columnar in-memory data format for data. It is designed to eliminate the need for data serialization and reduce the overhead of copying.
Forwarded from Записки админа
🛠 Live Patching Tools.
Сравнение livepatch-утилит, которые позволяют патчить ядро налету, уменьшая, тем самым, количество необходимых ребутов.
#livepatch #kernel #напочитать
Сравнение livepatch-утилит, которые позволяют патчить ядро налету, уменьшая, тем самым, количество необходимых ребутов.
#livepatch #kernel #напочитать
Forwarded from Полезняшки от "Разбора Полетов"
Troubleshooting CPU spike in a Major Trading Application
https://xn--r1a.website/iv?url=https://medium.com/@RamLakshmanan/troubleshooting-cpu-spike-in-a-major-trading-application-cb7ad222ff9f?source=rss------java-5&rhash=f5b2f9872995e4
https://xn--r1a.website/iv?url=https://medium.com/@RamLakshmanan/troubleshooting-cpu-spike-in-a-major-trading-application-cb7ad222ff9f?source=rss------java-5&rhash=f5b2f9872995e4
Medium
Troubleshooting CPU spike in a Major Trading Application
In this article we are going to discuss how we troubleshooted a CPU spike problem that surfaced in a major trading application in North…
Forwarded from L̶u̵m̶i̵n̷o̴u̶s̶m̶e̵n̵B̶l̵o̵g̵
Descriptive statistics will teach you the basic concepts used to describe the data sample
https://luminousmen.com/post/descriptive-and-inferential-statistics
https://luminousmen.com/post/descriptive-and-inferential-statistics
Blog | iamluminousmen
Data Science. Descriptive and Inferential Statistics
Discover the foundations of Descriptive and Inferential Statistics in Data Science. Learn how to analyze data samples effectively. Start understanding your data today!
Forwarded from Sergey
никто на это не напоролся ? https://www.opennet.ru/opennews/art.shtml?num=53061
проблема 2020-го года :)
проблема 2020-го года :)
www.opennet.ru
Устаревание корневого сертификата AddTrust привело к сбоям в системах с OpenSSL и GnuTLS
30 мая истёк 20-летний срок действия корневого сертификата AddTrust, который применялся для формирования перекрёстной подписи (cross-signed) в сертификатах одного из крупнейших удостоверяющих центров Sectigo (Comodo). Перекрёстная подпись позволяла обеспечить…
Forwarded from Мониторим ИТ
Метрики, логи и трейсы — три вещи, которые покажут всё о здоровье приложения. В этом видео разработчик Grafana Labs рассказывает как настроить корреляцию этих трёх субстанций в Grafana на основе лейлблов. Это ещё одно выступление с Fosdem 2020.
Forwarded from Nikolay Markov
вот вам годнота https://hackernoon.com/another-reason-why-your-docker-containers-may-be-slow-d37207dec27f
Hackernoon
Another reason why your Docker containers may be slow
In my last <a href="https://hackernoon.com/kubernetes-for-dev-infrastructure-40b9175cb8c0" target="_blank">blog post</a> I was talking about Kubernetes and how <a href="https://thoughtspot.com" target="_blank">ThoughtSpot</a> uses it for its dev infrastructure…
Forwarded from Записки админа
⚒ Monitoror.
Продолжаю открывать для себя мир простых мониторингов. Сегодня вот на Monitoror внимание обратил.
Интересный инструмент - один бинарник, две сущности - core и ui, простое (на первый взгляд во всяком случае) конфигурирование, приятный вид, открытый код.
Вобщем, если как и я пропустили ранее - познакомьтесь с этим мониторингом обязательно.
#monitoring #github
Продолжаю открывать для себя мир простых мониторингов. Сегодня вот на Monitoror внимание обратил.
Интересный инструмент - один бинарник, две сущности - core и ui, простое (на первый взгляд во всяком случае) конфигурирование, приятный вид, открытый код.
Вобщем, если как и я пропустили ранее - познакомьтесь с этим мониторингом обязательно.
#monitoring #github