Nike-Inc/koheesio
Python framework for building efficient data pipelines. It promotes modularity and collaboration, enabling the creation of complex pipelines from simple, reusable components.
Language:Python
Total stars: 162
Stars trend:
#python
#dataengineering, #deltalake, #pydantic, #pyspark, #python
Python framework for building efficient data pipelines. It promotes modularity and collaboration, enabling the creation of complex pipelines from simple, reusable components.
Language:Python
Total stars: 162
Stars trend:
3 Jun 2024
10pm ▏ +1
11pm +0
4 Jun 2024
12am +0
1am +0
2am ▏ +1
3am +0
4am +0
5am ▋ +5
6am ████▋ +37
7am ██▌ +20
8am ██▉ +23#python
#dataengineering, #deltalake, #pydantic, #pyspark, #python
eugeneyan/applied-ml
📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.
Language:
Total stars: 26245
Stars trend:
#applieddatascience, #appliedmachinelearning, #computervision, #datadiscovery, #dataengineering, #dataquality, #datascience, #deeplearning, #machinelearning, #naturallanguageprocessing, #production, #recsys, #reinforcementlearning, #search
📚 Papers & tech blogs by companies sharing their work on data science & machine learning in production.
Language:
Total stars: 26245
Stars trend:
18 Jun 2024
6pm ▍ +3
7pm █▌ +12
8pm █▍ +11
9pm █▎ +10
10pm ▌ +4
11pm █▏ +9
19 Jun 2024
12am ▋ +5
1am ▊ +6
2am ▋ +5
3am ▊ +6
4am █ +8#applieddatascience, #appliedmachinelearning, #computervision, #datadiscovery, #dataengineering, #dataquality, #datascience, #deeplearning, #machinelearning, #naturallanguageprocessing, #production, #recsys, #reinforcementlearning, #search
GokuMohandas/Made-With-ML
Learn how to design, develop, deploy and iterate on production-grade ML applications.
Language:Jupyter Notebook
Total stars: 36368
Stars trend:
#jupyternotebook
#dataengineering, #dataquality, #datascience, #deeplearning, #distributedml, #distributedtraining, #llms, #machinelearning, #mlops, #naturallanguageprocessing, #python, #pytorch, #ray
Learn how to design, develop, deploy and iterate on production-grade ML applications.
Language:Jupyter Notebook
Total stars: 36368
Stars trend:
19 Jun 2024
1am ▌ +4
2am ▍ +3
3am ▎ +2
4am ▉ +7
5am ▉ +7
6am ▉ +7
7am █▍ +11
8am █▎ +10
9am ▋ +5
10am ▋ +5
11am █▎ +10
12pm ▊ +6#jupyternotebook
#dataengineering, #dataquality, #datascience, #deeplearning, #distributedml, #distributedtraining, #llms, #machinelearning, #mlops, #naturallanguageprocessing, #python, #pytorch, #ray
open-metadata/OpenMetadata
OpenMetadata is a unified platform for discovery, observability, and governance powered by a central metadata repository, in-depth lineage, and seamless team collaboration.
Language:TypeScript
Total stars: 4556
Stars trend:
#typescript
#datacatalog, #datacollaboration, #datacontracts, #datadiscovery, #datagovernance, #datalineage, #dataobservability, #dataprofiling, #dataquality, #dataqualitychecks, #datascience, #datavalidation, #datacatalog, #datadiscovery, #dataengineering, #dataquality, #dbt, #metadata, #metadatamanagement, #snowflake
OpenMetadata is a unified platform for discovery, observability, and governance powered by a central metadata repository, in-depth lineage, and seamless team collaboration.
Language:TypeScript
Total stars: 4556
Stars trend:
19 Jun 2024
11am ▌ +4
12pm ▋ +5
1pm █▍ +11
2pm █ +8
3pm █ +8
4pm ▉ +7
5pm ▉ +7
6pm ▎ +2
7pm ▋ +5
8pm ▉ +7
9pm ▋ +5
10pm ▊ +6#typescript
#datacatalog, #datacollaboration, #datacontracts, #datadiscovery, #datagovernance, #datalineage, #dataobservability, #dataprofiling, #dataquality, #dataqualitychecks, #datascience, #datavalidation, #datacatalog, #datadiscovery, #dataengineering, #dataquality, #dbt, #metadata, #metadatamanagement, #snowflake
👍2
PrefectHQ/prefect
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
Language:Python
Total stars: 16541
Stars trend:
#python
#automation, #data, #dataengineering, #dataops, #datascience, #infrastructure, #mlops, #observability, #orchestration, #pipeline, #prefect, #python, #workflow, #workflowengine
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
Language:Python
Total stars: 16541
Stars trend:
9 Nov 2024
4am ▎ +2
5am ▏ +1
6am ▍ +3
7am ▏ +1
8am ▎ +2
9am ▏ +1
10am ▍ +3
11am ██▉ +23
12pm ███ +24
1pm ████▍ +35#python
#automation, #data, #dataengineering, #dataops, #datascience, #infrastructure, #mlops, #observability, #orchestration, #pipeline, #prefect, #python, #workflow, #workflowengine
apache/airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Language:Python
Total stars: 37799
Stars trend:
#python
#airflow, #apache, #apacheairflow, #automation, #dag, #dataengineering, #dataintegration, #dataorchestrator, #datapipelines, #datascience, #elt, #etl, #machinelearning, #mlops, #orchestration, #python, #scheduler, #workflow, #workflowengine, #workfloworchestration
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Language:Python
Total stars: 37799
Stars trend:
19 Dec 2024
8pm ▉ +7
9pm ▉ +7
10pm ▋ +5
11pm ▉ +7
20 Dec 2024
12am ▏ +1
1am █▏ +9
2am ▉ +7
3am ▍ +3
4am █▎ +10
5am ▉ +7
6am ▊ +6
7am █▎ +10#python
#airflow, #apache, #apacheairflow, #automation, #dag, #dataengineering, #dataintegration, #dataorchestrator, #datapipelines, #datascience, #elt, #etl, #machinelearning, #mlops, #orchestration, #python, #scheduler, #workflow, #workflowengine, #workfloworchestration
DataTalksClub/data-engineering-zoomcamp
Free Data Engineering course!
Language:Jupyter Notebook
Total stars: 27135
Stars trend:
#jupyternotebook
#dataengineering, #dbt, #docker, #kafka, #kestra, #spark
Free Data Engineering course!
Language:Jupyter Notebook
Total stars: 27135
Stars trend:
13 Jan 2025
5am ▏ +3
6am +2
7am ▎ +5
8am +1
9am ▊ +15
10am ▏ +3
11am ▏ +3
12pm ▍ +7
1pm ▍ +9
2pm ▎ +5
3pm ▍ +8
4pm ██████████████████▊ +342#jupyternotebook
#dataengineering, #dbt, #docker, #kafka, #kestra, #spark
pyper-dev/pyper
Concurrent Python made simple
Language:Python
Total stars: 136
Stars trend:
#python
#asyncio, #concurrency, #data, #datacollection, #dataengineering, #datapipelines, #dataprocessing, #multiprocessing, #parallelcomputing, #python, #threading
Concurrent Python made simple
Language:Python
Total stars: 136
Stars trend:
15 Jan 2025
2am █▌ +12
3am ██▊ +22
4am ██▊ +22
5am ██▋ +21
6am ██▏ +17#python
#asyncio, #concurrency, #data, #datacollection, #dataengineering, #datapipelines, #dataprocessing, #multiprocessing, #parallelcomputing, #python, #threading
DataExpert-io/data-engineer-handbook
This is a repo with links to everything you'd ever want to learn about data engineering
Language:Jupyter Notebook
Total stars: 24911
Stars trend:
#jupyternotebook
#apachespark, #awesome, #bigdata, #data, #dataengineering, #sql
This is a repo with links to everything you'd ever want to learn about data engineering
Language:Jupyter Notebook
Total stars: 24911
Stars trend:
22 Jan 2025
9am ▏ +1
10am ▊ +6
11am ▉ +7
12pm ▊ +6
1pm █▏ +9
2pm █▍ +11
3pm █ +8
4pm █▏ +9
5pm █ +8
6pm ▍ +3
7pm █ +8#jupyternotebook
#apachespark, #awesome, #bigdata, #data, #dataengineering, #sql
🔥1
cocoindex-io/cocoindex
ETL framework to turn your data AI-ready - with realtime incremental updates and support custom logic like lego.
Language:Rust
Total stars: 672
Stars trend:
#rust
#ai, #changedatacapture, #data, #dataengineering, #dataindexing, #datainfrastructure, #dataprocessing, #dataflow, #etl, #helpwanted, #indexing, #knowledgegraph, #llm, #pipeline, #python, #rag, #realtime, #rust, #semanticsearch, #streaming
ETL framework to turn your data AI-ready - with realtime incremental updates and support custom logic like lego.
Language:Rust
Total stars: 672
Stars trend:
20 Apr 2025
3pm █▉ +15
4pm ▍ +3
5pm ▋ +5
6pm ▉ +7
7pm ▋ +5
8pm ▋ +5
9pm ▊ +6
10pm ▊ +6
11pm █ +8
21 Apr 2025
12am ▍ +3
1am ▉ +7
2am ▉ +7#rust
#ai, #changedatacapture, #data, #dataengineering, #dataindexing, #datainfrastructure, #dataprocessing, #dataflow, #etl, #helpwanted, #indexing, #knowledgegraph, #llm, #pipeline, #python, #rag, #realtime, #rust, #semanticsearch, #streaming
cocoindex-io/cocoindex
Real-time data transformation framework for AI. Ultra performant, with incremental processing.
Language:Rust
Total stars: 1459
Stars trend:
#rust
#ai, #changedatacapture, #data, #dataengineering, #dataindexing, #datainfrastructure, #dataprocessing, #dataflow, #etl, #helpwanted, #indexing, #knowledgegraph, #llm, #pipeline, #python, #rag, #realtime, #rust, #semanticsearch, #streaming
Real-time data transformation framework for AI. Ultra performant, with incremental processing.
Language:Rust
Total stars: 1459
Stars trend:
20 May 2025
1am ▊ +6
2am ▍ +3
3am ▊ +6
4am █▎ +10
5am █▎ +10
6am ▋ +5
7am █▎ +10
8am ▎ +2
9am ▌ +4
10am █▎ +10
11am ▉ +7
12pm █ +8#rust
#ai, #changedatacapture, #data, #dataengineering, #dataindexing, #datainfrastructure, #dataprocessing, #dataflow, #etl, #helpwanted, #indexing, #knowledgegraph, #llm, #pipeline, #python, #rag, #realtime, #rust, #semanticsearch, #streaming
DataExpert-io/data-engineer-handbook
This is a repo with links to everything you'd ever want to learn about data engineering
Language:Jupyter Notebook
Total stars: 28322
Stars trend:
#jupyternotebook
#apachespark, #awesome, #bigdata, #data, #dataengineering, #sql
This is a repo with links to everything you'd ever want to learn about data engineering
Language:Jupyter Notebook
Total stars: 28322
Stars trend:
1 Jun 2025
6pm ▏ +1
7pm ▏ +1
8pm ▏ +1
9pm +0
10pm +0
11pm ▊ +6
2 Jun 2025
12am ████ +32
1am █▋ +13
2am █▉ +15
3am █▋ +13#jupyternotebook
#apachespark, #awesome, #bigdata, #data, #dataengineering, #sql
Eventual-Inc/Daft
Distributed query engine providing simple and reliable data processing for any modality and scale
Language:Rust
Total stars: 3615
Stars trend:
#rust
#artificialintelligence, #bigdata, #dataengineering, #distributedcomputing, #machinelearning, #multimodal, #python, #rust
Distributed query engine providing simple and reliable data processing for any modality and scale
Language:Rust
Total stars: 3615
Stars trend:
6 Sep 2025
7am ▎ +2
8am +0
9am ▏ +1
10am ▌ +4
11am █▍ +11
12pm ▊ +6
1pm █▏ +9
2pm █▍ +11
3pm █▌ +12
4pm █▎ +10
5pm ▊ +6
6pm █▎ +10#rust
#artificialintelligence, #bigdata, #dataengineering, #distributedcomputing, #machinelearning, #multimodal, #python, #rust
ruc-datalab/DeepAnalyze
DeepAnalyze is the first agentic LLM for autonomous data science.
Language:Python
Total stars: 172
Stars trend:
#python
#agent, #agentic, #agenticai, #chatbot, #data, #dataanalysis, #dataengineering, #datascience, #datavisualization, #gpt, #llm
DeepAnalyze is the first agentic LLM for autonomous data science.
Language:Python
Total stars: 172
Stars trend:
21 Oct 2025
12pm ▏ +1
1pm ▏ +1
2pm ▎ +2
3pm ▊ +6
4pm ▎ +2
5pm ▏ +1
6pm +0
7pm █ +8
8pm ▌ +4
9pm ▎ +2
10pm ▎ +2
11pm ▍ +3#python
#agent, #agentic, #agenticai, #chatbot, #data, #dataanalysis, #dataengineering, #datascience, #datavisualization, #gpt, #llm
kimtth/software-architect-mindmap
🧠Mindmap of 🗺️Software Architecture, Software engineering: An Overview of Software Terminologies and Concepts.
Language:Python
Total stars: 162
Stars trend:
#python
#architecturalpatterns, #architecture, #aws, #azure, #dataengineering, #datascience, #deeplearning, #development, #gcp, #machinelearning, #microservice, #mindmap, #softwarearchitecture, #softwareengineering, #webdevelopment
🧠Mindmap of 🗺️Software Architecture, Software engineering: An Overview of Software Terminologies and Concepts.
Language:Python
Total stars: 162
Stars trend:
21 Nov 2025
7pm ▎ +2
8pm +0
9pm ▏ +1
10pm ▍ +3
11pm ▏ +1
22 Nov 2025
12am ▎ +2
1am ▍ +3
2am ▌ +4
3am ▎ +2
4am ▌ +4
5am ▌ +4#python
#architecturalpatterns, #architecture, #aws, #azure, #dataengineering, #datascience, #deeplearning, #development, #gcp, #machinelearning, #microservice, #mindmap, #softwarearchitecture, #softwareengineering, #webdevelopment
andkret/Cookbook
The Data Engineering Cookbook
Language:Python
Total stars: 14758
Stars trend:
#python
#bestpractices, #bigdata, #cookbook, #dataengineer, #dataengineering
The Data Engineering Cookbook
Language:Python
Total stars: 14758
Stars trend:
25 Nov 2025
4am ▊ +6
5am █ +8
6am ▌ +4
7am ▍ +3
8am ▎ +2
9am ▏ +1
10am +0
11am ▏ +1
12pm ▏ +1
1pm +0
2pm ▏ +1
3pm ▋ +5#python
#bestpractices, #bigdata, #cookbook, #dataengineer, #dataengineering
taz4et/bitcoin-etl
A Python toolkit for extracting and transforming Bitcoin blockchain data into structured formats.
Language:Python
Total stars: 169
Stars trend:
#python
#apachebeam, #bitcoin, #bitcoinetl, #bitcoincash, #bitcoincashmining, #crypto, #cryptocurrency, #dash, #dataanalysis, #dataengineering, #dogecoin, #etl, #onchainanalysis, #zcash
A Python toolkit for extracting and transforming Bitcoin blockchain data into structured formats.
Language:Python
Total stars: 169
Stars trend:
29 Nov 2025
3pm ▍ +3
4pm ▌ +4
5pm ▊ +6
6pm ▎ +2
7pm █▏ +9
8pm ▊ +6
9pm ▉ +7
10pm ▎ +2
11pm ▉ +7#python
#apachebeam, #bitcoin, #bitcoinetl, #bitcoincash, #bitcoincashmining, #crypto, #cryptocurrency, #dash, #dataanalysis, #dataengineering, #dogecoin, #etl, #onchainanalysis, #zcash
DataTalksClub/data-engineering-zoomcamp
Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. The next cohort starts in January 2026. Join the course here 👇🏼
Language:Jupyter Notebook
Total stars: 37250
Stars trend:
#jupyternotebook
#course, #dataengineering, #dbt, #docker, #free, #kafka, #kestra, #spark
Data Engineering Zoomcamp is a free 9-week course on building production-ready data pipelines. The next cohort starts in January 2026. Join the course here 👇🏼
Language:Jupyter Notebook
Total stars: 37250
Stars trend:
25 Jan 2026
10pm ▎ +2
11pm ▎ +2
26 Jan 2026
12am ▏ +1
1am ▏ +1
2am ▏ +1
3am ██▌ +20
4am █▏ +9
5am ▌ +4
6am ▎ +2
7am ▌ +4
8am ▌ +4
9am ▌ +4#jupyternotebook
#course, #dataengineering, #dbt, #docker, #free, #kafka, #kestra, #spark
❤2
ashishps1/awesome-engineering-articles
A curated collection of 300+ engineering blog articles from top tech companies. Learn how the best engineering teams solve real-world problems at scale.
Language:
Total stars: 199
Stars trend:
#ai, #dataengineering, #database, #distributedsystems, #engineering, #infrastructure, #ml, #systemdesign
A curated collection of 300+ engineering blog articles from top tech companies. Learn how the best engineering teams solve real-world problems at scale.
Language:
Total stars: 199
Stars trend:
1 Mar 2026
4am █▊ +14
5am ██▍ +19
6am █▊ +14
7am ▌ +4
8am ▋ +5#ai, #dataengineering, #database, #distributedsystems, #engineering, #infrastructure, #ml, #systemdesign
👍1
open-metadata/OpenMetadata
OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.
Language:TypeScript
Total stars: 9334
Stars trend:
#typescript
#datacatalog, #datacollaboration, #datacontracts, #datadiscovery, #datagovernance, #datalineage, #dataobservability, #dataprofiling, #dataquality, #dataqualitychecks, #datavalidation, #datadiscovery, #dataengineering, #dataquality, #hacktoberfest, #mcp, #mcpserver, #metadata, #metadatamanagement, #snowflake
OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.
Language:TypeScript
Total stars: 9334
Stars trend:
2 Apr 2026
3am ▏ +1
4am ▋ +5
5am █▏ +9
6am ▏ +1
7am +0
8am ▍ +3
9am ▌ +4
10am ▎ +2
11am ▍ +3
12pm ▏ +1
1pm █▏ +9#typescript
#datacatalog, #datacollaboration, #datacontracts, #datadiscovery, #datagovernance, #datalineage, #dataobservability, #dataprofiling, #dataquality, #dataqualitychecks, #datavalidation, #datadiscovery, #dataengineering, #dataquality, #hacktoberfest, #mcp, #mcpserver, #metadata, #metadatamanagement, #snowflake