DataExpert-io/data-engineer-handbook
This is a repo with links to everything you'd ever want to learn about data engineering
Language:
Total stars: 7342
Stars trend:
#apachespark, #awesome, #bigdata, #data, #dataengineering, #sql
This is a repo with links to everything you'd ever want to learn about data engineering
Language:
Total stars: 7342
Stars trend:
20 Mar 2024
7pm ██▎ +18
8pm ███▉ +31
9pm █▉ +15
10pm █▉ +15#apachespark, #awesome, #bigdata, #data, #dataengineering, #sql
kestra-io/kestra
Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.
Language:Java
Total stars: 5704
Stars trend:
#java
#data, #dataengineering, #dataintegration, #dataorchestration, #dataorchestrator, #datapipeline, #dataquality, #elt, #etl, #lowcode, #orchestration, #pipeline, #reverseetl, #scheduler, #workflow, #workflowengine
Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.
Language:Java
Total stars: 5704
Stars trend:
26 Mar 2024
8am ▏ +1
9am +0
10am +0
11am +0
12pm +0
1pm +0
2pm ▏ +1
3pm ███▏ +25
4pm ██▏ +17
5pm ██▎ +18
6pm █▋ +13#java
#data, #dataengineering, #dataintegration, #dataorchestration, #dataorchestrator, #datapipeline, #dataquality, #elt, #etl, #lowcode, #orchestration, #pipeline, #reverseetl, #scheduler, #workflow, #workflowengine
spiceai/spiceai
A unified SQL query interface and portable runtime to locally materialize, accelerate, and query data tables sourced from any database, data warehouse, or data lake.
Language:Rust
Total stars: 913
Stars trend:
#rust
#artificialintelligence, #data, #developers, #infrastructure, #machinelearning, #sql, #timeseries
A unified SQL query interface and portable runtime to locally materialize, accelerate, and query data tables sourced from any database, data warehouse, or data lake.
Language:Rust
Total stars: 913
Stars trend:
28 Mar 2024
5pm ▏ +1
6pm ▌ +4
7pm ██▌ +20
8pm ██▍ +19
9pm ██▋ +21
10pm █▉ +15#rust
#artificialintelligence, #data, #developers, #infrastructure, #machinelearning, #sql, #timeseries
mendableai/firecrawl
🔥 Turn entire websites into LLM-ready markdown
Language:TypeScript
Total stars: 106
Stars trend:
#typescript
#ai, #crawler, #data, #htmltomarkdown, #llm, #markdown, #rag, #scraper, #scraping, #webcrawler
🔥 Turn entire websites into LLM-ready markdown
Language:TypeScript
Total stars: 106
Stars trend:
16 Apr 2024
5pm ▋ +5
6pm ██▍ +19
7pm ██▎ +18
8pm █▊ +14
9pm █▎ +10
10pm █ +8
11pm ▎ +2
17 Apr 2024
12am ██▎ +18#typescript
#ai, #crawler, #data, #htmltomarkdown, #llm, #markdown, #rag, #scraper, #scraping, #webcrawler
flyteorg/flyte
Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
Language:Go
Total stars: 4948
Stars trend:
#go
#data, #dataanalysis, #datascience, #dataops, #declarative, #finetuning, #flyte, #golang, #grpc, #kubernetes, #kubernetesoperator, #llm, #machinelearning, #mlops, #orchestrationengine, #production, #productiongrade, #python, #scale, #workflow
Scalable and flexible workflow orchestration platform that seamlessly unifies data, ML and analytics stacks.
Language:Go
Total stars: 4948
Stars trend:
22 May 2024
5pm ▏ +1
6pm +0
7pm +0
8pm ▏ +1
9pm ▏ +1
10pm +0
11pm +0
23 May 2024
12am █▉ +15
1am ███▏ +25
2am ███▍ +27
3am █▍ +11#go
#data, #dataanalysis, #datascience, #dataops, #declarative, #finetuning, #flyte, #golang, #grpc, #kubernetes, #kubernetesoperator, #llm, #machinelearning, #mlops, #orchestrationengine, #production, #productiongrade, #python, #scale, #workflow
amphi-ai/amphi-etl
Low-code ETL for structured and unstructured data. Generates Python code you can deploy anywhere.
Language:TypeScript
Total stars: 131
Stars trend:
#typescript
#data, #datapipelines, #etl, #ragpipeline, #structureddata, #unstructureddata
Low-code ETL for structured and unstructured data. Generates Python code you can deploy anywhere.
Language:TypeScript
Total stars: 131
Stars trend:
18 Jun 2024
6pm ▏ +1
7pm +0
8pm +0
9pm +0
10pm +0
11pm +0
19 Jun 2024
12am ▏ +1
1am █▋ +13
2am ███ +24
3am █▊ +14
4am ██▏ +17
5am █▊ +14#typescript
#data, #datapipelines, #etl, #ragpipeline, #structureddata, #unstructureddata
lakehq/sail
LakeSail's computation framework with a mission to unify stream processing, batch processing, and compute-intensive (AI) workloads.
Language:Rust
Total stars: 168
Stars trend:
#rust
#arrow, #bigdata, #data, #datafusion, #pyspark, #python, #rust, #spark, #sql
LakeSail's computation framework with a mission to unify stream processing, batch processing, and compute-intensive (AI) workloads.
Language:Rust
Total stars: 168
Stars trend:
10 Sep 2024
3am ▏ +1
4am +0
5am ▏ +1
6am +0
7am ▉ +7
8am █▏ +9
9am ▋ +5
10am █ +8
11am ██▏ +17
12pm █▉ +15
1pm █▏ +9
2pm █▍ +11#rust
#arrow, #bigdata, #data, #datafusion, #pyspark, #python, #rust, #spark, #sql
ucbepic/docetl
A system for complex LLM-powered document processing
Language:Python
Total stars: 482
Stars trend:
#python
#data, #datapipelines, #elt, #etl, #llm, #python, #workflow
A system for complex LLM-powered document processing
Language:Python
Total stars: 482
Stars trend:
28 Sep 2024
7am ▌ +4
8am █▏ +9
9am ▋ +5
10am ▎ +2
11am █▎ +10
12pm ▌ +4
1pm █▏ +9
2pm ▊ +6
3pm █▎ +10
4pm █▏ +9
5pm ▌ +4
6pm ▍ +3#python
#data, #datapipelines, #elt, #etl, #llm, #python, #workflow
Litlyx/litlyx
All-in-one Analytics Solution. Setup in 30 seconds. Display all your data on an AI-powered dashboard. Fully self-hostable and GDPR compliant.
Language:Vue
Total stars: 700
Stars trend:
#vue
#ai, #analytics, #angular, #charts, #data, #dataanalysis, #datavisualization, #javascript, #metrics, #nextjs, #nodejs, #nuxt, #opensource, #react, #statistics, #typescript, #vue, #website
All-in-one Analytics Solution. Setup in 30 seconds. Display all your data on an AI-powered dashboard. Fully self-hostable and GDPR compliant.
Language:Vue
Total stars: 700
Stars trend:
3 Nov 2024
9am ▉ +7
10am ██ +16
11am ██▎ +18
12pm █▉ +15
1pm ▋ +5
2pm █▉ +15#vue
#ai, #analytics, #angular, #charts, #data, #dataanalysis, #datavisualization, #javascript, #metrics, #nextjs, #nodejs, #nuxt, #opensource, #react, #statistics, #typescript, #vue, #website
shayonj/pg_flo
Stream, transform, and route PostgreSQL data in real-time.
Language:Go
Total stars: 127
Stars trend:
#go
#data, #database, #etl, #go, #golang, #logicalreplication, #postgres, #postgresql, #stream
Stream, transform, and route PostgreSQL data in real-time.
Language:Go
Total stars: 127
Stars trend:
3 Nov 2024
5pm ▍ +3
6pm ▎ +2
7pm ██▉ +23
8pm ██▌ +20
9pm █▋ +13
10pm █▉ +15#go
#data, #database, #etl, #go, #golang, #logicalreplication, #postgres, #postgresql, #stream