Do not know how to speedup your code? Just distribute!
Hi all!
I have created just-distribute package aimed mainly at those at the beginning of their Python journey, but it may be handy also for advanced users.
https://github.com/jakubgajski/just\_distribute
It is basically wrapping up popular libraries / approaches to speeding up code into one handy decorator \\@distribute.
I would appreciate any suggestions and feedback! Hope it will help someone :)
/r/Python
https://redd.it/1dwzx4j
Hi all!
I have created just-distribute package aimed mainly at those at the beginning of their Python journey, but it may be handy also for advanced users.
pip install just-distributehttps://github.com/jakubgajski/just\_distribute
It is basically wrapping up popular libraries / approaches to speeding up code into one handy decorator \\@distribute.
I would appreciate any suggestions and feedback! Hope it will help someone :)
/r/Python
https://redd.it/1dwzx4j
GitHub
GitHub - jakubgajski/just_distribute: Have a function doing stuff too long? Just distribute!
Have a function doing stuff too long? Just distribute! - jakubgajski/just_distribute
Ultimate SQL Learning Resource: Case Studies, Projects, and Platform Solutions in One Place!
Hi everyone !!
Check out Faizan's SQL Portfolio on GitHub! 🚀
This comprehensive resource includes:
Case Studies: Real-world scenarios from Danny Ma's 8 Week SQL Challenge.
Platform Solutions: SQL problems & solutions from 7 different platforms including DataLemur, Leetcode, Hackerrank, Stratascratch and more.
Projects: Detailed SQL projects with data analysis techniques.
Resources: List of compiled SQL resources from different channels like YT, Books, Tutorials etc.
and much more!!
Perfect for students and professionals to enhance their SQL skills through practical applications. Explore, learn, and improve your SQL expertise!
🔗 https://github.com/faizanxmulla/sql-portfolio
Thank you so much for considering! If you would like to connect, feel free to reach out to me on LinkedIn.
Happy learning!
/r/flask
https://redd.it/1dwq55m
Hi everyone !!
Check out Faizan's SQL Portfolio on GitHub! 🚀
This comprehensive resource includes:
Case Studies: Real-world scenarios from Danny Ma's 8 Week SQL Challenge.
Platform Solutions: SQL problems & solutions from 7 different platforms including DataLemur, Leetcode, Hackerrank, Stratascratch and more.
Projects: Detailed SQL projects with data analysis techniques.
Resources: List of compiled SQL resources from different channels like YT, Books, Tutorials etc.
and much more!!
Perfect for students and professionals to enhance their SQL skills through practical applications. Explore, learn, and improve your SQL expertise!
🔗 https://github.com/faizanxmulla/sql-portfolio
Thank you so much for considering! If you would like to connect, feel free to reach out to me on LinkedIn.
Happy learning!
/r/flask
https://redd.it/1dwq55m
GitHub
GitHub - faizanxmulla/sql-portfolio: Repository of SQL projects, case studies, platform solutions, and learning resources to enhance…
Repository of SQL projects, case studies, platform solutions, and learning resources to enhance SQL skills through practical applications. Includes content from DataLemur, LeetCode, HackerRank, and...
Flask authentication and clean architecture
I am a beginner with flask. I am receiving auth token from a flutter firebase setup using google sso login. How to get the data encrypted using secret key in flask app. I am actually looking for good coding practices (clean architecture) developer uses generally on flask apps.
/r/flask
https://redd.it/1dx3v9j
I am a beginner with flask. I am receiving auth token from a flutter firebase setup using google sso login. How to get the data encrypted using secret key in flask app. I am actually looking for good coding practices (clean architecture) developer uses generally on flask apps.
/r/flask
https://redd.it/1dx3v9j
Reddit
From the flask community on Reddit
Explore this post and more from the flask community
Does anyone know of an extension that will auto create a client class for all my endpoints?
Say I’m using DRF with a bunch of model viewsets. I have a swagger gen extension but is there one that will collect all those same endpoints and create a Python class or file containing all of them? Like using requests or urllib or something like that and have all the endpoints auto implemented for you? I’m sure it’s possible (and opinionated) just haven’t come across anything I’m aware of or could find on google.
/r/django
https://redd.it/1dx33uk
Say I’m using DRF with a bunch of model viewsets. I have a swagger gen extension but is there one that will collect all those same endpoints and create a Python class or file containing all of them? Like using requests or urllib or something like that and have all the endpoints auto implemented for you? I’m sure it’s possible (and opinionated) just haven’t come across anything I’m aware of or could find on google.
/r/django
https://redd.it/1dx33uk
Reddit
From the django community on Reddit
Explore this post and more from the django community
Django cache does not support hash(redis)
I am studying django and i just learned that django's build in cache framework DOES NOT support redis's hash.
Is it strange? for such a mature framework but its built in cache does not support in-memory database's hash? i asked chatgpt and they confirmed it.
I manually created my own function that uses the built in cache.set and put a json.dumps(dict) in it and in my redis GUI it still shows up under the string. I am worried that my own custom function may not be as efficient as the redis's hash since its really considered as a string. Is it advisable to put a dict as string or should i just opt out of the django cache and use a third party library to handle redis?
/r/django
https://redd.it/1dwthj6
I am studying django and i just learned that django's build in cache framework DOES NOT support redis's hash.
Is it strange? for such a mature framework but its built in cache does not support in-memory database's hash? i asked chatgpt and they confirmed it.
I manually created my own function that uses the built in cache.set and put a json.dumps(dict) in it and in my redis GUI it still shows up under the string. I am worried that my own custom function may not be as efficient as the redis's hash since its really considered as a string. Is it advisable to put a dict as string or should i just opt out of the django cache and use a third party library to handle redis?
/r/django
https://redd.it/1dwthj6
Reddit
From the django community on Reddit
Explore this post and more from the django community
Migration issue after Wagtail Update
I have updated Wagtail from 5.2.2 to 6.1.2 and it's telling me that I need to run
I have no idea how to fix this.
I updated Wagtail with
/r/django
https://redd.it/1dx9fia
I have updated Wagtail from 5.2.2 to 6.1.2 and it's telling me that I need to run
migrate. When I do I get the below error.django.db.utils.OperationalError: (3821, "Check constraint 'permission_or_permission_type_not_null' is not found in the table.")I have no idea how to fix this.
I updated Wagtail with
pip install -upgrade wagtail/r/django
https://redd.it/1dx9fia
Reddit
From the django community on Reddit
Explore this post and more from the django community
Experienced django dev looking for opportunities
Hello everyone,
As mentioned in the title, I’m a software developer with good experience in Django, used it professionally for 3 year and also used it before that during my college for some projects. I’ve also answered queries on this community whenever I could. Now I’m looking for new opportunities, I’ve used Django with react and have good understanding of both, I do not want to limit myself to a specific stack because I’m also eager to learn new technologies. That said if any of you are looking to hire someone with this skill set and experience then please reach out
/r/django
https://redd.it/1dwst1j
Hello everyone,
As mentioned in the title, I’m a software developer with good experience in Django, used it professionally for 3 year and also used it before that during my college for some projects. I’ve also answered queries on this community whenever I could. Now I’m looking for new opportunities, I’ve used Django with react and have good understanding of both, I do not want to limit myself to a specific stack because I’m also eager to learn new technologies. That said if any of you are looking to hire someone with this skill set and experience then please reach out
/r/django
https://redd.it/1dwst1j
Reddit
From the django community on Reddit
Explore this post and more from the django community
Sunday Daily Thread: What's everyone working on this week?
# Weekly Thread: What's Everyone Working On This Week? 🛠️
Hello /r/Python! It's time to share what you've been working on! Whether it's a work-in-progress, a completed masterpiece, or just a rough idea, let us know what you're up to!
## How it Works:
1. Show & Tell: Share your current projects, completed works, or future ideas.
2. Discuss: Get feedback, find collaborators, or just chat about your project.
3. Inspire: Your project might inspire someone else, just as you might get inspired here.
## Guidelines:
Feel free to include as many details as you'd like. Code snippets, screenshots, and links are all welcome.
Whether it's your job, your hobby, or your passion project, all Python-related work is welcome here.
## Example Shares:
1. Machine Learning Model: Working on a ML model to predict stock prices. Just cracked a 90% accuracy rate!
2. Web Scraping: Built a script to scrape and analyze news articles. It's helped me understand media bias better.
3. Automation: Automated my home lighting with Python and Raspberry Pi. My life has never been easier!
Let's build and grow together! Share your journey and learn from others. Happy coding! 🌟
/r/Python
https://redd.it/1dx3ae2
# Weekly Thread: What's Everyone Working On This Week? 🛠️
Hello /r/Python! It's time to share what you've been working on! Whether it's a work-in-progress, a completed masterpiece, or just a rough idea, let us know what you're up to!
## How it Works:
1. Show & Tell: Share your current projects, completed works, or future ideas.
2. Discuss: Get feedback, find collaborators, or just chat about your project.
3. Inspire: Your project might inspire someone else, just as you might get inspired here.
## Guidelines:
Feel free to include as many details as you'd like. Code snippets, screenshots, and links are all welcome.
Whether it's your job, your hobby, or your passion project, all Python-related work is welcome here.
## Example Shares:
1. Machine Learning Model: Working on a ML model to predict stock prices. Just cracked a 90% accuracy rate!
2. Web Scraping: Built a script to scrape and analyze news articles. It's helped me understand media bias better.
3. Automation: Automated my home lighting with Python and Raspberry Pi. My life has never been easier!
Let's build and grow together! Share your journey and learn from others. Happy coding! 🌟
/r/Python
https://redd.it/1dx3ae2
Reddit
From the Python community on Reddit
Explore this post and more from the Python community
My jupyter notebook doesn't have the "In" (function?)?
I have a 64-bit PC with Windows 10. I installed the latest Anaconda download. And when I try running Jupyter notebook, I don't have the "In" function to the left of the cell that allows me to execute code
https://preview.redd.it/b6qcqhu2b1bd1.png?width=1254&format=png&auto=webp&s=29369cd401edcf9a672cc3ecdbfa2b6eb605f686
Took this screenshot from a video
https://preview.redd.it/kaglsch5b1bd1.png?width=1025&format=png&auto=webp&s=2baf582197f3858795fcda5e28d822bc2172ff3a
I tried asking ChatGPT why my Jupyter notebook doesn't have the "In" function, but none of its suggestions are fixing the problem
/r/IPython
https://redd.it/1dx9c6w
I have a 64-bit PC with Windows 10. I installed the latest Anaconda download. And when I try running Jupyter notebook, I don't have the "In" function to the left of the cell that allows me to execute code
https://preview.redd.it/b6qcqhu2b1bd1.png?width=1254&format=png&auto=webp&s=29369cd401edcf9a672cc3ecdbfa2b6eb605f686
Took this screenshot from a video
https://preview.redd.it/kaglsch5b1bd1.png?width=1025&format=png&auto=webp&s=2baf582197f3858795fcda5e28d822bc2172ff3a
I tried asking ChatGPT why my Jupyter notebook doesn't have the "In" function, but none of its suggestions are fixing the problem
/r/IPython
https://redd.it/1dx9c6w
D Self-Promotion Thread
Please post your personal projects, startups, product placements, collaboration needs, blogs etc.
Please mention the payment and pricing requirements for products and services.
Please do not post link shorteners, link aggregator websites , or auto-subscribe links.
--
Any abuse of trust will lead to bans.
Encourage others who create new posts for questions to post here instead!
Thread will stay alive until next one so keep posting after the date in the title.
--
Meta: This is an experiment. If the community doesnt like this, we will cancel it. This is to encourage those in the community to promote their work by not spamming the main threads.
/r/MachineLearning
https://redd.it/1dx5tpo
Please post your personal projects, startups, product placements, collaboration needs, blogs etc.
Please mention the payment and pricing requirements for products and services.
Please do not post link shorteners, link aggregator websites , or auto-subscribe links.
--
Any abuse of trust will lead to bans.
Encourage others who create new posts for questions to post here instead!
Thread will stay alive until next one so keep posting after the date in the title.
--
Meta: This is an experiment. If the community doesnt like this, we will cancel it. This is to encourage those in the community to promote their work by not spamming the main threads.
/r/MachineLearning
https://redd.it/1dx5tpo
Reddit
From the MachineLearning community on Reddit
Explore this post and more from the MachineLearning community
Flask, Django, or FastAPI?
From your experiences as a developer, which of these 3 frameworks would you guys recommend learning for the backend? What are some of the pro and con of each framework that you've notice? If you were to start over again, which framework will you choose to learn first?
/r/Python
https://redd.it/1dxcdiy
From your experiences as a developer, which of these 3 frameworks would you guys recommend learning for the backend? What are some of the pro and con of each framework that you've notice? If you were to start over again, which framework will you choose to learn first?
/r/Python
https://redd.it/1dxcdiy
Reddit
From the Python community on Reddit
Explore this post and more from the Python community
How to deploy selenium based flask app to VPS with gunicorn and nginx
Hi, I have a basic web scraping price comparison app. I wanted to deploy it on the oracle free tier for cloud hosting. I’m quite new to this so I have a few questions.
How can I set up multiple workers to service multiple requests simultaneously? What would I need for that?
What is the procedure for deploying this app to oracle cloud? I do have a domain but I don’t know how to make that work.
I’ve gone through way too many YouTube videos and it’s all quite confusing. I’d really appreciate any help or video links.
/r/flask
https://redd.it/1dxf932
Hi, I have a basic web scraping price comparison app. I wanted to deploy it on the oracle free tier for cloud hosting. I’m quite new to this so I have a few questions.
How can I set up multiple workers to service multiple requests simultaneously? What would I need for that?
What is the procedure for deploying this app to oracle cloud? I do have a domain but I don’t know how to make that work.
I’ve gone through way too many YouTube videos and it’s all quite confusing. I’d really appreciate any help or video links.
/r/flask
https://redd.it/1dxf932
Reddit
From the flask community on Reddit
Explore this post and more from the flask community
Just finished my first week learning Python!!
Hey guys! My name is Christian (22M). Ever since my sophomore year in high school, when I began to learn more and more about computers by taking a simple html class for my cs credit, I have wanted to do more in this field. However, between bouncing around from many different ideas of what I wanted to do for the rest of my life, I ended up landing myself graduating from college with an associate in applied science for Engineering Systems Technology.
Since I turned 18, I have continuously worked in a field with utility scale electrical equipment, focusing on the software/programming side for the components that control the flow/state of that energy. I.E. PLC, STM32, JTAG, DSP, Custom Mod-bus. I have a decent amount of experience in several different applications but could never say that I truly KNEW what to do beyond basic debugging and integration.
I have recently started the 100 Days of Code challenge course on Udemy which was one of the highest rated courses for learning Python. I seem to be picking it up pretty quick but and truly starting to wonder if even after completing this course will I actually be able to land a
/r/Python
https://redd.it/1dxkr3f
Hey guys! My name is Christian (22M). Ever since my sophomore year in high school, when I began to learn more and more about computers by taking a simple html class for my cs credit, I have wanted to do more in this field. However, between bouncing around from many different ideas of what I wanted to do for the rest of my life, I ended up landing myself graduating from college with an associate in applied science for Engineering Systems Technology.
Since I turned 18, I have continuously worked in a field with utility scale electrical equipment, focusing on the software/programming side for the components that control the flow/state of that energy. I.E. PLC, STM32, JTAG, DSP, Custom Mod-bus. I have a decent amount of experience in several different applications but could never say that I truly KNEW what to do beyond basic debugging and integration.
I have recently started the 100 Days of Code challenge course on Udemy which was one of the highest rated courses for learning Python. I seem to be picking it up pretty quick but and truly starting to wonder if even after completing this course will I actually be able to land a
/r/Python
https://redd.it/1dxkr3f
Reddit
From the Python community on Reddit: Just finished my first week learning Python!!
Posted by PitifulHoneydew7302 - 9 votes and 6 comments
How much data validation is healthy?
How much manual validation do you think is healthy in Python code?
I almost never do validation. I mean, when reading data from files or via an API, or from anywhere that I don’t control with my code, I would generally do validation via Pydantic or Pandera, depending on the type of data. But in all other cases, I usually supply type hints and I write functions in complete trust that the things that actually get passed live up to what they claim to be, especially because my point of view is that MyPy or Pyright should be part of a modern CI pipeline (and even if not, people get IDE support when writing calls). Sometimes you have to use # type: ignore, but then the onus is on the callers’ side to know what they’re doing. I would make some exception perhaps for certain libraries like pandas that have poor type support, in those cases it probably makes sense to be a little more defensive.
But I’ve seen code from colleagues that basically validates everything, so every function starts with checks for None or isinstance, and ValueErrors with nice messages are raised if conditions are violated. I really don’t like this
/r/Python
https://redd.it/1dxmp46
How much manual validation do you think is healthy in Python code?
I almost never do validation. I mean, when reading data from files or via an API, or from anywhere that I don’t control with my code, I would generally do validation via Pydantic or Pandera, depending on the type of data. But in all other cases, I usually supply type hints and I write functions in complete trust that the things that actually get passed live up to what they claim to be, especially because my point of view is that MyPy or Pyright should be part of a modern CI pipeline (and even if not, people get IDE support when writing calls). Sometimes you have to use # type: ignore, but then the onus is on the callers’ side to know what they’re doing. I would make some exception perhaps for certain libraries like pandas that have poor type support, in those cases it probably makes sense to be a little more defensive.
But I’ve seen code from colleagues that basically validates everything, so every function starts with checks for None or isinstance, and ValueErrors with nice messages are raised if conditions are violated. I really don’t like this
/r/Python
https://redd.it/1dxmp46
Reddit
From the Python community on Reddit
Explore this post and more from the Python community
Django book recommendations
Hello, I'm a new developer. Currently really proficient with the flutter framework and I have a rather solid understanding of OOP. I love to read on frameworks and coding I'm general as I am able to get the knowledge in more depth than I would in a video or tutorial.
Do any of you know an excellent Django book for me to get to know the framework.
P.S I prefer human recommendations than Google search results when it comes to book recommendations. Because with good seo and marketing even a not so good book can be a first result.
/r/django
https://redd.it/1dxk70j
Hello, I'm a new developer. Currently really proficient with the flutter framework and I have a rather solid understanding of OOP. I love to read on frameworks and coding I'm general as I am able to get the knowledge in more depth than I would in a video or tutorial.
Do any of you know an excellent Django book for me to get to know the framework.
P.S I prefer human recommendations than Google search results when it comes to book recommendations. Because with good seo and marketing even a not so good book can be a first result.
/r/django
https://redd.it/1dxk70j
Reddit
From the django community on Reddit
Explore this post and more from the django community
Should I go with Threading or Celery?
I have a Flask registration route that sends an email to the user after adding it the database.
Including the email sending function in the route takes time and therefore the response is delayed. I'd much rather send a "An email will be sent soon." rather than a "Welcome" message that takes a lot of time to show.
I know that I can do this with Threading but I've recently come across Celery and I don't know if it's an overkill or not, especially in this case.
Thank you.
/r/flask
https://redd.it/1dxlsao
I have a Flask registration route that sends an email to the user after adding it the database.
Including the email sending function in the route takes time and therefore the response is delayed. I'd much rather send a "An email will be sent soon." rather than a "Welcome" message that takes a lot of time to show.
I know that I can do this with Threading but I've recently come across Celery and I don't know if it's an overkill or not, especially in this case.
Thank you.
/r/flask
https://redd.it/1dxlsao
Reddit
From the flask community on Reddit
Explore this post and more from the flask community
Embedding a diffefent website into flask
Hi,
Im using a raspberrypi to host 2 diffefent Websites. The first (main) website is hosted locally with flask. The second Uses apache.
If i embed the second website as an iFrame i can use both websites perfectly fine. The Main Website works without issues and the embedded one can be used inside of it. Even when using the dev options in my browser to simulate the mobile view it works.
However if i use an actual mobile device i cannot click on Buttons in the embedded website.
Both run on different ports.
Does anyone know why it doesnt work on the Phone even tho it workes on the Desktop?
Im gratefull for any help since i have no idea what could cause this.
/r/flask
https://redd.it/1dxnii3
Hi,
Im using a raspberrypi to host 2 diffefent Websites. The first (main) website is hosted locally with flask. The second Uses apache.
If i embed the second website as an iFrame i can use both websites perfectly fine. The Main Website works without issues and the embedded one can be used inside of it. Even when using the dev options in my browser to simulate the mobile view it works.
However if i use an actual mobile device i cannot click on Buttons in the embedded website.
Both run on different ports.
Does anyone know why it doesnt work on the Phone even tho it workes on the Desktop?
Im gratefull for any help since i have no idea what could cause this.
/r/flask
https://redd.it/1dxnii3
Reddit
From the flask community on Reddit
Explore this post and more from the flask community
Monday Daily Thread: Project ideas!
# Weekly Thread: Project Ideas 💡
Welcome to our weekly Project Ideas thread! Whether you're a newbie looking for a first project or an expert seeking a new challenge, this is the place for you.
## How it Works:
1. **Suggest a Project**: Comment your project idea—be it beginner-friendly or advanced.
2. **Build & Share**: If you complete a project, reply to the original comment, share your experience, and attach your source code.
3. **Explore**: Looking for ideas? Check out Al Sweigart's ["The Big Book of Small Python Projects"](https://www.amazon.com/Big-Book-Small-Python-Programming/dp/1718501242) for inspiration.
## Guidelines:
* Clearly state the difficulty level.
* Provide a brief description and, if possible, outline the tech stack.
* Feel free to link to tutorials or resources that might help.
# Example Submissions:
## Project Idea: Chatbot
**Difficulty**: Intermediate
**Tech Stack**: Python, NLP, Flask/FastAPI/Litestar
**Description**: Create a chatbot that can answer FAQs for a website.
**Resources**: [Building a Chatbot with Python](https://www.youtube.com/watch?v=a37BL0stIuM)
# Project Idea: Weather Dashboard
**Difficulty**: Beginner
**Tech Stack**: HTML, CSS, JavaScript, API
**Description**: Build a dashboard that displays real-time weather information using a weather API.
**Resources**: [Weather API Tutorial](https://www.youtube.com/watch?v=9P5MY_2i7K8)
## Project Idea: File Organizer
**Difficulty**: Beginner
**Tech Stack**: Python, File I/O
**Description**: Create a script that organizes files in a directory into sub-folders based on file type.
**Resources**: [Automate the Boring Stuff: Organizing Files](https://automatetheboringstuff.com/2e/chapter9/)
Let's help each other grow. Happy
/r/Python
https://redd.it/1dxuucj
# Weekly Thread: Project Ideas 💡
Welcome to our weekly Project Ideas thread! Whether you're a newbie looking for a first project or an expert seeking a new challenge, this is the place for you.
## How it Works:
1. **Suggest a Project**: Comment your project idea—be it beginner-friendly or advanced.
2. **Build & Share**: If you complete a project, reply to the original comment, share your experience, and attach your source code.
3. **Explore**: Looking for ideas? Check out Al Sweigart's ["The Big Book of Small Python Projects"](https://www.amazon.com/Big-Book-Small-Python-Programming/dp/1718501242) for inspiration.
## Guidelines:
* Clearly state the difficulty level.
* Provide a brief description and, if possible, outline the tech stack.
* Feel free to link to tutorials or resources that might help.
# Example Submissions:
## Project Idea: Chatbot
**Difficulty**: Intermediate
**Tech Stack**: Python, NLP, Flask/FastAPI/Litestar
**Description**: Create a chatbot that can answer FAQs for a website.
**Resources**: [Building a Chatbot with Python](https://www.youtube.com/watch?v=a37BL0stIuM)
# Project Idea: Weather Dashboard
**Difficulty**: Beginner
**Tech Stack**: HTML, CSS, JavaScript, API
**Description**: Build a dashboard that displays real-time weather information using a weather API.
**Resources**: [Weather API Tutorial](https://www.youtube.com/watch?v=9P5MY_2i7K8)
## Project Idea: File Organizer
**Difficulty**: Beginner
**Tech Stack**: Python, File I/O
**Description**: Create a script that organizes files in a directory into sub-folders based on file type.
**Resources**: [Automate the Boring Stuff: Organizing Files](https://automatetheboringstuff.com/2e/chapter9/)
Let's help each other grow. Happy
/r/Python
https://redd.it/1dxuucj
YouTube
Build & Integrate your own custom chatbot to a website (Python & JavaScript)
In this fun project you learn how to build a custom chatbot in Python and then integrate this to a website using Flask and JavaScript.
Starter Files: https://github.com/patrickloeber/chatbot-deployment
Get my Free NumPy Handbook: https://www.python-engi…
Starter Files: https://github.com/patrickloeber/chatbot-deployment
Get my Free NumPy Handbook: https://www.python-engi…
This media is not supported in your browser
VIEW IN TELEGRAM
[R] Open-TeleVision: Teleoperation with Immersive Active Visual Feedback
/r/MachineLearning
https://redd.it/1dxtsiq
/r/MachineLearning
https://redd.it/1dxtsiq
Self-hosted webscraper
I have created a self-hosted webscraper, "Scraperr".
https://github.com/jaypyles/Scraperr
# What my Project does?
Currently you can:
- Scrape sites specifying elements using xpath
- View and download job results as csv
- Rerun scrape jobs
- Login to organize jobs
- Bulk download/delete jobs
# Target Audience
Users looking for an easy way to collect data from sites using a webscraper.
# Comparisons
The backend of the app is developed fully in Python with basedpyright helping me with typesafety, using FastAPI as my HTTP API library. I mostly see users make GUI based webscrapers, and compile them into a launchable exe or a .py script, but this is developed with NextJS as the frontend to be used as a web application and/or deployed on cloud/self-hosted, etc.
Feel free to leave suggestions, tips, etc.
/r/Python
https://redd.it/1dxw3r8
I have created a self-hosted webscraper, "Scraperr".
https://github.com/jaypyles/Scraperr
# What my Project does?
Currently you can:
- Scrape sites specifying elements using xpath
- View and download job results as csv
- Rerun scrape jobs
- Login to organize jobs
- Bulk download/delete jobs
# Target Audience
Users looking for an easy way to collect data from sites using a webscraper.
# Comparisons
The backend of the app is developed fully in Python with basedpyright helping me with typesafety, using FastAPI as my HTTP API library. I mostly see users make GUI based webscrapers, and compile them into a launchable exe or a .py script, but this is developed with NextJS as the frontend to be used as a web application and/or deployed on cloud/self-hosted, etc.
Feel free to leave suggestions, tips, etc.
/r/Python
https://redd.it/1dxw3r8
GitHub
GitHub - jaypyles/Scraperr: Self-hosted webscraper.
Self-hosted webscraper. Contribute to jaypyles/Scraperr development by creating an account on GitHub.