I wrote some flask code. Would anyone be willing to review the code I wrote. Everything works I just want to get someone opinion of the code. If yes I will private message you the code.
I wrote some flask code. Would anyone be willing to review the code I wrote. Everything works I just want to get someone opinion of the code. If yes I will private message you the code.
/r/flask
https://redd.it/1f3m6g7
I wrote some flask code. Would anyone be willing to review the code I wrote. Everything works I just want to get someone opinion of the code. If yes I will private message you the code.
/r/flask
https://redd.it/1f3m6g7
Reddit
From the flask community on Reddit
Explore this post and more from the flask community
Help designing model for including sem/year
I'm creating models to store questions and syllabus of different courses.
eg. program: Master of Fine Arts (MFA), courses: Sculpture, Visual arts
This is what I have in mind so far:
#django and postgresql
#from django.db import models
class Program(models.Model):
programid = models.IntegerField(unique=True)
programcode = models.CharField(maxlength=100)
programname = models.CharField(maxlength=100)
class Course(models.Model):
courseid = models.IntegerField(unique=True)
coursecode = models.CharField(maxlength=100)
coursename = models.CharField(maxlength=100)
coursecredit = models.IntegerField()
courseicon = models.CharField(maxlength=50)
program = models.ForeignKey(
Program, ondelete=models.CASCADE, relatedname="courses"
)
class Syllabus(models.Model):
course = models.ForeignKey(Course, ondelete=models.CASCADE, relatedname='syllabus')
topic = models.CharField(maxlength=100)
content = models.TextField()
hours = models.IntegerField()
/r/django
https://redd.it/1f3robj
I'm creating models to store questions and syllabus of different courses.
eg. program: Master of Fine Arts (MFA), courses: Sculpture, Visual arts
This is what I have in mind so far:
#django and postgresql
#from django.db import models
class Program(models.Model):
programid = models.IntegerField(unique=True)
programcode = models.CharField(maxlength=100)
programname = models.CharField(maxlength=100)
class Course(models.Model):
courseid = models.IntegerField(unique=True)
coursecode = models.CharField(maxlength=100)
coursename = models.CharField(maxlength=100)
coursecredit = models.IntegerField()
courseicon = models.CharField(maxlength=50)
program = models.ForeignKey(
Program, ondelete=models.CASCADE, relatedname="courses"
)
class Syllabus(models.Model):
course = models.ForeignKey(Course, ondelete=models.CASCADE, relatedname='syllabus')
topic = models.CharField(maxlength=100)
content = models.TextField()
hours = models.IntegerField()
/r/django
https://redd.it/1f3robj
Reddit
From the django community on Reddit
Explore this post and more from the django community
2023 Python Developers Survey Results
# 2023 Python Developers Survey
Results are in for the official Python Developers Survey, conducted in partnership with JetBrains!
The survey is a joint initiative between the Python Software Foundation and JetBrains.
Read more about it here.
/r/Python
https://redd.it/1f43acl
# 2023 Python Developers Survey
Results are in for the official Python Developers Survey, conducted in partnership with JetBrains!
The survey is a joint initiative between the Python Software Foundation and JetBrains.
Read more about it here.
/r/Python
https://redd.it/1f43acl
JetBrains: Developer Tools for Professionals and Teams
Python Developers Survey 2023 Results
Official Python Developers Survey 2023 Results by Python Software Foundation and JetBrains: more than 25k responses from almost 200 countries.
D Post any bginner questions to r/MLQuestions!
I have recently inherited the subreddit r/MLQuestions, as the other mods had been innactive for 10 months and 4 years respectively. I have been sprucing up the sub, adding flairs, rules, etc, and I am trying to increase engagement and make it more useful for those who want to ask questions. Basically, stackoverflow but dedicated to beginn\r questions about ML. So if any of you have questions that your are too embarrased to ask here, ask at r/MLQuestions! I will also be introducing a system similar to r/changemyview, where each question someone answers, they get an increment to their user flair that shows how many questions they have answered!
BTW the mods gave me permission to post this, so thank you guys for this, very cool.
/r/MachineLearning
https://redd.it/1f3yfjg
I have recently inherited the subreddit r/MLQuestions, as the other mods had been innactive for 10 months and 4 years respectively. I have been sprucing up the sub, adding flairs, rules, etc, and I am trying to increase engagement and make it more useful for those who want to ask questions. Basically, stackoverflow but dedicated to beginn\r questions about ML. So if any of you have questions that your are too embarrased to ask here, ask at r/MLQuestions! I will also be introducing a system similar to r/changemyview, where each question someone answers, they get an increment to their user flair that shows how many questions they have answered!
BTW the mods gave me permission to post this, so thank you guys for this, very cool.
/r/MachineLearning
https://redd.it/1f3yfjg
Reddit
From the MachineLearning community on Reddit
Explore this post and more from the MachineLearning community
How to add alt to img in django-summernote?
As title says. I am add blog post via admin panel. I have integrated django-summernote. I am not able to find any way to add alt tag to image uploaded inside post.
I followed this to integration https://djangocentral.com/integrating-summernote-in-django/
Thanks
/r/django
https://redd.it/1f48kyr
As title says. I am add blog post via admin panel. I have integrated django-summernote. I am not able to find any way to add alt tag to image uploaded inside post.
I followed this to integration https://djangocentral.com/integrating-summernote-in-django/
Thanks
/r/django
https://redd.it/1f48kyr
Djangocentral
Integrating Summernote WYSIWYG Editor in Django
A WYSIWYG (pronounced "wiz-ee-wig") editor or program is one that allows a developer to see what the result will look like while the interface or document is
Fileuploads blocking Workers
Hello everybody,
I encountered a problem which got pretty big for my App now. Sometimes the app becomes unresponsive for a long time and will answer the request after like 50 seconds.
I think it is because a lot of users are uploading images and this blocks my gunicorn workers. Users might have a bad internet connection for example. Like this when 12 images are uploaded with bad connection and I only have 10 workers, this will cause a problem. (I also use Azure Blob Storage as my file storage)
How would you handle that?
Thank you in advance for your advice :)
/r/django
https://redd.it/1f487e9
Hello everybody,
I encountered a problem which got pretty big for my App now. Sometimes the app becomes unresponsive for a long time and will answer the request after like 50 seconds.
I think it is because a lot of users are uploading images and this blocks my gunicorn workers. Users might have a bad internet connection for example. Like this when 12 images are uploaded with bad connection and I only have 10 workers, this will cause a problem. (I also use Azure Blob Storage as my file storage)
How would you handle that?
Thank you in advance for your advice :)
/r/django
https://redd.it/1f487e9
Reddit
From the django community on Reddit
Explore this post and more from the django community
Created CLI that writes your semantic commit messages in git and more.
I've created CLI, a tool that generates semantic commit messages in Git
Here's a breakdown:
What My Project Does Penify CLI is a command-line tool that:
1. Automatically generates semantic commit messages based on your staged changes.
2. Generates documentation for specified files or folders.
3. Hooks: If you wish to automate documentation generation
Key features:
`penify-cli commit`: Commits code with an auto-generated semantic message for staged files.
Installation:
Target Audience Penify CLI is aimed at developers who want to:
Maintain consistent, meaningful commit messages without the mental overhead.
Quickly generate documentation for their codebase. It's suitable for both personal projects and professional development environments where consistent commit practices are valued.
Comparison Github-Copilot, aicommit:
Penify CLI generates semantic commit messages automatically, reducing manual input. None does.
It integrates documentation generation, combining two common developer tasks in one tool.
Note: Currently requires signup at Penify (we're working on Ollama integration for local use).
Check it out:
PyPI: [https://pypi.org/project/penify-cli/](https://pypi.org/project/penify-cli/)
GitHub: https://github.com/SingularityX-ai/penify-cli
I'd love to hear your thoughts and feedback!
/r/IPython
https://redd.it/1f4b888
I've created CLI, a tool that generates semantic commit messages in Git
Here's a breakdown:
What My Project Does Penify CLI is a command-line tool that:
1. Automatically generates semantic commit messages based on your staged changes.
2. Generates documentation for specified files or folders.
3. Hooks: If you wish to automate documentation generation
Key features:
`penify-cli commit`: Commits code with an auto-generated semantic message for staged files.
penify-cli doc-gen: Generates documentation for specified files/folders.Installation:
pip install penify-cliTarget Audience Penify CLI is aimed at developers who want to:
Maintain consistent, meaningful commit messages without the mental overhead.
Quickly generate documentation for their codebase. It's suitable for both personal projects and professional development environments where consistent commit practices are valued.
Comparison Github-Copilot, aicommit:
Penify CLI generates semantic commit messages automatically, reducing manual input. None does.
It integrates documentation generation, combining two common developer tasks in one tool.
Note: Currently requires signup at Penify (we're working on Ollama integration for local use).
Check it out:
PyPI: [https://pypi.org/project/penify-cli/](https://pypi.org/project/penify-cli/)
GitHub: https://github.com/SingularityX-ai/penify-cli
I'd love to hear your thoughts and feedback!
/r/IPython
https://redd.it/1f4b888
PyPI
penify-cli
A penify cli tool to generate Documentation, Commit-summary and Hooks to automate git workflows.
Friday Daily Thread: r/Python Meta and Free-Talk Fridays
# Weekly Thread: Meta Discussions and Free Talk Friday 🎙️
Welcome to Free Talk Friday on /r/Python! This is the place to discuss the r/Python community (meta discussions), Python news, projects, or anything else Python-related!
## How it Works:
1. Open Mic: Share your thoughts, questions, or anything you'd like related to Python or the community.
2. Community Pulse: Discuss what you feel is working well or what could be improved in the /r/python community.
3. News & Updates: Keep up-to-date with the latest in Python and share any news you find interesting.
## Guidelines:
All topics should be related to Python or the /r/python community.
Be respectful and follow Reddit's Code of Conduct.
## Example Topics:
1. New Python Release: What do you think about the new features in Python 3.11?
2. Community Events: Any Python meetups or webinars coming up?
3. Learning Resources: Found a great Python tutorial? Share it here!
4. Job Market: How has Python impacted your career?
5. Hot Takes: Got a controversial Python opinion? Let's hear it!
6. Community Ideas: Something you'd like to see us do? tell us.
Let's keep the conversation going. Happy discussing! 🌟
/r/Python
https://redd.it/1f4hgd2
# Weekly Thread: Meta Discussions and Free Talk Friday 🎙️
Welcome to Free Talk Friday on /r/Python! This is the place to discuss the r/Python community (meta discussions), Python news, projects, or anything else Python-related!
## How it Works:
1. Open Mic: Share your thoughts, questions, or anything you'd like related to Python or the community.
2. Community Pulse: Discuss what you feel is working well or what could be improved in the /r/python community.
3. News & Updates: Keep up-to-date with the latest in Python and share any news you find interesting.
## Guidelines:
All topics should be related to Python or the /r/python community.
Be respectful and follow Reddit's Code of Conduct.
## Example Topics:
1. New Python Release: What do you think about the new features in Python 3.11?
2. Community Events: Any Python meetups or webinars coming up?
3. Learning Resources: Found a great Python tutorial? Share it here!
4. Job Market: How has Python impacted your career?
5. Hot Takes: Got a controversial Python opinion? Let's hear it!
6. Community Ideas: Something you'd like to see us do? tell us.
Let's keep the conversation going. Happy discussing! 🌟
/r/Python
https://redd.it/1f4hgd2
Redditinc
Reddit Rules
Reddit Rules - Reddit
Looking for freelance Django/Python Dev, how much should I pay?
Hello hello,
Not a Django dev myself but need someone with Python, Django, and API / Rest experience. I'd say probably mid-level for about \~12 hours a month, only to go up from there. Any idea on what a good hourly rate for such a freelancer would be?
/r/django
https://redd.it/1f4dkin
Hello hello,
Not a Django dev myself but need someone with Python, Django, and API / Rest experience. I'd say probably mid-level for about \~12 hours a month, only to go up from there. Any idea on what a good hourly rate for such a freelancer would be?
/r/django
https://redd.it/1f4dkin
Reddit
From the django community on Reddit
Explore this post and more from the django community
How to properly record and send audio data from react to flask backend
I want to send audio data from react in a interval of approx. 10sec to my flask backend. Here is my code, it is working but music format is not write.
react code snippet
useEffect(() => {
socket.connect();
function sendData(data) {
var form = new FormData();
form.append("file", data, "data.mp3");
form.append("title", "data.mp3");
axios
.post("http://127.0.0.1:5000/save-record", form, { headers })
.then((response) => {
console.log(response.data);
})
.catch((error) => {
console.error(error);
/r/flask
https://redd.it/1f4978m
I want to send audio data from react in a interval of approx. 10sec to my flask backend. Here is my code, it is working but music format is not write.
react code snippet
useEffect(() => {
socket.connect();
function sendData(data) {
var form = new FormData();
form.append("file", data, "data.mp3");
form.append("title", "data.mp3");
axios
.post("http://127.0.0.1:5000/save-record", form, { headers })
.then((response) => {
console.log(response.data);
})
.catch((error) => {
console.error(error);
/r/flask
https://redd.it/1f4978m
Reddit
From the flask community on Reddit: How to properly record and send audio data from react to flask backend
Explore this post and more from the flask community
How to Build a Line Graph in Matplotlib | Python Data Visualization Tuto...
https://youtube.com/watch?v=tWBoDCnj5Ck&si=mE3sEVGy4Ki0Jurm
/r/IPython
https://redd.it/1f4nd6s
https://youtube.com/watch?v=tWBoDCnj5Ck&si=mE3sEVGy4Ki0Jurm
/r/IPython
https://redd.it/1f4nd6s
YouTube
How to Build a Line Graph in Matplotlib | Python Data Visualization Tutorial 📊
📊 How to Build a Line Graph in Matplotlib | Python Data Visualization Tutorial 📊#datascience #dataanalysis #datavisualization
In this video, you'll learn how to build a line graph in Matplotlib, a powerful Python library for data visualization. This Matplotlib…
In this video, you'll learn how to build a line graph in Matplotlib, a powerful Python library for data visualization. This Matplotlib…
Question on Macros vs Partial Templates
Hi,
Question on using macros vs partial templates.
Is there a preference or difference between the two? It seems like with the latest jinja updates, we can just pass variables to the partial template as well.
{% extends "home/home_base.html" %}
{% from "home/macros/nav_bar_macros.html" import nav_bar%}
{% block content %}
<div class="h-full">
<nav id="nav-bar" class="flex p-7 justify-between items-center">
<img src="{{ url_for('static', filename='images/logo.svg') }}">
<div>
{{ nav_bar(current_page)}}
</div>
</nav>
<div id="main-container" class="w-10/12 mx-auto mb-12">
{% include 'home/marketplace/partials/_recommended.html' with context %}
{% include 'home/marketplace/partials/_explore.html' with context %}
</div>
</div>
{% endblock %}
Per the code block above, i am using a macro for my dynamic nav bar, and also using partial templates. Both seem to do the same thing and my server can return a macro (via get_template_attribute) or just
/r/flask
https://redd.it/1f42zkf
Hi,
Question on using macros vs partial templates.
Is there a preference or difference between the two? It seems like with the latest jinja updates, we can just pass variables to the partial template as well.
{% extends "home/home_base.html" %}
{% from "home/macros/nav_bar_macros.html" import nav_bar%}
{% block content %}
<div class="h-full">
<nav id="nav-bar" class="flex p-7 justify-between items-center">
<img src="{{ url_for('static', filename='images/logo.svg') }}">
<div>
{{ nav_bar(current_page)}}
</div>
</nav>
<div id="main-container" class="w-10/12 mx-auto mb-12">
{% include 'home/marketplace/partials/_recommended.html' with context %}
{% include 'home/marketplace/partials/_explore.html' with context %}
</div>
</div>
{% endblock %}
Per the code block above, i am using a macro for my dynamic nav bar, and also using partial templates. Both seem to do the same thing and my server can return a macro (via get_template_attribute) or just
/r/flask
https://redd.it/1f42zkf
Reddit
From the flask community on Reddit
Explore this post and more from the flask community
Django API Manager
Hey guys,
I'm working on a platform that connects to a large amount of different APIs to extract different types of data. Instead of managing all those API calls from the code side. I want to manage them in the Django admin interface where I can add a button to "Add a new API connection" where I will provide details etc.. etc.. you know the drill.
Is there anything out there I could repurpose for this?
Thanks in advance!
/r/django
https://redd.it/1f4nl8p
Hey guys,
I'm working on a platform that connects to a large amount of different APIs to extract different types of data. Instead of managing all those API calls from the code side. I want to manage them in the Django admin interface where I can add a button to "Add a new API connection" where I will provide details etc.. etc.. you know the drill.
Is there anything out there I could repurpose for this?
Thanks in advance!
/r/django
https://redd.it/1f4nl8p
Reddit
From the django community on Reddit
Explore this post and more from the django community
Battleship TUI: a terminal-based multiplayer game
# What My Project Does
The good old Battleship reinvented as a TUI (Text User Interface) application. Basically, you can play Battleship in your terminal. More than that, you can play via the Internet! You can also track your performance (like the shooting accuracy and the win/loss rate) and customize the UI.
Here’s a screenshot of the game screen.
# Target Audience
Anyone who’s familiar with the terminal and has Python installed (or curious enough to try it out).
# Comparison
I didn’t find other Battleship implementations for the terminal that support multiplayer mode. Looks like it’s one of a kind. Let me know if I’m wrong!
# A bit of history
The project took me about a year to get to the alpha release. When I started in August 2023 I was on a sabbatical and things were moving fast. During August and September I created most of the domain model and tinkered a bit with Textual. It took some time to figure out what components should be there, what are their responsibilities, etc.
From there it took about three weeks to develop some kind of a visual design and implement the whole UI. Working with Textual was really a joy, though coming from VueJS background I was
/r/Python
https://redd.it/1f4flrj
# What My Project Does
The good old Battleship reinvented as a TUI (Text User Interface) application. Basically, you can play Battleship in your terminal. More than that, you can play via the Internet! You can also track your performance (like the shooting accuracy and the win/loss rate) and customize the UI.
Here’s a screenshot of the game screen.
# Target Audience
Anyone who’s familiar with the terminal and has Python installed (or curious enough to try it out).
# Comparison
I didn’t find other Battleship implementations for the terminal that support multiplayer mode. Looks like it’s one of a kind. Let me know if I’m wrong!
# A bit of history
The project took me about a year to get to the alpha release. When I started in August 2023 I was on a sabbatical and things were moving fast. During August and September I created most of the domain model and tinkered a bit with Textual. It took some time to figure out what components should be there, what are their responsibilities, etc.
From there it took about three weeks to develop some kind of a visual design and implement the whole UI. Working with Textual was really a joy, though coming from VueJS background I was
/r/Python
https://redd.it/1f4flrj
Ibis: Farewell pandas, and thanks for all the fish.
https://ibis-project.org/posts/farewell-pandas/
> TL; DR: we are deprecating the
/r/Python
https://redd.it/1f41ol2
https://ibis-project.org/posts/farewell-pandas/
> TL; DR: we are deprecating the
pandas and dask backends and will be removing them in version 10.0./r/Python
https://redd.it/1f41ol2
Ibis
Farewell pandas, and thanks for all the fish. – Ibis
the portable Python dataframe library
MinusX: AI assistant for Jupyter
MinusX in Action
Hey Folks!
I'm Vivek, building MinusX (https://minusx.ai) . It is an AI assistant for Jupyter. It is a chrome extension that adds a side chat to analytics apps. When you give it an instruction, it operates the tool - by clicking and typing - just like you would, to analyze data and answer queries. I was a research engineer at comma.ai (used to use jupyter everyday) for the last 3 years, and this project was born out of a personal itch.
You can inter-operate with the "agent" in your notebooks and take back control anytime. Our architecture makes the agent tool agnostic, and we're looking to support more tools!
We just did a beta release of MinusX recently. You can try it on your own data on your jupyter instances right now (https://minusx.ai/chrome-extension)! I'd love to hear any feedback you may have!
PS: We're open sourcing our entire application this week!
/r/JupyterNotebooks
https://redd.it/1f4fb95
MinusX in Action
Hey Folks!
I'm Vivek, building MinusX (https://minusx.ai) . It is an AI assistant for Jupyter. It is a chrome extension that adds a side chat to analytics apps. When you give it an instruction, it operates the tool - by clicking and typing - just like you would, to analyze data and answer queries. I was a research engineer at comma.ai (used to use jupyter everyday) for the last 3 years, and this project was born out of a personal itch.
You can inter-operate with the "agent" in your notebooks and take back control anytime. Our architecture makes the agent tool agnostic, and we're looking to support more tools!
We just did a beta release of MinusX recently. You can try it on your own data on your jupyter instances right now (https://minusx.ai/chrome-extension)! I'd love to hear any feedback you may have!
PS: We're open sourcing our entire application this week!
/r/JupyterNotebooks
https://redd.it/1f4fb95
PExploring Practical Uses of Machine Learning: How It's Revamping My Note-Taking Process
I’ve always struggled with note-taking because on one hand, I love keeping everything organized, but on the other, the constant need for categorizing and formatting just drives me nuts! Can anyone relate? I kept wishing for an AI tool that could handle all that for me, and that’s when I decided to create one myself.
I’ve come up with a basic version of the tool I dreamed up, called Stackie, because it helps me keep stacks of information organized—just type in whatever you need, and Stackie automatically sorts and structures every note (and yes, it gets natural language!).
Just the other day, I was looking into different mice for my sore wrist, popped the info into Stackie, and it neatly organized everything into my "Mouse Comparison" stack.
I’m also trying out new ways to use it, like tracking my calorie intake since I’ve put on a few pounds and want to get healthier. It’s funny how a tool I created out of laziness has turned out to be super useful.
I’m still brainstorming other ways to use Stackie and could really use some fresh ideas. So, I’d love to pick your brains and see if you can help me think outside the box. I’ve
/r/MachineLearning
https://redd.it/1f4owc0
I’ve always struggled with note-taking because on one hand, I love keeping everything organized, but on the other, the constant need for categorizing and formatting just drives me nuts! Can anyone relate? I kept wishing for an AI tool that could handle all that for me, and that’s when I decided to create one myself.
I’ve come up with a basic version of the tool I dreamed up, called Stackie, because it helps me keep stacks of information organized—just type in whatever you need, and Stackie automatically sorts and structures every note (and yes, it gets natural language!).
Just the other day, I was looking into different mice for my sore wrist, popped the info into Stackie, and it neatly organized everything into my "Mouse Comparison" stack.
I’m also trying out new ways to use it, like tracking my calorie intake since I’ve put on a few pounds and want to get healthier. It’s funny how a tool I created out of laziness has turned out to be super useful.
I’m still brainstorming other ways to use Stackie and could really use some fresh ideas. So, I’d love to pick your brains and see if you can help me think outside the box. I’ve
/r/MachineLearning
https://redd.it/1f4owc0
Reddit
From the MachineLearning community on Reddit: [P]Exploring Practical Uses of Machine Learning: How It's Revamping My Note-Taking…
Posted by [Deleted Account] - 8 votes and 5 comments
How to Create a 4-Digit Non-Repetitive OTP Authentication System in Django/DRF?
I was recently asked an interesting question during an interview: How would you design an authentication system that generates a 4-digit OTP without repeating any digits when a user tries to log in or sign up?
The requirements were pretty specific:
1. The OTP should be 4 digits long.
2. Each digit must be unique (no repetition of digits).
3. The OTP should be sent to the user for authentication during login or sign-up.
4. The system should be implemented using Django/DRF.
/r/django
https://redd.it/1f4s9li
I was recently asked an interesting question during an interview: How would you design an authentication system that generates a 4-digit OTP without repeating any digits when a user tries to log in or sign up?
The requirements were pretty specific:
1. The OTP should be 4 digits long.
2. Each digit must be unique (no repetition of digits).
3. The OTP should be sent to the user for authentication during login or sign-up.
4. The system should be implemented using Django/DRF.
/r/django
https://redd.it/1f4s9li
Reddit
From the django community on Reddit
Explore this post and more from the django community
Deploying a Django app with as few moving parts as possible
https://www.bugsink.com/installation-simplification-journey/
/r/django
https://redd.it/1f4qq64
https://www.bugsink.com/installation-simplification-journey/
/r/django
https://redd.it/1f4qq64
Bugsink
Deploying a Django app with as few moving parts as possible
How we simplified the installation process for Bugsink, our self-hosted error tracking tool built in Django, by removing as many unnecessary components as we could.
Optimizing Parallel Processing and other improvements along with Deployment Strategies for a Django-based Web Scraping Application
I have created a Django project with Redis and Celery in a Docker Compose setup for development. The project allows users to input a search keyword and select a country from a drop down list which is loaded initially from country model using js. Users can provide multiple search inputs and corresponding countries before submitting. After submission, a process (
Additionally, I have a dashboard page that displays the status of the task. In the
However, I’ve noticed that the parallel execution is taking more time than expected. I’m looking for the correct way to implement parallel processing efficiently.
For production, I’m considering deployment options
/r/django
https://redd.it/1f4sy58
I have created a Django project with Redis and Celery in a Docker Compose setup for development. The project allows users to input a search keyword and select a country from a drop down list which is loaded initially from country model using js. Users can provide multiple search inputs and corresponding countries before submitting. After submission, a process (
scraper.py) will start, extracting information from Google and Yelp searches by constructing URLs from the provided inputs using Python's requests library. This process is handled by Celery as a task. I also have a model for task status. Additionally, I have a dashboard page that displays the status of the task. In the
scraper.py file, I am using Python's multiprocessing module with a maximum of 15 workers. This allows the scraper to run in parallel when the user submits multiple sets of search and country inputs. Once the scraping is completed, a download option will be enabled on the dashboard page for the respective task, allowing users to download the resulting CSV file.However, I’ve noticed that the parallel execution is taking more time than expected. I’m looking for the correct way to implement parallel processing efficiently.
For production, I’m considering deployment options
/r/django
https://redd.it/1f4sy58
Reddit
From the django community on Reddit
Explore this post and more from the django community