Tuesday Daily Thread: Advanced questions
# Weekly Wednesday Thread: Advanced Questions 🐍
Dive deep into Python with our Advanced Questions thread! This space is reserved for questions about more advanced Python topics, frameworks, and best practices.
## How it Works:
1. **Ask Away**: Post your advanced Python questions here.
2. **Expert Insights**: Get answers from experienced developers.
3. **Resource Pool**: Share or discover tutorials, articles, and tips.
## Guidelines:
* This thread is for **advanced questions only**. Beginner questions are welcome in our [Daily Beginner Thread](#daily-beginner-thread-link) every Thursday.
* Questions that are not advanced may be removed and redirected to the appropriate thread.
## Recommended Resources:
* If you don't receive a response, consider exploring r/LearnPython or join the [Python Discord Server](https://discord.gg/python) for quicker assistance.
## Example Questions:
1. **How can you implement a custom memory allocator in Python?**
2. **What are the best practices for optimizing Cython code for heavy numerical computations?**
3. **How do you set up a multi-threaded architecture using Python's Global Interpreter Lock (GIL)?**
4. **Can you explain the intricacies of metaclasses and how they influence object-oriented design in Python?**
5. **How would you go about implementing a distributed task queue using Celery and RabbitMQ?**
6. **What are some advanced use-cases for Python's decorators?**
7. **How can you achieve real-time data streaming in Python with WebSockets?**
8. **What are the
/r/Python
https://redd.it/1jdrtck
# Weekly Wednesday Thread: Advanced Questions 🐍
Dive deep into Python with our Advanced Questions thread! This space is reserved for questions about more advanced Python topics, frameworks, and best practices.
## How it Works:
1. **Ask Away**: Post your advanced Python questions here.
2. **Expert Insights**: Get answers from experienced developers.
3. **Resource Pool**: Share or discover tutorials, articles, and tips.
## Guidelines:
* This thread is for **advanced questions only**. Beginner questions are welcome in our [Daily Beginner Thread](#daily-beginner-thread-link) every Thursday.
* Questions that are not advanced may be removed and redirected to the appropriate thread.
## Recommended Resources:
* If you don't receive a response, consider exploring r/LearnPython or join the [Python Discord Server](https://discord.gg/python) for quicker assistance.
## Example Questions:
1. **How can you implement a custom memory allocator in Python?**
2. **What are the best practices for optimizing Cython code for heavy numerical computations?**
3. **How do you set up a multi-threaded architecture using Python's Global Interpreter Lock (GIL)?**
4. **Can you explain the intricacies of metaclasses and how they influence object-oriented design in Python?**
5. **How would you go about implementing a distributed task queue using Celery and RabbitMQ?**
6. **What are some advanced use-cases for Python's decorators?**
7. **How can you achieve real-time data streaming in Python with WebSockets?**
8. **What are the
/r/Python
https://redd.it/1jdrtck
Discord
Join the Python Discord Server!
We're a large community focused around the Python programming language. We believe that anyone can learn to code. | 412982 members
[P] I fine-tuned Qwen 2.5 Coder on a single repo and got a 47% improvement in code completion accuracy
Hey all,
Just wanted to share an interesting experiment I ran to see what kind of performance gains can be achieved by fine-tuning a coding model to code from a single repo.
**Tl;dr**: The fine-tuned model achieves a 47% improvement in the code completion task (tab autocomplete). Accuracy goes from 25% to 36% (exact match against ground truth) after a short training run of only 500 iterations on a single RTX 4090 GPU.
https://preview.redd.it/3borsuk1bape1.png?width=1063&format=png&auto=webp&s=8d9510517f3d04163b3ac6b1de6765158886e489
This is interesting because it shows that there are significant gains to be had by fine-tuning to your own code.
**Highlights of the experiment**:
* Model: qwen2.5-coder 14b, 4-bit quantized
* Training data: Svelte source files from this repo: [https://github.com/hcengineering/platform](https://github.com/hcengineering/platform)
* Unsloth for LoRA training with rank 16, 4096 sequence length
* GPU: single RTX 4090
* 500 iterations with effective batch size 8
/r/MachineLearning
https://redd.it/1jdiafd
Hey all,
Just wanted to share an interesting experiment I ran to see what kind of performance gains can be achieved by fine-tuning a coding model to code from a single repo.
**Tl;dr**: The fine-tuned model achieves a 47% improvement in the code completion task (tab autocomplete). Accuracy goes from 25% to 36% (exact match against ground truth) after a short training run of only 500 iterations on a single RTX 4090 GPU.
https://preview.redd.it/3borsuk1bape1.png?width=1063&format=png&auto=webp&s=8d9510517f3d04163b3ac6b1de6765158886e489
This is interesting because it shows that there are significant gains to be had by fine-tuning to your own code.
**Highlights of the experiment**:
* Model: qwen2.5-coder 14b, 4-bit quantized
* Training data: Svelte source files from this repo: [https://github.com/hcengineering/platform](https://github.com/hcengineering/platform)
* Unsloth for LoRA training with rank 16, 4096 sequence length
* GPU: single RTX 4090
* 500 iterations with effective batch size 8
/r/MachineLearning
https://redd.it/1jdiafd
What is the convention for and when it comes to OOP?
Is it a convention in Python that \_ in class method or variable name signifies a private variable, while a _ signifies a protected variable?
I knew it was a convention to use it to signify that a variable or method wasn't to be used outside of the class, but I didn't know about this distinction of private and protected.
For context, I stumbled upon this question when Perplexity AI told me this was the case. I asked it to give me the sources for this but was unable to produce nothing outside a couple of blogs and articles.
So here I am asking the community, what do you think? I think it sounds interesting, to say the least. I have never though about using both __ and _ in the same piece of code, for the sake of consistency (I also thought it was discouraged), but now I am of the opinion that this distinction could actually be useful when designing more complex OOP systems.
/r/Python
https://redd.it/1jdtab6
Is it a convention in Python that \_ in class method or variable name signifies a private variable, while a _ signifies a protected variable?
I knew it was a convention to use it to signify that a variable or method wasn't to be used outside of the class, but I didn't know about this distinction of private and protected.
For context, I stumbled upon this question when Perplexity AI told me this was the case. I asked it to give me the sources for this but was unable to produce nothing outside a couple of blogs and articles.
So here I am asking the community, what do you think? I think it sounds interesting, to say the least. I have never though about using both __ and _ in the same piece of code, for the sake of consistency (I also thought it was discouraged), but now I am of the opinion that this distinction could actually be useful when designing more complex OOP systems.
/r/Python
https://redd.it/1jdtab6
Reddit
From the Python community on Reddit
Explore this post and more from the Python community
I wrote a script to simulate this years March Madness
Here’s the code: https://gist.github.com/CoreyMSchafer/27fcf83e5a0e5a87f415ff19bfdd2a4c
Also made a YouTube walkthrough here: https://youtu.be/4TFQD0ok5Ao
The script uses the inverse of the seeds to weight the teams. There is commented out code that you can adjust to give seeds more/less of an advantage. If you’d like to weight each team individually, you could also add a power attribute to the Team dataclass and at those individually when instantiating the first round.
/r/Python
https://redd.it/1jdsi9u
Here’s the code: https://gist.github.com/CoreyMSchafer/27fcf83e5a0e5a87f415ff19bfdd2a4c
Also made a YouTube walkthrough here: https://youtu.be/4TFQD0ok5Ao
The script uses the inverse of the seeds to weight the teams. There is commented out code that you can adjust to give seeds more/less of an advantage. If you’d like to weight each team individually, you could also add a power attribute to the Team dataclass and at those individually when instantiating the first round.
/r/Python
https://redd.it/1jdsi9u
Gist
March Madness Simulator
March Madness Simulator. GitHub Gist: instantly share code, notes, and snippets.
str2speech: A wrapper for all the new TTS models being released
GitHub: str2speech
# What my project does
Makes it easier to use all the latest TTS models.
I currently support:
- Zyphra Zonos
- Sesame CSM-1B
- Kokoro-82M
- Suno's Bark
- and a few older MMS-TTS models from Facebook
I'll be adding more models soon, like Spark TTS.
# Target Audience
Everyone who's interested in trying out/comparing TTS models. Could be used in production by YouTubers, podcasters, etc. Should work reasonably well in Shell scripts too.
# Comparison
The closest project I can think of is coqui-ai tts. It's a much larger project (supporting xtts, bark, tortoise, and many more models), but it seems to be abandoned. They haven't added support for any models released in the last couple of months.
My project tries to support more modern models. I try to focus on the CLI and ease of use on Google colab.
# Usage
Installation is easy:
And so is usage, I think:
More examples and full code available on GitHub: str2speech
/r/Python
https://redd.it/1jdrvwy
GitHub: str2speech
# What my project does
Makes it easier to use all the latest TTS models.
I currently support:
- Zyphra Zonos
- Sesame CSM-1B
- Kokoro-82M
- Suno's Bark
- and a few older MMS-TTS models from Facebook
I'll be adding more models soon, like Spark TTS.
# Target Audience
Everyone who's interested in trying out/comparing TTS models. Could be used in production by YouTubers, podcasters, etc. Should work reasonably well in Shell scripts too.
# Comparison
The closest project I can think of is coqui-ai tts. It's a much larger project (supporting xtts, bark, tortoise, and many more models), but it seems to be abandoned. They haven't added support for any models released in the last couple of months.
My project tries to support more modern models. I try to focus on the CLI and ease of use on Google colab.
# Usage
Installation is easy:
pip install str2speech
And so is usage, I think:
str2speech --text "Kokoro is nice" --model "kokoro"
More examples and full code available on GitHub: str2speech
/r/Python
https://redd.it/1jdrvwy
GitHub
GitHub - hathibelagal-dev/str2speech: An easy-to-use library and command-line tool for TTS
An easy-to-use library and command-line tool for TTS - hathibelagal-dev/str2speech
Building a Multi-Tenant Automation System in a Django CRM – Seeking Advice
Hi all,
I'm working on a SaaS CRM built with Django/DRF that centers around leads and deals, and I'm looking to implement a robust automation system. The idea is to allow for dynamic, multi-tenant automations that can be triggered by events on leads and deals, as well as scheduled tasks (like daily or weekly operations).
I'm using Django-tenants and django-q2
At a high level, the system should let users set up rules that include triggers, conditions, and actions, with everything stored in the database to avoid hardcoding. I'm considering a design that includes event-driven triggers (using Django signals or an equivalent) and a task queue for longer-running processes, but I'm curious about potential performance pitfalls and best practices when scaling these systems.
I'm interested in hearing from anyone who's built something similar or has experience with automations in a multi-tenant environment. Any advice, pitfalls to watch out for, or suggestions on design and architecture would be greatly appreciated!
Thanks in advance for your help.
/r/django
https://redd.it/1jdzkb8
Hi all,
I'm working on a SaaS CRM built with Django/DRF that centers around leads and deals, and I'm looking to implement a robust automation system. The idea is to allow for dynamic, multi-tenant automations that can be triggered by events on leads and deals, as well as scheduled tasks (like daily or weekly operations).
I'm using Django-tenants and django-q2
At a high level, the system should let users set up rules that include triggers, conditions, and actions, with everything stored in the database to avoid hardcoding. I'm considering a design that includes event-driven triggers (using Django signals or an equivalent) and a task queue for longer-running processes, but I'm curious about potential performance pitfalls and best practices when scaling these systems.
I'm interested in hearing from anyone who's built something similar or has experience with automations in a multi-tenant environment. Any advice, pitfalls to watch out for, or suggestions on design and architecture would be greatly appreciated!
Thanks in advance for your help.
/r/django
https://redd.it/1jdzkb8
Reddit
From the django community on Reddit
Explore this post and more from the django community
Just Built & Deployed a Video Platform MVP ( saketmanolkar.me ) — Looking for Feedback
https://redd.it/1je82un
@pythondaily
https://redd.it/1je82un
@pythondaily
Reddit
From the django community on Reddit: Just Built & Deployed a Video Platform MVP ( saketmanolkar.me ) — Looking for Feedback
Explore this post and more from the django community
PySide6 + Nuitka is very impressive (some numbers and feedback inside)
In preparation for releasing a new version of Flowkeeper I decided to try replacing PyInstaller with Nuitka. My main complaint about PyInstaller was that I could never make it work with MS Defender, but that's a topic for another time.
I've never complained about the size of the binaries that PyInstaller generated. Given that it had to bundle Python 3 and Qt 6, \~100MB looked reasonable. So you can imagine how surprised I was when instead of spitting out a usual 77MB for a standalone / portable Windows exe file it produced... a 39MB one! It is twice smaller, seemingly because Nuitka's genius C compiler / linker could shed unused Qt code so well.
Flowkeeper is a Qt Widgets app, and apart from typical QtCore, QtGui and QtWidgets it uses QtMultimedia, QtChart, QtNetwork, QtWebSockets and some other modules from PySide6_Addons. It also uses Fernet cryptography package, which in turn bundles hazmat. Finally, it includes a 10MB mp3 file, as well as \~2MB of images and fonts as resources. So all of that fits into a single self-contained 40MB exe file, which I find mighty impressive, especially if you start comparing it against Electron. Oh yes, and that's with the latest stable Python
/r/Python
https://redd.it/1je94cf
In preparation for releasing a new version of Flowkeeper I decided to try replacing PyInstaller with Nuitka. My main complaint about PyInstaller was that I could never make it work with MS Defender, but that's a topic for another time.
I've never complained about the size of the binaries that PyInstaller generated. Given that it had to bundle Python 3 and Qt 6, \~100MB looked reasonable. So you can imagine how surprised I was when instead of spitting out a usual 77MB for a standalone / portable Windows exe file it produced... a 39MB one! It is twice smaller, seemingly because Nuitka's genius C compiler / linker could shed unused Qt code so well.
Flowkeeper is a Qt Widgets app, and apart from typical QtCore, QtGui and QtWidgets it uses QtMultimedia, QtChart, QtNetwork, QtWebSockets and some other modules from PySide6_Addons. It also uses Fernet cryptography package, which in turn bundles hazmat. Finally, it includes a 10MB mp3 file, as well as \~2MB of images and fonts as resources. So all of that fits into a single self-contained 40MB exe file, which I find mighty impressive, especially if you start comparing it against Electron. Oh yes, and that's with the latest stable Python
/r/Python
https://redd.it/1je94cf
Reddit
From the Python community on Reddit
Explore this post and more from the Python community
R Jagged Flash Attention Optimization
Meta researchers have introduced Jagged Flash Attention, a novel technique that significantly enhances the performance and scalability of large-scale recommendation systems. By combining jagged tensors with flash attention, this innovation achieves up to 9× speedup and 22× memory reduction compared to dense attention, outperforming even dense flash attention with 3× speedup and 53% better memory efficiency.
Read the full paper write up here: https://www.shaped.ai/blog/jagged-flash-attention-optimization
/r/MachineLearning
https://redd.it/1je93sv
Meta researchers have introduced Jagged Flash Attention, a novel technique that significantly enhances the performance and scalability of large-scale recommendation systems. By combining jagged tensors with flash attention, this innovation achieves up to 9× speedup and 22× memory reduction compared to dense attention, outperforming even dense flash attention with 3× speedup and 53% better memory efficiency.
Read the full paper write up here: https://www.shaped.ai/blog/jagged-flash-attention-optimization
/r/MachineLearning
https://redd.it/1je93sv
www.shaped.ai
Jagged Flash Attention Optimization | Shaped Blog
Meta researchers have introduced Jagged Flash Attention, a novel technique that significantly enhances the performance and scalability of large-scale recommendation systems. By combining jagged tensors with flash attention, this innovation achieves up to…
Python Quirks I Secretly Like
Hi there,
I’ve always wanted to create YouTube content about programming languages, but I’ve been self-conscious about my voice (and mic, lol). Recently, I made a pilot video on the Zig programming language, and afterward, I met a friend here on Reddit, u/tokisuno, who has a great voice and offered to do the voiceovers.
So, we’ve put together a video on Python — I hope you’ll like it:
https://www.youtube.com/watch?v=DZtdkZV6hYM
/r/Python
https://redd.it/1je98cv
Hi there,
I’ve always wanted to create YouTube content about programming languages, but I’ve been self-conscious about my voice (and mic, lol). Recently, I made a pilot video on the Zig programming language, and afterward, I met a friend here on Reddit, u/tokisuno, who has a great voice and offered to do the voiceovers.
So, we’ve put together a video on Python — I hope you’ll like it:
https://www.youtube.com/watch?v=DZtdkZV6hYM
/r/Python
https://redd.it/1je98cv
YouTube
You don't know Python
Voiced by @tokisuno ... pay his channel a visit.
- Talk by Gary Bernhardt: "WAT. A lightning talk by Gary Bernhardt from CodeMash 2012"
Time codes:
0:00 - Intro
0:19 - Classic
0:59 - Walrus
1:36 - Is not?
1:59 - Is 1
3:06 - Is 2
3:41 - Plus equal
4:16…
- Talk by Gary Bernhardt: "WAT. A lightning talk by Gary Bernhardt from CodeMash 2012"
Time codes:
0:00 - Intro
0:19 - Classic
0:59 - Walrus
1:36 - Is not?
1:59 - Is 1
3:06 - Is 2
3:41 - Plus equal
4:16…
Hypercorn VS Uvicorn VS Daphne+Gunicorn? (Behind Nginx)
Hi folks,
I have a setup with Websockets (HTMX and older code) that I'm progressively dumping for Server Sent events with HTMX.
One thing I realized when starting to use SSE is that the http/1.1 protocol seems to limit the number of open connections, so after i open a couple tabs, nothing loads in the newer tabs until I close the first ones. Using runserver in developement made me chase that bug for 2 days.
By using an http/2 compatible server like hypercorn, I was able to get rid of the issue.
Now, for production... I'm behind Nginx and I have http/2 working properly on it. Couple questions for experienced devops here:
\- Is there a performance difference between Nginx with http2 + an http/1.1 server behind like uvicorn? Should I aim for http2 all the way?
\- What are your general insights for performance when it comes to an app with SSE? Should I keep Gunicorn in WSGI and send SSE traffic to an ASGI server? Or should I just use nginx+uvicorn everywhere.
Any insight appreciated
/r/django
https://redd.it/1jegny2
Hi folks,
I have a setup with Websockets (HTMX and older code) that I'm progressively dumping for Server Sent events with HTMX.
One thing I realized when starting to use SSE is that the http/1.1 protocol seems to limit the number of open connections, so after i open a couple tabs, nothing loads in the newer tabs until I close the first ones. Using runserver in developement made me chase that bug for 2 days.
By using an http/2 compatible server like hypercorn, I was able to get rid of the issue.
Now, for production... I'm behind Nginx and I have http/2 working properly on it. Couple questions for experienced devops here:
\- Is there a performance difference between Nginx with http2 + an http/1.1 server behind like uvicorn? Should I aim for http2 all the way?
\- What are your general insights for performance when it comes to an app with SSE? Should I keep Gunicorn in WSGI and send SSE traffic to an ASGI server? Or should I just use nginx+uvicorn everywhere.
Any insight appreciated
/r/django
https://redd.it/1jegny2
Reddit
From the django community on Reddit
Explore this post and more from the django community
Wednesday Daily Thread: Beginner questions
# Weekly Thread: Beginner Questions 🐍
Welcome to our Beginner Questions thread! Whether you're new to Python or just looking to clarify some basics, this is the thread for you.
## How it Works:
1. Ask Anything: Feel free to ask any Python-related question. There are no bad questions here!
2. Community Support: Get answers and advice from the community.
3. Resource Sharing: Discover tutorials, articles, and beginner-friendly resources.
## Guidelines:
This thread is specifically for beginner questions. For more advanced queries, check out our [Advanced Questions Thread](#advanced-questions-thread-link).
## Recommended Resources:
If you don't receive a response, consider exploring r/LearnPython or join the Python Discord Server for quicker assistance.
## Example Questions:
1. What is the difference between a list and a tuple?
2. How do I read a CSV file in Python?
3. What are Python decorators and how do I use them?
4. How do I install a Python package using pip?
5. What is a virtual environment and why should I use one?
Let's help each other learn Python! 🌟
/r/Python
https://redd.it/1jejwwv
# Weekly Thread: Beginner Questions 🐍
Welcome to our Beginner Questions thread! Whether you're new to Python or just looking to clarify some basics, this is the thread for you.
## How it Works:
1. Ask Anything: Feel free to ask any Python-related question. There are no bad questions here!
2. Community Support: Get answers and advice from the community.
3. Resource Sharing: Discover tutorials, articles, and beginner-friendly resources.
## Guidelines:
This thread is specifically for beginner questions. For more advanced queries, check out our [Advanced Questions Thread](#advanced-questions-thread-link).
## Recommended Resources:
If you don't receive a response, consider exploring r/LearnPython or join the Python Discord Server for quicker assistance.
## Example Questions:
1. What is the difference between a list and a tuple?
2. How do I read a CSV file in Python?
3. What are Python decorators and how do I use them?
4. How do I install a Python package using pip?
5. What is a virtual environment and why should I use one?
Let's help each other learn Python! 🌟
/r/Python
https://redd.it/1jejwwv
Discord
Join the Python Discord Server!
We're a large community focused around the Python programming language. We believe that anyone can learn to code. | 412982 members
Why drf not implemented schema into their api rather than use serialization who have performance issue?
I read some article about drf vs django ninja and find weird, If schema pydantic is better in term performance and validation, why drf implement serialization? Is there the info that I miss?
/r/django
https://redd.it/1jekc2g
I read some article about drf vs django ninja and find weird, If schema pydantic is better in term performance and validation, why drf implement serialization? Is there the info that I miss?
/r/django
https://redd.it/1jekc2g
Reddit
From the django community on Reddit
Explore this post and more from the django community
P Question about server GPU needs for for DeepLabCut for high throughput
Hi all,
Currently working on a project that uses DeepLabCut for pose estimation. Trying to figure out how much server GPU VRAM I need to process videos. I believe my footage would be 1080x1920p. I can downscale to 3fps for my application if that helps increase the analysis throughput.
If anyone has any advice, I would really appreciate it!
TIA
Edit:
From my research I saw a 1080ti was doing ~60fps with 544x544p video. A 4090 is about 200% faster but due to the increase in the footage size it only does 20 fps if you scale it relatively to the 1080ti w/ 544p footage size.
Wondering if that checks out from anyone that has worked with it.
/r/MachineLearning
https://redd.it/1jejg5d
Hi all,
Currently working on a project that uses DeepLabCut for pose estimation. Trying to figure out how much server GPU VRAM I need to process videos. I believe my footage would be 1080x1920p. I can downscale to 3fps for my application if that helps increase the analysis throughput.
If anyone has any advice, I would really appreciate it!
TIA
Edit:
From my research I saw a 1080ti was doing ~60fps with 544x544p video. A 4090 is about 200% faster but due to the increase in the footage size it only does 20 fps if you scale it relatively to the 1080ti w/ 544p footage size.
Wondering if that checks out from anyone that has worked with it.
/r/MachineLearning
https://redd.it/1jejg5d
Reddit
From the MachineLearning community on Reddit
Explore this post and more from the MachineLearning community
Searching for a Coding Partner
Hi, I’m a 6+ yrs Python (main) developer and I have been working for several companies projects but always freelanced, I have experience in website automation, data scraping, network requests analysis, e-commerce, website creation, trading and more. I am looking for a well determined and motivated guy that wants to work with me to start a business and make some money together. Looking for opportunities
Regards
/r/Python
https://redd.it/1jesga2
Hi, I’m a 6+ yrs Python (main) developer and I have been working for several companies projects but always freelanced, I have experience in website automation, data scraping, network requests analysis, e-commerce, website creation, trading and more. I am looking for a well determined and motivated guy that wants to work with me to start a business and make some money together. Looking for opportunities
Regards
/r/Python
https://redd.it/1jesga2
Reddit
From the Python community on Reddit
Explore this post and more from the Python community