Easily Customize LLM Pipelines with YAML templates.

Hey everyone,

I’ve been working on productionizing Retrieval-Augmented Generation (RAG) applications, especially when dealing with data sources that frequently change (like files being added, updated, or deleted by multiple team members).

For Python devs who aren’t deep into Gen AI, RAG is a common way to extend Gen AI models by connecting them to external data sources for info beyond their training data. Building a quick pilot is often straightforward, but the real challenge comes in making it production-ready.

However, spending time tweaking application scripts is a hassle. For example, if you have swap a model or change the type of index.

To tackle this, we’ve created an open-source repository that provides YAML templates to simplify RAG deployment without the need to modify code each time. You can check it out here: [llm-app GitHub Repo](https://github.com/pathwaycom/llm-app).

Here’s how it helps:

* **Swap components easily**, like switching data sources from local files to SharePoint or Google Drive, changing models, or swapping indexes from a vector index to a hybrid index.
* **Change parameters in RAG pipelines via readable YAML files.**
* **Keep configurations clean and organized**, making it easier to manage and update.

For more details, there’s also a [blog post](https://pathway.com/blog/llm-yaml-templates) and a [detailed guide](https://pathway.com/developers/user-guide/llm-xpack/yaml-templates) that explain how to customize the templates.

This

/r/Python
https://redd.it/1glq4jd

Keeping a thread alive

I never thought simply running a task every hour would turn out to be an issue.

Context: I have a Flask API deployed to a Windows machine using IIS w/ wfastcgi and I want the program to also run a process every hour.

I know I can just use Task scheduler through windows to run my Python program every hour, but I spent all this time merging my coworker’s project into my Flask api project and really wanted only a single app to manage.

I thought at the start of the program it could be executed, but I realized I had multiple workers and so multiple instances would start, which is not okay for the task.

So I created an api endpoint to initiate the job, and figured it could run a thread asynchronously where this asynchronous thread would run a “while True:” loop where the thread would sleep for an hour in between executions… but when I ran the program it would never restart after an hour, and from the logs it is clear the thread is just stopping - poof!

So I figure what about 15 minutes?? Still stops.

What about 1 minute? Success!

So I get the clever idea to make the

/r/Python
https://redd.it/1glpfj5

Django Cotton: Bringing component-based design to Django templates.


* [https://django-cotton.com/](https://django-cotton.com/)
* [https://github.com/wrabit/django-cotton](https://github.com/wrabit/django-cotton)
* **Creator:** [https://x.com/willrabbott/status/1813558563610227073](https://x.com/willrabbott/status/1813558563610227073)

**Does this package look promising?**

/r/django
https://redd.it/1gllbup

Experienced Django devs, I have a question: How can I improve my database design skills?

Help Needed with Database Design

Hey everyone,

I’m really struggling with database design. It’s been a while since I started working on a small social media app using Flutter and Firebase. The app includes features like news, comments, likes, dislikes, and user rankings based on likes and dislikes. I managed to write about half of the project, but then I realized that my data model was flawed. I became stuck trying to figure out how to implement batch write for likes and dislikes, so I ended up abandoning the project after three months.

Now, a friend of mine has asked me to create a web app for diet tracking. I’m fairly comfortable with Django, as I've completed a couple of projects using it. However, this new project feels quite large, and I’m worried that I might get confused again in the middle of development.



How can I improve my database design skills? Is it okay to use tools like ChatGPT for assistance? I tried it once before, but the outcome was a complete mess.



Thank you very much for any advice!

/r/django
https://redd.it/1glnwi1

Multi-Tenancy with Separate Databases ?



i have an application and it's using django and react js and in this application i want to do school management for mutliple school but i want to make for each school it's own database that gonna be created automatic of course there will be an admin superior now what i want to do is to tell me what do you understand by this, and for each school her own database will be created like it gonna have the same structure same tables like teacher,student,planning,absence ,.... and other tables

what are the possible solutions please

/r/django
https://redd.it/1glt7l7

Thank you again r/Python - I'm opening up my Python course for those who missed it before

A bit of background - loads of people joined my Python course for beta testing via this community, and shared lots of valuable feedback, which I’ve been able to incorporate.

I’m thrilled to share that since then, the course has started bringing in a small but meaningful amount of income.

This is a big milestone for me, especially as it was my first course. I’m now moving forward with my next course, this time focused on simulation in Python.

So, as a thank you to this community, I have just generated 1000 free vouchers for the course: https://www.udemy.com/course/python-for-engineers-scientists-and-analysts/?couponCode=5DAYFREEBIE

This is the most which I am allowed to generate, and Udemy rules mean they will expire in 5 days. Sharing with this community is a real win-win, since you guys get something that you hopefully find helpful, and I get more people enrolling in the course, which helps the algorithms in Udemy promote my course in the future (meaning I'm more likely to be able to actually make a living lol).

So please take a voucher if the course might be of value to you. You don't need to do the course right away as you will have lifetime access, so you could do it

/r/Python
https://redd.it/1glxbrj

How to Deploy a Django Project
https://www.thedevspace.io/community/django-deploy

/r/djangolearning
https://redd.it/1glukuh

9x model serving performance without changing hardware

Project

https://github.com/martynas-subonis/model-serving

Extensive write-up available here.

What My Project Does

This project uses ONNX-Runtime with various optimizations (implementations both in Python and Rust) to benchmark performance improvements compared to naive PyTorch implementations.

Target Audience

ML engineers, serving models in production.

Comparison

This project benchmarks basic PyTorch serving against ONNX Runtime in both Python and Rust, showcasing notable performance gains. Rust’s Actix-Web with ONNX Runtime handles 328.94 requests/sec, compared to Python ONNX at 255.53 and PyTorch at 35.62, with Rust's startup time of 0.348s being 4x faster than Python ONNX and 12x faster than PyTorch. Rust’s Docker image is also 48.3 MB—6x smaller than Python ONNX and 13x smaller than PyTorch. These numbers highlight the efficiency boost achievable by switching frameworks and languages in model-serving setups.

/r/Python
https://redd.it/1gm0flj

Friday Daily Thread: r/Python Meta and Free-Talk Fridays

# Weekly Thread: Meta Discussions and Free Talk Friday 🎙️

Welcome to Free Talk Friday on /r/Python! This is the place to discuss the r/Python community (meta discussions), Python news, projects, or anything else Python-related!

## How it Works:

1. Open Mic: Share your thoughts, questions, or anything you'd like related to Python or the community.
2. Community Pulse: Discuss what you feel is working well or what could be improved in the /r/python community.
3. News & Updates: Keep up-to-date with the latest in Python and share any news you find interesting.

## Guidelines:

All topics should be related to Python or the /r/python community.
Be respectful and follow Reddit's Code of Conduct.

## Example Topics:

1. New Python Release: What do you think about the new features in Python 3.11?
2. Community Events: Any Python meetups or webinars coming up?
3. Learning Resources: Found a great Python tutorial? Share it here!
4. Job Market: How has Python impacted your career?
5. Hot Takes: Got a controversial Python opinion? Let's hear it!
6. Community Ideas: Something you'd like to see us do? tell us.

Let's keep the conversation going. Happy discussing! 🌟

/r/Python
https://redd.it/1gm53zx

Oracle forms builder alternate

Oracle forms builder alternate

Hi All,
My employer recently upgraded from Oracle 11g to 19c..there was a reporting module that was built out of Oracle 6i and now with the upgrade the reporting module is breaking as there is no compatible version of Oracle forms builder with 19c.

So we have been asked to find alternates.I am thinking of suggesting Django with html as the requirement mainly focuses on generating excel docs by querying the Oracle tables.they need an UI component just to trigger the Excel generation process.

Now am from completely java background and have very minimal knowledge in Django.But I did start leaning python and found the file operations are much more clean and minimal code in python when compared to java and hence thinking of suggesting python with Django for a quick turnaround.

Is this good suggestion or Is there anything else out there that am completely missing for this scenario?

Tech stack preferred is java,springboot,angular,python and Django or flask

P.S it has to be open source.When I say open source I mean it should be free of cost

Thanks In advance

/r/Python
https://redd.it/1gm6hp1

I would like feedback

I am creating an application template for django and I would like to know your opinions, the only condition is that when making django-admin startapp --template=

The app must be created within the django project in a folder called Apps

I leave you the link: https://github.com/simuel/DjangoHexTemplate.git

/r/django
https://redd.it/1gmcbat

Best practice to save some variable in between calls? (no session, no db)

Hello,

I am using Flask to build a desktop app, together with pywebview and other libraries. It's a desktop app, so there will be only one user (it uses the camera, a second screen, tensorflow, opencv, so not something that would be moved to the cloud). I use pywebview to take advantage of the web browser to display a nice interface and use SVG canvas a lot. That's for the context.

Now, there are a lot of internal variables that I track between the different calls to Flask routes. At the beginning I tried to used sessions to record them, but many object are to big in size to store in session, and many are not appropriately serialized to store in cookies (like a keras model for instance). So at the moment, I just store them as global variables, and use the \`global\` keyword here and there to modify their content.

It works fine, but it does not look good. What would be the best practices to store and reuse those variables in my specific case?

Edit: Eventually, I ended up wrapping all those variable in the Flask application variable. Something like this:

```
class Application(Flask):
def __init__(self, *args, **kwargs):


/r/flask
https://redd.it/1gkuggn

I finally found a currently-maintained version of Whoosh, a text search library

Sygil-Dev/whoosh-reloaded: Whoosh is a fast, featureful full-text indexing and searching library implemented in pure Python.

Whoosh 3.0.0 documentation — Whoosh-Reloaded 3.0.0 documentation

/r/Python
https://redd.it/1gm8ovf

Feature Friday: The querystring tag!

Today's Feature Friday is about {% querystring %}!

The new {% querystring %} template tag in Django 5.1 makes it easier to access and modify the query string in your Django templates, letting you work with links that use query parameters.

Previously, if you wanted to add or change a single value in the query string, you would need to write a lot of code:

{# Linebreaks added for readability, this should be one, long line. #}
<a href="?{% for key, values in request.GET.iterlists %}
{% if key != "page" %}
{% for value in values %}
{{ key }}={{ value }}&amp;
{% endfor %}
{% endif %}
{% endfor %}page={{ page.nextpagenumber }}">Next page</a>

With {% querystring %} you can replace all of that with this single line:

{% querystring page=http://page.nextpagenumber %}

The {% querystring %} tag is particularly useful for things like filters and pagination—where you want to pass through most of the query parameters but modify one

/r/django
https://redd.it/1gmjgep

🎉 Introducing dj-data-generator! 🎉

We’re thrilled to announce the release of dj-data-generator, a new tool designed to simplify generating customizable test data for Django projects. Whether you’re setting up demo environments, populating databases for testing, or running performance tests, dj-data-generator offers efficient, built-in support to meet your needs—all without third-party packages.



Key Features:



\- Generate any number of records for your project models and save them directly to the database with a simple django command

\- Easy customization for model fields

\- Handles unique and related fields



Check it out, and let us know what you think! Feedback, contributions, and suggestions are welcome as we continue to build.



📥 Check it out on PyPI:

https://pypi.org/project/dj-data-generator/



💻 Source Code and Docs on GitHub:

https://github.com/Lazarus-org/dj-data-generator

/r/django
https://redd.it/1gmj9nl

Django Tip explaining.only()/.defer() n+1

https://redd.it/1gmo33c
@pythondaily

Choosing a Backend Framework for ML Prediction and Matching



Hey, if you’re going with some ML for prediction and matching—nothing very complicated—and implementing it in a web app, what sort of backend framework should I use, and what workflow should I implement? Usually, I’ve worked with Express.js.

/r/django
https://redd.it/1gme0iv

A search engine for all your memes (v2.0 updates)

The app is open source 👉 https://github.com/neonwatty/meme\_search

# What My Project Does

The open source engine indexes your memes by their visual content and text, making them easily searchable. Drag and drop recovered memes into any messager.

Addittional features rolling out with the new "pro" version include:

1. Auto-Generate Meme Descriptions: Target specific memes for auto-description generation (instead of applying to your entire directory).
2. Manual Meme Description Editing: Edit or add descriptions manually for better search results, no need to wait for auto-generation if you don't want to.
3. Tags: Create, edit, and assign tags to memes for better organization and search filtering.
4. Faster Vector Search: Powered by Postgres and pgvector, enjoy faster keyword and vector searches with streamlined database transactions.
5. Keyword Search: Pro adds traditional keyword search in addition to semantic/vector search.
6. Directory Paths: Organize your memes across multiple subdirectories—no need to store everything in one folder.
7. New Organizational Tools: Filter by tags, directory paths, and description embeddings, plus toggle between keyword and vector search for more control.

# Target Audience

This is a toy project. Open source and made for fun.

# Comparison

immich: great open source image organizer
other local photo apps: some allow for indexing but not quite

/r/Python
https://redd.it/1gmkv55