HELP Ensuring complete transactions with long running tasks and API requests with SQLAlchemy

Hello, I am having some trouble with my Flask App having to wait long periods of time for to obtain a read write lock on database entries, that are simultaneously being read / written on by long running celery tasks (\~1 minute).

For context, I have a Flask App, and a Celery App, both interacting with the same database.

I have a table that I use to track jobs that are being ran by the Celery app. Lets call these objects JobDBO.

1. I send a request to Flask to create the Job, and trigger the Celery task.

2. Celery runs the job (\~1 minute)

3. During the 1 minute job I send a request to cancel the job. (This sets a flag on the JobDBO). However, this request stalls because the Celery task has read that same JobDBO and is keeping 1 continuous SQLAlchemy session

4. The task finally completes. The original request to cancel the job is fulfilled (or times out by now waiting to obtain a lock) and both the request and celery tasks SQL operations are fulfilled.


Now I understand that this could obviously be solved by keeping short lived sql alchemy sessions, and only opening when reading or writing quickly,

/r/flask
https://redd.it/1ms7bgc

Sunday Daily Thread: What's everyone working on this week?

# Weekly Thread: What's Everyone Working On This Week? 🛠️

Hello /r/Python! It's time to share what you've been working on! Whether it's a work-in-progress, a completed masterpiece, or just a rough idea, let us know what you're up to!

## How it Works:

1. Show & Tell: Share your current projects, completed works, or future ideas.
2. Discuss: Get feedback, find collaborators, or just chat about your project.
3. Inspire: Your project might inspire someone else, just as you might get inspired here.

## Guidelines:

Feel free to include as many details as you'd like. Code snippets, screenshots, and links are all welcome.
Whether it's your job, your hobby, or your passion project, all Python-related work is welcome here.

## Example Shares:

1. Machine Learning Model: Working on a ML model to predict stock prices. Just cracked a 90% accuracy rate!
2. Web Scraping: Built a script to scrape and analyze news articles. It's helped me understand media bias better.
3. Automation: Automated my home lighting with Python and Raspberry Pi. My life has never been easier!

Let's build and grow together! Share your journey and learn from others. Happy coding! 🌟

/r/Python
https://redd.it/1msc4g4

Help me choose Django a Niches.

Hello, guys. I've recently learned Django and python. Now, I want to do freelancing in Upwork but the competition is really high plus I also have no reviews.

So, I want to pick a niche and master it for less competition. The problem is I'm not sure what to choose. Can anyone suggest me? or give me some career advice? TIA

/r/django
https://redd.it/1ms4qy5

Where to Run DB Migrations with Shared Models Package?

I have two apps (A and B) sharing a single database. Both apps use a private shared-models package (separate repo) for DB models.

Question: Where should migrations live, and which app (or package) should run them?

1. Should migrations be in shared-models or one of the apps?
2. Should one app’s CI/CD run migrations (e.g., app A deploys → upgrades DB), or should shared-models handle it?

How have you solved this? Thanks!

/r/flask
https://redd.it/1mscy1u

How many of you use S3 for static files?

I’ve been using Django for years but have a basic newb query. Our frontend is Nextjs. We don’t need static files cached or with an external object store. But having them in S3 has been convenient especially when swapping out environments (we have some custom css/js but very few).

Convenient but does add one more step to DevOps to collect static files and have to manage permissions etc. What do you guys do when not using HTMX, Django Templates, etc?

/r/django
https://redd.it/1mrjiix

What theme and IDE are you using for Python development?

I’ve been curious about the setups people in the community use for Python development. Personally, I feel like the theme and IDE you use can make a big difference in productivity and overall coding comfort.
So I wanted to start a little discussion:
• Which editor/IDE do you usually use for Python (PyCharm, VS Code, Vim, Sublime, Jupyter, etc.)?
• Which theme do you prefer (dark, light, custom setups like Dracula, Monokai, Solarized, etc.)?
• And if you’ve customized your environment a lot, what’s your favourite tweak that makes coding smoother?

For me:
• IDE: I mostly use VS Code and PyCharm.
• Theme: I stick with the default Dark+ theme in VS Code.
• Favourite Setup: Honestly nothing too fancy — I just like keeping it clean and minimal so I can focus on the code.
Curious to see what others are using — maybe I’ll discover a new theme or setup to try out! 🚀

/r/Python
https://redd.it/1mshy89

Creating a web application using Python

Hello Everyone, I need some help with the following ? I am creating a very basic python web application. I will be writing the application in Python , what I have some doubts as how will I run it in a website as MVP. I don't know Angular JS and Javascript.

1. What front end should I use
2. What backend should I use
3. How many components will it take to run the Python application on a website..



/r/Python
https://redd.it/1msmalu

Meerschaum v3.0 released

For the last five years, I’ve been working on an open-source ETL framework in Python called [**Meerschaum**](https://github.com/bmeares/Meerschaum), and version 3.0 was just released. This release brings performance improvements, new features, and of course bugfixes across the board.

# What My Project Does

Meerschaum is an ETL framework, optimized for time-series and SQL workloads, that lets you build and organize your pipes, connectors, and scripts (actions). It's CLI-first and also includes a web console web application.

Meerschaum is extendable with plugins (Python modules), allowing you to add connectors, dash web pages, and actions in a tightly-knit environment.

# Target Audience

* Developers storing data in databases, looking for something less cumbersome than an ORM
* Data engineers building data pipelines and materializing views between databases
* Hobbyists experimenting with syncing data
* Sysadmins looking to consolidate miscellaneous scripts

# Usage

Install with `pip`:

pip install meerschaum

Install the plugin `noaa`:

mrsm install plugin noaa

Bootstrap a new pipe:

mrsm bootstrap pipe -i sql:local

Sync pipes:

mrsm sync pipes -i sql:local

Here's the same process as above but via the Python API:

```python
import meerschaum as mrsm

mrsm.entry('install plugin noaa')

pipe = mrsm.Pipe(
'plugin:noaa', 'weather',
columns={


/r/Python
https://redd.it/1msgnpe

CDC with Debezium on Real-Time theLook eCommerce Data

We've built a Python-based project that transforms the classic [**theLook eCommerce**](https://console.cloud.google.com/marketplace/product/bigquery-public-data/thelook-ecommerce) dataset into a **real-time data stream**.

What it does:

* Continuously generates simulated user activity
* Writes data into PostgreSQL in real time
* Serves as a great source for CDC pipelines with Debezium + Kafka

Repo: https://github.com/factorhouse/examples/tree/main/projects/thelook-ecomm-cdc

If you're into data engineering + Python, this could be a neat sandbox to explore!

/r/Python
https://redd.it/1msoeam

Django tip Hijack Users For Better Customer Support

/r/django
https://redd.it/1mspday

Database transaction and atomic operation

I was reading Django documentation to know exactly how the database transaction and atomic() operation work in Django. I'm not sure if I misunderstood, but does django actually use atomic transaction automatically? and if yes why should someone add them again? if not when should we use them and where exactly?



/r/django
https://redd.it/1msoc6h

XML/PEPPOL/UBL Validator. What would you add?

I'm building a tool that validates XML files and translates parser errors into human-readable messages.

Currently it only checks XML structure, but I want to add PEPPOL/UBL invoice validation next.

Questions:

\- What Python libraries work well for XSD schema validation?

\- Should I add file history/sessions?

\- Any security concerns with file uploads?

/r/flask
https://redd.it/1msrxw0

Help needed

Hey so I was using this library dj-rest-auth, followed the docs carefully, and set up everything as it should.

However I got this error whenever I try to send requests to the /registration endpoint:

### AttributeError at /dj-rest-auth/registration/
#### 'RegisterSerializer' object has no attribute '_has_phone_field'

So my first instinct was to extend the RegisterSerializer built into the library, and change the register serializer in settings.py into my custom serializer:

from rest_framework import serializers
from dj_rest_auth.registration.serializers import RegisterSerializer

class RegSerializer(RegisterSerializer):
    phone = serializers.CharField(required = False)
    
    def get_cleaned_data(self):
        data= super().get_cleaned_data()
        data['phone']=self.validated_data.get("phone","")
        return data

But still, none of this worked, would appreciate some help here :)

/r/django
https://redd.it/1mruz5x

Monday Daily Thread: Project ideas!

# Weekly Thread: Project Ideas 💡

Welcome to our weekly Project Ideas thread! Whether you're a newbie looking for a first project or an expert seeking a new challenge, this is the place for you.

## How it Works:

1. **Suggest a Project**: Comment your project idea—be it beginner-friendly or advanced.
2. **Build & Share**: If you complete a project, reply to the original comment, share your experience, and attach your source code.
3. **Explore**: Looking for ideas? Check out Al Sweigart's ["The Big Book of Small Python Projects"](https://www.amazon.com/Big-Book-Small-Python-Programming/dp/1718501242) for inspiration.

## Guidelines:

* Clearly state the difficulty level.
* Provide a brief description and, if possible, outline the tech stack.
* Feel free to link to tutorials or resources that might help.

# Example Submissions:

## Project Idea: Chatbot

**Difficulty**: Intermediate

**Tech Stack**: Python, NLP, Flask/FastAPI/Litestar

**Description**: Create a chatbot that can answer FAQs for a website.

**Resources**: [Building a Chatbot with Python](https://www.youtube.com/watch?v=a37BL0stIuM)

# Project Idea: Weather Dashboard

**Difficulty**: Beginner

**Tech Stack**: HTML, CSS, JavaScript, API

**Description**: Build a dashboard that displays real-time weather information using a weather API.

**Resources**: [Weather API Tutorial](https://www.youtube.com/watch?v=9P5MY_2i7K8)

## Project Idea: File Organizer

**Difficulty**: Beginner

**Tech Stack**: Python, File I/O

**Description**: Create a script that organizes files in a directory into sub-folders based on file type.

**Resources**: [Automate the Boring Stuff: Organizing Files](https://automatetheboringstuff.com/2e/chapter9/)

Let's help each other grow. Happy

/r/Python
https://redd.it/1mt6qhc

What is a Python thing you slept on too long?

I only recently heard about alternative json libraries like orjson, ujson etc, or even msgspec. There are so many things most of us only learn about if we see it mentioned.

Curious what other tools, libraries, or features you wish you’d discovered earlier?





/r/Python
https://redd.it/1mt5hun

unfold dashboard

I recently integrated django-unfold into my Django admin, and it works great. However, before I discovered Unfold, I had already built my own custom dashboard.

Now I’m wondering:

Is it possible to add my existing dashboard into the Unfold-powered admin?
Or would it be better to just rebuild/replicate the dashboard using Unfold’s features?

Has anyone here tried merging a custom dashboard with Unfold, or is the recommended approach to stick with Unfold’s way of doing things?

/r/django
https://redd.it/1mt8xit

[R] Bing Search API is Retiring - What’s Your Next Move?

/r/MachineLearning
https://redd.it/1msw9vd

Why does Django's documentation look like it's design is stuck in 2010?

Today I decided to start learning backend development in Python, choosing Django as the framework. But honestly, I was absolutely disappointed with the appearance of the documentation.

It feels like the design was never tested from the perspective of a regular user. The dark theme palette is poorly chosen, the text area is unnecessarily small, and to read anything comfortably you constantly need to zoom in. And seriously - who thought it was a good idea to make the font color gray?

The content itself might be fine, but the reading experience is frustrating enough that I couldn't spend more than an hour with it. And in the end, the way the documentation looks completely kills the motivation to stay on the site and continue learning Django

/r/django
https://redd.it/1ms1gu9

Tuitka - A TUI for Nuitka

Hi folks, I wanted to share a project I've been working on in my free time - **Tuitka**

# What My Project Does

Tuitka simplifies the process of compiling Python applications into standalone executables by providing an intuitive TUI instead of wrestling with complex command-line flags.

Additionally, Tuitka does a few things differently than Nuitka. We will use your requirements.txt, pyproject.toml or PEP 723 metadata, and based on this, we will leverage `uv` to create a clean environment for your project and run it only with the dependencies that the project might need.

# Target Audience

This is for Python developers who need to distribute their applications to users who don't have Python installed on their systems.

# Installation & Usage

You can download it via `pip install tuitka`

**Interactive TUI mode:**

tuitka


Since most people in my experience *just* want their executables packaged into onefile or standalone, I've decided to allow you to point directly at the file you want to compile:**Direct compilation mode:**

tuitka my_script.py

The direct mode automatically uses sensible defaults:

* `--onefile` (single executable file)
* `--assume-yes-for-downloads` (auto-downloads plugins)
* `--remove-output` (cleans up build artifacts)

# Why PEP 723 is Preferred

When you're working in a development environment, you often accumulate libraries that aren't actually needed by your specific script - things

/r/Python
https://redd.it/1mteev1

UV python image building does not seem to be completely in sync with python releases

Had a pipeline errors this weekend because of:

```

1.615 error: No download found for request: cpython-3.13.7-linux-x86_64-gnu

```

local testing:

```

uv python install 3.13.7 -v

DEBUG uv 0.8.11 (f892276ac 2025-08-14)

DEBUG Acquired lock for `C:\\Users\\mobj\\AppData\\Roaming\\uv\\python`

DEBUG Released lock at `C:\\Users\\mobj\\AppData\\Roaming\\uv\\python\\.lock`

error: No download found for request: cpython-3.13.7-windows-x86_64-none



uv python install 3.13.6 -v

DEBUG uv 0.8.11 (f892276ac 2025-08-14)

DEBUG Acquired lock for `C:\\Users\\mobj\\AppData\\Roaming\\uv\\python`

DEBUG No installation found for request `3.13.6 (cpython-3.13.6-windows-x86_64-none)`

DEBUG Found download `cpython-3.13.6-windows-x86_64-none` for request `3.13.6 (cpython-3.13.6-windows-x86_64-none)`

DEBUG Using request timeout of 30s

DEBUG Downloading https://github.com/astral-sh/python-build-standalone/releases/download/20250814/cpython-3.13.6%2B20250814-x86\_64-pc-windows-msvc-install\_only\_stripped.tar.gz

DEBUG Extracting cpython-3.13.6-20250814-x86_64-pc-windows-msvc-install_only_stripped.tar.gz to temporary location: C:\\Users\\mobj\\AppData\\Roaming\\uv\\python\\.temp\\.tmpWQNy1c

Downloading cpython-3.13.6-windows-x86_64-none (download) (20.1MiB)

/r/Python
https://redd.it/1mtgpa3

A high-level Cloudflare Queues consumer library for Python

Hey everyone,

I built a high-level Python-based Cloudflare queue consumer package!

Cloudflare has some great products with amazing developer experiences. However, their architecture is primarily built on the V8 runtime, which means their services are optimized for JavaScript.

They do have a beta version of their Workers for Python, but it doesn’t support some key packages that I need for an application I’m working on. So, I decided to build CFQ, to provide an easy interface for consuming messages from Cloudflare Queues in Python environments.

# What My Project Does

Lets you easily consume messages from a Cloudflare queue in pure Python environments.

# Comparison

I couldn’t find many alternatives, which is why I created this package. The only other option was to use Cloudflare’s Python SDK, which is more low-level.

# Target Audience


Developers who want to consume messages from a Cloudflare queue but can’t directly bind a Python-based Worker to the queue.


Github: https://github.com/jpjacobpadilla/cfq


Hope some of you also find it useful!

/r/Python
https://redd.it/1mtivrf