Dendrite: Interact with websites with natural language instead of using css selectors

What my project does:

Dendrite is a simple framework for interacting with websites using natural language. Interact and extract without having to find brittle css selectors or xpaths like this:

browser.click(“the sign in button”)

For the developers who like their code typed, specify what data you want with a Pydantic BaseModel and Dendrite returns it in that format with one simple function call. Built on top of playwright for a robust experience. This is an easy way to give your AI agents the same web browsing capabilities as humans have. Integrates easily with frameworks such as  Langchain, CrewAI, Llamaindex and more. 

We are planning on open sourcing everything soon as well so feel free to reach out to us if you’re interested in contributing!

Github: https://github.com/dendrite-systems/dendrite-python-sdk

Overview

Authenticate Anywhere: Dendrite Vault, our Chrome extension, handles secure authentication, letting your agents log in to almost any website.
Interact Naturally: With natural language commands, agents can click, type, and navigate through web elements with ease.
Extract and Manipulate Data: Collect structured data from websites, return data from different websites in the same structure without having to maintain different scripts.
Download/Upload Files: Effortlessly manage file interactions to and from websites, equipping agents to handle documents,

/r/Python
https://redd.it/1gkg23q

Avoid Counting in Django Pagination
https://testdriven.io/blog/django-avoid-counting/

/r/django
https://redd.it/1gk8ubx

ParScrape v0.4.7 Released

# What My project Does:

Scrapes data from sites and uses AI to extract structured data from it.

# Whats New:

* BREAKING CHANGE: --pricing cli option now takes a string value of 'details', 'cost', or 'none'.
* Added pool of user agents that gets randomly pulled from.
* Updating pricing data.
* Pricing token capture and compute now much more accurate.
* Faster startup

# Key Features:

* Uses Playwright / Selenium to bypass most simple bot checks.
* Uses AI to extract data from a page and save it various formats such as CSV, XLSX, JSON, Markdown.
* Has rich console output to display data right in your terminal.

# GitHub and PyPI

* PAR Scrape is under active development and getting new features all the time.
* Check out the project on GitHub or for full documentation, installation instructions, and to contribute: [https://github.com/paulrobello/par\_scrape](https://github.com/paulrobello/par_scrape)
* PyPI [https://pypi.org/project/par\_scrape/](https://pypi.org/project/par_scrape/)

# Comparison:

I have seem many command line and web applications for scraping but none that are as simple, flexible and fast as ParScrape

# Target Audience

AI enthusiasts and data hungry hobbyist

/r/Python
https://redd.it/1gkhl3c

How to Integrate Tailwind with Django
https://www.freecodecamp.org/news/how-to-integrate-tailwind-with-django/

/r/django
https://redd.it/1gkse82

Any solutions

/r/IPython
https://redd.it/1gkq5mg

D Want to move away from coding heavy ML but still want to complete the PhD

Hi Folks,

I come from a tradition electrical engineering background doing things like industrial automation and computer vision. I decided to pursue a PhD in ML as I thought it will be a good field to enter given my past experience. Now I have been doing the PhD for the past three years. While I like my group and research, I am getting discouraged/depressed by (1) The publication rat race (2) post graduation opportunities mostly being coding heavy (3) the inability to carve a name for myself in the field given how crowded the field has become.

Thus, ideally I would like to complete my PhD and move into a more relaxed paced (even if it is not as high paying as ML jobs) non coding heavy but technical job, where I do not have to constantly up-skill myself. Do you folks have any suggestion on what jobs I can look into or would you suggest dropping the PhD and doing something else?

TLDR: 4th year ML PhD student unsure of sticking with the PhD as they desire a non coding heavy technical job in the industry post graduation. Seeking advice on what to do.

/r/MachineLearning
https://redd.it/1gkx6o7

Prototyping with Nanodjango, uv and ninja
https://www.youtube.com/watch?v=0-iuJgfQMOw

/r/django
https://redd.it/1gkzl07

Thursday Daily Thread: Python Careers, Courses, and Furthering Education!

# Weekly Thread: Professional Use, Jobs, and Education 🏢

Welcome to this week's discussion on Python in the professional world! This is your spot to talk about job hunting, career growth, and educational resources in Python. Please note, this thread is not for recruitment.

---

## How it Works:

1. Career Talk: Discuss using Python in your job, or the job market for Python roles.
2. Education Q&A: Ask or answer questions about Python courses, certifications, and educational resources.
3. Workplace Chat: Share your experiences, challenges, or success stories about using Python professionally.

---

## Guidelines:

- This thread is not for recruitment. For job postings, please see r/PythonJobs or the recruitment thread in the sidebar.
- Keep discussions relevant to Python in the professional and educational context.

---

## Example Topics:

1. Career Paths: What kinds of roles are out there for Python developers?
2. Certifications: Are Python certifications worth it?
3. Course Recommendations: Any good advanced Python courses to recommend?
4. Workplace Tools: What Python libraries are indispensable in your professional work?
5. Interview Tips: What types of Python questions are commonly asked in interviews?

---

Let's help each other grow in our careers and education. Happy discussing! 🌟

/r/Python
https://redd.it/1gld3ic

An article on lazy fetching in Django

I published my article about lazy fetching today on medium for Django developers especially those new to the framework. I wrote everything based on personal experience.

https://medium.com/@mikyrola8/understanding-lazy-fetching-in-django-a-deep-dive-8159c4822cd4

/r/django
https://redd.it/1gl4ajx

Keep your code snippets in README up-to-date!

# Code-Embedder

Links: GitHub, GitHub Actions Marketplace

What My Project Does

Code Embedder is a GitHub Action that automatically updates code snippets in your markdown (README) files. It finds code blocks in your README that reference specific scripts, then replaces these blocks with the current content of those scripts. This keeps your documentation in sync with your code.

✨ Key features

🔄 Automatic synchronization: Keep your README code examples up-to-date without manual intervention.
🛠️ Easy setup: Simply add the action to your GitHub workflow and format your README code blocks.
📝 Section support: Update only specific sections of the script in the README.
🧩 Object support: Update only specific objects (functions, classes) in the README. The latest version v0.5.1 supports only 🐍 Python objects (other languages to be added soon).

Find more information in GitHub 🎉

Target Audience

It is a production-ready, tested Github Action that can be part of you CICD workflow to keep your READMEs up-to-date.

Comparison

It is a light-weight package with primary purpose to keep your code examples in READMEs up-to-date. MkDocs is a full solution to creating documentation as a code, which also offers embedding external files. Code-Embedder is a light-weight package that can be used for projects with or without MkDocs. It offers additional functionality to sync not only full scripts, but also

/r/Python
https://redd.it/1gl1hla

Meerkat: Monitor data sources and track changes over time from the terminal

What My Project Does

Meerkat is a fully asynchronous Python library for monitoring data sources and tracking changes over time. Inspired by the constant watchfulness of meerkats, this tool helps you stay aware of shifts in your data—whether it’s new entries, updates, or deletions. Originally created to track job postings, it’s designed to handle any type of data source, making it versatile for various real-world applications.

Meerkat’s CLI module provides an easy way to view changes in your data as text in the terminal, which is especially useful for quickly setting up simple visualizations. However, Meerkat isn’t limited to logging: it can be used to trigger any arbitrary actions in response to data changes, thanks to its action executor. This flexibility lets you define custom workflows, making it more than just a data logger.

Meerkat comes with an example use case—tracking job postings—so you can get a quick start and see the library in action (though you will need to implement the job fetchers yourself).

Target Audience

Meerkat is ideal for developers who need efficient, lightweight tools for monitoring data sources. It’s well-suited to hobby projects, prototyping, or small-scale production applications where regular change detection is required.

Comparison

I’m not aware of a direct comparison, but if

/r/Python
https://redd.it/1glbo1o

Beginner web dev, I need some help understanding something regarding Flask and Angular

Hello everyone. I'm sorry in advance if this belongs on the Angular subreddit, I'll be posting it there as well. I'm a (very) rookie web dev and I'm currently trying to build a website with a Flask backend and Angular frontend. My group and I currently have a very basic Flask app up and running, but I wanted to get started on the Angular early so that we can make the website look good while we work instead of all at the end.

However, I'm very confused as to how I'm supposed to "link" (for lack of a better word) flask and angular together. That is, use Flask and Angular on the same project, same HTML files, etc. I've found this video but that seems to be for an earlier version of Angular, as the overall file structure is different since Angular doesn't automatically make modules anymore, and there's no "dist" folder being made. I also found this reddit post but I can't really make heads or tails of it, and I dont even know if that's even what im looking for in the first place.

The attached picture is our current file structure, currently the angular stuff is all

/r/flask
https://redd.it/1glifpf

Talk Python has moved to Hetzner

See the full article. Performance comparisons to Digital Ocean too. If you've been considering one the new Hetzner US data centers, I think this will be worth your while.

https://talkpython.fm/blog/posts/we-have-moved-to-hetzner/

/r/Python
https://redd.it/1glixwh

Whispr: A multi-vault secret injection tool completely written in Python

What My Project Does ?

Whispr is a CLI tool to safely inject secrets from your favorite secret vault (Ex: AWS Secrets Manager, Azure Key Vault etc.) into your app's environment. You can run a local web server or application with secrets (DB credentials etc.) pulled from a secure vault only when needed. It avoids storing secrets in `.env` files for local software development.

Project link: https://github.com/narenaryan/whispr

Whispr is written completely in Python (100%)

Target Audience: Developers & Engineers

Comparison: Whispr can be compared to client SDKs of various cloud providers, but with extra powers of injection into app environment or standard input.

/r/Python
https://redd.it/1gljize

A Python script to gain remote access to Metasploitable.

A Python script to connect to a Metasploitable machine using SSH and FTP protocols. This tool allows users to execute commands interactively over SSH and manage files via FTP.

Remote\_Access

/r/Python
https://redd.it/1glgg9x

Easily Customize LLM Pipelines with YAML templates.

Hey everyone,

I’ve been working on productionizing Retrieval-Augmented Generation (RAG) applications, especially when dealing with data sources that frequently change (like files being added, updated, or deleted by multiple team members).

For Python devs who aren’t deep into Gen AI, RAG is a common way to extend Gen AI models by connecting them to external data sources for info beyond their training data. Building a quick pilot is often straightforward, but the real challenge comes in making it production-ready.

However, spending time tweaking application scripts is a hassle. For example, if you have swap a model or change the type of index.

To tackle this, we’ve created an open-source repository that provides YAML templates to simplify RAG deployment without the need to modify code each time. You can check it out here: [llm-app GitHub Repo](https://github.com/pathwaycom/llm-app).

Here’s how it helps:

* **Swap components easily**, like switching data sources from local files to SharePoint or Google Drive, changing models, or swapping indexes from a vector index to a hybrid index.
* **Change parameters in RAG pipelines via readable YAML files.**
* **Keep configurations clean and organized**, making it easier to manage and update.

For more details, there’s also a [blog post](https://pathway.com/blog/llm-yaml-templates) and a [detailed guide](https://pathway.com/developers/user-guide/llm-xpack/yaml-templates) that explain how to customize the templates.

This

/r/Python
https://redd.it/1glq4jd

Keeping a thread alive

I never thought simply running a task every hour would turn out to be an issue.

Context: I have a Flask API deployed to a Windows machine using IIS w/ wfastcgi and I want the program to also run a process every hour.

I know I can just use Task scheduler through windows to run my Python program every hour, but I spent all this time merging my coworker’s project into my Flask api project and really wanted only a single app to manage.

I thought at the start of the program it could be executed, but I realized I had multiple workers and so multiple instances would start, which is not okay for the task.

So I created an api endpoint to initiate the job, and figured it could run a thread asynchronously where this asynchronous thread would run a “while True:” loop where the thread would sleep for an hour in between executions… but when I ran the program it would never restart after an hour, and from the logs it is clear the thread is just stopping - poof!

So I figure what about 15 minutes?? Still stops.

What about 1 minute? Success!

So I get the clever idea to make the

/r/Python
https://redd.it/1glpfj5

Django Cotton: Bringing component-based design to Django templates.


* [https://django-cotton.com/](https://django-cotton.com/)
* [https://github.com/wrabit/django-cotton](https://github.com/wrabit/django-cotton)
* **Creator:** [https://x.com/willrabbott/status/1813558563610227073](https://x.com/willrabbott/status/1813558563610227073)

**Does this package look promising?**

/r/django
https://redd.it/1gllbup

Experienced Django devs, I have a question: How can I improve my database design skills?

Help Needed with Database Design

Hey everyone,

I’m really struggling with database design. It’s been a while since I started working on a small social media app using Flutter and Firebase. The app includes features like news, comments, likes, dislikes, and user rankings based on likes and dislikes. I managed to write about half of the project, but then I realized that my data model was flawed. I became stuck trying to figure out how to implement batch write for likes and dislikes, so I ended up abandoning the project after three months.

Now, a friend of mine has asked me to create a web app for diet tracking. I’m fairly comfortable with Django, as I've completed a couple of projects using it. However, this new project feels quite large, and I’m worried that I might get confused again in the middle of development.



How can I improve my database design skills? Is it okay to use tools like ChatGPT for assistance? I tried it once before, but the outcome was a complete mess.



Thank you very much for any advice!

/r/django
https://redd.it/1glnwi1

Multi-Tenancy with Separate Databases ?



i have an application and it's using django and react js and in this application i want to do school management for mutliple school but i want to make for each school it's own database that gonna be created automatic of course there will be an admin superior now what i want to do is to tell me what do you understand by this, and for each school her own database will be created like it gonna have the same structure same tables like teacher,student,planning,absence ,.... and other tables

what are the possible solutions please

/r/django
https://redd.it/1glt7l7