Any must-have extensions for working with Python in VSCode/VSCodium?

Suggestions for extensions that help in general and not just with Python, are also welcome.

/r/Python
https://redd.it/13h2xuc

Multiparadigmatic Web Scraping Tool!

Hello! Hope you're all doing fine! I recently developed a multiparadigmatic web scraping tool with web crawling included as you can define the max hops it takes inside every link it finds inside a webpage!

It uses Async/IO to give maximum performance! Wanted to use multithreading to boost the HTML Parsing part since it's more of a CPU Bound part, but I benchmarked using multiprocessing and the overhead of creating multiple processes just for parsing wasn't worth it, since I'm quite experienced with C I might create a simple wrapper around this tool.

It's great to embed in a Browser Extension or just for any general purpose data mining on the web you might need. Instead of writing purpose specific scrapers for that particular web page you can just use this library.

Give me your sincere opinion since it is my first time publishing and authoring a python library, and it's my first OSS getting actually 16 stars. With this post I hope to get good insight and to learn and improve my python knowledge! My first programming language was C!
I know I got to automate the publishing process with GitHub Actions, the same way I have it for

/r/Python
https://redd.it/13hkrgc

Release 0.6.0 of FastKafka adds Redditors' requested features

FastKafka is a powerful and easy-to-use Python library for building asynchronous web services that interact with Kafka topics. Built on top of Pydantic, AIOKafka and AsyncAPI, FastKafka simplifies the process of writing producers and consumers for Kafka topics and automatically generates documentation for such microservices.


This release contains s number of feature requests from Redditors. Please take a look and let us know how we can make it better for you and your use cases.

https://github.com/airtai/fastkafka

/r/Python
https://redd.it/13i0eaz

A short bugfix story about UnicodeDecodeError: 'utf-8' codec can't decode byte

Hi everyone. Hope you are doing well.

I just wanted to share a short story with you about my yesterday's journey of bug fixing.

I am not a professional programmer but I have a working live webservice made on flask.

It appears, that when return render_template("about.html") happens, inside Flask there is some check happens whether passed html code has utf-8 symbols or not.

Error was: UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position ... which was hard to debug because track trace was pointing out to my return render_template code and that was it. Position error was pointing to something inside a Flask files, not in my html. Other errors inside track trace were happening also inside Flask files.

What I normally do, when I update my code locally in PyCharm, I push it to private github and then just copy a text version of a file (code) to my production server via WinSCP, just via editing same file on server. I don't do a lot of changes so it is just easier for me, I guess.

So when I did slight change to my about.html on my local machine when everything worked fine, I copied code, opened remote SSH connection via WinSCP to

/r/flask
https://redd.it/13fe6rt

How do you maintain a development environment as well as a production version of your Django projects?

New to Web Dev and Django. I have created my first Django project and have deployed it to a website that I own. I am struggling to make a nice workflow to allow me to both develop and test my app, as well as deploy it when I am happy.

My current workflow is to write new code in the production version of my app which is currently deployed to my website. I write the code locally on VS Code, push to Gitlab, jump onto my server, and pull the latest code. This takes a while and isn't a nice way to develop and test quickly.

I can't seem to find any guidance online about managing and developing a running website. Does anyone have any suggestions on improving my workflow, and maintaining a separate space for development?

/r/django
https://redd.it/13i9h19

A Python package to retrieve data from Steam

Hi guys, I recently just build this lightweight package called steamcrawl after seeing that people are having trouble accessing/querying certain information on Steam. It simply makes a query to Steam, sorts the important information from the messy JSON then returns a pandas DataFrame. Here are some features that my package has been able to do so far:

\- Get your listing/buying/selling/cancelling on the community market.
\- Get all app id
\- Get the price history of an item.
\- Get the item id of an item.
\- Get buy/sell orders of an item.
...and more.

If you are interested, please give it a visit at: https://github.com/Hungreeee/steamcrawl
Note: the code is completely open, you can freely look into it or take it for personal use, I really don't mind that. I think it can be useful for educational purposes, especially for anyone who is trying to crawl Steam data but just doesn't know where to start.

/r/Python
https://redd.it/13i7oyy

I interviewed Joris Schellekens, author of the Borb PDF library, about his progress on building a business around an open source library. “If you want a library that is regularly updated by a team of developers, someone needs to pay those developers.”
https://codecodeship.com/blog/2023-05-15-joris-schellekens?utm_source=reddit

/r/Python
https://redd.it/13i6wo4

Scaling Django Applications: Best Practices and Strategies

Hey everyone! 😊

I'm excited to share my latest blog post with you all - "Scaling Django Applications: Best Practices and Strategies". I've attempted to be as precise as possible, and not too verbose. However, if you're interested in diving deeper into any specific topic, let me know and I'll consider writing another article with more detail.

In this comprehensive guide, I delve into the world of scaling Django applications, covering everything from understanding scalability, identifying bottlenecks, database strategies, caching, to best practices. It's a lot, but remember - every big journey begins with a single step.

So, whether you're a seasoned Django developer or just starting out, I hope you'll find some value in it. And remember, the path to success is always under construction. So, let's keep learning and growing together!

You can read the full article here: https://danielbuilescu.com/blogs/learn-python/scaling-django-applications-best-practices-and-strategies

/r/django
https://redd.it/13iaqzg

May 2023 Grants
https://www.djangoproject.com/weblog/2023/may/15/may-2023-grants/

/r/django
https://redd.it/13ipnkb

Tuesday Daily Thread: Advanced questions

Have some burning questions on advanced Python topics? Use this thread to ask more advanced questions related to Python.

If your question is a beginner question we hold a beginner Daily Thread tomorrow (Wednesday) where you can ask any question! We may remove questions here and ask you to resubmit tomorrow.

This thread may be fairly low volume in replies, if you don't receive a response we recommend looking at r/LearnPython or joining the Python Discord server at https://discord.gg/python where you stand a better chance of receiving a response.

/r/Python
https://redd.it/13ip05k

Introducing seaborn-polars, a package allowing to use Polars DataFrames and LazyFrames with Seaborn

In the last few months I've been using Polars more and more with only major inconvenience being that when doing exploratory data analysis, Polars dataframes are not supported by any of the common plotting packages. So it was either switching to Pandas or having a whole lot of boilerplate. For example, creating a scatterplot using pandas df is simply:

import seaborn as sns
sns.scatterplot(df, x='rating', y='votes', hue='genre')

But with Polars you'd have to do:

x = df.select(pl.col('rating')).to_numpy().ravel()
y = df.select(pl.col('votes')).to_numpy().ravel()
hue = df.select(pl.col('genre')).to_numpy().ravel()
sns.scatterplot(x=x, y=y, hue=hue)

That's quite a lot of boilerplate so I wrote this small package that is a wrapper around seaborn plotting functions and allows for them to be used with Polars DataFrames and LazyFrames using the same syntax as with Pandas dfs:

import polars as pl
import seaborn_polars as snl

df = pl.scan_csv('data.txt')
snl.scatterplot(df, x='rating', y='votes', hue='genre')

The code creates a deepcopy of the original dataframe so your source LazyFrames will remain lazy after plotting.

The package is available on PyPI: https://pypi.org/project/seaborn-polars/

If you want to contribute or interested in source code, the repository is here: https://github.com/pavelcherepan/seaborn\_polars

/r/Python
https://redd.it/13ittrl

Django 5.x Steering Council Election Results
https://www.djangoproject.com/weblog/2023/may/16/django-5x-steering-council-election-results/

/r/django
https://redd.it/13ixoyt

Python Flask routing return 404 in my hosting (Centos 7 Cpanel)



Hi,

I’ve got an issue in terms of using send_from_directory in my app.py :

​

from flask import Flask, render_template, request, jsonify, send_from_directory

import json

app = Flask(__name__)

u/app.route("/")

def home():

return render_template("index.html")

u/app.route("/newIndex", methods=["GET", "POST"])

def newIndex():

Appdata = extractApp(appUrl)

json.dumps(Appdata)

filename = Appdata["nama_file"].replace("./static/assets/save\\", "")

permitted_directory = "static/assets/save/"

return send_from_directory(

directory=permitted_directory, path=filename, as_attachment=True

)

u/app.route("/riwu")

def riwu():

return "<h1>ini test saja 123</h1>"

if __name__ == "__main__":

`app.run`(debug=True)

​



However, when i deploy to my hosting (using centos 7 in Cpanel), the @app.route(“/”) it’s works as well as if i type manually in http://[my_website\]/riwu. but not for @app.route(“/newIndex”).

it shows error “Not found : The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.”.

I already setup the .htaccess and httpd.conf to change value from :

​



<Directory /var/www/html/public>

...

AllowOverride None

...

</Directory>

​

to

​



<Directory /var/www/html/public>

...

AllowOverride All

...

</Directory>

​



but still does not work.

I’ve tried in my local server using Visual Code (Windows 10) and it’s works normally.

note : @ app.route(“/newIndex”) is does not have any redirect new .html , It just to download file from send_from_directory.

​

is there any configuration that i missed?

Thanks

/r/flask
https://redd.it/13j7jv3

I made a small Website where everyone can leave a little note. I am not the most experienced, so this project was a lot of fun.

/r/flask
https://redd.it/13imus9

Python’s super() considered super! (Raymond Hettinger)
https://rhettinger.wordpress.com/2011/05/26/super-considered-super/

/r/Python
https://redd.it/13j39al

RPython-based emulator speeds up RISC-V simulation over 15x

RPython, the python sublanguage behind PyPy, has been used to create a RISC-V emulator. It beats the C version by over 15x when booting linux. https://www.pypy.org/posts/2023/05/rpython-used-to-speed-up-risc-v-simulation-over-15x.html

/r/Python
https://redd.it/13j9i06

Wednesday Daily Thread: Beginner questions

New to Python and have questions? Use this thread to ask anything about Python, there are no bad questions!

This thread may be fairly low volume in replies, if you don't receive a response we recommend looking at r/LearnPython or joining the Python Discord server at https://discord.gg/python where you stand a better chance of receiving a response.

/r/Python
https://redd.it/13jlfti

Easiest way to add write-through cache for authuser queries?

Hello! I'm new to django. I have an existing app with quite a lot of scale, and I'm seeing that simple reads from the auth\user table (ie SELECT .. WHERE id=$1) account for a huge amount of load on database. I'd like to put a write-through cache in front of that. It's very read heavy so I think it should effectively reduce that load to near-zero.

What's the easiest path to do this? Is there any way to override just the queries for the select and the insert/update? I've tried googling, but there are a lot of similar topics that make it hard to hone in on the best path, so I thought I'd bring it to y'all :-)

/r/django
https://redd.it/13jbzs1

Flask hosting for specific purpose

Hello. I am sorry if this comes off as an oblivious, and perhaps frequently asked, question. I am quite new to this, and am overwhelmed with the different options.

For my bachelor project, I will need to post a specialized survey, which I have created as a flask application, containing the survey questions, as well as some background code to show randomized context each time the survey is taken, and then storing the survey responses.

Right now, I simply have the server hosted on a free Repl.it server, but as the application will need to properly manage upwards of 300-500 concurrent requests at a time, I will need to host it on a more capable server to avoid latency problems/errors, but that allows for the survey responses to be stored and continuously updated, either in a dedicated database, or simply writing them into a JSON file or something like that.

As such, I am looking for the best/simplest place to host this application properly. I won't expect this to be entirely free, but am hoping that it is not too expensive, especially since the website only needs to be up for a few weeks.

/r/flask
https://redd.it/13jdslm

Issues with form submission and form.validate_on_submit()

I am having issues with a form submission using wtforms and form.validate_on_submit(). Here is the code from the following files:

myapp/forms.py:

from flask\_wtf import FlaskForm
from wtforms import StringField, PasswordField, BooleanField, SubmitField
from wtforms.validators import DataRequired

class LoginForm(FlaskForm):
username = StringField('Username', validators=[DataRequired()])
password = PasswordField('Password', validators=[DataRequired()])
remember_me = BooleanField('Remember Me')
submit = SubmitField('Sign In')



myapp/routes.py:

from flask import render_template, flash, redirect, url_for
from myapp import app
from myapp.forms import LoginForm

@app.route('/login', methods=['GET', 'POST'])
def login():
if current_user.is_authenticated:
return redirect(url_for('index'))

form = LoginForm()

if form.validate_on_submit():
flash("{}".format(form.username.data))
return redirect(url_for('index'))

flash("setup")
return render_template('login.html', form = form)




myapp/templates/login.html:

{% extends "clientbase.html" %}

{% block content %}
<main id="main">
<section id="breadcrumbs" class="breadcrumbs">
<div class="container">
<div class="d-flex justify-content-between align-items-center">
<h2>Login</h2>
<ol>
<li><a href="{{ url_for('index') }}">Home</a>
<li>Login</li>
</ol>
</div>
</div>
</section>
<section>
<div class="container">
<div class="row">
<div class="col-lg-11 col-md-4 text-lg-left">
<h3 data-aos="fade-up">Login</h3>
{%

/r/flask
https://redd.it/13jd01c