The Pythonic Guide to Logging
https://timber.io/blog/the-pythonic-guide-to-logging/

/r/Python
https://redd.it/8sm80g

Convert Django ORM query with large IN clause to table value constructor


I have a bit of Django code that builds a relatively complicated query in a programmatic fashion, with various filters getting applied to an initial dataset through a series of filter and exclude calls:

for filter in filters:
if filter['name'] == 'revenue':
accounts = accounts.filter(account_revenue__in: filter['values'])
if filter['name'] == 'some_other_type':
if filter['type'] == 'inclusion':
accounts = accounts.filter(account__some_relation__in: filter['values'])
if filter['type'] == 'exclusion':
accounts = accounts.exclude(account__some_relation__in: filter['values'])
...etc

return accounts
For most of these conditions, the possible values of the filters are relatively small and contained, so the IN clauses that Django's ORM generates are performant enough. However there are a few cases where the IN clauses can be much larger (10K - 100K items).

In plain postgres I can make this query much more optimal by using a table value constructor, e.g.:

SELECT domain
FROM accounts
INNER JOIN (
VALUES ('somedomain.com'), ('anotherdomain.com'), ...etc 10K more times
) VALS(v) ON accounts.domain=v

With a 30K+ IN clause in the original query it can take 60+ seconds to run, while the table value version of the query takes 1 second, a huge difference.

But I cannot figure out how to get Django ORM to build the query like I want, and because of the way all the other filters are constructed from ORM filters I can't really write the entire thing as raw SQL.

I was thinking I could get the raw SQL that Django's ORM is going to run, regexp parse it, but that seems very brittle (and surprisingly difficult to get the actual SQL that is about to be run, because of parameter handling etc). I don't see how I could annotate with RawSQL since I don't want to add a column to select, but instead want to add a join condition. Is there a simple solution I am missing?

/r/django
https://redd.it/8sp2p5

Open source Python libraries for Embedded and IoT: sensors, actuators, industrial protocols, cloud services and board definitions
https://www.zerynth.com/blog/python-libraries-for-embedded-and-iot-over-100-repositories-on-zerynth-github/

/r/Python
https://redd.it/8sqhi9

New Django extension for VS Code

I just published version 0.8 of my try to write a very good Django extension for Visual Studio Code!

And at the same time, `vscode-icons` was released with support of icons for Django templates!

[https://marketplace.visualstudio.com/items?itemName=batisteo.vscode-django](https://marketplace.visualstudio.com/items?itemName=batisteo.vscode-django)

I put some effort into syntax, so that we can see tags and variables within HTML `class` or any attributes, highlighting known tags/filters and contribs, adding more scopes etc.

A support for non-HTML templates is also available (called `django-txt`), typically for emails.

From today it is also possible to click through (aka Go To Definiton) on the path in a `{% extends %}` or `{% include %}` tag.

And of course a bunch of snippets, some with support for pasting. Still needs some love and polishing.

[https://github.com/vscode-django/vscode-django](https://github.com/vscode-django/vscode-django)

Next steps:

* Having scoped Python snippets: model fields if in a models(/\*\*).py file, form fields on forms, same for admin…
* Autogenerating HTML snippets from Django introspection, so that’s easier to have up to date the extension
* Better support for Jinja and [Tera](https://github.com/Keats/tera/)
* Make you like it (please submit an issue if it’s not at your taste)

I hope VS Code users will enjoy it

/r/django
https://redd.it/8sqmoy

Using SelectFields with flask-wtforms !HELP!

I'm having issues using Select Field in flask . I'm unable to submit get data from the select fields , also when i use them with normal input fields none of the fields return any data . But the input fields work and return data if I remove the selectFields . Here's the code :

Models and form :

class City(db.Model):
id = db.Column(db.Integer, primary\_key=True)
city = db.Column(db.String(30), unique=True, nullable=False)
state = db.Column(db.String(30), nullable=False)
country = db.Column(db.String(30), nullable=False)
class CityForm(FlaskForm):
city = StringField('city', validators=\[InputRequired()\])
state = SelectField('state' , validators=\[InputRequired()\] , coerce = str)
country = SelectField('country' , validators=\[InputRequired()\] , coerce = str)

the select fields are populated in the views , like this :

form\_city.state.choices = \[(row\[0\],row\[0\]) for row in db.session.query(login\_model.State.state)\]
form\_city.country.choices = \[(row\[0\],row\[0\]) for row in db.session.query(login\_model.Country.country)\]

\~\~\~\~\~

if form\_city.validate\_on\_submit():
mssg = ""
prod = login\_model.City.query.filter\_by(city=form\_city.city.data).first()
print('okaaay')
if prod :
mssg = "Duplicate Data "
flash(mssg)
return redirect(url\_for('basic\_master'))
else:
new\_data = login\_model.City(city=form\_city.city.data.upper())
try:
db.session.add(new\_data)
db.session.commit()
mssg = "Data Successfully added 👍"
flash(mssg)
return redirect(url\_for('basic\_master'))

except Exception as e:
mssg = "Error occured while adding data 😵. Here's the error : "+str(e)
flash(mssg)
return redirect(url\_for('basic\_master'))
Nothing works if I have selectfields in my Form class , and If i remove them the normal input field works .

Any help is much appreciated thanks.

\#flask #sqlalchemy #wtforms #python

/r/flask
https://redd.it/8srquo

How to upload image file via binary file Not using form-data

Hi, I am trying to upload image files using postman binary file. I read \`[request.stream.read](https://request.stream.read)()\` but how can i save this file on specific folder.

/r/flask
https://redd.it/8sq8il

A web application that allows you to test in real time which site is blocked in China.

[ezscale.science](http://ezscale.science)

This is my first django project. I hosted this project on a server in Beijing, since websites have to register domain in compliance with Chinese law, so I redirected to IP address from a remote server in Germany. This site uses ajax calls to remain on the same page and utilize a ping backend to test whether the website you called is blocked.

You can submit your own website to the list to see if they're blocked.

This is only the test phase - I have not yet been able to hook it on to apache - the tutorials online didn't work for me. Mobile doesn't work for real time ajax - I used :hover to show elements but I can't seem to get :active working on mobile.. Also, I'll be very appreciative if you find any bugs! Thanks in advance for any help.

EDIT: I currently reviewing all websites, if the new websites are not random they'll be added to the main lists. You'll be seeing new website you added but they only appear before you refresh the page..

EDIT2: I'm going to sleep and this might suffer a Reddit Hug of Death - I'm running this on a very small ECS. 1GB ram and 1mbps of link. But I'll hopefully get it on Apache tomorrow and if successful I'll enlarge the ECS and officially release it.

/r/Python
https://redd.it/8srnef

PEP 572 and decision-making in Python
https://lwn.net/SubscriberLink/757713/2118c7722d957926/

/r/Python
https://redd.it/8stu3u

Would you benefit from a Django Rest Framework and VueJS tutorial?

Hi everyone,

I have been developing with Django and VueJS for years now and when I started out I had a rough time getting everything linked together and finding the best way to create SPAs. There are some tutorials available right now here and there, but some aren’t up to date with the latest versions and some are just incomplete. I also noticed quite a few questions about that here and on the subreddit djangolearning, which I have been trying to answer as much as possible.

I have had this idea in my mind for a while to build a step-by-step course on building Django and VueJS applications. Before I actually start working on this, I want to ask you:

* Would you benefit from a course like this? Basically holding hands from installing Django to building the application to deploying it.
* Would you be willing to pay for that? I would providing support to any issues you have during the course and keep it up to date with the latest versions. I am aware of Udemy, but this would probably be priced higher than the average Udemy course. I found that a lot of Udemy courses don't get updated and some that I bought weren't great with little to no support.
* Would you want a written version or a video version of this? Or perhaps both?

Again, I am not trying to sell anything here. I am just trying to find out if it would be useful to you. I don’t want to spend countless hours on building a course that no ones is going to use. As for the idea of the course, I was thinking of building a status page with realtime incident and uptime updates through the course, like you have probably seen with almost any tech company. Using Redis, Celery, Django Rest Framework, Django channels, custom admin forms and what not.

Thanks,

Stan

/r/django
https://redd.it/8su1p4

Single Input for Multiple Model Formset Instances

I'm trying to figure out how I would go about creating a form that has a single multiplechoicefield with Author's name, exactly 9 empty text fields for Book names, and a single save button.

The goal being to select the author once, and add 9 book names to the database. This is relatively straight forward if I use 9 author dropdowns with 9 Book text fields. But how would I go about using a single Author dropdown to populate 9 Book instances?

class Author(models.Model):
name = models.CharField(max_length=100)

def __str__(self):
return self.name

class Book(models.Model):
name = models.CharField(max_length=100)
authors = models.ManyToManyField(Author)

class BookForm(ModelForm):
class Meta:
model = Book
fields = ['name', 'authors']

def add_two_books(request):
BookFormSet = modelformset_factory(Book, form=BookForm, extra=9)
formset = BookFormSet(request.POST or None, queryset=Author.objects.none())

/r/djangolearning
https://redd.it/8sqzqg

Writing Python Extensions In Rust Using PyO3
https://www.benfrederickson.com/writing-python-extensions-in-rust-using-pyo3/

/r/Python
https://redd.it/8svfkz

GeoPandas and pandas.HDFStore() method incompatible.
https://www.reddit.com/r/gis/comments/8stckr/geopandas_and_pandashdfstore_method_incompatible/

/r/pystats
https://redd.it/8stg6r

Handling secret keys while sharing repo

Hi there, I've got a bit of stupid question: If I want to create a public django project and hide the secret key and other sensitive information, how do I hide it effectively if I want someone else to be able to run it locally? I mean, if I create a project and then hide its secret key in an env variable on my machine, how do I then allow someone else to run the same project on their machine without putting the secret key in source control?

If I were building a production app I'd just launch on a PAAS provider and set an environment variable on the server in production to be read in with `os.environ.get('SECRET_KEY')`, but in the case of someone cloning my repo they won't have that env variable set. So, how do I get them the key so they can set it, or do they just generate a new one to use in their local development environment somehow?

Thanks!!



/r/django
https://redd.it/8swv3e

Python for Finance?

Hey there, I'm looking to learn financial analysis through python for personal enjoyment/see if I want to pursue it long term. I am an engineering student, decent at python and I understand basic finance/economics, but pretty beginner overall.

Currently, I'm improving my python and finance knowledge concurrently. But I want to see them in action together, and I'm looking for something where I can see how to interact with finance using python, preferably something beginner friendly.

Does anyone have experience with this kinda thing and can make some recommendations?

/r/Python
https://redd.it/8svyjo

How can I present Pandas to my peers?

I am going to hold a seminar on Pandas to my colleagues next week and I am thinking of the best way to introduce the framework and present its most useful features in one hour or little more.

We are all software engineers and we mainly use Java for the backend. We decided to introduce Airflow and Python for wrangling data and producing reports and I was chosen to be a pioneer for this new path so I studied and used Pandas a lot in the last month.

First of all I am wondering what would be more effective for the audience: using a Power Point presentation? or sharing my screen as I pick a dataset and play around with Pandas in a Jupyter notebook?

I am also writing down the topics I think it would be cool to present / demonstrate, such as

* ease of import of a csv or json file
* handling missing data
* groupby, value_counts, aggregation
* features of DateTimeIndex, like selecting by date or by hour of day, resampling, automatic handling of days of the week
* merging two dataframes
* plotting

Do you guys have any suggestions on the best format and what to present?

Thanks!

/r/Python
https://redd.it/8sws81

Made a simple World Cup 2018 Score and Schedule using Selenium webdriver and scraped ESPN World Cup 2018 data

Nothing fancy. Just a simple webscrape using Selenium webdriver and ESPN World Cup 2018 data.

If the game is finished, shows score and who won. If the game has not played yet, shows the scheduled time and date. Also put in flags for each country

[https://github.com/spaceguy01/DataScience/tree/master/ESPN\_World\_Cup\_Score\_Scrape](https://github.com/spaceguy01/DataScience/tree/master/ESPN_World_Cup_Score_Scrape)

If it doesn't show, just click on 'Desktop version' on bottom right

/r/Python
https://redd.it/8sw0h0

AJAX web application. No need for django-webpack-loader, right?

We are trying to build a complex web application - similar to Facebook in some sense. Before we didn't have an API server, so we used the django-webpack-loader and used webpack bundles selectively on Django templates.

Now that we have a full API support for our clients and we are going AJAX for everything, I don't see much value in django-webpack-loader. I am planning to make webpack compile to Django's static directory and then run collectstatic command to push the output files to s3 for CDN.

We are going away with the cached pages because we want a smooth page loading user experience. So the question is, Is there optimization that can be done that I am overlooking here by ditching django-webpack-loader and Django template?

Also if there are hybrid approaches to routing users' page requests via reactJS and Django, please let me know!

/r/django
https://redd.it/8szraj

Install and Configure – Django Virtualenv, Gunicorn, Nginx and Supervisor | All in one Article!
https://opensourceprojects.org/install-and-configure-django-virtualenv-gunicorn-nginx-and-supervisor/

/r/Python
https://redd.it/8sz9qu

Help with displaying object data (use a form or not)?

I have a Django app with a list of tiles on a page, where a user can click on tile to view/edit the details behind it. When clicked there is a side bar menu that slides out so the user can still all of the tiles. I initially used an AJAX request to pull the data into divs, but then when I started thinking about how to make it editable using a form just made more sense. However, I'm now starting to wonder though if this is the really the best way to approach this.

One other issue I'm working through is that each tile can have multiple files attached to it. Using the form approach above, I had to add a button that triggers a modal window load with a list of all of the files and then an upload button within that. So what seemed like a better approach a few days ago I feel is now starting to spiral into a mess.

Is displaying the data via a form the right method or not in this situation? If not, how would I go about showing a list of information but making it easily editable (like clicking an edit button)? Any guidance here is appreciated!

/r/django
https://redd.it/8t1fnq

I made a timelapse of me making a simple game in pyglet after 2 days of learning it
https://www.youtube.com/watch?v=XLFabo0IVho

/r/Python
https://redd.it/8t0esv