Complete scrapy tutorial : get data from reddit, amazon, pexel, steam, airbnb
https://tanpham.org/

/r/Python
https://redd.it/7ebcd9

Deploying a Flask app into Docker Swarm (xpost /r/python)

[Link first because this is what everyone wants](https://github.com/justanr/flaskbb-swarm)

I recently set up a docker swarm in my home lab and I realized that just putting tooling (traefik, portainer, registry, etc) on it was silly. So I decided to document how I went about deploying a non-trivial application: [FlaskBB](https://github.com/sh4nks/flaskbb) -- which disclosure I'm a maintainer on. I'm also defining "non-trivial" as has multiple moving parts that work together in interesting ways (for some definition of interesting) -- in this case, there is the actual application container, a celery container, redis, a postgres database and finally [an nginx container](https://i.imgflip.com/1z9w62.jpg).

The entrypoint is Traefik running as a Swarm service, which then hands off traffic to the Nginx container which decides to serve either static content or pass to the uWSGI container (in the future, I plan on using frontend rules to partition this on the Traefik layer rather than the nginx layer). Docker networks are used to partition which services are allowed to talk to each other. Secrets and Configs are used to separate configuration logic from the code running in containers.

The nuts and bolts are specific to FlaskBB but hopefully can provide a template for usage with other applications. I tried to be thorough without delving down rabbit holes too much, but let me know if I missed something or should've been clearer some where. :)

/r/flask
https://redd.it/7cmwpz

Automatically downloading PDF using Django running code on multiple ports (X-post from /r/djangolearning)

When a user uploads a PDF, it is modified using ReportLab and the modified PDF is automatically downloaded. Below is my code for processing the files and returning them.

def process_report(request):

# Handle file upload
if request.method == 'POST':
report_file = request.FILES.pop('file')[0]

merged_pdf, namefinal = pdfAnalyzingFunction(report_file)

response = HttpResponse(content_type='application/pdf')

response['Content-Disposition'] = 'attachment; filename="{}"'.format(namefinal)
response.write(merged_pdf.getvalue())

return response

else:
return HttpResponseBadRequest('Must be Post')

I am using Django to render and return content. The above code worked and was able to download files automatically when I just had some simple HTML on a localhost:8000 port. However, after building a frontend on a different port, localhost:3000, I was able to successfully run and generate a modified PDF but it no longer downloaded automatically (with the PDF modifying code running on the localhost:8000 port still).

On my localhost:3000 port network tab I can see the uploaded file, and am wondering why it is not downloading.

Request URL: http://localhost:8000/process_report
Request Method : POST
Satus Code: 200 OK
Host: localhost:8000
Origin: http://localhost:3000
Referer: http://localhost:3000/

Unclear as to why it appears my PDF is rendering, but not automatically downloading now. Is there something I need to know about generating a response object on one port and attempting to download it automatically on a separate port?


/r/django
https://redd.it/7ees5t

Powerpoint to PDF Converter with python
https://www.youtube.com/attribution_link?a=uS_0zNR_JZw&u=%2Fwatch%3Fv%3DoWbVobteULo%26feature%3Dshare

/r/Python
https://redd.it/7efeee

Abstract Classes

Im not sure I fully understand the usages of abstract classes. I know that they are useful when you want subclasses to inherit repeating fields, but how does this make working with repeating fields useful despite just making you type less? What kind of special actions can you perform on abstract classes? Am I missing something

/r/django
https://redd.it/7efjuk

[AF] Passing Python variables on the server to React component

Hi folks, a bit late to the party.

A bit of background: I'm practising with Flask + React in [this repo](https://github.com/CamiloGarciaLaRotta/fullstack_app_boilerplate/tree/vars_as_props).

Here is my issue on [SO](https://stackoverflow.com/questions/47256340/flask-passing-python-variables-on-the-server-to-react-component?noredirect=1#comment81469485_47256340).

But I figured I could ask you guys, in a more general sense, what are the heuristics when one wants to pass variables from the server side to a component at the first rendering of the html template:

* Do devs usually avoid this situation altogether?
* Is it advised to use other component mounting approaches for other than through a main index.jsx?
* What are the DO's and DONT's when working in a mixed environment like this? (i.e. Flask using templates and all the jsx components linked to the html in a single bundle)



Thank you very much in advance,

any advice is appreciated


/r/flask
https://redd.it/7cmsb3

[N] GCP lowers GPU pricing by up to 36%: K80 at 40c/hr and P100 at $1.46/hr
https://cloudplatform.googleblog.com/2017/11/new-lower-prices-for-GPUs-and-preemptible-Local-SSDs.html

/r/MachineLearning
https://redd.it/7eckwf

JWT Authorization in Flask

Hello guys! I just published a tutorial on how you can create JWT Authorization in Flask. Enjoy and ask any question if you have :)

https://codeburst.io/jwt-authorization-in-flask-c63c1acf4eeb

/r/flask
https://redd.it/7eikat

What's everyone working on this week?

Tell /r/python what you're working on this week! You can be bragging, grousing, sharing your passion, or explaining your pain. Talk about your current project or your pet project; whatever you want to share.


/r/Python
https://redd.it/7ei2o7

Best strategies for incorporating data from a RESTful api: forms vs serializers?

**tl;dr**
When pulling data into my django database from an api, should I use forms, or a [django rest framework](http://www.django-rest-framework.org/) serializer?

**What I did**
Based on some discussion in [another thread](https://www.reddit.com/r/django/comments/7e0olm/how_to_save_json_api_response_in_database/), I built an ultra-simple example of how to extract data from an api, in this case the api for the video game Destiny 2:
https://github.com/cortical-iv/restit

Note this is essentially a port of UltimateDjango's [How to Consume REST APIs with Django & Python Requests](https://ultimatedjango.com/blog/how-to-consume-rest-apis-with-django-python-reques/).

My goal was to follow Section 12.1 of Two Scoops, and *Validate all incoming data with django forms.* But what I actually did was a two step process that I'm now thinking was sort of dumb (I pasted the code below the fold):

1. Bind data about path parameters in the url into a form.
2. After extracting the needed data from the response to `requests`, use a django rest framework *serializer* to validate and save that data.

Now that this is up and working, I realize I didn't actually really follow the advice from Two Scoops of importing through forms. I imported required *path* information (user name) through a form, but then did all the heavy lifting with the serializer. I now think it would be better to replace step 2 with another form (a SaveUser form) that itself includes all the fields I need in my database.

**Why I don't like what I did**
Why am I thinking my approach smells? A couple of reasons:

1. The serializer is overkill. I shouldn't need to invoke the django rest framework to do what I am doing. Why not stick with django forms until I need something more powerful?
2. I am doing so much processing on the Destiny 2 end points, that I can't even just apply the serializer to the output of `requests`. This may not be obvious from the simple example at my project.
3. Forms are awesome and I'm not even using them. Once I make some fields unique, calling `serializer.save()` just triggers a validation error. OTOH, using forms, you can feed it an instance to simply *update* the db row without having to do a lot of extra work. Forms have lots of cool stuff like this so it isn't clear, again, why a serializer from DRF is that helpful.

On the other hand, I am a django noob, so maybe there are benefits to pulling data in via a serializer that I don't appreciate. So I'm asking for advice: when pulling data from an api into a db (using `requests`), should I use a serializer, or a form? If a form, which I'm starting to think is the right answer, I'm a little confused about why the project I copied, from UltimateDjango, even used a serializer?

What conditions for that embedly api consumer made it smarter in his case, but not my case?

-----
Here is the `save_user` function I'm using, which contains the main logic:

def add_user(save_user_form):
"""
Validate form data containing 'display_name', request their info from server,
and pull in via a serializer. Returns dictionary with two keys:
'flag': 1 if successful, 0 if not, and None if that user is not a D2 player in PS4.
'message': message about the outcome (success, validation error, etc).
"""
add_result = {'flag': 0, 'message': ''}
if save_user_form.is_valid():
user_name = save_user_form.cleaned_data['display_name']
search_url = search_destiny_player_url(user_name)
search_response = requests.get(search_url, headers = {"X-API-Key": D2_KEY})
user_data = search_response.json()['Response']
if user_data:
submission_data = {'display_name': user_data[0]['displayName'], 'user_id': user_data[0]['membershipId']}
try:
serializer = SearchPlayerSerializer(data = submission_data)
print(repr(serializer)) #for debu

gging
if serializer.is_valid():
serializer.save() #save data to database
add_result['flag'] = 1
add_result['message'] = 'Successfully added {0}'.format(user_name)
else:
msg = "Invalid serializer. serializer.errors: {0}".format(serializer.errors)
add_result['message'] = msg
except Exception as e:
msg = "Exception using serializer: {0}. Exception type: {1}.".format(e, e.__class__.__name__)
add_result['message'] = msg
else:
msg = "'{0}' is not a Destiny2 player on PS4".format(user_name)
add_result['message'] = msg
else: #not valid
msg = "save_user_form not valid. error: {0}".format(save_user_form.errors)
add_result['message'] = msg

return add_result




/r/django
https://redd.it/7eigs7

ActivityWatch - An open source automated time-tracker that is cross-platform, extensible and privacy-focused.
https://github.com/ActivityWatch/activitywatch

/r/Python
https://redd.it/7ei5fx

Memoization in Python: How to Cache Function Results
https://dbader.org/blog/python-memoization#intro

/r/Python
https://redd.it/7eiqq0

Building a CRUD application with Flask and SQLAlchemy
https://www.codementor.io/garethdwyer/building-a-crud-application-with-flask-and-sqlalchemy-dm3wv7yu2

/r/flask
https://redd.it/7bslae

username collisions

We are building a site where our users upload files. Django's API uses the 'username.'

When we handle registration we just use email. It seems that Django (or maybe it is AllAuth) just truncates the email to build the username:

email: name@gmail.com --> username: name

This is fine for the most part. And Django may even handle username collisions because because it actually works off of the ID and just displays 'username' for human friendly reasons (IDK).

The issue I foresee is as follows: If we build file paths based on username, e.g. all of user1's files are in files/user1, we have a potential collision because at this point, on say AWS S3 we are no longer using Django's (potential) collision handling.

TL DR: what happens if we have two users, one with username1@gmail.com and the other with username1@yahoo.com AND Django sets username to be a truncated email AND we build file paths off of username?

/r/django
https://redd.it/7emdcg

[D] Optimization Algorithms: Math and Code

A colleague of mine had mentioned that they were getting asked quite a few questions about optimization algorithms in their interviews for deep learning positions. I decided to make a quick post about the important ones over at 3dbabove: https://3dbabove.com/2017/11/14/optimizationalgorithms/

/r/MachineLearning
https://redd.it/7ehxky

How to create multiple user registration forms with flask-user

Hello
I understand that flask-user allows users with various roles to login. Based on the different roles that are available to users, each user can access routes that are available to that user and role.


However, I am trying to set up multiple registration pages for each user-role.
E.g. users with admin role have one signup page which asks for a lot of information up front , whereas users with regular role will have a simplified signup page just asking for basic details.
How can this be achieved using flask-user.


One good example would be Employers and Candidates on a job portal. How can we have 2 different forms - one for employers to sign up and another one for candidate sign up - for example..

Thanks

/r/flask
https://redd.it/7b5z77

Open Offer to Mentor
https://www.reddit.com/r/learnpython/comments/7eniv1/open_offer_to_mentor/

/r/djangolearning
https://redd.it/7enj21

Our Jupyter/IPython client for iPad now includes introductory notebooks on Python, NumPy, Matplotlib and SciPy that you can run without any preliminary configuration. Sign up for beta to run on your iPad!
https://juno.sh/juno-v-1-0-6-beta/

/r/IPython
https://redd.it/7embl0

I want to learn Python web development but I'm not sure if I should learn Flask or Django framework.



/r/Python
https://redd.it/7ena2q

Integrate bar chart in Flask app with database

I created a Flask app with a flask-sqlalchemy database. I would like one of the pages to show a bar chart which fetches data from said database, but Google hasn't helped much so far. Maybe someone has some hot tip? :)

/r/flask
https://redd.it/7eps8j