Vim Python IDE

/r/Python
https://redd.it/7e6pes

Using custom bootstrap instead of Django templates

I have created a web app using Django, and while I find the default Admin backend useful, its a headache to customize in certain cases. I want to develop a dashboard like interface and do away with Django's Admin templates. I want to end up with something like [this](https://adminlte.io/themes/AdminLTE/index2.html) I have a REST API in place using DjangoRestFramework, but I don't understand how this could work well with plain html pages. I still want Django WSGI to "serve" my pages through Apache. What's the best way to go with this?

/r/django
https://redd.it/7e1tk3

Python code is running too slow, do I need to find a way to allocate more memory to python process?

I have python code that can extract a zip file (tar.gz) . If I use the code, it takes a really long time (~1.5 hour) but if I unzip it directly using 7zip it takes less than 5 minutes, so I am guessing there is something impeding the processing power of python.

I am trying to run this code https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/udacity/1_notmnist.ipynb

for convenience, here a version of the code that doesn't require numpy, scripy, etc. EDIT. The formatting is messed up. In meanwhile just look at the stack over flow implementation.

https://stackoverflow.com/questions/47392697/python-code-is-running-too-slow-do-i-need-to-find-a-way-to-allocate-more-memory

> url = 'https://commondatastorage.googleapis.com/books1000/'

> last_percent_reported = None

> data_root = '.' # Change me to store data elsewhere
>
>
> def download_progress_hook(count, blockSize, totalSize):

> """A hook to report the progress of a download. This is mostly intended for users with

> slow internet connections. Reports every 5% change in download progress.

> """

> global last_percent_reported

> percent = int(count * blockSize * 100 / totalSize)
>
> if last_percent_reported != percent:

> if percent % 5 == 0:

> sys.stdout.write("%s%%" % percent)

> sys.stdout.flush()

> else:

> sys.stdout.write(".")

> sys.stdout.flush()
>
> last_percent_reported = percent
>
>
> def maybe_download(filename, expected_bytes, force=False):

> """Download a file if not present, and make sure it's the right size."""

> dest_filename = os.path.join(data_root, filename)

> if force or not os.path.exists(dest_filename):

> print('Attempting to download:', filename)

> filename, _ = urlretrieve(url + filename, dest_filename, reporthook=download_progress_hook)

> print('\nDownload Complete!')

> statinfo = os.stat(dest_filename)

> if statinfo.st_size == expected_bytes:

> print('Found and verified', dest_filename)

> else:

> raise Exception(

> 'Failed to verify ' + dest_filename + '. Can you get to it with a browser?')

> return dest_filename
>
>
> train_filename = maybe_download('notMNIST_large.tar.gz', 247336696)

> test_filename = maybe_download('notMNIST_small.tar.gz', 8458043)
>
> num_classes = 10
>
> def maybe_extract(filename, force=False):

> root = os.path.splitext(os.path.splitext(filename)[0])[0] # remove .tar.gz

> if os.path.isdir(root) and not force:

> # You may override by setting force=True.

> print('%s already present - Skipping extraction of %s.' % (root, filename))

> else:

> print('Extracting data for %s. This may take a while. Please wait.' % root)

> tar = tarfile.open(filename)

> sys.stdout.flush()

> tar.extractall(data_root)

> tar.close()

> data_folders = [

> os.path.join(root, d) for d in sorted(os.listdir(root))

> if os.path.isdir(os.path.join(root, d))]

> if len(data_folders) != num_classes:

> raise Exception(

> 'Expected %d folders, one per class. Found %d instead.' % (

> num_classes, len(data_folders)))

> print(data_folders)

> return data_folders

>
> train_folders = maybe_extract(train_filename)

> test_folders = maybe_extract(test_filename)

I am using 64-bit 3.6.3 python on a 64-bit windows 10 laptop, with 12 gigs of RAM.

/r/Python
https://redd.it/7e8nni

Taskmap: A python framework for managing task dependencies and executing coroutines across multiple cores
https://github.com/n-s-f/taskmap

/r/Python
https://redd.it/7eag9s

Python 3.6:Drawing LINEGRAPHS and Saving it as a PDF!
https://youtu.be/PdbLVbzoXYI

/r/pystats
https://redd.it/7ebo5u

If I built almost exactly the same site using pure Django vs DRF + VueJS, what would be the performance differences ?

Say I have an online shop that gets about 1000 hits per day.

One version uses Django for the templating, one uses Vue with Django-Rest-Framework for everything under the hood.

What would be the performance differences ? What is the cost of templating with Django? Would the gap be significant enough to warrant changing between one or the other ? Or is the impact minimal, leaving the decision be based on other factors like UX ?

/r/django
https://redd.it/7ebqq4

website review

I just made this website using django 1.8. I want to get some reviews and suggestions??
www.frontboard.ml

/r/djangolearning
https://redd.it/7ebqvh

How to save JSON API response in database?

Hi, I am new to django and am currently trying to connect to third-party JSON API, retrieve information and save it into the database. I am able to retrieve the data from views.py but I want to run a API query every 1 minute and save updated results into the database. Can anyone help me with it?

/r/django
https://redd.it/7e0olm

Complete scrapy tutorial : get data from reddit, amazon, pexel, steam, airbnb
https://tanpham.org/

/r/Python
https://redd.it/7ebcd9

Deploying a Flask app into Docker Swarm (xpost /r/python)

[Link first because this is what everyone wants](https://github.com/justanr/flaskbb-swarm)

I recently set up a docker swarm in my home lab and I realized that just putting tooling (traefik, portainer, registry, etc) on it was silly. So I decided to document how I went about deploying a non-trivial application: [FlaskBB](https://github.com/sh4nks/flaskbb) -- which disclosure I'm a maintainer on. I'm also defining "non-trivial" as has multiple moving parts that work together in interesting ways (for some definition of interesting) -- in this case, there is the actual application container, a celery container, redis, a postgres database and finally [an nginx container](https://i.imgflip.com/1z9w62.jpg).

The entrypoint is Traefik running as a Swarm service, which then hands off traffic to the Nginx container which decides to serve either static content or pass to the uWSGI container (in the future, I plan on using frontend rules to partition this on the Traefik layer rather than the nginx layer). Docker networks are used to partition which services are allowed to talk to each other. Secrets and Configs are used to separate configuration logic from the code running in containers.

The nuts and bolts are specific to FlaskBB but hopefully can provide a template for usage with other applications. I tried to be thorough without delving down rabbit holes too much, but let me know if I missed something or should've been clearer some where. :)

/r/flask
https://redd.it/7cmwpz

Automatically downloading PDF using Django running code on multiple ports (X-post from /r/djangolearning)

When a user uploads a PDF, it is modified using ReportLab and the modified PDF is automatically downloaded. Below is my code for processing the files and returning them.

def process_report(request):

# Handle file upload
if request.method == 'POST':
report_file = request.FILES.pop('file')[0]

merged_pdf, namefinal = pdfAnalyzingFunction(report_file)

response = HttpResponse(content_type='application/pdf')

response['Content-Disposition'] = 'attachment; filename="{}"'.format(namefinal)
response.write(merged_pdf.getvalue())

return response

else:
return HttpResponseBadRequest('Must be Post')

I am using Django to render and return content. The above code worked and was able to download files automatically when I just had some simple HTML on a localhost:8000 port. However, after building a frontend on a different port, localhost:3000, I was able to successfully run and generate a modified PDF but it no longer downloaded automatically (with the PDF modifying code running on the localhost:8000 port still).

On my localhost:3000 port network tab I can see the uploaded file, and am wondering why it is not downloading.

Request URL: http://localhost:8000/process_report
Request Method : POST
Satus Code: 200 OK
Host: localhost:8000
Origin: http://localhost:3000
Referer: http://localhost:3000/

Unclear as to why it appears my PDF is rendering, but not automatically downloading now. Is there something I need to know about generating a response object on one port and attempting to download it automatically on a separate port?


/r/django
https://redd.it/7ees5t

Powerpoint to PDF Converter with python
https://www.youtube.com/attribution_link?a=uS_0zNR_JZw&u=%2Fwatch%3Fv%3DoWbVobteULo%26feature%3Dshare

/r/Python
https://redd.it/7efeee

Abstract Classes

Im not sure I fully understand the usages of abstract classes. I know that they are useful when you want subclasses to inherit repeating fields, but how does this make working with repeating fields useful despite just making you type less? What kind of special actions can you perform on abstract classes? Am I missing something

/r/django
https://redd.it/7efjuk

[AF] Passing Python variables on the server to React component

Hi folks, a bit late to the party.

A bit of background: I'm practising with Flask + React in [this repo](https://github.com/CamiloGarciaLaRotta/fullstack_app_boilerplate/tree/vars_as_props).

Here is my issue on [SO](https://stackoverflow.com/questions/47256340/flask-passing-python-variables-on-the-server-to-react-component?noredirect=1#comment81469485_47256340).

But I figured I could ask you guys, in a more general sense, what are the heuristics when one wants to pass variables from the server side to a component at the first rendering of the html template:

* Do devs usually avoid this situation altogether?
* Is it advised to use other component mounting approaches for other than through a main index.jsx?
* What are the DO's and DONT's when working in a mixed environment like this? (i.e. Flask using templates and all the jsx components linked to the html in a single bundle)



Thank you very much in advance,

any advice is appreciated


/r/flask
https://redd.it/7cmsb3

[N] GCP lowers GPU pricing by up to 36%: K80 at 40c/hr and P100 at $1.46/hr
https://cloudplatform.googleblog.com/2017/11/new-lower-prices-for-GPUs-and-preemptible-Local-SSDs.html

/r/MachineLearning
https://redd.it/7eckwf

JWT Authorization in Flask

Hello guys! I just published a tutorial on how you can create JWT Authorization in Flask. Enjoy and ask any question if you have :)

https://codeburst.io/jwt-authorization-in-flask-c63c1acf4eeb

/r/flask
https://redd.it/7eikat

What's everyone working on this week?

Tell /r/python what you're working on this week! You can be bragging, grousing, sharing your passion, or explaining your pain. Talk about your current project or your pet project; whatever you want to share.


/r/Python
https://redd.it/7ei2o7

Best strategies for incorporating data from a RESTful api: forms vs serializers?

**tl;dr**
When pulling data into my django database from an api, should I use forms, or a [django rest framework](http://www.django-rest-framework.org/) serializer?

**What I did**
Based on some discussion in [another thread](https://www.reddit.com/r/django/comments/7e0olm/how_to_save_json_api_response_in_database/), I built an ultra-simple example of how to extract data from an api, in this case the api for the video game Destiny 2:
https://github.com/cortical-iv/restit

Note this is essentially a port of UltimateDjango's [How to Consume REST APIs with Django & Python Requests](https://ultimatedjango.com/blog/how-to-consume-rest-apis-with-django-python-reques/).

My goal was to follow Section 12.1 of Two Scoops, and *Validate all incoming data with django forms.* But what I actually did was a two step process that I'm now thinking was sort of dumb (I pasted the code below the fold):

1. Bind data about path parameters in the url into a form.
2. After extracting the needed data from the response to `requests`, use a django rest framework *serializer* to validate and save that data.

Now that this is up and working, I realize I didn't actually really follow the advice from Two Scoops of importing through forms. I imported required *path* information (user name) through a form, but then did all the heavy lifting with the serializer. I now think it would be better to replace step 2 with another form (a SaveUser form) that itself includes all the fields I need in my database.

**Why I don't like what I did**
Why am I thinking my approach smells? A couple of reasons:

1. The serializer is overkill. I shouldn't need to invoke the django rest framework to do what I am doing. Why not stick with django forms until I need something more powerful?
2. I am doing so much processing on the Destiny 2 end points, that I can't even just apply the serializer to the output of `requests`. This may not be obvious from the simple example at my project.
3. Forms are awesome and I'm not even using them. Once I make some fields unique, calling `serializer.save()` just triggers a validation error. OTOH, using forms, you can feed it an instance to simply *update* the db row without having to do a lot of extra work. Forms have lots of cool stuff like this so it isn't clear, again, why a serializer from DRF is that helpful.

On the other hand, I am a django noob, so maybe there are benefits to pulling data in via a serializer that I don't appreciate. So I'm asking for advice: when pulling data from an api into a db (using `requests`), should I use a serializer, or a form? If a form, which I'm starting to think is the right answer, I'm a little confused about why the project I copied, from UltimateDjango, even used a serializer?

What conditions for that embedly api consumer made it smarter in his case, but not my case?

-----
Here is the `save_user` function I'm using, which contains the main logic:

def add_user(save_user_form):
"""
Validate form data containing 'display_name', request their info from server,
and pull in via a serializer. Returns dictionary with two keys:
'flag': 1 if successful, 0 if not, and None if that user is not a D2 player in PS4.
'message': message about the outcome (success, validation error, etc).
"""
add_result = {'flag': 0, 'message': ''}
if save_user_form.is_valid():
user_name = save_user_form.cleaned_data['display_name']
search_url = search_destiny_player_url(user_name)
search_response = requests.get(search_url, headers = {"X-API-Key": D2_KEY})
user_data = search_response.json()['Response']
if user_data:
submission_data = {'display_name': user_data[0]['displayName'], 'user_id': user_data[0]['membershipId']}
try:
serializer = SearchPlayerSerializer(data = submission_data)
print(repr(serializer)) #for debu

gging
if serializer.is_valid():
serializer.save() #save data to database
add_result['flag'] = 1
add_result['message'] = 'Successfully added {0}'.format(user_name)
else:
msg = "Invalid serializer. serializer.errors: {0}".format(serializer.errors)
add_result['message'] = msg
except Exception as e:
msg = "Exception using serializer: {0}. Exception type: {1}.".format(e, e.__class__.__name__)
add_result['message'] = msg
else:
msg = "'{0}' is not a Destiny2 player on PS4".format(user_name)
add_result['message'] = msg
else: #not valid
msg = "save_user_form not valid. error: {0}".format(save_user_form.errors)
add_result['message'] = msg

return add_result




/r/django
https://redd.it/7eigs7

ActivityWatch - An open source automated time-tracker that is cross-platform, extensible and privacy-focused.
https://github.com/ActivityWatch/activitywatch

/r/Python
https://redd.it/7ei5fx

Memoization in Python: How to Cache Function Results
https://dbader.org/blog/python-memoization#intro

/r/Python
https://redd.it/7eiqq0