How to access Google Drive files from a Jupyter server?

I'm working on a project that involves consolidating 100+ .csv files into one file and running specific queries on this data. However, all of these .csv files are located in Google Drive. I don't want to manually download all 100 .csv files and upload them to my Jupyter server, so I'm wondering if there is a way to access them easily and store them in a folder in the server.

I'm aware of the github code that allows you do to this, but my access was denied, so I'm hoping there's another way to do this.

/r/IPython
https://redd.it/7p85kj

How to run ipython notebook and save straight to pdf w/o viewing?

I've been creating reports using ipython notebooks. When I want to present them to others I print to pdf and then share. Is it possible to run all the cells in a notebook and save the resulting notebook to pdf w/o opening the notebook?

/r/IPython
https://redd.it/7p9ci4

What's everyone working on this week?

Tell /r/python what you're working on this week! You can be bragging, grousing, sharing your passion, or explaining your pain. Talk about your current project or your pet project; whatever you want to share.


/r/Python
https://redd.it/7p7b6z

Python Release Python 3.7.0a4
https://www.python.org/downloads/release/python-370a4/

/r/Python
https://redd.it/7p7yi4

Collaborating on a project.

Hello! I'm looking to create a project and I wanted to see if anyone else is interested in joining. I've gone through a couple books so I'd consider myself a step above beginner in terms of ability and this project wouldn't be anything serious, just an exercise to get better at Django.



/r/djangolearning
https://redd.it/7pd3k2

Python Open Source Projects of the Year
https://medium.com/@Mybridge/30-amazing-python-projects-for-the-past-year-v-2018-9c310b04cdb3

/r/Python
https://redd.it/7pbe9z

[AF] POST request on one of multiple defined endpoints returns 404

EDIT: The error ended up being with NGINX, all I needed to do was change the ownership of the /var/lib/nginx directory from the nginx user to the user running the Flask application. Doh

Hi, I was having the darndest time debugging this and couldn't find anything similar online so hopefully you guys can help me get to the bottom of this

I have two endpoints, a login and a post submitter. Both are GET and POST request endpoints, both are defined, both work locally. Deployed, however, the login endpoint works fine but POST requests on the post submitter endpoint always return a 404. I don't believe it's an issue with NGINX or gunicorn as the logs show nothing out of the ordinary and the login endpoint works flawlessly. I've spent almost two days debugging this and I just can't for the life of me figure it out. Relevant code snippets are posted below.

@app.route('/newpost', methods=['GET', 'POST'])
@login_required
def new_post():
form = BlogForm()
#No code in this conditional is ever actually executed, 404's first
if form.validate_on_submit():
#Omitted code for getting form data and creating form object
try:
#Omitted code for database actions
return redirect(url_for('index', _external=True))
except IntegrityError:
flash("Error inserting into database")
return('', 203)
else:
#Omitted some code for template rendering
return render_template('new_post.html')


@app.route('/login', methods=['GET', 'POST'])
def login():
form = LoginForm()
userAuth = current_user.is_authenticated
if form.validate_on_submit():
flash("Login attempt logged")
user = User.query.filter_by(username=form.userID.data).first()
if user:
checkPass = form.password.data.encode('utf-8')

if bcrypt.checkpw(checkPass, user.password):
user.authenticated = True
login_user(user, remember=form.remember_me.data)
return redirect(url_for('index', _external=True))
else:
form.errors['password'] = 'false'
else:
form.errors['username'] = 'false'

else:
return render_template('login.html',
title='Log In',
loggedIn=userAuth,
form=form)

/r/flask
https://redd.it/7p1g11

Python script as Windows service only working in DEBUG mode

Hi r/python, I'm trying to run a dummy Python script as a service in Windows 7. It just writes the current time to a file in disk. My code goes like this:

import os
import time

import win32serviceutil
import win32service
import win32event
import servicemanager
import socket


class AppServerSvc (win32serviceutil.ServiceFramework):
_svc_name_ = "DummyService"
_svc_display_name_ = "Dummy App"

def __init__(self, args):
win32serviceutil.ServiceFramework.__init__(self, args)
self.hWaitStop = win32event.CreateEvent(None, 0, 0, None)
socket.setdefaulttimeout(60)

def SvcStop(self):
self.ReportServiceStatus(win32service.SERVICE_STOP_PENDING)
win32event.SetEvent(self.hWaitStop)

def SvcDoRun(self):
self.ReportServiceStatus(win32service.SERVICE_START_PENDING)
self.ReportServiceStatus(win32service.SERVICE_RUNNING)
servicemanager.LogMsg(servicemanager.EVENTLOG_INFORMATION_TYPE,
servicemanager.PYS_SERVICE_STARTED,
(self._svc_name_, ''))
self._logger.info("Service Is Starting")
dummy()


def dummy():
while True:
dummy_file = open('C:\dummy_text.txt', 'w+')
trace = str(time.time()) + '\n'
dummy_file.write(trace)
dummy_file.close()
time.sleep(40)

if __name__ == '__main__':
win32serviceutil.HandleCommandLine(AppServerSvc)

When I go to the directory containing the script and run *python dummyproc.py install* it tells me *Service installed*. I can debug the process with *python dummyproc.py debug* and it works fine (writes the file). But if I run *python dummyproc.py start* it gives me an error (it tells me that the service timed out against the start or control request). If I go to Windows 7 services and try to run it manually it also tells me the error code: 1053. I've been googling so far and tried many solutions but none of them worked for me. I included python and their Scripts and DLLs folders to the system path, I incremented the ServicesPipeTimeout value on regedit, I tried to install it as my user, tried to run not as a python script but as an executable compiled with pyinstaller... It did not work. Does anybody know what could be happening?? I'm using Windows 7 Professional 64 bits and Python 2.7.13.

Thank you in advanced!

/r/Python
https://redd.it/7pedd5

pandas-profiling 1.4.1 released - Create beautiful HTML profiling reports from pandas DataFrame objects
https://github.com/pandas-profiling/pandas-profiling

/r/pystats
https://redd.it/7pg7h0

[Q] Need help in understanding how a part of flask works.

Hi r/Flask

I was following Miguel Grinberg refreshed flask mega tutorial and had a small doubt in one section.

for whatever reason, I was unable to post a comment there so I decided to seek the expertise of you folks here.

so here's the question (and relevant code snippet).

@app.route('/login', methods=['GET', 'POST'])
def login():
form = LoginForm()
if form.validate_on_submit():
flash('Login requested for user {}, remember_me={}'.format(
form.username.data, form.remember_me.data))
return redirect('/index')
return render_template('login.html', title='Sign In', form=form)


I don't completely understand the flow of the program when the login url is visited.

Could someone please explain how the " form.validate_on_submit() " is evaluated before the user enters data into the form and clicks the submit button, which I believe is only possible when the login function has returned and rendered the login template ?

Thanks !

/r/flask
https://redd.it/7pfz3w

Changing jinja2 configuration with flask trim_blocks and lstrip_blocks

So I want to change `trim_blocks` and `lstrip_blocks` to true to remove the ugly whitespace you get when you render a template but I can't seem to figure out how to it correctly. I found that I can make a Jinja environment with my own config but this makes it so I have to do `environment.get_template('index.html').render(var=var)` instead of simply `render_template('index.html', var=var)`. Also with this I need to specify the loader for it to know where to find the templates and it breaks flask-bootstrap.

/r/flask
https://redd.it/7ph5na

What a surprise: had some Python swag in the mail today!

/r/Python
https://redd.it/7pid7p

Has anyone used vaex? Out-of-core dataframes

Recently discovered [vaex](https://docs.vaex.io/en/latest/index.html). I was curious how it compares to using Dask.

/r/pystats
https://redd.it/7pjmi0

Why form.isValid() always returns false ? HELP

/r/django
https://redd.it/7pklwd

What would be an alternative to matplotlib in python for visualisation?

I've been using matplotlib and plotly for visualisation in python for some time. matplotlib feels very primitive, plots look crude. Plotly has been a nice ride but it's not free and had to give it up. Are there any better alternatives?

I'd mostly be working in bivariate analysis and time series analysis and more than often want to produce geographical overlays/plots.

/r/Python
https://redd.it/7pg3nu

Achoo (the predictive asthma alert system built with the raspberry pi) Beta version 0.1 is now available!

Find the blog post [here](https://tmthyjames.github.io/tools/prediction/Achoo-beta-0.1/).

The app is at [achoo.us](https://achoo.us/).

The github page is [here](https://github.com/tmthyjames/Achoo)

I have been accepted to give a talk about this project at this year's [AnacondaCON](https://anacondacon.io)!

Also, there will inevitably be bugs. This app is still under heavy development and bug fixes will be top priority.

I am grateful for any feedback you have. There are "contact me" buttons on the app and feel free to DM me here too :)

/r/Python
https://redd.it/7pl4ro

Curated collection of python snippets that you can understand in 30 seconds or less (work in progress)
https://github.com/kriadmin/30-seconds-of-python-code

/r/Python
https://redd.it/7pn0qm

[AF] advice about streamlining long running processes

Hi,

I made a Python program that processes large data files. I was asked to make the program accessible through a web interface. I would like to use Flask for this, but i have no experience with Flask yet.

The program flow would be that my colleagues can upload a file and set parameters for the processing > add the job to a queue > process the job > send an email when the job is done > they can download the processed file.

Processing the job is usually done between 5 -15 minutes. What is the preferred way to set this up? I'm worried that because of to the processing time, the website would freeze up and becomes unresponsive.
Is it better to handle the processing separate from my Flask application?

Any advice is greatly appreciated.

/r/flask
https://redd.it/7pn0rd

How to pass an already defined function to "@app.rotue("path")"?

I'm not super good with decorators but I kinda know how they work, but can't get it work.

so I don't want to define my functions and routes in the same file.


@app.route("/")
def home():
return render_template("home.html")

I wanna seperate those in different files.
I thought I can do


def home():
return render_template("home.html")

then

from .controller import home
app.route("/", home)

but it does not work

I hope my question make sense.


/r/flask
https://redd.it/7oyj7t

Retrieving lat/long coordinates from the client without requiring a form POST

Is it possible to retrieve the lat/long coordinates immediately after the client "allows" access to their location?

Currently upon agreement to share their location, we'll have access to their coordinates, but only in the browser. How would I send this information server-side ASAP in order to do some processing?

Essentially, how do I skip the POST step after the user agrees to share their coordinates?



/r/django
https://redd.it/7pli92

Co-Worker & Philosophical Differences

I am curious how others have dealt with co-workers who may have had fundamental difference in best practices with co-workers in the past.

Some context:

I have a co-worker (with several years of seniority at this job) who looks at things quite differently than I. While we both agree on KISS, our perspective on what is simple is completely different. We work on primarily "web-applications" but tend to integrate with a lot of external systems.

Some examples:

- The co-worker wants to keep the dependencies as minimal as possible, almost without exception. I agree with not adding complexity needlessly, but am also opposed to reinventing the wheel on the job. Example: limiting ourselves to a document store when the data is very relational in nature. On my end, this adds significant code complexity to perform joins, Cascade deletes, verify relationships are still valid, etc... While data can be structured in a way that makes sense here in some cases, the nesting became too deep for Mongo.

- When dealing with code that would run slower than is acceptable in a single request/response cycle, I would tend to push the logic to Celery, use caching and/or other techniques along those lines. My co-worker does things like spin off a thread from the gunicorn worker or replicates an entire large, external database.

- The co-worker isn't remotely open to the idea of using ORMs on principle. The claim is that you can't optimize if needed (the truth to that statement can be debated) and it adds additional complexity co-worker has to wrap their brain around. However, our applications tend to be small, our deadlines short, and maintainability has been our largest issue historically. I think introducing an ORM (e.g. sqlalchemy to avoid including Django and stay somewhat slim) would totally work for us, significantly decrease the time to build out MVPs, make our code easier to reason about, and consequently make it easier to maintain our applications in the long-run.

- There are times we needed a cache. The co-worker absolutely refuses to inclide Redis (as a counter-point they have no problem with using elastic search for searching/indexing text) on principle of staying slim. We have historically used Mongo as our primary DB so I wrote a cache decorator to compare Mongo vs. Redis. Mongo is not accurate with TTLs, but more importantly Redis was returning results in a 10th of the time. This, to me, is a case of using the right tool for the job.

- There is background processing we do in several applications. A previous co-worker wrote a distributed cron-runner (avoids running the same job on multiple servers simultaneously) which is not a bad solution. However, these tasks sometimes need to be kicked off when a request is made. I think Celery and Celerybeat is a great tool to fit this job; again, that idea isn't well received.

There are other examples of being given unsolicited feedback on even the most minor design decisions. I can't tell if the co-worker absolutely thinks I am incompetent, if they think on a different wavelength, or if I am missing something important. I've been at this for around 11 years but only 1 with the current company. I am not coming from a perspective of trends or laziness but adapting my previous experience on many, many projects with the needs of my current team. I am open to the idea that co-worker knows many things I don't, but the constant resistance to "everything" is making it very difficult to work with them and run my own projects successfully. I do wish I could understand co-worker's perspective better but I am at a loss.

Anyways, I just wanted to see if anyone else has ran into times where they had to work with someone who was very opinionated, it felt like they were not empowered, and how they handled it. I am asking here, because in the end 90% of our work is writing Python applications.

/r/Python
https://redd.it/7pngyz