[AF] Import issues with my project. Unsure how to approach.

I have a flask app that relies on web scraping to display data to the end user and let them download it. Right now, the various scrapers I have are sitting in the root level of the *myapp/* package/project so that they can write to app.db, a SQLite database.

Here is the current structure of the app:

my_app/
- app/
--- __init__.py
--- routes.py
--- models.py
--- templates/
--- static/
- scrapers/ (*note: my goal is create a folder like this)
--- scraper1
--- scraper2
--- scraper3
- config.py
- app.db
- scrape.py (*note: where my scraper currently is)


In the scrape.py file, I'm importing with the following import statement:

from app.models import db, table_name

My goal is to move scrape.py (and other scraping scripts) into a single directory where they can import **db** and **table_name** from app/models.py, but relative imports haven't worked for me.

Am I approaching this wrong?

/r/flask
https://redd.it/8nsv2a

Easier data analysis in Python with pandas (30+ videos)
http://www.dataschool.io/easier-data-analysis-with-pandas/

/r/Python
https://redd.it/8nxfwf

What's up with all the dict key->attribute frameworks (python-box, dataclasses, namedtuple etc.)?

### Data Classes are a-comin'

So, now that we're getting [data classes](https://www.python.org/dev/peps/pep-0557/), we've finally got a concise and convenient way to avoid huge `__init__` methods full of `self.foo = foo`. That's great, that's fantastic, that's overdue. I'm sick of writing an entire `__init__` method for things that have like four attributes.

But it made me think -- why are there so many frameworks for taking things that would ordinarily be dictionaries and making their 'keys' available as attributes?

### The alternatives
####Namedtuple

`Namedtuple` kind of has a reason to exist other than as a key->attribute interface. Even with dataclasses, it provides the only super-easy framework to represent a datatype or small object:

Employee = namedtuple('Employee', ['id', 'name', 'salary'])

I like Namedtuple. I think that the syntax is a little silly because of that mandatory first argument, but that's just adds character. Namedtuple instances are immutable, too, and that's useful in its own way.

####Box
Box (`python-box` on pypi) is something I only just used for the first time today, and it's basically the same kind of thing, but more magic:

>>> bands = Box({
"My Awesome Band": {
"Albums": ["Foo, Vol 1.", "Bar Baz Blues"],
"Artists": ["Mr. Example", "John Doe"]
}
})

>>> print(bands.My_Awesome_Band.artists[0])

Mr. Example

It's super awesome and I immediately started using it to keep track of a deep, multi-layer dictionary I got courtesy of some JSON API. Having ['many']['layers']['of']['these']['selectors'] can get annoying.

Even dataclasses can't beat it: a `dict` in a dataclass is still dataclass.mydict['and']['then']['you']['come']['back']['here']. A `dict` in a `Box` has its keys exposed too.

### A clear trend

This entire situation reminds me of type hints, where despite the fact that Python will officially always be dynamically typed, the Python maintainers eventually sighed and admitted maybe there was some use for type annotations in programming, even if they were just so much syntactic sugar for Python itself.

Isn't it pretty apparent by this point that people like to access attributes with dots? Accessing them by ['string keys'] is fun and all, but it's ugly-looking and repetitive. Dots are cleaner. A dot-access class might lose case-sensitivity or might require you to use no special characters in keys or whatever, but those are just quibbles.

Would the community support adding something more like Box to the standard library? Should there be some kind of `collections.DottedDict`? Should we just admit, yeah, dots are nice, and while they're not the official syntax for dictionaries, they're available if you like 'em?


/r/Python
https://redd.it/8nx4sk

Newbie ForeignKey list problem

Hello,

Again I'm struggling with something quite simple.

I have two models. Products and ParoductCategory

*class* ProductCategory\(*models**.Mode*l\):
category\_name = models.CharField\(*max\_length*=40\)
slug = models.SlugField\(*max\_length*=40,* uniqu*e=True\)
*class* Product\(*models**.Mode*l\):
product\_name = models.CharField\(*max\_length*=255\)
product\_amount = models.IntegerField\(\)
updated = models.DateTimeField\(*auto\_now*=True\)
timestamp = models.DateTimeField\(*auto\_now\_add*=True\)
category = models.ForeignKey\('ProductCategory', *on\_delete*=models.CASCADE,* related\_nam*e='cat'\)
additional\_info = models.TextField\(*blank*=True\)

I would like to iterate over ProductCategory and create list of items in each productcategory model. Help me guys plz, I won't sleep if i won't understand where is my problem.

I' trying to use CBV, and I was playing with get\_context\_data method in view.

/r/djangolearning
https://redd.it/8nwr6z

I'd like to create a convenience app that works across all apps--where do I begin?

Vague title, I know. Lemme explain:
I'm trying to create a text translator that works across not only the browser, but on the desktop or any other app--as long as the text is 'highliteable', I want to be able to hold a key down and hover over the text and have a little pop-up appear and translate the text.

So what exactly am I looking for? A desktop app using tkinter? How do I sense that the mouse is hovering over text and that it's highlighted? Pyautogui?
All I know so far is that it'll involve a translation API.
What other libraries will be involved?


Feedback much appreciated!

/r/Python
https://redd.it/8nyn8l

Django bugfix release: 2.0.6
https://www.djangoproject.com/weblog/2018/jun/01/bugfix-release/

/r/django
https://redd.it/8ntc2r

When is a project due for a complete re-write?

I built a project that has been in production for about 4 years now and is integral to the daily operations of the business. Its original design and functionality was for a fairly narrow set of processes. Over the years as the company has grown, I've tacked on features, built in new work arounds, and expanded the project beyond its initial intent. It's all working fairly well, but I'm definitely seeing how it's aging. Right now I'm working on adding a new feature that, were the project designed differently/better initially, I wouldn't even have to write this new code \-\- the project would already be able to handle the new requirement.

Over the last year or two, I've often wondered if there's a valid case for "we need to rebuild this beast". Management is reluctant \(understandably\) since their business rides on the project working in its current state.

Any of you have experience on this front?

/r/django
https://redd.it/8nvw8c

[N] UC Berkeley Open-Sources 100k Driving Video Database
https://medium.com/@Synced/uc-berkeley-open-sources-100k-driving-video-database-dce09ff7cf78

/r/MachineLearning
https://redd.it/8ns7vv

[AF]Why is my flask/SQLAlchemy simple app not respecting data constraints from db.Column parameters?

Hi.

I feel like this is more of SQLAlchemy question but I would assume that a lot of people here have used it as well in their Flask app, so this is why I'm asking here.


(Whole code at the end of the post)


The way I understand it, when creating the db using SQLAlchemy, we define the data that should be accepted by this column of the database using db.Column, for example:

field0 = db.Column("f0", db.String(3), unique=False, nullable=False, primary_key=True)


Here, the **field0** column is named *f0*, should accept only *string* data that is at most *3* characters long, should accept *non-unique* entries, but should not accept *empty* inputs.


However, once I run my app, I can enter literally anything into the respective columns, regardless of the constrains imposed during creation of the db.


So, either I did something wrong, or I don't understand how this whole thing should work.


I do understand that the db is defined the moment we create it, and not the moment the app is started. I made sure to create a fresh db with the correct column definitions.


**the_app.py**

____________________________

import os, datetime
from flask import Flask, Markup, render_template, request
from flask_sqlalchemy import SQLAlchemy

project_dir = os.path.dirname(os.path.abspath(__file__))
app = Flask(__name__)

database_file_uri = "sqlite:///{}".format(os.path.join(project_dir, "databse.db"))
app.config["SQLALCHEMY_DATABASE_URI"] = database_file_uri
db = SQLAlchemy(app)

class MyTable(db.Model):
field0 = db.Column("f0", db.String(3), unique=False, nullable=False, primary_key=True)
field1 = db.Column("f1", db.Integer, unique=False, nullable=False, primary_key=True)

def __repr__(self):
return " string representation".format(self.title)

def __init__(self, field0=field0, field1=field1):
self.field0 = field0
self.field1 = field1

@app.route("/", methods=["GET", "POST"])
def home():
if request.form:
newRow = MyTable(field0 = request.form.get("f0"), field1 = request.form.get("f1"))
db.session.add(newRow)
db.session.commit()

data_from_db = MyTable.query.all()
return render_template("home.html", data_from_db=data_from_db)

if __name__ == '__main__':
app.run(host='0.0.0.0', port=8080)


**templates\home.html**
____________________________

<html>
<body>
<h1>Add entry</h1>
<form method="POST" action="/">
<input type="text" name="f0">
<input type="text" name="f1">
<input type="submit" value="Add">
</form>
<h1>Values</h1>
{% for elem in data_from_db %}
<p>{{elem.field0}} - {{elem.field1}}</p>
{% endfor %}
</body>
</html>


**db initialization**
____________________________

from the_app import db
db.create_all()
exit()

To recap, column constraints:

field0 = db.Column("f0", db.String(3), unique=False, nullable=False, primary_key=True)
field1 = db.Column("f1", db.Integer, unique=False, nullable=False, primary_key=True)

But, when I run the app and go to http://localhost:8080/, I am:

- able to insert very long strings into field0 (greater in length than 3)

- able to insert strings into field1 (which should only accept integers)

- able to insert empty fields (despite nullable=False)

- unable to insert identical fields (despite unique=False)

Can someone explain why db seems to just completely ignore the column constrains I'm using? Thanks for any replies in advance.

/r/flask
https://redd.it/8o0mfd

Flask SQLAlchemy tenant data isolation

hello,

I'm currently moving my flask application from a multi instance to multi-tenant architecture. I want to isolate customer data to its own db rather than comingle it with context in a single db. I have found a couple of learning resources so far, like this bit on using binds for table isolation

SQLALCHEMY_DATABASE_URI = 'postgres://localhost/main'
SQLALCHEMY_BINDS = {
'users': 'mysqldb://localhost/users',
'appmeta': 'sqlite:////path/to/appmeta.db'
}
[Source](http://flask-sqlalchemy.pocoo.org/2.3/binds/#binds)

My trouble is whether this is the best way of isolation. I'm wondering how to update the binds and inject the new database URI as the application gains new tenants. The best resource I've found is
[this talk](https://www.infoq.com/presentations/saas-python).

I'm wondering if someone can help point me towards learning resources or help me ask this question in a better way because I'm stuck.

/r/flask
https://redd.it/8o15hq

A game of tokens: write an interpreter in Python with TDD - Part 4
http://blog.thedigitalcatonline.com/blog/2018/06/02/a-game-of-tokens-write-an-interpreter-in-python-with-tdd-part-4/

/r/Python
https://redd.it/8o0o8h

how to use a variable in a template other than via : return render_template('template.html',variable=variable)

my program currently look like this and is becoming more and more unreadable as i need to display things from my database in my template because i dont know how to do it other than this currently

app= Flask(__name__)
app.config['MYSQL_HOST'] = 'localhost'
app.config['MYSQL_USER'] = 'root'
app.config['MYSQL_PASSWORD'] = ''
app.config['MYSQL_DB'] = 'database1'
mysql = MySQL(app)

@app.route('/1')
def route1():
var=function_getting_something_from_database(mysql)
var2=function_getting_something_else_from_database(mysql)
var3=other_function_getting_something_from_database(mysql)
(...)
return render_template('home.html',variable=var,variable2=var2,variable3=var3)

@app.route('/2')
def route2():
var=some_function_getting_something_from_database(mysql)
var2=function_getting_something_else_from_database(mysql)
var3=other_function_getting_something_from_database(mysql)
(...)
return render_template('home.html',variable=var,variable2=var2,variable3=var3)

if ((__name__) == '__main__'):
app.run()

home.html

<p>{{ variable }}</p>
<p>{{ variable2 }}</p>
<p>{{ variable3 }}</p>

/r/flask
https://redd.it/8nzwsb

[N] Google Will Not Renew Project Maven Contract
https://www.nytimes.com/2018/06/01/technology/google-pentagon-project-maven.html

/r/MachineLearning
https://redd.it/8o0j19

Very Peculiar Flask Issue

Hey guys. I'm really confused.

I'm trying to open a URL in Flask in an embedded browser (X-Frame), the problem is the URL is not allowed to be opened in the X-Frame so comes up blank.

What I'm trying to do (very unsuccesfully) is simply just redirect from the embedded page to the regular version (non-embedded) of the billing page I want to visit.

When I put in the URL of the billing page manually into the browser, it works fine.

Any thoughts as to how to get this to work WITHOUT opening a new tab?

/r/flask
https://redd.it/8nw952

Any Django 2.0 tutorials?

I went through the docs but I still don't feel like good at Django. I've been trying to find some other resources but all of them weren't on 2.0 and did things differently than what I learnt. Any resources you would recommend?

/r/django
https://redd.it/8o3p8g

Django Celery Periodic Task at specific time

I asked a question on stackoverflow. [this](https://stackoverflow.com/questions/50649919/django-celery-periodic-task-at-specific-time) is the link to the question.

Complete Question:

I am using `celery==4.1.1` in my project. In my `settings.py`, I have the following:

from celery.schedules import crontab

CELERY_BROKER_URL = "redis://127.0.0.1:6379/1"
CELERY_TIMEZONE = 'Asia/Kolkata'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_RESULT_BACKEND = "redis://127.0.0.1:6379/1"


CELERY_BEAT_SCHEDULE = {
'task-number-one': {
'task': 'mathematica.core.tasks.another_test',
'schedule': crontab(minute=45, hour=00)
},
'task-number-two': {
'task': 'mathematica.core.tasks.test',
'schedule': crontab(hour='*/1')
}
}

The second task mentioned in `CELERY_BEAT_SCHEDULE` is running perfectly. However, the first task `mathematica.core.tasks.another_test` which is a simple function returning a string is not running at the specified time, `00:45 (45 minutes past midnight)`. I have tried a number of ways to run a function at a given time each day but failed to achieve the same.

Please suggest ways/hints to achieve the same results.


Edit:

My test function looks like the following

from celery import task

@task
def another_test():
return "another test"

/r/djangolearning
https://redd.it/8o1hvq

Animate points in two numpy arrays with matplotlib

hi,

i have two numpy arrays X and Y with same length describing a circumference. The points are created with a function that takes as inputs the radius and the number of points i want and return the already mentioned X and Y arrays.

i want a animation to show 1 pair of points at the time but i have not being able to achieve this using matplotlib. i would show my animation code but truly is useless garbage. Can you guys please point me in the right direction on how to make this work?

Thanks.


/r/IPython
https://redd.it/8o5vft

Can I use a value from a foreign key in the __string__ of a model?

Edit: I mean the \_\_str\_\_ method. Not \_\_string\_\_

I have an mailing address table that has a foreign key to a User. On the admin page, when I'm looking at the list of Addresses, instead of saying "Address Object" I'd like it to say something like "John Smith: 1234 SomeStreet".

When defining the Address model can I access the name from the User model? Or do I need to have a name field in the Address that duplicates the users name?

/r/djangolearning
https://redd.it/8nvlzx

ML - Linear & Logistic Regression, K Means, Decision Trees & Random Forests easily explained in Python
https://www.youtube.com/watch?v=0kH6-Bafrto&list=PL998lXKj66MrJc80jS0E1htuEo5zI-tYG

/r/Python
https://redd.it/8o34n9

Postgres JSONField how to validate?

I am trying to create a user profile schema for accounts and I have profile information stored in a separate table. In the profile schema there are some fields that are basically array of objects. For example, "experiences" is has a schema like this (array of objects):

[
{ title, employer, location, duration: { from, to } }
...
]

I am currently using JSONField to store these types of data but I want a way to validate the contents of the data. For example, I want to be able to check that when appending a new entry to JSON field, the mode validates that there are **only** "title", "employee", "location", and "duration" keys and "duration" key is an object with "from" and "to" keys.

I just need a way to define a function that can validate these fields.

/r/django
https://redd.it/8o1q3k