[AF] Please share advice for testing with existing database

I am new to flask and marginally familiar with databases and testing. I am working on a project that mainly deals with an Oracle database and it isn't clear to me how I should to test my project. I need help getting my mindset right.

Currently the application draws a chart of selected data, the user categorizes points in the chart and saves the categories to another table. The site is then updated in a way so other uses know that group of data has been examined.

At the moment the dev database is a full copy with hundreds of thousands of rows in the main table. At the moment relations are simple and I know exactly which tables I need and their relation to one another.

What advice or resources do you all have?

The popular advice from just searching is that I should build a new version of the database up for each test, but that seems unwieldy for the amount of data. Do I take a subset? Are there established ways of copying the database's schema using sql-alchemy so it can be regenerated quickly? Even then what do I test, that I get the correct data from the database?

The other advice I have seen is to avoid CRUD tests altogether and just test data manipulation using mocks or example data. This seems like the easy way out as almost all the business logic will live in the database.

Thanks






/r/flask
https://redd.it/5kuxfn

Jinja 2.8.1 Released with a Security Fix for str.format in Sandbox Mode
https://www.palletsprojects.com/blog/jinja-281-released/

/r/Python
https://redd.it/5kwvr6

Flask author Armin Ronacher: "Flask for Fun and Profit" at PyBay2016
https://www.youtube.com/watch?v=1ByQhAM5c1I&index=24&list=WL

/r/flask
https://redd.it/5kynb4

Notebook Gallery -- Most Viewed Notebooks!
http://nb.bianp.net/sort/views/

/r/JupyterNotebooks
https://redd.it/47mjzk

The Iterator Protocol: How for Loops Work in Python
http://treyhunner.com/2016/12/python-iterator-protocol-how-for-loops-work/

/r/Python
https://redd.it/5kzopj

[D] r/MachineLearning's 2016 Best Paper Award!

Hi guys!

Welcome to /r/MachineLearning's 2016 Best Paper Award!

The idea is to have a community-wide vote for the best papers of this year.

I hope you find this to be a good idea, mods please tell me if this breaks any rules/if you had something like this in store.

---

## How does it work?

**Nominate** by commenting on the dedicatd top level comments. Please provide a (paywall free) link. Feel free to justify your choice. Also if you're one of the author, be courteous and indicate it.

**Vote** by upvoting the nominees.

The **results** will be announced **by the end of next week** (6-7th of Jan.). Depending on the participation/interest I might change it.

It's that simple!

There are some simple rules to make sure everything runs smoothly, you can find them below, please read them before commenting.

---

## Categories

- [Best Paper of the year](https://www.reddit.com/r/MachineLearning/comments/5kxfkb/d_rmachinelearnings_2016_best_paper_award/dbrapti/)

> No rules! Any research paper you feel had the greatest impact/had top writing, any criterion is good.

- [Best student paper](https://www.reddit.com/r/MachineLearning/comments/5kxfkb/d_rmachinelearnings_2016_best_paper_award/dbraqkv/)

> Papers from a student, grad/undergrad/highschool, everyone who doesn't have a phd and goes to school. The student must be first author of course. Provide evidence if possible.

- [Best paper name](https://www.reddit.com/r/MachineLearning/comments/5kxfkb/d_rmachinelearnings_2016_best_paper_award/dbrar08/)

> Try to beat [this](http://www.oneweirdkerneltrick.com/spectral.pdf)

- [Best paper from academia](https://www.reddit.com/r/MachineLearning/comments/5kxfkb/d_rmachinelearnings_2016_best_paper_award/dbraroz/)

> Papers where the first author is from a university / a state research organization (eg INRIA in France).

- [Best paper from the industry](https://www.reddit.com/r/MachineLearning/comments/5kxfkb/d_rmachinelearnings_2016_best_paper_award/dbrasnz/)

> Great paper from a multi-billion tech company (or more generally a research lab sponsored by privat funds, eg. openai)

- [Best rejected paper](https://www.reddit.com/r/MachineLearning/comments/5kxfkb/d_rmachinelearnings_2016_best_paper_award/dbrat9t/)

> A chance of redemption for good papers that didn't make it trough peer review. Please provide evidence that the paper was rejected if possible.

- [Best unpublished preprint](https://www.reddit.com/r/MachineLearning/comments/5kxfkb/d_rmachinelearnings_2016_best_paper_award/dbratyb/)

> A category for those yet to be published (e.g. papers from the end of the year). This may or may not be redundant with the rejected paper category, we'll see.

- [Best theoretical paper](https://www.reddit.com/r/MachineLearning/comments/5kxfkb/d_rmachinelearnings_2016_best_paper_award/dbrauan/)

> Keep the math coming

- [Best non Deep Learning paper](https://www.reddit.com/r/MachineLearning/comments/5kxfkb/d_rmachinelearnings_2016_best_paper_award/dbraumv/)

> Because gaussian processes, random forests and kernel methods deserve a chance amid the DL hype train

---

## Rules

1. Only one nomination by comment. You can nominate multiple papers in different comments/categories.
2. Nominations should include a **link to the paper**. In case of an arxiv link, please link to the arxiv page and not the pdf directly. Please do not link paywalled articles.
3. Only **research paper** are to be nominated. This means no book, no memo or no tutorial/blog post for instance. This could be adressed in a separate award or category if there is enough demand.
4. For the sake of clarity, there are some rules on commenting :
- ***Do NOT comment on the main thread***. For discussion, use the [*discussion* thread](https://www.reddit.com/r/MachineLearning/comments/5kxfkb/d_rmachinelearnings_2016_best_paper_award/dbrap6u/)
- ***Please ONLY comment the other threads with nominations***. You can discuss individual nominations in child comments. However 1rst level comments on each thread should be nominations only.

5. Respect reddit and this sub's rules.

I am not a mod so I have no way of enforcing these rules, please follow them to keep the thread clear. Of course, suggestions are welcome [here](https://www.reddit.com/r/MachineLearning/comments/5kxfkb/d_rmachinelearnings_2016_best_paper_award/dbrap6u/).

---

That's it, have fun!

/r/MachineLearning
https://redd.it/5kxfkb

Be Careful with Python's New-Style String Format
http://lucumr.pocoo.org/2016/12/29/careful-with-str-format/

/r/Python
https://redd.it/5kzhnn

[R] Image Synthesis From Text With Deep Learning | Two Minute Papers
https://youtu.be/rAbhypxs1qQ

/r/MachineLearning
https://redd.it/5l00re

Script that sends you an email with your new IP address when It changes.
http://www.ryansadowski.xyz/posts/raspi-phone-home

/r/Python
https://redd.it/5kzpec

[P] Hip-Hop: Generating lyrics with RNNs
http://affinelayer.com/hiphop

/r/MachineLearning
https://redd.it/5l1bgb

Comparing python clustering algorithms
http://nbviewer.ipython.org/github/lmcinnes/hdbscan/blob/master/notebooks/Comparing%20Clustering%20Algorithms.ipynb

/r/pystats
https://redd.it/3o3emg

Using Python 3.6 in Ubuntu 16.04 - Anaconda way

### Notice:
This requires starting a virtual environment whenever you wish to use Python 3.6. However this does solve the f string highlighting issue that comes with using pip.

### Installation Instructions:

Download [Full Anaconda](https://www.continuum.io/downloads#linux) (Python 3.5 version)

Install Anaconda

bash Anaconda3-4.2.0-Linux-x86_64.sh
conda update conda

Create a virtual environment with Python 3.6

conda create --name snowflakes python=3.6
source activate snowflakes

Install a python3.6 kernel in Jupyter

python3.6 -m pip install ipykernel
python3.6 -m ipykernel install --user

### Note:
How this is different from the pip installation instructions is that you will end up with version 4.2.3 of the notebook server using Anaconda and version 4.2.1 using pip. This update fixes the f string formatting issue.

### Using the Jupyter Notebook:
Unlike when installing with pip, if you use Anaconda, you will need to have an active virtual environment within a terminal to use Python 3.6.

I use `jupyter notebook --no-browser` so that I can choose the browser I run the notebook in, so I use the following commands in the terminal.

source activate snowflakes
jupyter notebook --no-browser

When I'm done ctrl-C twice to close the notebook and `source deactivate snowflake` exit the virtual environment.

/r/IPython
https://redd.it/5l0ai7

A really nice Pandas cheat sheet, made by the Pandas guys themselves
https://github.com/pandas-dev/pandas/blob/master/doc/cheatsheet/Pandas_Cheat_Sheet.pdf

/r/Python
https://redd.it/5l0tsd

Why is always a with statement needed for working with files?

I have a very basic query about python. I've seen most examples use the `with` statement to read from or write to a file, something like this:

with open('output.txt','w') as f:
f.write(html_doc)

Is it not a better idea to simply call `open('output.txt','w').write(html_doc)` considering simplicity is very much preferred in python? Is closing file handles absolutely necessary, because similar i/o functions in `php` don't require you to close the file handles.

/r/Python
https://redd.it/5l2ld9

Small lib for logging objects in human-readable form.
https://github.com/oleg-golovanov/unilog

/r/Python
https://redd.it/5l2huz

What special features or batteries does Django ship with that Flask doesn't has?

I've always used only flask for all the web apps I've built and surprisingly, I've never needed something like django. For database interaction, I always had `sqlalchemy`, for other stuff like the google api client library or something, I usually just `pip install` stuff that I need.

As a result, I've never felt the need to look much deeper into something like django. Nevertheless, since django is so much popular than flask and everyone seem to sing praises of it, I'm curious to know what all batteries are included in it that makes it more "fully featured" as a lot of developers say, when compared to flask.

/r/Python
https://redd.it/5l353d

Top 10 Django Projects Started in 2016 on Github — End of Year Review
https://medium.com/@raiderrobert/top-10-django-projects-started-2016-end-of-year-review-8e4efc88ac43#.dpm0969el

/r/django
https://redd.it/5kzkes

From iPython itself--a gallery of interesting notebooks
https://github.com/ipython/ipython/wiki/A-gallery-of-interesting-IPython-Notebooks

/r/JupyterNotebooks
https://redd.it/47mkn4

Pandas Cheat Sheet
https://github.com/pandas-dev/pandas/blob/master/doc/cheatsheet/Pandas_Cheat_Sheet.pdf

/r/IPython
https://redd.it/5l3wfz