I need help with my first genetic algorithm

Hi I develop open source cryptocurrency trading software at www.tradewave.net. I have written python versions of many financial technical indicators, you can find them in the tradewave forums.

This is my first attempt at a genetic algo and I'm not sure where I went wrong. When I optimize the bot manually... backtesting each possible allele state I get better results than when I let the algo run through my evolve() definition and do the same optimization automatically on the same data set.

You can backtest the strategy free here (you have to create an account - its free):

https://tradewave.net/strategy/unlgoZdiYF

or just read the syntax highlighted code here:

http://pastebin.com/nLdy2CiY

The whole algo is about 400 lines, but I'm pretty sure the problem is in `def evolve()` which is less than 50 lines without comments. The rest of the code works fine if I pre train the genome and skip the evolution process. There is a description of the algo at the top of the pastebin and everything is well commented.

Can you help my find my error?

much thanks!

`def evolve():`


log('evolve() attempt at locus: %s' % storage.locus)

# Define simulated backtest window
depth = 5000; width = 200

# create deep closing price arrays for each of 3 trading pairs
ltcbtc_dc, ltcusd_dc, btcusd_dc = close_arrays(depth)

#matrix of closing price numpy arrays
price_matrix = [ltcbtc_dc, ltcusd_dc, btcusd_dc]

# find the current value of locus to be mutated
original_state = storage.genome[storage.locus]

# create 3 simulated starting portfolios w/ same start balance
portfolio = np.array([ [0.0, 1.0, 0.0],
[0.0, 1.0, 0.0],
[0.0, 1.0, 0.0] ])
# [usd, btc, ltc]

# empty any previous data in storage.macd_n_smooth from previous evolution
storage.macd_0_smooth = [[],[]] #ltcbtc
storage.macd_1_smooth = [[],[]] #ltcusd
storage.macd_2_smooth = [[],[]] #btcusd
# log('DEBUG 1 %s' % portfolio)

# run a simulated backtest for each allele state
# possible allele states are 0=usd; 1=btc; 2=ltc

for allele in range(0,3): # do this for each possible allele state (0,1,2)
#log('DEBUG 2 %s' % allele)
# for every possible slice[-200:] of 2000 deep closing array; oldest first:
for cut in range(depth, width, -1):
#if cut == depth: log('DEBUG 3 %s' % allele)
# create 2d matrix for market slices and macd for each currency pair
market_slice = [[],[],[]]
macd = [[],[],[]]
# create 1d matrix for last price
last = [0,0,0]
for n in range (0,3): # do this for each currency pair
#if (n == 0) and (cut==depth): log('DEBUG 4 %s' % allele)
#take a slice of the market for each currency pair
# slice notation reference
# z[-3:] # only the last 3
# z[:3] # only the first 3
market_slice[n] = (price_matrix[n][:cut])[-width:]
#calculate an macd value, and previous macd value for each currency pair
#try:
macd[n] = ta.MACD(market_slice[n], PERIOD1, PERIOD2)[-1]
#except Exception as e: log('talib fail %s' % e)
# smooth the macd for each currency pair
label = 'macd_' + str(n)
macd[n] = smooth(label, macd[n], AGGREGATION, 10)
# price normalize each macd by sma30
mean = (sum((market_slice[n])[-30:])/30)/100
macd[n] = macd[n]/mean
#extract last closing price
last[n] = market_slice[n][-1]

# calculate all_bull and all_bear from sim macd arrays for each instrument
all_bull, all_bear = all_bull_all_bear(macd[0], macd[1], macd[2])
#if (cut == depth): log('normalized macd %.5f %

.5f %.5f' % (macd[0][-1], macd[1][-1], macd[2][-1]))

# Temporarily MODIFY THE GENOME at the current locus
storage.genome[storage.locus] = allele
# use genome definition to return ideal position
sim_position = genome(
macd[0][-1], macd[1][-1], macd[2][-1], all_bull, all_bear)
# Undo MODIFY THE GENOME
storage.genome[storage.locus] = original_state

# check what each simulation is holding
sim_state = np.argmax(portfolio, axis=1)

# if the the simulator wants to change position
if sim_position != sim_state[allele]:
#if (cut == depth): log('allele %s simulator trade signal: to' % allele)

#update simulated portfolio via the last price
#last[0] 'ltcbtc close'
#last[1] 'ltcusd close'
#last[2] 'btcusd close'
if sim_position == 0:
#if (cut == depth): log('move to usd')
if sim_state[allele] == 1:
#if (cut == depth): log('via btcusd')
portfolio[allele][0] = portfolio[allele][1] * last[2]
portfolio[allele][1] = 0
if sim_state[allele] == 2:
#if (cut == depth): log('via ltcusd')
portfolio[allele][0] = portfolio[allele][2] * last[1]
portfolio[allele][2] = 0

if sim_position == 1:
#if (cut == depth): log('move to btc')
if sim_state[allele] == 0:
#if (cut == depth): log('via btcusd')
portfolio[allele][1] = portfolio[allele][0] / last[2]
portfolio[allele][0] = 0
if sim_state[allele] == 2:
#if (cut == depth): log('via ltcbtc')
portfolio[allele][1] = portfolio[allele][2] * last[0]
portfolio[allele][2] = 0

if sim_position == 2:
#if (cut == depth): log('move to ltc')
if sim_state[allele] == 0:
#if (cut == depth): log('via ltcusd')
portfolio[allele][2] = portfolio[allele][0] / last[1]
portfolio[allele][0] = 0
if sim_state[allele] == 1:
#if (cut == depth): log('via ltcbtc')
portfolio[allele][2] = portfolio[allele][1] / last[0]
portfolio[allele][1] = 0
#if (cut == depth):
# log('sim_position %s' % sim_position)
# log('sim_state %s' % sim_state)
# log(portfolio)

# move postion back to usd at the end of the simulated backtest
sim_state = np.argmax(portfolio, axis=1)
if sim_state[allele] != 0:
if sim_state[allele] == 1:
#if (cut == depth): log('via btcusd')
portfolio[allele][0] = portfolio[allele][1] * last[2]
portfolio[allele][1] = 0
if sim_state[allele] == 2:
#if (cut == depth): log('via ltcusd')
portfolio[allele][0] = portfolio[allele][2] * last[1]
portfolio[allele][2] = 0

log(portfolio)

# determine which allele has highest USD ROI
winner = -1
if portfolio[0][0] > max([portfolio[1][0], portfolio[2][0]]): winner = 0
if portfolio[1][0] > max([portfolio[0][0], portfolio[2][0]]): winner = 1
if portfolio[2][0] > max([portfolio[0][0], portfolio[1][0]]): winner = 2


# if mutation improves ROI
if (winner != original_state) and (winner > -1):
# evolve genome at this locus to winning allele
storag

[AF] UTC/Timezone in Database and onupdate functions. Can I get current_user.timezone?

Howdy!

I have been venturing in and out of the terrible timezone conundrum. I am on "US/Pacific". I have been trying to store all writes to the database in UTC but my web application is basically centered around time. A very import query below, will query based on the users hotel, filter messages posted today or updated today, and then filter out posts that even though they were wrote or updated today, since they post_date is in the future, not to show in the current day's message queue. I included a Base/Meta class that all the DB Models inherit from. The onupdate function are amazing. But with "US/Pacific" being -8 hr behind UTC there is problems in that gap in the below query.

Question: I know it is a major design flaw/risk that I am will to take. Does anyone know how to feed Flask-Login's current_user variable so I could replace datetime based on current_user.timzone instead of UTC. Example below. I know it is not it will take a little more code to make that actually happen then the below code but just and interested in anyone's initial thoughts and if that is possible. Basically I would like anytime this updates, to update based on the current_user.timezone rather than UTC. Thanks in advance for any help or advice.

What I would like:
timestamp = db.Column(db.DateTime, default=db.func.now()) #current
timestamp = db.Column(db.DateTime, default=current_user.timezone) #if possible, how?


def query_messages(date=None, tz=None, hotelcode=None, role='user'):
"""Query database for messages. I find it easier if query
is for 1 day instead of a range. .filter(Message.post_date <=date)
prevents messages scheduled for the future from displaying in current
days messages.
TODO: add flexibility if role='admin' or for role for multiple hotels
"""
print("DB query! username={}, hotelcode={}, timezone={}, is searching for {}".\
format(current_user.username, current_user.hotelcode, current_user.timezone, date))

q = Message.query.filter(Message.hotelcode == hotelcode)\
.filter(or_(Message.post_date==date, Message.last_updated_date_only==date))\
.filter(Message.post_date <= date)\
.order_by(Message.last_updated.desc()).all()
return q


class Base(db.Model):
"Base/Meta SQL table to inherit from"

__abstract__ = True
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
timestamp = db.Column(db.DateTime, default=db.func.now())
last_updated = db.Column(db.DateTime, default=db.func.now(), onupdate=db.func.now())
last_updated_date_only = db.Column(db.Date, default=db.func.current_date(), onupdate=db.func.current_date())
created_by = db.Column(db.String(40), default=None)
base_hotel = db.Column(db.String(5), default=None)
archive = db.Column(db.Boolean, default=False)


Class User(Base):
"""SQL Table"""

__tablename__ = 'user'

username = db.Column(db.String(64), unique=True)
_password = db.Column(db.String(128))
email = db.Column(db.String(128), default=None)
authenticated = db.Column(db.Boolean, default=True)
department = db.Column(db.String, default=None)
base_hotel = db.Column(db.String, default=None)
timezone = db.Column(db.String, default=None)
role = db.Column(db.String, default='user')





/r/flask
https://redd.it/5q63rw

Is there an easier way to use buttons and perform actions in Django?

Currently I have buttons on my "site" that perform actions based on the "device". The buttons show up as forms, and then when clicked, they perform login on the homepage to determine what to do. Are "form" buttons, the best way to do buttons?

here are the buttons that I'm using:

https://imgur.com/a/b9Xh9

Please see here

https://github.com/vektorhalili/networkninja/blob/master/devices/views.py


def devices(request):
##if post it means we are passing the name of device here to get arp
if request.method == "POST" and "arp:" in (request.POST['name']):
print(request.POST)
form = ArpDevice(request.POST)
if form.is_valid():
devicename = form.cleaned_data['name']
devicename = devicename.split(':')
devicename = devicename[1]
try:
device = Device.objects.get(name__iexact=devicename)
devicename = device.name
ipadd = device.ipadd
type = device.type
user = device.user
password = device.password
device = get_device_arp(ipadd, type, user, password)
return render(request, 'devicearp.html',{'device': device,'devicename': devicename})
except Exception:
raise Http404
elif request.method == "POST" and "config:" in (request.POST['name']):
#if post and contains config it means we clicked config button
form = ConfigDevice(request.POST)
if form.is_valid():
devicename = form.cleaned_data['name']
devicename = devicename.split(':')
devicename = devicename[1]
try:
device = Device.objects.get(name__iexact=devicename)
devicename = device.name
ipadd = device.ipadd
type = device.type
user = device.user
password = device.password
driver = get_network_driver(type)
device = driver(ipadd, user, password)
device.open()
config = device.get_config()
config = config['startup']
device.close()
return render(request, 'deviceconfig.html',{'config': config,'devicename': devicename})
except Exception:
raise Http404



Notice how I chose to handle the request.POST. Is that the best way to do this or am I creating too much work? This is my first django site so please be gentle.

I think I know the answer to this, it might be to use a specific action url and then trim that url and perform an action in the view.


I know what some of you might say regarding the buttons, that they take too long to process and are sync, I just learned about celery today so I incorporated that in to other pages however the buttons are meant for "DO THIS NOW" type of actions. While celery populates the postgresql backend every 15 min.






/r/django
https://redd.it/7khvnd

Registration form shows too many fields after I make custom register form model

I'm trying to make a custom registration form, but am running into trouble.

My `forms.py` has this:

class RegisterForm(UserCreationForm):
'''
Form that makes user using just email as username
'''
error_messages= {
"password_mismatch": _("Passwords do not match."),
"duplicate_email": _("Email already exists."),
"unique": _("Email already exists"),
}
register_username= forms.EmailField(label=_("Email"), widget=forms.TextInput(attrs={"placeholder":"Email"}))
register_password1= forms.CharField(label=_("Password"), widget=forms.PasswordInput(attrs={"placeholder":"Password"}))
register_password2= forms.CharField(label=_("Password confirmation"), widget=forms.PasswordInput(attrs={"placeholder":"Confirm password"}))

def clean_username(self):
username = self.cleaned_data["username"]
try:
User._default_manager.get(username=username)

#if the user exists, then let's raise an error message
raise forms.ValidationError(self.error_messages['duplicate_email'], #user my customized error message
code='duplicate_email', #set the error message key
)
except User.DoesNotExist:
return username # great, this user does not exist so we can continue the registration process

class Meta:
model= User
fields= ("username",)

My `views.py` looks like this:

def login_register(request, template="pages/login_register.html"):
registration_form= RegisterForm()
return render(request, template, {"registration_form": registration_form})


Which leads to my registration form in `login_register.html` rendering like this:

<form method="post">
<input type="hidden" name="csrfmiddlewaretoken" value="CSRF-TOKEN-HERE">
<p><input autofocus="" id="id_username" maxlength="150" name="username" type="text" required=""></p>
<p><input id="id_password1" name="password1" type="password" required=""></p>
<p><input id="id_password2" name="password2" type="password" required=""></p>
<p><input id="id_register_username" name="register_username" placeholder="Email" type="text" required=""></p>
<p><input id="id_register_password1" name="register_password1" placeholder="Password" type="password" required=""></p>
<p><input id="id_register_password2" name="register_password2" placeholder="Confirm password" type="password" required=""></p>
<button class="btn btn-primary" type="submit">Register</button>
</form>


I only want the last three `input` tags to be used. My goal is for my registration form to have three fields: Email, password, and password confirmation. Why is Django loading the three extra top `input` fields and how do I prevent that?

/r/django
https://redd.it/7xn7pe

Help with using public APIs

Edit-

day=datetime.datetime.today().strftime('%Y-%m-%d')#Get Todays Date, Daily Setting

my problem is below this

url = "https://demo.docusign.net/restapi/v2/accounts/"+accountId+"/envelopes"

querystring = {"from_date":Date,"status":"completed"}#if Envelope is completed

headers = { 'X-DocuSign-Authentication': "{\"Username\":\""+ email +"\",\"Password\":\""+Password+"\",\"IntegratorKey\": \""+IntegratorKey+"\"}", 'Content-Type': "application/json", 'Cache-Control': "no-cache", 'Postman-Token': "e53ceaba-512d-467b-9f95-1b89f6f65211" }

**my problem above this **

''' Envelope Response ''' response = requests.request("GET", url, headers=headers, params=querystring) envelopes=response.text

I currently have a python 3 program on my desktop. I run it with idle and everything is how I want it. What I want to do with Django is use this code to print its outputs on a webpage and have the user download it’s additional csv file output. I have managed to make a Django localhost and I am stuck at that point. I do not know how to use my python 3 code to run to webpage. The code is made up of api calls , I use postman to help me with sending the right parameters. I will add a example of code. All I want is for user to enter value such as “accountID” so that the api can complete the request and give them data for their own request. I would highly appreciate help on this

I have been trying for hours, from my understanding I have to make a view then send to HTML template

/r/django
https://redd.it/8ri846

How to download YouTube captions via API

I've been trying to work out how to download YouTube captions via the API but the official instructions are tailored towards command line code whereas I'm trying to do this in Python Shell.

Currently I've been following this page to no avail - [https://developers.google.com/youtube/v3/docs/captions/list](https://developers.google.com/youtube/v3/docs/captions/list)

What seems to trip me up is the Storage and args pieces of code which even after much googling doesn't make any sense to me.

See below:

storage = Storage("%s-oauth2.json" % sys.argv[0])
#nowhere on the page does it refer to this oauth2.json file 
credentials = storage.get()
if credentials is None or credentials.invalid:   
credentials = run_flow(flow, storage, args)
#if credentials is none why would storage still be needed as an argument?

In the second half of the page, it's all adding arguments to args which I assume is command line stuff that I don't want to use as I'm working from Python Shell.

&#x200B;

I've also been reading the official page on getting authorization OAuth 2.0 so I can rewrite this code myself but I can't get past this part: [https://developers.google.com/api-client-library/python/auth/web-app](https://developers.google.com/api-client-library/python/auth/web-app)

## *Step 5: Exchange

/r/Python
https://redd.it/9jvm8k

What is the proper way to approach this problem?

What I **want** to do is the following:

1. Client presses button on website
2. Page appears as "loading" as websites waits for a command from my PC
3. When webpage receives the command from my PC (POST request to website), it then loads a specific page DEPENDING on what is the string that I sent from my PC.

So far, what I've **done** is the following:

* Have a specific page to receive the POST request
* When server receives POST request, it will change a global variable named "status"
* When the user presses the button, webpage constantly checks for that global until it has been changed. When it does, it redirects user to a page depending on what that global is now.

This is the code for when the ***user*** presses the *submit button*

if request.method == "POST":
while True:
if status == "no changes yet": #If the global "status" is "no changes yet", wait
time.sleep(1)

elif

/r/flask
https://redd.it/kgwpoi

Video Understanding if name == 'main' in Python in 2 Minutes

In this quick 2-minute video, we'll demystify a fundamental concept in Python programming that's often a source of confusion for newcomers and even some experienced developers.

We'll explore the purpose and practical application of the if __name__ == '__main__' construct in Python scripts. No jargon, just clear explanations to help you gain a solid understanding of how this simple line of code can make your Python scripts more organized and versatile.

Video Link: https://youtu.be/WfPwvUjIZtE?si=ODo0DYZq51s\_nVct

If you have any suggestions or feedback, then don't hesitate.

/r/Python
https://redd.it/17rflks

A Visual Basic for Applications precompiler written in python.

I’ve just created an official release of a VBA precompiler written in python.

# What my Project Does

For those who don’t know, in VBA you can have precompiler blocks to change things for different OSs or VBA versions.
For example, to change the function signature for a new windows version:

#if Win64 Then
Function foo(Bar, Baz)
#Else
Function Foo(Bar)
#Endif

The problem with this is, if you want to scan raw source code, you can’t just ignore the precompiler lines, because now the code looks like you’ve tried to define the same function twice, and one was never ended.

This tool takes environment variables that the user provides, and will comment out any lines that need to be skipped, creating 100% valid precompiled code. It even performs the comparisons, and arithmetic operations specified in the VBA specification.

# Target Audience

People interested in malware prevention, static analysis, and Linting may find it helpful. Also, useful if you are interested in learning about compilers, ANTLR, and code parsing.

# Limitations
It is currently missing the standard library functions, like Cbool(), Abs(), etc. I’m guessing these are never called by users in the precompiler phase.

/r/Python
https://redd.it/1ay6lt1

What could I have done better here?

Hi, I'm pretty new to Python, and actual scripting in general, and I just wanted to ask if I could have done anything better here. Any critiques?

import time
import colorama
from colorama import Fore, Style

color = 'WHITE'
colorvar2 = 'WHITE'

#Reset colors
print(Style.RESETALL)

#Get current directory (for debugging)
#print(os.getcwd())

#Startup message
print("Welcome to the ASCII art reader. Please set the path to your ASCII art below.")

#Bold text
print(Style.BRIGHT)

#User-defined file path
path = input(Fore.BLUE + 'Please input the file path to your ASCII art: ')
color = input('Please input your desired color (default: white): ' + Style.RESETALL)

#If no ASCII art path specified
if not path:
print(Fore.RED +

/r/Python
https://redd.it/1p4ffmh