Selenium over scrapy
I keep seeing posts about using selenium to scrape pages and I’m curious why people prefer that over a library like scrapy
I’ve worked with both and absolutely prefer scrapy — just wondering out loud
Thank you
/r/Python
https://redd.it/137zeq8
I keep seeing posts about using selenium to scrape pages and I’m curious why people prefer that over a library like scrapy
I’ve worked with both and absolutely prefer scrapy — just wondering out loud
Thank you
/r/Python
https://redd.it/137zeq8
Reddit
r/Python on Reddit: Selenium over scrapy
Posted by u/geekluv - 4 votes and 11 comments
[D] Google "We Have No Moat, And Neither Does OpenAI": Leaked Internal Google Document Claims Open Source AI Will Outcompete Google and OpenAI
https://www.semianalysis.com/p/google-we-have-no-moat-and-neither
/r/MachineLearning
https://redd.it/137rxgw
https://www.semianalysis.com/p/google-we-have-no-moat-and-neither
/r/MachineLearning
https://redd.it/137rxgw
Semianalysis
Google "We Have No Moat, And Neither Does OpenAI"
Leaked Internal Google Document Claims Open Source AI Will Outcompete Google and OpenAI
What IDE do y’all use
I’m the process of learning python. I used net beans for Java
/r/Python
https://redd.it/1383eox
I’m the process of learning python. I used net beans for Java
/r/Python
https://redd.it/1383eox
Reddit
r/Python on Reddit: What IDE do y’all use
Posted by u/youngnebsi - 47 votes and 192 comments
Behave Driven Development Python library
There are Python BDD frameworks out there, most notably `behave` (https://github.com/behave/behave).
They are nice. However, from time to time I find myself fighting against some design choices made by their authors. I thought that things can be made more lightweight, flexible and less constrained.
I thought that a tool with more of a library-like architecture, rather than a framework, might be fit for the purpose.
`Rumex` is my work-in-progress implementation of such a tool: https://github.com/uigctaw/rumex
I would appreciate any feedback. Be it minor things or fundamental ones - like perhaps I'm misguided in my assessment of perceived inflexibility of the existing tools.
/r/Python
https://redd.it/13874wi
There are Python BDD frameworks out there, most notably `behave` (https://github.com/behave/behave).
They are nice. However, from time to time I find myself fighting against some design choices made by their authors. I thought that things can be made more lightweight, flexible and less constrained.
I thought that a tool with more of a library-like architecture, rather than a framework, might be fit for the purpose.
`Rumex` is my work-in-progress implementation of such a tool: https://github.com/uigctaw/rumex
I would appreciate any feedback. Be it minor things or fundamental ones - like perhaps I'm misguided in my assessment of perceived inflexibility of the existing tools.
/r/Python
https://redd.it/13874wi
GitHub
GitHub - behave/behave: BDD, Python style.
BDD, Python style. Contribute to behave/behave development by creating an account on GitHub.
Hosting recommendation
Hi all. I have a couple of dedi servers at inmotion, but they all lack the "Linux Cloud" feature and want to charge me rather high amounts to enable this feature. Ideally I was going to deploy via the "Setup Python App' in cpanel, but of course, this requires Cloud Linux.
Trying to do this without it via WHM has proven to be a nightmare and I was hoping somebody could recommend a suitable alternative for a domain (with sub domains), static IP and easy deployment of django apps?
/r/django
https://redd.it/138h1y4
Hi all. I have a couple of dedi servers at inmotion, but they all lack the "Linux Cloud" feature and want to charge me rather high amounts to enable this feature. Ideally I was going to deploy via the "Setup Python App' in cpanel, but of course, this requires Cloud Linux.
Trying to do this without it via WHM has proven to be a nightmare and I was hoping somebody could recommend a suitable alternative for a domain (with sub domains), static IP and easy deployment of django apps?
/r/django
https://redd.it/138h1y4
Reddit
r/django on Reddit: Hosting recommendation
Posted by u/bedroomsport - No votes and no comments
Concantantion of strings from fields of model
I'm trying to combine values passed to fields name and surname to create a string to be stored in full\_name field. When l was looking for solutions, I found suggestion to overwrite Django's save method. I tried, but get no expected results. Instead after making a few instances of Author class and trying to loop over them in Django shell. I get following results:
> \--snip--/models.py", line 13, in __str__
>
> """Override save method."""
>
>AttributeError: 'Author' object has no attribute 'full_name'
​
Code is here:
https://pastebin.com/Kx9ea6x4
/r/djangolearning
https://redd.it/138f5wx
I'm trying to combine values passed to fields name and surname to create a string to be stored in full\_name field. When l was looking for solutions, I found suggestion to overwrite Django's save method. I tried, but get no expected results. Instead after making a few instances of Author class and trying to loop over them in Django shell. I get following results:
> \--snip--/models.py", line 13, in __str__
>
> """Override save method."""
>
>AttributeError: 'Author' object has no attribute 'full_name'
​
Code is here:
https://pastebin.com/Kx9ea6x4
/r/djangolearning
https://redd.it/138f5wx
Pastebin
from django.db import models# Create your models here.class Author(model - Pastebin.com
Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.
When do you use Q?
A question that just came to my mind. I wrote:
etfs = ETF.objects.filter(Q(updated_at__lte=timezone.now() -timezone.timedelta(days=1)))
but to be honest, it's not a complex query. However, I am tending to automatically use a Q object. When do you use it, would you use it here?
I hope to read some opinions
/r/django
https://redd.it/138kki6
A question that just came to my mind. I wrote:
etfs = ETF.objects.filter(Q(updated_at__lte=timezone.now() -timezone.timedelta(days=1)))
but to be honest, it's not a complex query. However, I am tending to automatically use a Q object. When do you use it, would you use it here?
I hope to read some opinions
/r/django
https://redd.it/138kki6
Reddit
r/django on Reddit: When do you use Q?
Posted by u/palmy-investing - 3 votes and 4 comments
Can you monitor multiple urlpatterns with Django's runserver autoreload
In the project I'm working on, I'd like to have it run 2 apps on one runserver for development purposes (standalone in production), to utilize amongs others autoreload options.
What I'd like to do is have wsgi application, and asgi application by implementing Django channels next to it. For that, I feel like I need to have 2 ROOT_URLCONFs basically, as one would apply to wsgi, and other to asgi application. I am interested is there a way to make runserver pay attention to multiple ROOT_URLCONFs or urlpatterns whatever you want to name them? And does this approach make sense in the first place, as from what I've seen on the web, it's not such an uncommon setup, but I haven't seen anyone placing it under one runserver.
For the ease of development setup it would be one server, as there's already an overhead on startup of the project development.
/r/django
https://redd.it/138x6u5
In the project I'm working on, I'd like to have it run 2 apps on one runserver for development purposes (standalone in production), to utilize amongs others autoreload options.
What I'd like to do is have wsgi application, and asgi application by implementing Django channels next to it. For that, I feel like I need to have 2 ROOT_URLCONFs basically, as one would apply to wsgi, and other to asgi application. I am interested is there a way to make runserver pay attention to multiple ROOT_URLCONFs or urlpatterns whatever you want to name them? And does this approach make sense in the first place, as from what I've seen on the web, it's not such an uncommon setup, but I haven't seen anyone placing it under one runserver.
For the ease of development setup it would be one server, as there's already an overhead on startup of the project development.
/r/django
https://redd.it/138x6u5
Reddit
r/django on Reddit: Can you monitor multiple urlpatterns with Django's runserver autoreload
Posted by u/Competitive_Worth263 - No votes and no comments
Deploy Django app using Docker or not...
So on my previous project (which right now has 300,000 page views per month) I tried using docker but kept having issues so I quickly gave up.
Instead, I ended up deploying it in AWS by using an EC2 Launch Template, so whenever a new instance is needed the template will launch and set up the instance (updates yum, installs Python, and Code Deploy agent). Then the Code Pipeline will deploy and run my application using the Code Deploy agent.
I also have a NextJs frontend application that gets deployed in the same EC2 instance. So whenever there is any autoscaling, both Django and Nextjs get scaled at the same time.
All the infrastructure is set up using a Cloudformation template which took me almost 1 month to figure out since it was the first time I was dealing CloudFormation, Code Pipeline, Launch templates, autoscaling, etc.
Okay that's it for my current architecture for deploying my Django Application.
​
For my current project I'm considering using Docker to deploy it on ECS. Here are the current reasons why I'm reconsidering Docker once again.
1. People have mentioned that deploying Django directly in EC2 server (manually or through launch template)
/r/django
https://redd.it/138xurn
So on my previous project (which right now has 300,000 page views per month) I tried using docker but kept having issues so I quickly gave up.
Instead, I ended up deploying it in AWS by using an EC2 Launch Template, so whenever a new instance is needed the template will launch and set up the instance (updates yum, installs Python, and Code Deploy agent). Then the Code Pipeline will deploy and run my application using the Code Deploy agent.
I also have a NextJs frontend application that gets deployed in the same EC2 instance. So whenever there is any autoscaling, both Django and Nextjs get scaled at the same time.
All the infrastructure is set up using a Cloudformation template which took me almost 1 month to figure out since it was the first time I was dealing CloudFormation, Code Pipeline, Launch templates, autoscaling, etc.
Okay that's it for my current architecture for deploying my Django Application.
​
For my current project I'm considering using Docker to deploy it on ECS. Here are the current reasons why I'm reconsidering Docker once again.
1. People have mentioned that deploying Django directly in EC2 server (manually or through launch template)
/r/django
https://redd.it/138xurn
Reddit
r/django on Reddit: Deploy Django app using Docker or not...
Posted by u/adrenaline681 - 9 votes and 6 comments
Saturday Daily Thread: Resource Request and Sharing! Daily Thread
Found a neat resource related to Python over the past week? Looking for a resource to explain a certain topic?
Use this thread to chat about and share Python resources!
/r/Python
https://redd.it/1395uno
Found a neat resource related to Python over the past week? Looking for a resource to explain a certain topic?
Use this thread to chat about and share Python resources!
/r/Python
https://redd.it/1395uno
Reddit
r/Python on Reddit: Saturday Daily Thread: Resource Request and Sharing! Daily Thread
Posted by u/Im__Joseph - 1 vote and no comments
My Python-based Code Smells Catalog Paper was finally published in Springer! 🎉
Hey! Some time ago, I posted here my Code Smells Catalog website, which has a big pile of aggregated general Code Smells with examples & solutions written in Python.
Since then, the website is still reporting a daily activity of at least \~50 users and has an all-time peak of 8500, which is incredible! People seem to be using it both as a knowledge source and as a pointer for Code Reviews (referring to a particular Code Smell page).
Last month, my paper finally got published in Springer, which means a lot to me, and because of the occasion, I wanted to re-share the website catalog and paper (*pre-print* is free!) for those who missed it around a year ago.
Thank you all for your appreciation! It is so lovely that it brings at least a tiny bit of value to some of you! ❤️
/r/Python
https://redd.it/138u9yf
Hey! Some time ago, I posted here my Code Smells Catalog website, which has a big pile of aggregated general Code Smells with examples & solutions written in Python.
Since then, the website is still reporting a daily activity of at least \~50 users and has an all-time peak of 8500, which is incredible! People seem to be using it both as a knowledge source and as a pointer for Code Reviews (referring to a particular Code Smell page).
Last month, my paper finally got published in Springer, which means a lot to me, and because of the occasion, I wanted to re-share the website catalog and paper (*pre-print* is free!) for those who missed it around a year ago.
Thank you all for your appreciation! It is so lovely that it brings at least a tiny bit of value to some of you! ❤️
/r/Python
https://redd.it/138u9yf
luzkan.github.io
Code Smells Catalog
A collection of Bad Code Smells in a Catalog form for Developers & Researchers. Code Smell is a typical bad code implementation, and learning these concepts immiedietly makes you a better developer!
[P] The first RedPajama models are here! The 3B and 7B models are now available under Apache 2.0, including instruction-tuned and chat versions. These models aim replicate LLaMA as closely as possible.
https://www.together.xyz/blog/redpajama-models-v1
/r/MachineLearning
https://redd.it/13965sq
https://www.together.xyz/blog/redpajama-models-v1
/r/MachineLearning
https://redd.it/13965sq
www.together.ai
Releasing 3B and 7B RedPajama-INCITE family of models including base, instruction-tuned & chat models
Checkout the tool I coded to generate a multiple choice quizz from the content of any uploaded PDF.
It’s a Streamlit Python App.The langchain GPT template is in French so make sure you translate it in your language for better result!
https://github.com/fbellame/pdf-to-quizz
/r/Python
https://redd.it/139l9da
It’s a Streamlit Python App.The langchain GPT template is in French so make sure you translate it in your language for better result!
https://github.com/fbellame/pdf-to-quizz
/r/Python
https://redd.it/139l9da
GitHub
GitHub - fbellame/pdf-to-quizz: Upload a PDF and generate a quizz
Upload a PDF and generate a quizz. Contribute to fbellame/pdf-to-quizz development by creating an account on GitHub.
Context Managers And The 'with' Statement In Python: A Comprehensive Guide With Examples
​
https://preview.redd.it/u6xczndjw7ya1.png?width=1600&format=png&auto=webp&v=enabled&s=373ea2365d45780156dd6d63bc86b70864513bce
Resource management is critical in any programming language, and the use of system resources in programs is common.
Assume we are working on a project where we need to establish a database connection or perform file operations; these operations consume resources that are limited in supply, so they must be released after use; otherwise, issues such as running out of memory or file descriptors, or exceeding the maximum number of connections or network bandwidth can arise.
**Context managers** come to the rescue in these situations; they are used to prepare resources for use by the program and then free resources when the resources are no longer required, even if exceptions have occurred.
Context managers provide a mechanism for the setup and teardown of the resources associated with the program. It improves the readability, conciseness, and maintainability of the code.
The context managers can be used with Python's `with` statement to handle the **setup** and **teardown** of resources in the program. However, we can create our own custom context manager by implementing the **enter(setup)** logic and **exit(teardown)** logic within a Python class.
In this article, we'll learn:
* **What is context manager and why they are used**
* **Using context manager with the** `with` **statement**
* **Implementing context management
/r/Python
https://redd.it/139prq4
​
https://preview.redd.it/u6xczndjw7ya1.png?width=1600&format=png&auto=webp&v=enabled&s=373ea2365d45780156dd6d63bc86b70864513bce
Resource management is critical in any programming language, and the use of system resources in programs is common.
Assume we are working on a project where we need to establish a database connection or perform file operations; these operations consume resources that are limited in supply, so they must be released after use; otherwise, issues such as running out of memory or file descriptors, or exceeding the maximum number of connections or network bandwidth can arise.
**Context managers** come to the rescue in these situations; they are used to prepare resources for use by the program and then free resources when the resources are no longer required, even if exceptions have occurred.
Context managers provide a mechanism for the setup and teardown of the resources associated with the program. It improves the readability, conciseness, and maintainability of the code.
The context managers can be used with Python's `with` statement to handle the **setup** and **teardown** of resources in the program. However, we can create our own custom context manager by implementing the **enter(setup)** logic and **exit(teardown)** logic within a Python class.
In this article, we'll learn:
* **What is context manager and why they are used**
* **Using context manager with the** `with` **statement**
* **Implementing context management
/r/Python
https://redd.it/139prq4
I recorded a crash course on Polars library of Python (Great library for working with big data) and uploaded it on Youtube
Hello everyone, I created a crash course of Polars library of Python and talked about data types in Polars, reading and writing operations, file handling, and powerful data manipulation techniques. I am leaving the link, have a great day!
https://www.youtube.com/watch?v=aiHSMYvoqYE
/r/Python
https://redd.it/139wsvp
Hello everyone, I created a crash course of Polars library of Python and talked about data types in Polars, reading and writing operations, file handling, and powerful data manipulation techniques. I am leaving the link, have a great day!
https://www.youtube.com/watch?v=aiHSMYvoqYE
/r/Python
https://redd.it/139wsvp
YouTube
Master Python Polars for Efficient Big Data Handling: 22-Minute Crash Course!
Unlock the power of Python Polars for efficient big data handling in this 22-minute crash course. Dive into practical techniques and real-world examples to streamline your data processing workflows and maximize performance. Perfect for data enthusiasts and…
How to handle the return code of the confirm() javascript result?
Hi, so this is probably a very simple question but I don't know where to start looking (well, google obviously but I need the right terms to look for).
I want to show a little javascript popup where the user needs to press OK or cancel and confirm() seems like the easiest way to do that but I don't understand how to handle the return code in Flask/Python. The w3schools site gives a nice example of how to handle it in plain HTML but I need to execute 'actual' code instead of just show some text and I have no idea how start as it seems there's no real communication back to the server and it's only client-side..
/r/flask
https://redd.it/139hov9
Hi, so this is probably a very simple question but I don't know where to start looking (well, google obviously but I need the right terms to look for).
I want to show a little javascript popup where the user needs to press OK or cancel and confirm() seems like the easiest way to do that but I don't understand how to handle the return code in Flask/Python. The w3schools site gives a nice example of how to handle it in plain HTML but I need to execute 'actual' code instead of just show some text and I have no idea how start as it seems there's no real communication back to the server and it's only client-side..
/r/flask
https://redd.it/139hov9
W3Schools
W3Schools offers free online tutorials, references and exercises in all the major languages of the web. Covering popular subjects like HTML, CSS, JavaScript, Python, SQL, Java, and many, many more.
FrontEngine can play videos, GIFs, webpages, and more on your screen foreground.
I create this side project that can play videos, GIFs, webpages, and more on your screen foreground.
FrontEngine can draw below type on front of your screen.
You can use your computer as usual without being disturbed.
Video.
Website.
GIF & WEBP.
Image.
Play sound.
Text
GitHub repo: Intergration-Automation-Testing/FrontEngine (github.com)
Youtube showcase: https://youtu.be/fewogcb3b8Y
/r/Python
https://redd.it/139vgm7
I create this side project that can play videos, GIFs, webpages, and more on your screen foreground.
FrontEngine can draw below type on front of your screen.
You can use your computer as usual without being disturbed.
Video.
Website.
GIF & WEBP.
Image.
Play sound.
Text
GitHub repo: Intergration-Automation-Testing/FrontEngine (github.com)
Youtube showcase: https://youtu.be/fewogcb3b8Y
/r/Python
https://redd.it/139vgm7
GitHub
GitHub - Intergration-Automation-Testing/FrontEngine
Contribute to Intergration-Automation-Testing/FrontEngine development by creating an account on GitHub.
Is it possible pickle a function with its dependencies?
I am trying to pickle Python functions with their dependencies (e.g. numpy, etc) and execute the pickled function on worker nodes without installing dependencies on the worker node. Is this possible?
If not, what are some other approaches to accomplish this task?
/r/Python
https://redd.it/139vz41
I am trying to pickle Python functions with their dependencies (e.g. numpy, etc) and execute the pickled function on worker nodes without installing dependencies on the worker node. Is this possible?
If not, what are some other approaches to accomplish this task?
/r/Python
https://redd.it/139vz41
Reddit
r/Python on Reddit: Is it possible pickle a function with its dependencies?
Posted by u/FourSigma - 7 votes and 17 comments
Redirect with token in header using href
I'm trying to include an auth header that includes a token using a href in a link like below:
<a href="/some/route/{{ data }}"></a>
Because I want this route to only be available to users with a valid auth token, but I can't figure out how to include the token in the header from here. I've tried using a GET request with AJAX like below:
$.ajax(
{
type: "GET",
url: "/some/route/" + data,
headers: {
Authorization: get_token()
},
async: true,
success: function()
{
console.log("worked");
},
error: function(xhr, status, error)
{
/r/flask
https://redd.it/13a2f7f
I'm trying to include an auth header that includes a token using a href in a link like below:
<a href="/some/route/{{ data }}"></a>
Because I want this route to only be available to users with a valid auth token, but I can't figure out how to include the token in the header from here. I've tried using a GET request with AJAX like below:
$.ajax(
{
type: "GET",
url: "/some/route/" + data,
headers: {
Authorization: get_token()
},
async: true,
success: function()
{
console.log("worked");
},
error: function(xhr, status, error)
{
/r/flask
https://redd.it/13a2f7f
Reddit
r/flask on Reddit: Redirect with token in header using href
Posted by u/Zalosath - 1 vote and 7 comments