AmazonMe
Hello everyone,
I have been working on this scraping project to polish my scraping skills and wanted to share it with this awesome Python and web scraping community. It's an Amazon scraper that scrapes the product information according to the URL provided by the user. The scraper utilizes playwright automation to automate and extract the data. (Will add more functionalities)
The code is very beginner friendly and will help beginners to get their hands dirty on websites like Amazon. Your feedback is highly appreciated
/r/Python
https://redd.it/10gekp4
Hello everyone,
I have been working on this scraping project to polish my scraping skills and wanted to share it with this awesome Python and web scraping community. It's an Amazon scraper that scrapes the product information according to the URL provided by the user. The scraper utilizes playwright automation to automate and extract the data. (Will add more functionalities)
The code is very beginner friendly and will help beginners to get their hands dirty on websites like Amazon. Your feedback is highly appreciated
/r/Python
https://redd.it/10gekp4
GitHub
GitHub - sushil-rgb/AmazonMe: Introducing AmazonMe, a Python-based web scraper designed to extract data from amazon.com using theβ¦
Introducing AmazonMe, a Python-based web scraper designed to extract data from amazon.com using the requests and beautifulSoup libraries. It simplifies navigation and makes it easy to gather inform...
for example, I want to develop a whatsapp-like application for ios. is it possible to write the design and interface with swift and all the remaining codes with python?
I wanna learn it
/r/Python
https://redd.it/10gr2z6
I wanna learn it
/r/Python
https://redd.it/10gr2z6
reddit
for example, I want to develop a whatsapp-like application for...
I wanna learn it
Appending data to a file....
Hello,
I am trying to append data to a file that I am reading from in the path "
And here is my code..
When I run this code, I get the following in the file "
​
With Python, I am still "on my trainer wheels on my bike". and so how
/r/Python
https://redd.it/10gr475
Hello,
I am trying to append data to a file that I am reading from in the path "
01Resources/name.json", and this file contains the following data..{"Name" : "Robert", "Gender" : "Male","age" :"23"}{"Name" : "peter", "Gender" : "Male","age" :"24"}{"Name" : "james", "Gender" : "Male","age" :"25"}{"Name" : "janet", "Gender" : "Female","age" :"26"}{"Name" : "mary", "Gender" : "Female","age" :"27"}And here is my code..
#################### # 0. Data to be appended. data = {"Name": "Roberta", "Gender": "FeMale","age":"35"}#################### # 1/ read the the JSON data and append by writing new data in.fdataOut1 = open('01Resources/name.json', "a") data_strng = str(fdataOut1) # the data must be string to write to a file fdataOut1.write(data_strng) #fdataOut1.write("This is test of appended data") fdataOut1.close() fdataIn1 = open('01Resources/name.json', "r") data1 = fdataIn1.read() print("") print("####################") print("Now the file has more content") print("contents of whole input data file: " + data1) fdataIn1.close()When I run this code, I get the following in the file "
01Resources/name.json"....{"Name" : "Robert", "Gender" : "Male","age" :"23"}{"Name" : "peter", "Gender" : "Male","age" :"24"}{"Name" : "james", "Gender" : "Male","age" :"25"}{"Name" : "janet", "Gender" : "Female","age" :"26"}{"Name" : "mary", "Gender" : "Female","age" :"27"}<_io.TextIOWrapper name='01Resources/name.json' mode='a' encoding='UTF-8'>​
With Python, I am still "on my trainer wheels on my bike". and so how
/r/Python
https://redd.it/10gr475
reddit
Appending data to a file....
Hello, I am trying to append data to a file that I am reading from in the path "`01Resources/name.json`", and this file contains the following...
I'm kinda noob in Portfolio's World
So I'm trying to get into portfolio's world and I made a website all by myself with Python and Django, if someone could help me with tips to improve, features that would be nice to implement on backend or just saying that is good already it would help much. Thank you :)
My Portfolio
/r/django
https://redd.it/10gsyez
So I'm trying to get into portfolio's world and I made a website all by myself with Python and Django, if someone could help me with tips to improve, features that would be nice to implement on backend or just saying that is good already it would help much. Thank you :)
My Portfolio
/r/django
https://redd.it/10gsyez
Creating a database before starting django project
Is it possible to use an existing database in a django project or does the data base have to be created as part of making the django app?
​
Thanks for your time.
/r/django
https://redd.it/10gspzs
Is it possible to use an existing database in a django project or does the data base have to be created as part of making the django app?
​
Thanks for your time.
/r/django
https://redd.it/10gspzs
reddit
Creating a database before starting django project
Is it possible to use an existing database in a django project or does the data base have to be created as part of making the django app?...
How to code it is so that section 1 automatically links to discussion a. I'm using topic 1,2,3 to organize the discussions and the {%for post in page.get_parent%} in the template to make a navigation system. I want section 1 to always link to discussion a.
/r/django
https://redd.it/10gsn1x
/r/django
https://redd.it/10gsn1x
Best way to load initial data and maintain it in production ?
We develop surveys for regulatory purposes and we expose them via an API (DRF).
Each survey has questions and we'll have different set of questions for each of our client (wording and number of questions can vary).
We are going live soon, and we haven't settled yet on how to initially load thatdata and how to maintain it (ie. wording may change over time, new question can be added, etc...).
Here's our feeling so far:
Fixtures: We use them extensively in our unit tests and our test environments but it doesn't feel right for production for several reasons:
No tracking of what has been run or what has not yet been run
Force us to deal with PKs
One fixtures can easily be forgotten
Migrations with raw SQL: idea would be to have a folder for each client with the different scripts to be run
Harder to generate and maintain (no loaddata / dumpdata)
What's your take on this ?
Thanks
/r/django
https://redd.it/10gcrw8
We develop surveys for regulatory purposes and we expose them via an API (DRF).
Each survey has questions and we'll have different set of questions for each of our client (wording and number of questions can vary).
We are going live soon, and we haven't settled yet on how to initially load thatdata and how to maintain it (ie. wording may change over time, new question can be added, etc...).
Here's our feeling so far:
Fixtures: We use them extensively in our unit tests and our test environments but it doesn't feel right for production for several reasons:
No tracking of what has been run or what has not yet been run
Force us to deal with PKs
One fixtures can easily be forgotten
Migrations with raw SQL: idea would be to have a folder for each client with the different scripts to be run
Harder to generate and maintain (no loaddata / dumpdata)
What's your take on this ?
Thanks
/r/django
https://redd.it/10gcrw8
reddit
Best way to load initial data and maintain it in production ?
We develop surveys for regulatory purposes and we expose them via an API (DRF). Each survey has questions and we'll have different set of...
Aggregate posts from your Mastodon timeline (built with Django)
Hey all, I built https://fediview.com/ with Django. It surfaces posts from your Mastodon timeline with a very simple algorithm -- super useful to just "catch up" on posts you might have missed.
Hopefully it's useful if anyone is on Mastodon. I'd love to hear any feedback!
/r/django
https://redd.it/10glm2u
Hey all, I built https://fediview.com/ with Django. It surfaces posts from your Mastodon timeline with a very simple algorithm -- super useful to just "catch up" on posts you might have missed.
Hopefully it's useful if anyone is on Mastodon. I'd love to hear any feedback!
/r/django
https://redd.it/10glm2u
Fediview
fediview - The algorithmic timeline for Mastodon
Generate an algorithmic summary for Mastodon timelines
Jupyter + copilot
Anyone had success setting these two up together?
/r/JupyterNotebooks
https://redd.it/10gnx7k
Anyone had success setting these two up together?
/r/JupyterNotebooks
https://redd.it/10gnx7k
reddit
Jupyter + copilot
Anyone had success setting these two up together?
How long will data be stored in RAM?
Since I use notebook as a demo for images and plots when I do data analysis, I often have my jupyternotebook on and leave it on for weeks incase I need to use/check certain notebooks ocassionally.
​
In the jupyter notebook, there are often multiple cells, and there are some variables being stored and pass to the next cell. And my question is that, how long will the intermediate data be kept, and can I run the cell even after weeks and trust the output as long as there is no error reported?
​
My guess is that if the RAM throws away certain groups of data, then I should not be able to run the cell since the intermediate data it needs is no longer available, which means, as long as it can run, the data is still there.
Also, I am using m1 Macbook, which I know will use the hard drive as RAM in certain cases, not sure if this means the intermediate data will be kept on some temporary files on the hard drive, which sounds to be a safer place to store.
/r/JupyterNotebooks
https://redd.it/10gjv8j
Since I use notebook as a demo for images and plots when I do data analysis, I often have my jupyternotebook on and leave it on for weeks incase I need to use/check certain notebooks ocassionally.
​
In the jupyter notebook, there are often multiple cells, and there are some variables being stored and pass to the next cell. And my question is that, how long will the intermediate data be kept, and can I run the cell even after weeks and trust the output as long as there is no error reported?
​
My guess is that if the RAM throws away certain groups of data, then I should not be able to run the cell since the intermediate data it needs is no longer available, which means, as long as it can run, the data is still there.
Also, I am using m1 Macbook, which I know will use the hard drive as RAM in certain cases, not sure if this means the intermediate data will be kept on some temporary files on the hard drive, which sounds to be a safer place to store.
/r/JupyterNotebooks
https://redd.it/10gjv8j
reddit
How long will data be stored in RAM?
Since I use notebook as a demo for images and plots when I do data analysis, I often have my jupyternotebook on and leave it on for weeks incase I...
Today I re-learned: Python function default arguments are retained between executions
https://www.valentinog.com/blog/tirl-python-default-arguments/
/r/Python
https://redd.it/10gt7tv
https://www.valentinog.com/blog/tirl-python-default-arguments/
/r/Python
https://redd.it/10gt7tv
Valentino Gagliardi's Blog
Today I re-learned: Python function default arguments are retained between executions
A short story about Python function default arguments, and why testing even simple things is important.
Parsing and validating
The upcoming release of msgspec
(my fast & friendly
serialization library) adds builtin support for YAML & TOML formats, on top of
the existing JSON & msgpack support.
To demonstrate this, I wrote up a quick example of parsing and validating
https://jcristharif.com/msgspec/examples/pyproject-toml.html
It applies the following standard msgspec workflow to the
format:
1. Define a schema using standard python type annotations. If you're already a
dataclasses/attrs/pydantic user this should feel pretty familiar.
supports a wide selection of common stdlib
types. Any
additional needed types can also be added via an extension mechanism.
2. Pass that schema to one of the supported
using
input data matches the schema. If it's valid the specified type is returned,
otherwise a user-friendly error message is raised detailing where the data
is malformed.
The same technique can be applied for any of the formats msgspec supports,
allowing msgspec to be a one-stop-shop for serialization & validation in
Python.
Note that unlike the existing JSON & msgpack support, these new formats rely on
external parser libraries (
has a
/r/Python
https://redd.it/10gzbgo
pyproject.toml files with msgspecThe upcoming release of msgspec
(my fast & friendly
serialization library) adds builtin support for YAML & TOML formats, on top of
the existing JSON & msgpack support.
To demonstrate this, I wrote up a quick example of parsing and validating
pyproject.toml files using msgspec:https://jcristharif.com/msgspec/examples/pyproject-toml.html
It applies the following standard msgspec workflow to the
pyproject.toml fileformat:
1. Define a schema using standard python type annotations. If you're already a
dataclasses/attrs/pydantic user this should feel pretty familiar.
msgspecsupports a wide selection of common stdlib
types. Any
additional needed types can also be added via an extension mechanism.
2. Pass that schema to one of the supported
decode functions (here we'reusing
msgspec.toml.decode). The decoder will parse and validate that theinput data matches the schema. If it's valid the specified type is returned,
otherwise a user-friendly error message is raised detailing where the data
is malformed.
The same technique can be applied for any of the formats msgspec supports,
allowing msgspec to be a one-stop-shop for serialization & validation in
Python.
Note that unlike the existing JSON & msgpack support, these new formats rely on
external parser libraries (
msgspec includes a fast, custom JSON parser). Thishas a
/r/Python
https://redd.it/10gzbgo
Getting Started With Property-Based Testing in Python π With Hypothesis and Pytest - Semaphore
https://semaphoreci.com/blog/property-based-testing-python-hypothesis-pytest
/r/Python
https://redd.it/10gv7b7
https://semaphoreci.com/blog/property-based-testing-python-hypothesis-pytest
/r/Python
https://redd.it/10gv7b7
Semaphore
Getting Started With Property-Based Testing in Python With Hypothesis and Pytest - Semaphore
In this tutorial, we will be learning about the concepts behind property-based testing, and then we will put those concepts to practice.
Saturday Daily Thread: Resource Request and Sharing! Daily Thread
Found a neat resource related to Python over the past week? Looking for a resource to explain a certain topic?
Use this thread to chat about and share Python resources!
/r/Python
https://redd.it/10hdaol
Found a neat resource related to Python over the past week? Looking for a resource to explain a certain topic?
Use this thread to chat about and share Python resources!
/r/Python
https://redd.it/10hdaol
reddit
Saturday Daily Thread: Resource Request and Sharing! Daily Thread
Found a neat resource related to Python over the past week? Looking for a resource to explain a certain topic? Use this thread to chat about and...
Pynecone: New Features and Performance Improvements β‘οΈ
Hi everyone, wanted to give a quick update on Pynecone because there have been major improvements in the past month since our initial release.
For those who have never heard of Pynecone, it is a way to build full-stack web apps in pure Python. The framework is easy to get started with even without previous web dev experience, and is entirely open source / free to use.
# Improvements:
Here are some of the notable improvements we implemented. Along with these were many bug fixes to get Pynecone more stable.
Components/Features:
πͺ Added Windows support!
π Added built-in graphing libraries using Victory.
Added Dynamic Routes.
Performance:
β‘οΈSwitched to WebSockets (No more new requests for every event!)
Compiler improvements to speed up event processing.
Community:
βοΈ Grown from \~30 to \~2400 Github stars.
70 [Discord](https://discord.gg/T5WSbC2YtQ) members.
13 More contributors.
Testing:
β Improved unit test coverage and added integration tests for all PRs.
Next Steps:
Add components such as upload and date picker.
Show how to make your own Pynecone 3rd party libraries.
And many more features!
/r/Python
https://redd.it/10h6l7e
Hi everyone, wanted to give a quick update on Pynecone because there have been major improvements in the past month since our initial release.
For those who have never heard of Pynecone, it is a way to build full-stack web apps in pure Python. The framework is easy to get started with even without previous web dev experience, and is entirely open source / free to use.
# Improvements:
Here are some of the notable improvements we implemented. Along with these were many bug fixes to get Pynecone more stable.
Components/Features:
πͺ Added Windows support!
π Added built-in graphing libraries using Victory.
Added Dynamic Routes.
Performance:
β‘οΈSwitched to WebSockets (No more new requests for every event!)
Compiler improvements to speed up event processing.
Community:
βοΈ Grown from \~30 to \~2400 Github stars.
70 [Discord](https://discord.gg/T5WSbC2YtQ) members.
13 More contributors.
Testing:
β Improved unit test coverage and added integration tests for all PRs.
Next Steps:
Add components such as upload and date picker.
Show how to make your own Pynecone 3rd party libraries.
And many more features!
/r/Python
https://redd.it/10h6l7e
GitHub
GitHub - reflex-dev/reflex: πΈοΈ Web apps in pure Python π
πΈοΈ Web apps in pure Python π. Contribute to reflex-dev/reflex development by creating an account on GitHub.
Migrating from Flask to FastAPI
Part of the work I've done with Forethought is lead an effort to migrate from Flask to FastAPI. π
Here's the first blog post out of 3, with all the tips and tricks to migrate a real-life, huge, production code base. π€
I hope it's useful! ππ
https://engineering.forethought.ai/blog/2022/12/01/migrating-from-flask-to-fastapi-part-1/
/r/Python
https://redd.it/10h9fb5
Part of the work I've done with Forethought is lead an effort to migrate from Flask to FastAPI. π
Here's the first blog post out of 3, with all the tips and tricks to migrate a real-life, huge, production code base. π€
I hope it's useful! ππ
https://engineering.forethought.ai/blog/2022/12/01/migrating-from-flask-to-fastapi-part-1/
/r/Python
https://redd.it/10h9fb5
engineering.forethought.ai
Migrating from Flask to FastAPI, Part 1 - Forethought AI Engineering
Welcome to the Forethought AI engineering team's blog! We are a group of software engineers, data scientists, and machine learning experts who are committed to building innovative solutions to improve the efficiency and effectiveness of customer service teams.
NiceGUI now has a subreddit.
Hi Folks... Just a heads up that NiceGUI now has it's own subreddit at r/nicegui. If you want to do really fast and efficient web page development using Python this is definitely worth looking at. Cheers!
/r/Python
https://redd.it/10hg2i4
Hi Folks... Just a heads up that NiceGUI now has it's own subreddit at r/nicegui. If you want to do really fast and efficient web page development using Python this is definitely worth looking at. Cheers!
/r/Python
https://redd.it/10hg2i4
Reddit
r/Python on Reddit: NiceGUI now has a subreddit.
Posted by u/ParallaxRay - 47 votes and 5 comments
Add Watermarks To PDF, JPG & PNG files with no restrictive licensing
I created a small Python package to add watermarks to PDF, JPG & PNG files.
**Why?** Most Python PDF packages have licensing that requires you to release your source code which isn't ideal for everyone. In this package, I've utilised PIL (open source HPND License) & Pypdfium2 ( either Apache-2.0 or BSD-3-Clause, at your choice. ) and the code itself is released under MIT license.
Check it out - [https://github.com/bowespublishing/pythonwatermark](https://github.com/bowespublishing/pythonwatermark)
It can probably be massively improved but it works :)
## Installation
Installing the latest PyPI release (recommended)
python3 -m pip install -U pythonwatermark
This will use a pre-built wheel package, the easiest way of installing pythonwatermark.
## Dependencies
pythonwatermark uses two awesome open source python packages to work it's magic they are...
Pillow - [https://github.com/python-pillow/Pillow](https://github.com/python-pillow/Pillow)
Like PIL, Pillow is licensed under the open source HPND License
Pypdfium2 - [https://github.com/pypdfium2-team/pypdfium2](https://github.com/pypdfium2-team/pypdfium2)
PDFium and pypdfium2 are available by the terms and conditions of either Apache-2.0 or BSD-3-Clause, at your choice.
These are both fantastic packages and are liberally licensed meaning unlike with other options you don't need to release your source code to the public.
## Usage
Import watermark utils
from pythonwatermark import watermarkutils
Add watermark to file
watermarkutils.put_watermark(inputfile, outputfile, watermark,
/r/Python
https://redd.it/10hfz75
I created a small Python package to add watermarks to PDF, JPG & PNG files.
**Why?** Most Python PDF packages have licensing that requires you to release your source code which isn't ideal for everyone. In this package, I've utilised PIL (open source HPND License) & Pypdfium2 ( either Apache-2.0 or BSD-3-Clause, at your choice. ) and the code itself is released under MIT license.
Check it out - [https://github.com/bowespublishing/pythonwatermark](https://github.com/bowespublishing/pythonwatermark)
It can probably be massively improved but it works :)
## Installation
Installing the latest PyPI release (recommended)
python3 -m pip install -U pythonwatermark
This will use a pre-built wheel package, the easiest way of installing pythonwatermark.
## Dependencies
pythonwatermark uses two awesome open source python packages to work it's magic they are...
Pillow - [https://github.com/python-pillow/Pillow](https://github.com/python-pillow/Pillow)
Like PIL, Pillow is licensed under the open source HPND License
Pypdfium2 - [https://github.com/pypdfium2-team/pypdfium2](https://github.com/pypdfium2-team/pypdfium2)
PDFium and pypdfium2 are available by the terms and conditions of either Apache-2.0 or BSD-3-Clause, at your choice.
These are both fantastic packages and are liberally licensed meaning unlike with other options you don't need to release your source code to the public.
## Usage
Import watermark utils
from pythonwatermark import watermarkutils
Add watermark to file
watermarkutils.put_watermark(inputfile, outputfile, watermark,
/r/Python
https://redd.it/10hfz75
GitHub
GitHub - bowespublishing/pythonwatermark: Easily add watermarks to PDF, JPG & PNG files with no restrictive licensing
Easily add watermarks to PDF, JPG & PNG files with no restrictive licensing - GitHub - bowespublishing/pythonwatermark: Easily add watermarks to PDF, JPG & PNG files with no restric...
Experimental MiniJinja Bindings for Python
https://pypi.org/project/minijinja/0.30.1/
/r/Python
https://redd.it/10hcmdj
https://pypi.org/project/minijinja/0.30.1/
/r/Python
https://redd.it/10hcmdj
PyPI
minijinja
An experimental Python binding of the Rust MiniJinja template engine.