Take the tools out of 'Data', but don't take the data out of the tools
https://csvbase.com/blog/1

/r/Python
https://redd.it/109vtdk

D The Open Deep Learning Toolkit for Robotics v2.0 was just released

The Open Deep Learning Toolkit for Robotics version 2.0 was just released! This new version of the toolkit includes several improvements, such as new tools for object detection, efficient continual inference, tracking, emotion estimation and high-resolution pose estimation. Furthermore, this version includes a refined ROS interface, along with support for ROS2.

You can download it here: https://github.com/opendr-eu/opendr

We look forward to receiving your feedback, bug reports, and suggestions for improvements!

/r/MachineLearning
https://redd.it/109w09k

R Git is for Data (CIDR 2023) - Extending Git to Support Large-Scale Data

Paper: https://www.cidrdb.org/cidr2023/papers/p43-low.pdf

Abstract:

Dataset management is one of the greatest challenges to the application of machine learning (ML) in the industry. Although scaling and performance have often been highlighted as the significant ML challenges, development teams are bogged down by the contradictory requirements of supporting fast and flexible data iteration while maintaining stability, provenance, and reproducibility. For example, blobstores are used to store datasets for maximum flexibility, but their unmanaged access patterns limit reproducibility. Many ML pipeline solutions to ensure reproducibility have been devised, but all introduce a degree of friction and reduce flexibility.

In this paper, we propose that the solution to the dataset management challenges is simple and apparent: Git. As a source control system, as well as an ecosystem of collaboration and developer tooling, Git has enabled the field of DevOps to provide both speed of iteration and reproducibility to source code. Git is not only already familiar to developers, but is also integrated into existing pipelines, which facilitates adoption. However, as we (and others) demonstrate, Git, as designed today, does not scale to the needs of ML dataset management. In this paper, we propose XetHub; a system that retains the Git user experience and ecosystem, but can scale to

/r/MachineLearning
https://redd.it/10a4mns

D Microsoft ChatGPT investment isn't about Bing but about Cortana

I believe that Microsoft's 10B USD investment in ChatGPT is less about Bing and more about turning Cortana into an Alexa for corporates.
Examples: Cortana prepare the new T&Cs... Cortana answer that client email... Cortana prepare the Q4 investor presentation (maybe even with PowerBI integration)... Cortana please analyze cost cutting measures... Cortana please look up XYZ...

What do you think?

/r/MachineLearning
https://redd.it/1095os9

D Are there any papers on optimization-based approaches which combine learned parameter initializations with learned optimisers?

There are quite a few papers on optimisation-based meta-learning approaches for learning parameter initialisations (i.e. MAML and its derivatives) [1, 2\], and there are also many papers on learning optimisers [3\].

Question: Are there any papers which combine the two?

I am aware of some papers such as [4, 5\] which achieve this in some capacity indirectly/implicitly, but wondering if there are any other papers that I am not aware of, or do this explicitly? Thanks in advance.

\---

[1\] Finn, C., et al. (2017. Model-agnostic meta-learning for fast adaptation of deep networks. ICML.)

[2\] Nichol, A., et al. (2018. On first-order meta-learning algorithms.)

[3\] Andrychowicz, M., et al. (2016. Learning to learn by gradient descent by gradient descent.) NIPS

[4\] Li, Z., et al. (2017. Meta-sgd: Learning to learn quickly for few-shot learning.)

[5\] Ravi, S., & Larochelle, H. (2016. Optimization as a model for few-shot learning. ICLR.)

/r/MachineLearning
https://redd.it/109xqw3

Why Polars uses less memory than Pandas
https://pythonspeed.com/articles/polars-memory-pandas/

/r/Python
https://redd.it/10a2tjg

Is it still relevant today to take a Python and Pandas course that was created two years ago?

I'm learning Python nowadays and I'm taking this course on freecodecamp

Is is still relevant today or a lot of things have changed in Python and Pandas?

I'm coming from R language so I have experience in R, SQL and Tableau. I chose this one because I liked the way they teach the course (there are projects).

I really appreciate your suggestions for other courses.

The interest here is in Data Analysis/Science and Automation. (If someone has taken this path before)

https://www.freecodecamp.org/learn/data-analysis-with-python/

​

I really appreciate any help and thank you for your time.

/r/Python
https://redd.it/10a1zir

Train a language model from scratch
https://neuml.hashnode.dev/train-a-language-model-from-scratch

/r/Python
https://redd.it/10a72bh

Python Folium: Create Web Maps From Your Data – Real Python
https://realpython.com/python-folium-web-maps-from-data/

/r/Python
https://redd.it/10a0t7w

Analyzing the Chemical Trends of Cannabis Sativa with Python
https://www.youtube.com/watch?v=ECvHWVrkY80

/r/Python
https://redd.it/10ajm1j

Console application that automates participation in giveaways on SteamGifts.com using PHPSESSID
https://github.com/PalmaLuv/SteamGiftBot

/r/Python
https://redd.it/10ah47o

Do you want to easily fetch weather data in Python for your Data Science projects?

Have a look at this medium article:

https://pub.towardsai.net/how-to-create-a-python-package-for-fetching-weather-data-b17614627f30

And the corresponding repository:

https://github.com/stavrostheocharis/weather\_data\_retriever

/r/Python
https://redd.it/10a6dj2

Build a ChatGPT-like SMS Chatbot with OpenAI and Python
https://www.twilio.com/blog/sms-chatbot-openai-python

/r/Python
https://redd.it/10aasev

Create web app to manage sql tables

I am wondering if there is an out of the box solution to this for django already.

I want to give users of our organization an easy way to manipulate sql tables through a django interface. The things I would need it to do are these:

1. Have CRUD capabilities and connect to databases with sql tables.
2. Use our windows Active Directory for user sign in.

I was looking at using .net core 6 and blazor as an alternative as well. Any thoughts?

/r/django
https://redd.it/10amcgu

Django Values queryset returns table id instead of the actual text in template

Hope everyone is doing great,

I have a queryset :

price = Price.objects.values('part', 'seller').annotate(latest_report=Max('added'))

returned values :

<QuerySet {'part': 1, 'seller': 1, 'latest_report': datetime.datetime(2023, 1, 12, 17, 47, 29, 645857, tzinfo=datetime.timezone.utc)}, {'part': 1, 'seller': 2, 'l
atest_report': datetime.datetime(2023, 1, 12, 16, 57, 48, 598057, tzinfo=datetime.timezone.utc)}, {'part': 1, 'seller': 3, 'latest_report': datetime.datetime(2023,
1, 12, 16, 55, 54, 662815, tzinfo=datetime.timezone.utc)}, {'part': 1, 'seller': 4, 'latest_report': datetime.datetime(2023, 1, 12, 17, 46, 36, 110057, tzinfo=dat
etime.timezone.utc)}, {'part': 2, 'seller': 1, 'latest_report': datetime.datetime(2023, 1, 11, 20, 2, 2, 336492, tzinfo=datetime.timezone.utc)}, {'part': 2, 'selle
r': 2, 'latest_report': datetime.datetime(2023, 1, 11, 20, 2, 7, 701336, tzinfo=datetime.timezone.utc)}, {'part': 2, 'seller': 3, 'latest_report': datetime.datetim
e(2023, 1, 11, 20, 2, 12, 618087, tzinfo=datetime.timezone.utc)}, {'part': 2, 'seller': 4, 'latest_report': datetime.datetime(2023, 1, 11, 20, 2, 17, 900953, tzinf
o=datetime.timezone.utc)}>

I want to display part name "text" and part seller "text" in template, i tried :

{% for item in price %}


/r/django
https://redd.it/10afetm

How do you delete expired sessions?

I have a hosted django app and found out there are some old expired sessions as old as 2018.

Is it a good practice to delete expired sessions?

What is the usual strategy to delete expired sessions?
May be a CRON job, or run the "clearsessions" management command for every deployment, etc...?

Any there any problems/pitfalls with deleting expired sessions?

/r/django
https://redd.it/10anmjw

Unique field not throwing graceful ValidationError

Creating API app. Field in question:

phone = models.CharField(max_length=20, null=True, unique=True, error_messages={"unique": "A user with that phone number already exists, please login."})

I ran the migration and its clearly being enforced when I try to create a user with the same phone, but why not gracefully (see below) like email does?

OUTPUT

API: 422 output to API but no error message

Server log: [Long stack trace\] django.db.utils.IntegrityError: duplicate key value violates unique constraint + INTERNAL SERVER ERROR given

Thanks!

/r/django
https://redd.it/109zzk1

How could we do a reliable social authentication api with custom user model?

I've been wanting to make a vue/ionic mobile app with django as the backend and I wanted to make 3 sign in options (google, facebook, email).
I have implemented the email sign up/log in api with simpleJWT. I then tried to use django-allauth to make the facebook and google auth but I discovered that they have their own views and not very good documentation for customizing the views.

I found the dj-rest-auth as a method of making a rest auth api, unfortunately when using the documentation I have this error coming up.

is there a way to either fix this issue or maybe there's a different implementation I should do?

also how can I register custom users while using social auth? registering accounts with socialaccounts fills the user form automatically and I want to ask the user to add certain fields like gender and birthday etc.

any sort of help or sources or demos would be appreciated.

/r/django
https://redd.it/10aqfic

how to do ip tests locally

hello everyone, i'm studying django and i came across an exercise where it asks to send an alert message in case the user ip is different from the previous one. is there any way to test this? being in local i always get '127.0.0.1' back

/r/django
https://redd.it/109w89y

Extracting pdf page count corrupts attachment

I am developing a Django website, wherein I have a django form that collects some data and a file from the user, and then mail the response to a mail ID. If the file happens to be a PDF, the form is supposed to automatically get its page count and add it to the mail body.

I am able to successfully isolate pdf files and even get the page count value correctly, the process of getting page count seems to corrupt the pdf file when it is attached to the email.

Emailing is being handled with django email, and I have tried to read page count using PyPDF2 and pdfminer, but both give the same outcome. The file is not being stored to any database.

What should I do?

EDIT: Problem solved. Thanks all!

/r/django
https://redd.it/109nitk