Various types of test used in statistics for data science
T-test: used to test whether the means of two groups are significantly different from each other.
ANOVA: used to test whether the means of three or more groups are significantly different from each other.
Chi-squared test: used to test whether two categorical variables are independent or associated with each other.
Pearson correlation test: used to test whether there is a significant linear relationship between two continuous variables.
Wilcoxon signed-rank test: used to test whether the median of two related samples is significantly different from each other.
Mann-Whitney U test: used to test whether the median of two independent samples is significantly different from each other.
Kruskal-Wallis test: used to test whether the medians of three or more independent samples are significantly different from each other.
Friedman test: used to test whether the medians of three or more related samples are significantly different from each other.
T-test: used to test whether the means of two groups are significantly different from each other.
ANOVA: used to test whether the means of three or more groups are significantly different from each other.
Chi-squared test: used to test whether two categorical variables are independent or associated with each other.
Pearson correlation test: used to test whether there is a significant linear relationship between two continuous variables.
Wilcoxon signed-rank test: used to test whether the median of two related samples is significantly different from each other.
Mann-Whitney U test: used to test whether the median of two independent samples is significantly different from each other.
Kruskal-Wallis test: used to test whether the medians of three or more independent samples are significantly different from each other.
Friedman test: used to test whether the medians of three or more related samples are significantly different from each other.
👍6
Essential Tools and Libraries for Data Science Students
1. Programming Languages:
Python
R
SQL
2. Python Libraries:
NumPy: For numerical computations.
Pandas: For data manipulation and analysis.
Matplotlib: For basic data visualization.
Seaborn: For statistical data visualization.
Scikit-learn: For machine learning models.
TensorFlow: For deep learning.
PyTorch: For advanced neural networks.
3. R Libraries:
ggplot2: For data visualization.
dplyr: For data manipulation.
caret: For machine learning.
shiny: For building interactive web apps.
4. Data Visualization Tools:
Tableau
Power BI
Google Data Studio
5. Big Data Tools:
Apache Hadoop
Apache Spark
6. Cloud Platforms:
AWS (Amazon Web Services)
Google Cloud Platform (GCP)
Microsoft Azure
7. Statistical Software:
SAS
SPSS
8. Version Control System:
Git
9. Notebook Tools:
Jupyter Notebook
Google Colab
10. Data Sources for Practice:
Kaggle Datasets
UCI Machine Learning Repository
GitHub Repositories
Best Data Science & Machine Learning Resources: https://topmate.io/coding/914624
ENJOY LEARNING 👍👍
1. Programming Languages:
Python
R
SQL
2. Python Libraries:
NumPy: For numerical computations.
Pandas: For data manipulation and analysis.
Matplotlib: For basic data visualization.
Seaborn: For statistical data visualization.
Scikit-learn: For machine learning models.
TensorFlow: For deep learning.
PyTorch: For advanced neural networks.
3. R Libraries:
ggplot2: For data visualization.
dplyr: For data manipulation.
caret: For machine learning.
shiny: For building interactive web apps.
4. Data Visualization Tools:
Tableau
Power BI
Google Data Studio
5. Big Data Tools:
Apache Hadoop
Apache Spark
6. Cloud Platforms:
AWS (Amazon Web Services)
Google Cloud Platform (GCP)
Microsoft Azure
7. Statistical Software:
SAS
SPSS
8. Version Control System:
Git
9. Notebook Tools:
Jupyter Notebook
Google Colab
10. Data Sources for Practice:
Kaggle Datasets
UCI Machine Learning Repository
GitHub Repositories
Best Data Science & Machine Learning Resources: https://topmate.io/coding/914624
ENJOY LEARNING 👍👍
👍8❤2👏1
7 Free Kaggle Micro-Courses for Data Science Beginners with Certification
Python
https://www.kaggle.com/learn/python
Pandas
https://www.kaggle.com/learn/pandas
Data visualization
https://www.kaggle.com/learn/data-visualization
Intro to sql
https://www.kaggle.com/learn/intro-to-sql
Advanced Sql
https://www.kaggle.com/learn/advanced-sql
Intro to ML
https://www.kaggle.com/learn/intro-to-machine-learning
Advanced ML
https://www.kaggle.com/learn/intermediate-machine-learning
#datascienceprojects #kaggle
Python
https://www.kaggle.com/learn/python
Pandas
https://www.kaggle.com/learn/pandas
Data visualization
https://www.kaggle.com/learn/data-visualization
Intro to sql
https://www.kaggle.com/learn/intro-to-sql
Advanced Sql
https://www.kaggle.com/learn/advanced-sql
Intro to ML
https://www.kaggle.com/learn/intro-to-machine-learning
Advanced ML
https://www.kaggle.com/learn/intermediate-machine-learning
#datascienceprojects #kaggle
👍6❤2
The Data Science skill no one talks about...
Every aspiring data scientist I talk to thinks their job starts when someone else gives them:
1. a dataset, and
2. a clearly defined metric to optimize for, e.g. accuracy
But it doesn’t.
It starts with a business problem you need to understand, frame, and solve. This is the key data science skill that separates senior from junior professionals.
Let’s go through an example.
Example
Imagine you are a data scientist at Uber. And your product lead tells you:
We say that a user churns when she decides to stop using Uber.
But why?
There are different reasons why a user would stop using Uber. For example:
1. “Lyft is offering better prices for that geo” (pricing problem)
2. “Car waiting times are too long” (supply problem)
3. “The Android version of the app is very slow” (client-app performance problem)
You build this list ↑ by asking the right questions to the rest of the team. You need to understand the user’s experience using the app, from HER point of view.
Typically there is no single reason behind churn, but a combination of a few of these. The question is: which one should you focus on?
This is when you pull out your great data science skills and EXPLORE THE DATA 🔎.
You explore the data to understand how plausible each of the above explanations is. The output from this analysis is a single hypothesis you should consider further. Depending on the hypothesis, you will solve the data science problem differently.
For example…
Scenario 1: “Lyft Is Offering Better Prices” (Pricing Problem)
One solution would be to detect/predict the segment of users who are likely to churn (possibly using an ML Model) and send personalized discounts via push notifications. To test your solution works, you will need to run an A/B test, so you will split a percentage of Uber users into 2 groups:
The A group. No user in this group will receive any discount.
The B group. Users from this group that the model thinks are likely to churn, will receive a price discount in their next trip.
You could add more groups (e.g. C, D, E…) to test different pricing points.
1. Translating business problems into data science problems is the key data science skill that separates a senior from a junior data scientist.
2. Ask the right questions, list possible solutions, and explore the data to narrow down the list to one.
3. Solve this one data science problem
Every aspiring data scientist I talk to thinks their job starts when someone else gives them:
1. a dataset, and
2. a clearly defined metric to optimize for, e.g. accuracy
But it doesn’t.
It starts with a business problem you need to understand, frame, and solve. This is the key data science skill that separates senior from junior professionals.
Let’s go through an example.
Example
Imagine you are a data scientist at Uber. And your product lead tells you:
👩💼: “We want to decrease user churn by 5% this quarter”
We say that a user churns when she decides to stop using Uber.
But why?
There are different reasons why a user would stop using Uber. For example:
1. “Lyft is offering better prices for that geo” (pricing problem)
2. “Car waiting times are too long” (supply problem)
3. “The Android version of the app is very slow” (client-app performance problem)
You build this list ↑ by asking the right questions to the rest of the team. You need to understand the user’s experience using the app, from HER point of view.
Typically there is no single reason behind churn, but a combination of a few of these. The question is: which one should you focus on?
This is when you pull out your great data science skills and EXPLORE THE DATA 🔎.
You explore the data to understand how plausible each of the above explanations is. The output from this analysis is a single hypothesis you should consider further. Depending on the hypothesis, you will solve the data science problem differently.
For example…
Scenario 1: “Lyft Is Offering Better Prices” (Pricing Problem)
One solution would be to detect/predict the segment of users who are likely to churn (possibly using an ML Model) and send personalized discounts via push notifications. To test your solution works, you will need to run an A/B test, so you will split a percentage of Uber users into 2 groups:
The A group. No user in this group will receive any discount.
The B group. Users from this group that the model thinks are likely to churn, will receive a price discount in their next trip.
You could add more groups (e.g. C, D, E…) to test different pricing points.
In a nutshell
1. Translating business problems into data science problems is the key data science skill that separates a senior from a junior data scientist.
2. Ask the right questions, list possible solutions, and explore the data to narrow down the list to one.
3. Solve this one data science problem
👍8❤5