Image Segmentation: tips and tricks from 39 Kaggle competitions
this article gave you some background into #image #segmentation tips and tricks
also, collect some tools and frameworks that you can use to start competing
the author overview:
* architectures
* training tricks
* losses
* pre-processing
* post processing
* ensembling
* tools and frameworks
link here
this article gave you some background into #image #segmentation tips and tricks
also, collect some tools and frameworks that you can use to start competing
the author overview:
* architectures
* training tricks
* losses
* pre-processing
* post processing
* ensembling
* tools and frameworks
link here
neptune.ai
Blog - neptune.ai
Blog for ML practicioners with articles about MLOps, ML tools, and other ML-related topics. You'll find here guides, tutorials, case studies, tools reviews, and more.
Interview about how DS startups are being scouted, grown and then sold
Most notable highlights:
Where to dig β what specific areas or technologies in the near future? What project / team would you invest in after this interview on the channel?
We currently are really interested in Voice Processing and are in the search of Voice experts.
Particularly we are discussing creating a technology that allows you to change your voice into the voice of a celebrity in real time. We also consider options related to creating non-copyright photos on a given topic, Media Compression, Calorie Calculator, using TikTok algorithms. If people are experts in these fields they can me on telegram paul_shab.
How to sell your companies and ideas to someone strategic? Suppose I donβt want a middle man who will receive a share of the company. How can I achieve it myself without grinding through endless investment funds?
You can always sell companies, Selling ideas is not possible. you need middlemen β they are good π they help you do work that you should not waste your time and frustration on. Usually if someone helps to connect and close the deal, it can cost 1β5% from the deal amount. This is acceptable β you want to reward a person who helped anyway.
Link: https://medium.com/@timooxaaaa/questions-to-the-investor-machine-learning-is-our-future-ebb8e4046ff2
#wheretodig #dsventure #botan
Most notable highlights:
Where to dig β what specific areas or technologies in the near future? What project / team would you invest in after this interview on the channel?
We currently are really interested in Voice Processing and are in the search of Voice experts.
Particularly we are discussing creating a technology that allows you to change your voice into the voice of a celebrity in real time. We also consider options related to creating non-copyright photos on a given topic, Media Compression, Calorie Calculator, using TikTok algorithms. If people are experts in these fields they can me on telegram paul_shab.
How to sell your companies and ideas to someone strategic? Suppose I donβt want a middle man who will receive a share of the company. How can I achieve it myself without grinding through endless investment funds?
You can always sell companies, Selling ideas is not possible. you need middlemen β they are good π they help you do work that you should not waste your time and frustration on. Usually if someone helps to connect and close the deal, it can cost 1β5% from the deal amount. This is acceptable β you want to reward a person who helped anyway.
Link: https://medium.com/@timooxaaaa/questions-to-the-investor-machine-learning-is-our-future-ebb8e4046ff2
#wheretodig #dsventure #botan
Medium
Questions to the investor. Machine learning is our future.
Botan Investments is an investment fund aimed at supporting and investing in the field of mobile technologies and artificial intelligence.
Forwarded from ΠΠ°Ρ
ΠΎΠ΄ΠΊΠΈ Π² ΠΎΠΏΠ΅Π½ΡΠΎΡΡΠ΅
ββIn a chord diagram (or radial network), entities are arranged radially as segments with their relationships visualised by arcs that connect them. The size of the segments illustrates the numerical proportions, whilst the size of the arc illustrates the significance of the relationships1.
Chord diagrams are useful when trying to convey relationships between different entities, and they can be beautiful and eye-catching.
https://github.com/shahinrostami/chord
#python
Chord diagrams are useful when trying to convey relationships between different entities, and they can be beautiful and eye-catching.
https://github.com/shahinrostami/chord
#python
ββA tiny autograd engine
Andrej Karpathy recently released a library called micrograd which provides the ability to build & train a NN using a simple and intuitive interface.
In fact, he wrote the whole library in roughly 150 lines of code which he claims is the tiniest autograd engine there is. Ideally, such types of libraries can be used for educational purposes.
github: https://github.com/karpathy/micrograd
#karpathy #autograd
Andrej Karpathy recently released a library called micrograd which provides the ability to build & train a NN using a simple and intuitive interface.
In fact, he wrote the whole library in roughly 150 lines of code which he claims is the tiniest autograd engine there is. Ideally, such types of libraries can be used for educational purposes.
github: https://github.com/karpathy/micrograd
#karpathy #autograd
ββRethinking Generalization of Neural Models: A Named Entity Recognition Case Study
Authors use the NER task to analyze the generalization behavior of existing models from different perspectives. Experiments with in-depth analyses diagnose the bottleneck of existing neural NER models in terms of breakdown performance analysis, annotation errors, dataset bias, and category relationships, which suggest directions for improvement.
The authors also release two datasets for future research: ReCoNLL and PLONER.
The main findings of the paper:
β the performance of existing models (including the state-of-the-art model) heavily influenced by the degree to which test entities have been seen in the training set with the same label
β the proposed measure enables to detect human annotation errors.
Once these errors are fixed, previous models can achieve new state-of-the-art results
β authors introduce two measures to characterize the data bias and the cross-dataset generalization experiment shows that the performance of NER systems is influenced not only by whether the test entity has been seen in the training set but also by whether the context of the test entity has been observed
β providing more training samples is not a guarantee of better results. A targeted increase in training samples will make it more profitable
β the relationship between entity categories influences the difficulty of model learning, which leads to some hard test samples that are difficult to solve using common learning methods
Paper: https://arxiv.org/abs/2001.03844
Github: https://github.com/pfliu-nlp/Named-Entity-Recognition-NER-Papers
Website: http://pfliu.com/InterpretNER/
#nlp #generalization #NER #annotations #dataset
Authors use the NER task to analyze the generalization behavior of existing models from different perspectives. Experiments with in-depth analyses diagnose the bottleneck of existing neural NER models in terms of breakdown performance analysis, annotation errors, dataset bias, and category relationships, which suggest directions for improvement.
The authors also release two datasets for future research: ReCoNLL and PLONER.
The main findings of the paper:
β the performance of existing models (including the state-of-the-art model) heavily influenced by the degree to which test entities have been seen in the training set with the same label
β the proposed measure enables to detect human annotation errors.
Once these errors are fixed, previous models can achieve new state-of-the-art results
β authors introduce two measures to characterize the data bias and the cross-dataset generalization experiment shows that the performance of NER systems is influenced not only by whether the test entity has been seen in the training set but also by whether the context of the test entity has been observed
β providing more training samples is not a guarantee of better results. A targeted increase in training samples will make it more profitable
β the relationship between entity categories influences the difficulty of model learning, which leads to some hard test samples that are difficult to solve using common learning methods
Paper: https://arxiv.org/abs/2001.03844
Github: https://github.com/pfliu-nlp/Named-Entity-Recognition-NER-Papers
Website: http://pfliu.com/InterpretNER/
#nlp #generalization #NER #annotations #dataset
Tips for releasing research code in ML
with official NeurIPS 2020 recommendations
In repo you can find template that you can use for releasing ML research repositories. The sections in the template were derived by looking at existing repositories, seeing which had the best reception in the community, and then looking at common components that correlate with popularity.
The ML Code Completness Checklist consists of five items:
1 Specification of dependencies
2 Training code
3 Evaluation code
4 Pre-trained models
5 README file including table of results accompanied by precise commands to run/produce those results
Also, you can find additional awesome resources for releasing research code like: where to hosting pretrained models files, standardized model interfaces, results leaderboards, and etc.
github: https://github.com/paperswithcode/releasing-research-code
with official NeurIPS 2020 recommendations
In repo you can find template that you can use for releasing ML research repositories. The sections in the template were derived by looking at existing repositories, seeing which had the best reception in the community, and then looking at common components that correlate with popularity.
The ML Code Completness Checklist consists of five items:
1 Specification of dependencies
2 Training code
3 Evaluation code
4 Pre-trained models
5 README file including table of results accompanied by precise commands to run/produce those results
Also, you can find additional awesome resources for releasing research code like: where to hosting pretrained models files, standardized model interfaces, results leaderboards, and etc.
github: https://github.com/paperswithcode/releasing-research-code
GitHub
releasing-research-code/templates/README.md at master Β· paperswithcode/releasing-research-code
Tips for releasing research code in Machine Learning (with official NeurIPS 2020 recommendations) - paperswithcode/releasing-research-code
ββ#StyleGan2 applied to maps
Ever imagined what happens in Inception on bigger scale?
#mapdreamer #GAN
Ever imagined what happens in Inception on bigger scale?
#mapdreamer #GAN
Dear audience, if you want to share any links donβt hesitate to paste them into @opendatasciencebot
Forwarded from ΠΠ°Π±ΠΎΡΠ°ΡΠΎΡΠΈΡ ΠΠ (ΠΡΡΡΡ ΠΡΠΌΠ°Π΅Π²)
This media is not supported in your browser
VIEW IN TELEGRAM
PyBoy: A Game Boy emulator written in Python, focused on scripting, AI and learning
https://github.com/Baekalfen/PyBoy
https://github.com/Baekalfen/PyBoy
ββResNeSt: Split-Attention Networks
A novel variation of ResNet architecture that outperforms other networks with similar model complexities.
Usually, downstream applications use the ResNet or one of its variants as the backbone CNN. Its simple and modular design can be easily adapted to various tasks. However, since ResNet models are originally designed for image classification, they may not be suitable for various downstream applications because of the limited receptive-field size and lack of cross-channel interaction.
Main contributions of the paper:
- Split-Attention block. Each block divides the feature-map into several groups (along the channel dimension) and finer-grained subgroups or splits, where the feature representation of each group is determined via a weighted combination of the representations of its splits. By stacking several Split-Attention blocks, they get a ResNet-like network called ResNeSt (
- a lot of large scale benchmarks on image classification and transfer learning.
Models utilizing a ResNeSt backbone are able to achieve SOTA performance on several tasks, namely: image classification, object detection, instance segmentation, and semantic segmentation.
ResNeSt-50 achieves 81.13% top-1 accuracy on ImageNet using a single crop-size of 224 Γ 224, outperforming previous best ResNet variant by more than 1% accuracy
Paper: https://arxiv.org/abs/2004.08955
Github: https://github.com/zhanghang1989/ResNeSt
#computervision #deeplearning #resnet #image #backbone #downstream #sota
A novel variation of ResNet architecture that outperforms other networks with similar model complexities.
Usually, downstream applications use the ResNet or one of its variants as the backbone CNN. Its simple and modular design can be easily adapted to various tasks. However, since ResNet models are originally designed for image classification, they may not be suitable for various downstream applications because of the limited receptive-field size and lack of cross-channel interaction.
Main contributions of the paper:
- Split-Attention block. Each block divides the feature-map into several groups (along the channel dimension) and finer-grained subgroups or splits, where the feature representation of each group is determined via a weighted combination of the representations of its splits. By stacking several Split-Attention blocks, they get a ResNet-like network called ResNeSt (
S stands for βsplitβ). This architecture requires no more computation than existing ResNet-variants, and is easy to be adopted as a backbone for other vision tasks- a lot of large scale benchmarks on image classification and transfer learning.
Models utilizing a ResNeSt backbone are able to achieve SOTA performance on several tasks, namely: image classification, object detection, instance segmentation, and semantic segmentation.
ResNeSt-50 achieves 81.13% top-1 accuracy on ImageNet using a single crop-size of 224 Γ 224, outperforming previous best ResNet variant by more than 1% accuracy
Paper: https://arxiv.org/abs/2004.08955
Github: https://github.com/zhanghang1989/ResNeSt
#computervision #deeplearning #resnet #image #backbone #downstream #sota
COVID-19 Challenge final stage begins! Participants have to build an algorithm that most accurately predicts the dynamics of the number of reported cases of COVID-19 over the next 7 days. Last stage prizes are 1 000 000 RUB. 5 best forecasts + 5 best public solutions will be rewarded.
The objective of the competition is to draw attention to the forecasts of the coronavirus pandemic. Perhaps while solving this problem, you could find problems in the data sources or make a suitable forecast based on the most reliable data. Remember, we are developing an open science in ODS.ai by creating new and testing the existing forecasting methods. Only solving the tasks based on the open and public benchmark we can test and compare different approaches, as well as come to the best practices, and make them accessible to the entire research community.
β Click the link
π€ Act
π May the best win!
The objective of the competition is to draw attention to the forecasts of the coronavirus pandemic. Perhaps while solving this problem, you could find problems in the data sources or make a suitable forecast based on the most reliable data. Remember, we are developing an open science in ODS.ai by creating new and testing the existing forecasting methods. Only solving the tasks based on the open and public benchmark we can test and compare different approaches, as well as come to the best practices, and make them accessible to the entire research community.
β Click the link
π€ Act
π May the best win!
This media is not supported in your browser
VIEW IN TELEGRAM
Nvidia AI Noise Reduction
#Nvidia launches #KrispAI competitor Noise Reduction by AI on RTX Videocards.
Seems it works significantly better then other that kind of tools. But it needs to have Nvidia RTX officially.
But it possible to run it on older cards. The instruction is below. Or you can just download already hacked executable (also, below)
Setup Guide: https://www.nvidia.com/en-us/geforce/guides/nvidia-rtx-voice-setup-guide/
The instruction: https://forums.guru3d.com/threads/nvidia-rtx-voice-works-without-rtx-gpu-heres-how.431781/
Executable (use it on your own risk): https://mega.nz/file/CJ0xDYTB#LPorY_aPVqVKfHqWVV7zxK8fNfRmxt6iw6KdkHodz1M
#noisereduction #soundlearning #dl #noise #sound #speech #nvidia
#Nvidia launches #KrispAI competitor Noise Reduction by AI on RTX Videocards.
Seems it works significantly better then other that kind of tools. But it needs to have Nvidia RTX officially.
But it possible to run it on older cards. The instruction is below. Or you can just download already hacked executable (also, below)
Setup Guide: https://www.nvidia.com/en-us/geforce/guides/nvidia-rtx-voice-setup-guide/
The instruction: https://forums.guru3d.com/threads/nvidia-rtx-voice-works-without-rtx-gpu-heres-how.431781/
Executable (use it on your own risk): https://mega.nz/file/CJ0xDYTB#LPorY_aPVqVKfHqWVV7zxK8fNfRmxt6iw6KdkHodz1M
#noisereduction #soundlearning #dl #noise #sound #speech #nvidia
Training with quantization noise for extreme model compression
It is a new technique to enable extreme compression of models that still deliver high performance when deployed in practical applications mimics the effect of quantization during training time.
This method delivers performance that nearly matches that of the original uncompressed models while reducing the memory footprint by 10x to 20x. This significantly exceeds the 4x compression with int8 currently available in both PyTorch and Tensorflow. Quant-Noise can be used to shrink models even further β by more than 50x β in use cases where greater performance trade-offs are acceptable. Quant-Noise changes model training only by adding a regularization noise similar to dropout, with no impact on either the convergence rate or training speed.
At training time during the forward pass, it takes a subset of the weights and then randomly applies simulated quantization noise. This makes the model resilient to quantization and enables large compression ratios without much loss in accuracy.
Quant-Noise is applied to only a subset of the weights. This method has the advantage that the unbiased gradients still flow from the weights that are unaffected by the noise.
The authors demonstrated that their framework compresses the SOTA EfficientNet-B3 model from ~50 MB to 3.3 MB while achieving 80% top-1 accuracy on ImageNet, compared with 81.7% for the uncompressed model. Compress RoBERTa Base model from 480 MB to 14 MB while achieving 82.5% on MNLI, compared with 84.8% for the original model.
blogpost: https://ai.facebook.com/blog/training-with-quantization-noise-for-extreme-model-compression/
paper: https://arxiv.org/abs/2004.07320
github: https://github.com/pytorch/fairseq/tree/master/examples/quant_noise
#quantization #compression #shrinking
It is a new technique to enable extreme compression of models that still deliver high performance when deployed in practical applications mimics the effect of quantization during training time.
This method delivers performance that nearly matches that of the original uncompressed models while reducing the memory footprint by 10x to 20x. This significantly exceeds the 4x compression with int8 currently available in both PyTorch and Tensorflow. Quant-Noise can be used to shrink models even further β by more than 50x β in use cases where greater performance trade-offs are acceptable. Quant-Noise changes model training only by adding a regularization noise similar to dropout, with no impact on either the convergence rate or training speed.
At training time during the forward pass, it takes a subset of the weights and then randomly applies simulated quantization noise. This makes the model resilient to quantization and enables large compression ratios without much loss in accuracy.
Quant-Noise is applied to only a subset of the weights. This method has the advantage that the unbiased gradients still flow from the weights that are unaffected by the noise.
The authors demonstrated that their framework compresses the SOTA EfficientNet-B3 model from ~50 MB to 3.3 MB while achieving 80% top-1 accuracy on ImageNet, compared with 81.7% for the uncompressed model. Compress RoBERTa Base model from 480 MB to 14 MB while achieving 82.5% on MNLI, compared with 84.8% for the original model.
blogpost: https://ai.facebook.com/blog/training-with-quantization-noise-for-extreme-model-compression/
paper: https://arxiv.org/abs/2004.07320
github: https://github.com/pytorch/fairseq/tree/master/examples/quant_noise
#quantization #compression #shrinking
ββStarGAN v2 code release on GitHub
The better news is if you put a human into the animal model you do in fact get out a feline version of the human, and it's even wearing a suit.
GitHub: https://github.com/clovaai/stargan-v2
ArXiV: https://arxiv.org/abs/1912.01865
YouTube: https://www.youtube.com/watch?v=0EVh5Ki4dIY&feature=youtu.be
#GAN #StarGAN #PyTorch
The better news is if you put a human into the animal model you do in fact get out a feline version of the human, and it's even wearing a suit.
GitHub: https://github.com/clovaai/stargan-v2
ArXiV: https://arxiv.org/abs/1912.01865
YouTube: https://www.youtube.com/watch?v=0EVh5Ki4dIY&feature=youtu.be
#GAN #StarGAN #PyTorch
π1
Two more samples for dog-lovers. And it also seems that dog-transition works better.
ββThe Ingredients of Real World Robotic Reinforcement Learning
Blog post describing experiments on applying #RL in real world.
Blog post: https://bair.berkeley.edu/blog/2020/04/27/ingredients/
Paper: https://openreview.net/forum?id=rJe2syrtvS
#DL #robotics
Blog post describing experiments on applying #RL in real world.
Blog post: https://bair.berkeley.edu/blog/2020/04/27/ingredients/
Paper: https://openreview.net/forum?id=rJe2syrtvS
#DL #robotics
ββICASSP 2020 β FREE
45th International Conference on Acoustics, Speech, and Signal Processing
Registration includes full access to the virtual conference and all sessions, virtual patron and exhibitor experiences, as well as the conference app and any live and asynchronous discussion forums, and an electronic download of the conference proceedings.
link: https://cmsworkshops.com/ICASSP2020/Registration.asp
#icassp #conference
45th International Conference on Acoustics, Speech, and Signal Processing
Registration includes full access to the virtual conference and all sessions, virtual patron and exhibitor experiences, as well as the conference app and any live and asynchronous discussion forums, and an electronic download of the conference proceedings.
link: https://cmsworkshops.com/ICASSP2020/Registration.asp
#icassp #conference
ββScheduled DropHead: A Regularization Method for Transformer Models
In this paper introduced DropHead, a structured dropout method specifically designed for regularizing the multi-head attention mechanism, which is a key component of the transformer, a SOTA model for various NLP tasks.
In contrast to the conventional dropout mechanisms which randomly drop units or connections, the proposed DropHead is a structured dropout method. It drops entire attention heads during training and It prevents the multi-head attention model from being dominated by a small portion of attention heads while also reduces the risk of overfitting the training data, thus making use of the multi-head attention mechanism more efficiently.
paper: https://arxiv.org/abs/2004.13342
#nlp #regularization #transformer
In this paper introduced DropHead, a structured dropout method specifically designed for regularizing the multi-head attention mechanism, which is a key component of the transformer, a SOTA model for various NLP tasks.
In contrast to the conventional dropout mechanisms which randomly drop units or connections, the proposed DropHead is a structured dropout method. It drops entire attention heads during training and It prevents the multi-head attention model from being dominated by a small portion of attention heads while also reduces the risk of overfitting the training data, thus making use of the multi-head attention mechanism more efficiently.
paper: https://arxiv.org/abs/2004.13342
#nlp #regularization #transformer
ββππΆImproved audio generative model from OpenAI
Wow! OpenAI just released Jukebox β neural net and service that generates music from genre, artist name, and some lyrics that you can supply. It is can generate even some singing like from corrupted magnet compact cassette.
Some of the sounds seem it is from hell. Agonizing Michel Jakson for example or Creepy Eminiem or Celien Dion
#OpenAI 's approach is to use 3 levels of quantized variational autoencoders VQVAE-2 to learn discrete representations of audio and compress audio by 8x, 32x, and 128x and use the spectral loss to reconstruct spectrograms. And after that, they use sparse transformers conditioned on lyrics to generate new patterns and upsample it to higher discrete samples and decode it to the song.
The net can even learn and generates some solo parts during the track.
explore some creepy songs: https://jukebox.openai.com/
code: https://github.com/openai/jukebox/
paper: https://cdn.openai.com/papers/jukebox.pdf
blog: https://openai.com/blog/jukebox/
#openAI #music #sound #cool #fan #creepy #vae #audiolearning #soundlearning
Wow! OpenAI just released Jukebox β neural net and service that generates music from genre, artist name, and some lyrics that you can supply. It is can generate even some singing like from corrupted magnet compact cassette.
Some of the sounds seem it is from hell. Agonizing Michel Jakson for example or Creepy Eminiem or Celien Dion
#OpenAI 's approach is to use 3 levels of quantized variational autoencoders VQVAE-2 to learn discrete representations of audio and compress audio by 8x, 32x, and 128x and use the spectral loss to reconstruct spectrograms. And after that, they use sparse transformers conditioned on lyrics to generate new patterns and upsample it to higher discrete samples and decode it to the song.
The net can even learn and generates some solo parts during the track.
explore some creepy songs: https://jukebox.openai.com/
code: https://github.com/openai/jukebox/
paper: https://cdn.openai.com/papers/jukebox.pdf
blog: https://openai.com/blog/jukebox/
#openAI #music #sound #cool #fan #creepy #vae #audiolearning #soundlearning