Forwarded from Self Supervised Boy
Interactive Weak Supervision paper from ICLR 2021.
In contrast to classical active learning where experts are queried to assess individual samples, the idea of this paper is to assess labeling heuristics being automatically generated. Authors argue that since experts are good in writing such heuristics from scratch, they should be able to label auto-generated heuristics. To be able to rank non-assessed heuristics authors proposed to train an ensemble of models to predict the assessors' mark for the heuristic. As input for these models authors proposed to use fingerprint of the heuristic: concatenated predictions on some subset of data.
There is no very fancy results, there is some concerns raised by reviewers, and there are some strange notations in this paper. Yet the idea looks interesting to me.
With a bit deeper description (and one unanswered question) here.
Source (and rebuttal comments with important links) there.
In contrast to classical active learning where experts are queried to assess individual samples, the idea of this paper is to assess labeling heuristics being automatically generated. Authors argue that since experts are good in writing such heuristics from scratch, they should be able to label auto-generated heuristics. To be able to rank non-assessed heuristics authors proposed to train an ensemble of models to predict the assessors' mark for the heuristic. As input for these models authors proposed to use fingerprint of the heuristic: concatenated predictions on some subset of data.
There is no very fancy results, there is some concerns raised by reviewers, and there are some strange notations in this paper. Yet the idea looks interesting to me.
With a bit deeper description (and one unanswered question) here.
Source (and rebuttal comments with important links) there.
Ярослав's Notion on Notion
Interactive Weak Supervision: Learning Useful Heuristics for Data Labeling | Notion
Authors proposed a new pipeline for interactive weak supervision. Instead of asking users for sample labeling authors proposed to ask for labeling of the labeling functions (LF), e.g. regular expressions for text parsing. Authors argues that since experts…
CvT: Introducing Convolutions to Vision Transformers
Another improvement for Vision transformers! Inject inductive biases of CNNs (i.e. shift, scale, and distortion invariance) to the ViT architecture while maintaining the flexibility of Transformers.
❓How?
Main architectural novelties:
- Hierarchical architecture
- New convolutional token embedding
- Convolutional projections before self-attention instead of the linear which was used in ViT. This is where convolutions come into play.
✅ Results:
Almost SOTA on Imagenet 1K and 22K: 83.3% and 87.7%.
Almost because Swin Transformers with local window self-attention layers and downsampling layers are a bit stronger (see the image with results) and perhaps faster.
🤔 Looks like it is a trend now to incorporate useful structural properties of CNN into Transformers. I'm pretty sure, we will see more papers like this in the next few months.
📝 Paper arxiv.org/abs/2103.15808
Another improvement for Vision transformers! Inject inductive biases of CNNs (i.e. shift, scale, and distortion invariance) to the ViT architecture while maintaining the flexibility of Transformers.
❓How?
Main architectural novelties:
- Hierarchical architecture
- New convolutional token embedding
- Convolutional projections before self-attention instead of the linear which was used in ViT. This is where convolutions come into play.
✅ Results:
Almost SOTA on Imagenet 1K and 22K: 83.3% and 87.7%.
Almost because Swin Transformers with local window self-attention layers and downsampling layers are a bit stronger (see the image with results) and perhaps faster.
🤔 Looks like it is a trend now to incorporate useful structural properties of CNN into Transformers. I'm pretty sure, we will see more papers like this in the next few months.
📝 Paper arxiv.org/abs/2103.15808
New aggregator of trending papers🗞
It uses number of tweets as a paper hotness score.
You can also create your own reading lists there, add notes about papers and follow other users.
https://42papers.com/
Here is my profile https://42papers.com/u/gradient-dude-933
It uses number of tweets as a paper hotness score.
You can also create your own reading lists there, add notes about papers and follow other users.
https://42papers.com/
Here is my profile https://42papers.com/u/gradient-dude-933
StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery 🔥
Adobe Research
Contrastive Language-Image Pretraining (CLIP) models in order to navigate image editing by text queries.
1. Take pretrained CLIP, pretrained StyleGAN, and pretrained ArcFace network for face recognition.
2. Project an input image in StyleGAN latent vector
3. Now, given a source latent code
a) Distance between generated by StyleGAN image and the text query;
b) Regularization loss penalizing large deviation of the source vector
c) Identity loss, which makes sure that the identity of the generated face is the same as the original one. This is done by minimizing the distance between images in the ArcFace face recognition network embedding space.
Such an image editing process requires iterative optimization of the latent code
The overall idea of the paper is not super novel and has been around for some time already. But this is the first formal paper on such an image editing approach using CLIP and StyleGAN.
Other related papers, recently discussed in this channel:
▪️Paint by Word
▪️Using latent space regression to analyze and leverage compositionality in GANs
📝 StyleCLIP Paper
⚙️ StyleCLIp code
Adobe Research
Contrastive Language-Image Pretraining (CLIP) models in order to navigate image editing by text queries.
1. Take pretrained CLIP, pretrained StyleGAN, and pretrained ArcFace network for face recognition.
2. Project an input image in StyleGAN latent vector
w_s. 3. Now, given a source latent code
w_s∈ W+, and a directive in natural language, or a text prompt t, we iteratively minimize the sum of three losses by changing the latent code w: a) Distance between generated by StyleGAN image and the text query;
b) Regularization loss penalizing large deviation of the source vector
w_sc) Identity loss, which makes sure that the identity of the generated face is the same as the original one. This is done by minimizing the distance between images in the ArcFace face recognition network embedding space.
Such an image editing process requires iterative optimization of the latent code
w (usually 200-300 iterations) for several minutes. To make it faster authors propose a feed-forward method, where instead of optimization, another neural network predicts the residuals which are added to the latent code w to produce the desired image alterations.The overall idea of the paper is not super novel and has been around for some time already. But this is the first formal paper on such an image editing approach using CLIP and StyleGAN.
Other related papers, recently discussed in this channel:
▪️Paint by Word
▪️Using latent space regression to analyze and leverage compositionality in GANs
📝 StyleCLIP Paper
⚙️ StyleCLIp code
HaloNet: Scaling Local Self-Attention for Parameter Efficient Visual Backbones
Google research
Novel computer vision backbone - HaloNet. Yes, yet another one! 🤷🏼♂️
Authors develop a new family of parameter-efficient local self-attention models, HaloNets, that outperform EfficientNet in the parameter-accuracy tradeoff on ImageNet.
HaloNets show strong results on ImageNet-1k, and promising improvements (up to 4.4x inference speedups) over strong baselines when pretrained on ImageNet-21k with comparable settings.
The ideas are similar to Swin Transformers and CvT: local self-attention, attention-based downsampling layers, a mix of regular convolutions, and self-attention blocks.
In their previous work, the authors used pixel-centered windows, similar to convolutions. Here, they develop a block-centered formulation for better efficiency on matrix accelerators (GPU, TPU). They also introduce the attention downsampling layer
When applied to the detection and instance segmentation, the proposed local self-attention improves on top of strong convolutional baselines. Interestingly, local self-attention with 14x14 receptive fields performs nearly as well as 35x35.
📝 Paper
No code yet!
Google research
Novel computer vision backbone - HaloNet. Yes, yet another one! 🤷🏼♂️
Authors develop a new family of parameter-efficient local self-attention models, HaloNets, that outperform EfficientNet in the parameter-accuracy tradeoff on ImageNet.
HaloNets show strong results on ImageNet-1k, and promising improvements (up to 4.4x inference speedups) over strong baselines when pretrained on ImageNet-21k with comparable settings.
The ideas are similar to Swin Transformers and CvT: local self-attention, attention-based downsampling layers, a mix of regular convolutions, and self-attention blocks.
In their previous work, the authors used pixel-centered windows, similar to convolutions. Here, they develop a block-centered formulation for better efficiency on matrix accelerators (GPU, TPU). They also introduce the attention downsampling layer
When applied to the detection and instance segmentation, the proposed local self-attention improves on top of strong convolutional baselines. Interestingly, local self-attention with 14x14 receptive fields performs nearly as well as 35x35.
📝 Paper
No code yet!
Bored? Here is yet another Fast&Furious backbone for you!
New day - new SOTA on ImageNet🤯
New day - new SOTA on ImageNet🤯
Forwarded from Data Science by ODS.ai 🦜
EfficientNetV2: Smaller Models and Faster Training
A new paper from Google Brain with a new SOTA architecture called EfficientNetV2. The authors develop a new family of CNN models that are optimized both for accuracy and training speed. The main improvements are:
- an improved training-aware neural architecture search with new building blocks and ideas to jointly optimize training speed and parameter efficiency;
- a new approach to progressive learning that adjusts regularization along with the image size;
As a result, the new approach can reach SOTA results while training faster (up to 11x) and smaller (up to 6.8x).
Paper: https://arxiv.org/abs/2104.00298
Code will be available here:
https://github.com/google/automl/efficientnetv2
A detailed unofficial overview of the paper: https://andlukyane.com/blog/paper-review-effnetv2
#cv #sota #nas #deeplearning
A new paper from Google Brain with a new SOTA architecture called EfficientNetV2. The authors develop a new family of CNN models that are optimized both for accuracy and training speed. The main improvements are:
- an improved training-aware neural architecture search with new building blocks and ideas to jointly optimize training speed and parameter efficiency;
- a new approach to progressive learning that adjusts regularization along with the image size;
As a result, the new approach can reach SOTA results while training faster (up to 11x) and smaller (up to 6.8x).
Paper: https://arxiv.org/abs/2104.00298
Code will be available here:
https://github.com/google/automl/efficientnetv2
A detailed unofficial overview of the paper: https://andlukyane.com/blog/paper-review-effnetv2
#cv #sota #nas #deeplearning
This media is not supported in your browser
VIEW IN TELEGRAM
VR Mind Control from NextMind: decode the act of focusing
Sorry Elon, no need to drill skulls anymore!
The NextMind sensor is non-invasive and can read electrical signals from the brains' visual cortex using small electrodes attached to the skin. Then the machine learning is used to decode brain activity and pinpoint the object of focus, allowing you to control game actions with your mind in real-time.
The sensor itself is surprisingly small and light — it fits in the palm of your hand, with two arms that extend slightly beyond that. It easily fits under a baseball cap. You just need to ensure that the nine sets of two-pronged electrode sensors make contact with your skin
Currently, it's just a dev-kit that can be paired with 3rd party VR headsets including Oculus. The kit retails for $399 and can be already preordered. The functional is limited, but it is only the first step, I'm very excited to see the further development of this technology!
Full review is here.
Thanks @ai_newz for the pointer.
Sorry Elon, no need to drill skulls anymore!
The NextMind sensor is non-invasive and can read electrical signals from the brains' visual cortex using small electrodes attached to the skin. Then the machine learning is used to decode brain activity and pinpoint the object of focus, allowing you to control game actions with your mind in real-time.
The sensor itself is surprisingly small and light — it fits in the palm of your hand, with two arms that extend slightly beyond that. It easily fits under a baseball cap. You just need to ensure that the nine sets of two-pronged electrode sensors make contact with your skin
Currently, it's just a dev-kit that can be paired with 3rd party VR headsets including Oculus. The kit retails for $399 and can be already preordered. The functional is limited, but it is only the first step, I'm very excited to see the further development of this technology!
Full review is here.
Thanks @ai_newz for the pointer.
Self-supervised Learning for Medical images
Due to fixed imaging procedures, medical images like X-ray or CT scans are usually well aligned geometrically.
This gives an opportunity to utilize such an alignment to automatically mine similar pairs of image patches for self-supervised training.
The basic idea is to fix K random locations in the unlabeled medical images (K locations are the same for every image) and crop image patches across different images (which correspond to scans of different patients).
Now we create a surrogate classification task by assigning a unique pseudo-label to every location 1...K.
Authors combine the surrogate classification task with image restoration using a denoising autoencoder: they randomly perturb the cropped patches (color jittering, random noise, random cut-outs) and train a decoder to restore the original view.
However, sometimes the alignment between medical images is not perfect by default and images may depict different body parts. To make sure that the images are aligned, we train an autoencoder on full images (before cropping) and select only similar images by comparing the distances between them in the learned autoencoder latent space.
Authors show that their method is significantly better than other self-supervised learning approaches on medical data and can even be combined with existing self-supervised methods like RotNet (predicting image rotations). But unfortunately, the comparison is rather limited, and they didn't compare to Jigsaw Puzzle, SwaV, or recent contrastive self-supervised methods like MoCO, BYOL, and SimCLR.
📝 Paper
🛠 Code & Models
#paper_tldr #cv #self_supervised
Due to fixed imaging procedures, medical images like X-ray or CT scans are usually well aligned geometrically.
This gives an opportunity to utilize such an alignment to automatically mine similar pairs of image patches for self-supervised training.
The basic idea is to fix K random locations in the unlabeled medical images (K locations are the same for every image) and crop image patches across different images (which correspond to scans of different patients).
Now we create a surrogate classification task by assigning a unique pseudo-label to every location 1...K.
Authors combine the surrogate classification task with image restoration using a denoising autoencoder: they randomly perturb the cropped patches (color jittering, random noise, random cut-outs) and train a decoder to restore the original view.
However, sometimes the alignment between medical images is not perfect by default and images may depict different body parts. To make sure that the images are aligned, we train an autoencoder on full images (before cropping) and select only similar images by comparing the distances between them in the learned autoencoder latent space.
Authors show that their method is significantly better than other self-supervised learning approaches on medical data and can even be combined with existing self-supervised methods like RotNet (predicting image rotations). But unfortunately, the comparison is rather limited, and they didn't compare to Jigsaw Puzzle, SwaV, or recent contrastive self-supervised methods like MoCO, BYOL, and SimCLR.
📝 Paper
🛠 Code & Models
#paper_tldr #cv #self_supervised