A Recipe for Training #NeuralNetworks
Some few weeks ago I posted a tweet on “the most common neural net mistakes”, listing a few common gotchas related to training neural nets. The tweet got quite a bit more engagement than I anticipated (including a webinar :)). Clearly, a lot of people have personally encountered the large gap between “here is how a convolutional layer works” and “our convnet achieves state of the art results”.
Link
🔭 @DeepGravity
Some few weeks ago I posted a tweet on “the most common neural net mistakes”, listing a few common gotchas related to training neural nets. The tweet got quite a bit more engagement than I anticipated (including a webinar :)). Clearly, a lot of people have personally encountered the large gap between “here is how a convolutional layer works” and “our convnet achieves state of the art results”.
Link
🔭 @DeepGravity
karpathy.github.io
A Recipe for Training Neural Networks
Musings of a Computer Scientist.
A #DeepLearning framework for #neuroscience
Systems neuroscience seeks explanations for how the brain implements a wide variety of perceptual, #cognitive and motor tasks. Conversely, #ArtificialIntelligence attempts to design computational systems based on the tasks they will have to solve. In artificial #NeuralNetworks, the three components specified by design are the objective functions, the learning rules and the architectures. With the growing success of #deep learning, which utilizes brain-inspired architectures, these three designed components have increasingly become central to how we model, engineer and optimize complex artificial learning systems. Here we argue that a greater focus on these components would also benefit systems neuroscience. We give examples of how this optimization-based framework can drive theoretical and experimental progress in neuroscience. We contend that this principled perspective on systems neuroscience will help to generate more rapid progress.
Link
🔭 @DeepGravity
Systems neuroscience seeks explanations for how the brain implements a wide variety of perceptual, #cognitive and motor tasks. Conversely, #ArtificialIntelligence attempts to design computational systems based on the tasks they will have to solve. In artificial #NeuralNetworks, the three components specified by design are the objective functions, the learning rules and the architectures. With the growing success of #deep learning, which utilizes brain-inspired architectures, these three designed components have increasingly become central to how we model, engineer and optimize complex artificial learning systems. Here we argue that a greater focus on these components would also benefit systems neuroscience. We give examples of how this optimization-based framework can drive theoretical and experimental progress in neuroscience. We contend that this principled perspective on systems neuroscience will help to generate more rapid progress.
Link
🔭 @DeepGravity
Nature Neuroscience
A deep learning framework for neuroscience
A deep network is best understood in terms of components used to design it—objective functions, architecture and learning rules—rather than unit-by-unit computation. Richards et al. argue that this inspires fruitful approaches to systems neuroscience.
Gilbert Strang: #DeepLearning and #NeuralNetworks
Part of Lex Fridman conversation with Gilbert Strang
Gilbert Strang is a professor of mathematics at #MIT and perhaps one of the most famous and impactful teachers of #math in the world. His MIT OpenCourseWare lectures on linear algebra have been viewed millions of times.
🔭 @DeepGravity
Part of Lex Fridman conversation with Gilbert Strang
Gilbert Strang is a professor of mathematics at #MIT and perhaps one of the most famous and impactful teachers of #math in the world. His MIT OpenCourseWare lectures on linear algebra have been viewed millions of times.
🔭 @DeepGravity
YouTube
Gilbert Strang: Deep Learning and Neural Networks
Full episode with Gilbert Strang (Nov 2019): https://www.youtube.com/watch?v=lEZPfmGCEk0
Subscribe to this channel if you like clips and to the main channel if you like full length episodes: https://www.youtube.com/lexfridman
(more links below)
Podcast…
Subscribe to this channel if you like clips and to the main channel if you like full length episodes: https://www.youtube.com/lexfridman
(more links below)
Podcast…
#NeuralNetworks: Feedforward and #Backpropagation Explained & Optimization
What is neural networks? Developers should understand backpropagation, to figure out why their code sometimes does not work. Visual and down to earth explanation of the math of backpropagation.
Link
🔭 @DeepGravity
What is neural networks? Developers should understand backpropagation, to figure out why their code sometimes does not work. Visual and down to earth explanation of the math of backpropagation.
Link
🔭 @DeepGravity
Machine Learning From Scratch
Neural Networks: Feedforward and Backpropagation Explained
What is neural networks? Developers should understand backpropagation, to figure out why their code sometimes does not work. Visual and down to earth explanation of the math of backpropagation.
How #NeuralNetworks work—and why they’ve become a big business
The last decade has seen remarkable improvements in the ability of computers to understand the world around them. Photo software automatically recognizes people's faces. Smartphones transcribe spoken words into text. Self-driving cars recognize objects on the road and avoid hitting them.
Underlying these breakthroughs is an artificial intelligence technique called deep learning. Deep learning is based on neural networks, a type of data structure loosely inspired by networks of biological neurons. Neural networks are organized in layers, with inputs from one layer connected to outputs from the next layer.
Link
🔭 @DeepGravity
The last decade has seen remarkable improvements in the ability of computers to understand the world around them. Photo software automatically recognizes people's faces. Smartphones transcribe spoken words into text. Self-driving cars recognize objects on the road and avoid hitting them.
Underlying these breakthroughs is an artificial intelligence technique called deep learning. Deep learning is based on neural networks, a type of data structure loosely inspired by networks of biological neurons. Neural networks are organized in layers, with inputs from one layer connected to outputs from the next layer.
Link
🔭 @DeepGravity
Ars Technica
How neural networks work—and why they’ve become a big business
Neural networks have grown from an academic curiosity to a massive industry.
#Transferlearning in hybrid classical- #quantum #neuralNetworks
We extend the concept of transfer learning, widely applied in modern machine learning algorithms, to the emerging context of hybrid neural networks composed of classical and quantum elements. We propose different implementations of hybrid transfer learning, but we focus mainly on the paradigm in which a pre-trained classical network is modified and augmented by a final variational quantum circuit. This approach is particularly attractive in the current era of intermediate-scale quantum technology since it allows to optimally pre-process high dimensional data (e.g., images) with any state-of-the-art classical network and to embed a select set of highly informative features into a quantum processor. We present several proof-of-concept examples of the convenient application of quantum transfer learning for image recognition and quantum state classification. We use the cross-platform software library PennyLane to experimentally test a high-resolution image classifier with two different quantum computers, respectively provided by #IBM and Rigetti.
Paper
🔭 @DeepGravity
We extend the concept of transfer learning, widely applied in modern machine learning algorithms, to the emerging context of hybrid neural networks composed of classical and quantum elements. We propose different implementations of hybrid transfer learning, but we focus mainly on the paradigm in which a pre-trained classical network is modified and augmented by a final variational quantum circuit. This approach is particularly attractive in the current era of intermediate-scale quantum technology since it allows to optimally pre-process high dimensional data (e.g., images) with any state-of-the-art classical network and to embed a select set of highly informative features into a quantum processor. We present several proof-of-concept examples of the convenient application of quantum transfer learning for image recognition and quantum state classification. We use the cross-platform software library PennyLane to experimentally test a high-resolution image classifier with two different quantum computers, respectively provided by #IBM and Rigetti.
Paper
🔭 @DeepGravity
Neural #Quantum States
How #neuralnetworks can solve highly complex problems in quantum mechanics
Article
🔭 @DeepGravity
How #neuralnetworks can solve highly complex problems in quantum mechanics
Article
🔭 @DeepGravity
Medium
Neural Quantum States
How neural networks can solve highly complex problems in quantum mechanics
#Evolution of #NeuralNetworks
Today, #AI lives its golden age whereas neural networks make a great contribution to it. Neural networks change our lifes without even realizing it. It lies behind the image, face and speech recognition, also language translation, even in future predictions. However, it is not coming to the present form in a day. Let’s travel to the past and monitor its previous forms.
Link
🔭 @DeepGravity
Today, #AI lives its golden age whereas neural networks make a great contribution to it. Neural networks change our lifes without even realizing it. It lies behind the image, face and speech recognition, also language translation, even in future predictions. However, it is not coming to the present form in a day. Let’s travel to the past and monitor its previous forms.
Link
🔭 @DeepGravity
#TensorNetworks in #NeuralNetworks
Here, we have a small toy example of how to use a TN inside of a fully connected neural network.
Colab
🔭 @DeepGravity
Here, we have a small toy example of how to use a TN inside of a fully connected neural network.
Colab
🔭 @DeepGravity
Google
Google Colaboratory