To keep things easy this page will act as an index for all AI related resources.
External Resources
 Machine Learning Tutorials
 Machine Learning Getting Started Ideas
 Machine Learning Resources
 Top 100 Data Science Resources
 Learn Math Fast
News:
Contests:
Articles

Deep Learning Performance with AutoAugment
 Augmenting computer vision datasets by generating new images based on existing data in the set

Deep Reinforcement Learning Doesn’t Work Yet  February 14th, 2018
 Examples of why RL is hard to put into practice, the state of the industry, DeepRL

The Evolution and Core Concepts of Deep Learning & Neural Networks  August 2016
 Good primer on NN

AI at Google: our principles
 Google’s principles for AI development

Understanding the backward pass through Batch Normalization Layer
 Good explanation of how the derivative of the Batch Normalization layer is computed

A Beginner’s Guide to Recurrent Networks and LSTMs
 Good explanation of LSTMs and GRUs, plus an intuition of backpropagation through time

Optimization techniques comparison in Julia: SGD, Momentum, Adagrad, Adadelta, Adam
 Short explanations and performance comparison for SGD, SGD with momentum, AdaGrad, AdaDelta and Adam optimizers

googlenet in keras
 An implementation of GoogLeNet in Keras

Learning Rate Schedules and Adaptive Learning Rate Methods for Deep Learning
 Compares different methods of adjusting the learning rate of a NN dynamically

Building powerful image classification models using very little data
 Data augmentation in Keras; use VGG16 as a base for our image classifier

Profiling Top Kagglers: Martin Henze (AKA Heads or Tails), World’s First Kernels Grandmaster
 The journey to becoming a Kaggle Kernel grandmaster, with lots of tips

Machine learning megabenchmark: GPU providers (part 2)
 Comparison of Cloud GPU compute providers

Learning from humans: what is inverse reinforcement learning?
 Inverse reinforcement learning: given a policy or a set of data about an expert performing a task, try to extract a reward function
 Apprenticeship learning: given an expert policy, use it as a baseline from which to improve

Retro Contest: Results
 Short presentation of the winners of a reinforcement learning contest (they attempted to generalize from previous knowledge)

CNNs from different viewpoints
 Short post describing how to view CNNs as matrix multiplications or dense neural nets; informative
 1(https://eng.uber.com/deconstructinglotterytickets/)
 Using a binary mask to zero out many of the model’s weights leads to better accuracy
 2(https://eng.uber.com/coordconv/)
 Introducing coordinates to convolution filters to give CNNs a way to model the coordinate transform task (increased performance in RL, Object detection)
Tutorials and Guides
Tutorials:
 A Guide to TF Layers: Building a Convolutional Neural Network

Introduction to CNN Keras  Acc 0.997 (top 8%)
 A guide to a small but powerful CNN with a 99.671% accuracy on MNIST; uses data augmentation techniques

A Guide to TF Layers: Building a Convolutional Neural Network
 Tutorial about building CNNs in Tensorflow
Guides

Tutorial: Optimizing Neural Networks using Keras (with Image recognition case study
 Widen your net, deepen your net, add dropout layers, change activation function, change network arhitecture
 A guide to convolution arithmetic for deep learning

Which GPU(s) to Get for Deep Learning
 Basically aim for a GTX 1080/1080Ti if you can afford it, if not go for a GTX 1070

TF Estimators
 Amongst other things you can convert Keras models to TF estimators
 Understanding and Coding Inception Module in Keras
Papers

Generative Temporal Models with Spatial Memory for Partially Observed Environments
 Videos of the agent in action
 Generative Temporal Model with Spatial Memory, an actionconditioned generative model that uses a scalable nonparametric memory to store spatial and visual information; DND memory
 The agent learns an environment then can predict how the environment will look like after a number of time steps (hundreds) and moves

Synthesizing Programs for Images using Reinforced Adversarial Learning
 Video of demonstration
 Using GANs to train a RL one a cluster of machines; the algorithm learns to understand the creation of and recreate images it sees (drawing the digits of the MNIST dataset, characters from the POLIGLOT dataset) and even reconstruct 3D scenes

Backdrop: Stochastic Backpropagation
 Backdrop, a flexible and simpletoimplement method, intuitively described as dropout acting only along the backpropagation pipeline

Efficient Estimation of Word Representations in Vector Space
 Study of the quality of vector representations of words derived by various models on a collection of syntactic and semantic language tasks

Rethinking the Inception Architecture for Computer Vision
 Ideas for reducing computational complexity of CNNs, different Inception architectures

Curiositydriven Exploration bySelfsupervised Prediction
 RL with intrinsic curiosity reward manages to learn how to play Mario without extrinsic rewards, is able to generalize knowledge well; uses an Intrinsic Curiosity Module

L2 Regularization versus Batch and Weight Normalization
 These neural networks use L2 regularization, also called weight decay, ostensibly to prevent overfitting. However, we show that L2 regularization has no regularizing effect when combined with normalization
 Understanding the difficulty of training deep feedforward neural networks

Deep Residual Learning for Image Recognition
 ResNet architecture

Wide Residual Networks
 Wide ResNet architecture
 Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

How Does Batch Normalization Help Optimization?
 An investigation into why Batch Normalization works. It is also compared to L losses. Apparently it works because it smoothens the optimization landscape; it doesn’t have a large impact on ICS.

Verification Of Non Linear Specifications For Neural Networks
 Mathematical tools for proving the robustness of a model against adversarial attacks
 Going deeper with convolutions
 ImageNet Classification with Deep Convolutional Neural Networks
 Practical Recommendations for GradientBased Training of Deep Architectures
 CoupleNet: Paying Attention to Couples with Coupled Attention for Relationship Recommendation
 DARTS: Differentiable Architecture Search
 Learning Cognitive Models using Neural Networks
 Going deeper with convolutions
Courses and Books
Courses:

Coursera Machine Learning  taught by AndrewNG
 ML course, presents the mathematics and concepts behind supervised and unsupervised learning algorithms, and has a good introduction to neural networks
 Great primer into machine learning

Udacity Deep Learning
 Tensorflow Deep Learning course
 Good course for introducing a number of advanced concepts, but I feel it doesn’t spend enough time on some of them
 Has a lot of good assignments in Tensorflow

Udemy Zero to Deep Learning
 Keras Deep Learning course
 Good course with many interesting assigments; provides an intro to data preprocessing and augmentation as well
Books:
Tools
Existing work
Internal Resources
Learning ML
 Loss and Accuracy
 Debugging ML algorithms
 Neural Networks
 Types of Neural Networks
 Types of Machine Learning Problems
 Machine Learning Advice
Tools:
Projects
Discussion:
Guides: