Intro to optimization in deep learning: Gradient Descent


#1

Image Credits: O'Reilly Media

Deep Learning, to a large extent, is really about solving massive nasty optimization problems. A Neural Network is merely a very complicated function, consisting of millions of parameters, that represents a mathematical solution to a problem. Consider the task of image classification. AlexNet is a mathematical function that takes an array representing RGB values of an image, and produces the output as a bunch of class scores.


This is a companion discussion topic for the original entry at https://blog.paperspace.com/intro-to-optimization-in-deep-learning-gradient-descent/

#2

Hi, I’d like to translate this optimization series to Chinese. Can you give me the permission to translate it?

The translated text will be published at ai.jqr.com and related Chinese social network accounts.

Thanks.


#3

@weakish Definitely. As long as the post receives credit (link to blog mentioning it was originally posted there), then that’s fine.


#4

Chinese translation: https://www.jqr.com/article/000495

Attribution to author is given at the beginning (in translator’s note) and there is a backlink at the end of the translated text.

BTW, some possible typos encountered during translation:

classifying images of images of cats as humans

images of cats

we can infinite directions on this plane

can have infinite directions

or convergence has has taken place

has

with only one minima we can converge too

to

out earlier approach processed all examples in one single batch

our

while trying to converge to a global maximum

minima


#5

@weakish Thanks for the corrections, the post has been updated.