Deep Learning Professional Certificate Program (edX)

“Formal education makes a living, but self-education makes a fortune,” - Jim Rohn

Professional Certificate in Deep Learning

https://www.edx.org/professional-certificate/ibm-deep-learning

Due to the heavy use of Data Science when working on Deep Learning, I've decided take an entirely different certification course first...

• Professional Certificate in Python Data Science þ

    ·         oDeep Learning Fundamentals with Keras

o   Prerequisites:

§  Python Programming.

·        See prerequisites of next section for more Python related courses
·         https://www.edx.org/course/python-basics-for-data-science-2 þ

§  Machine Learning with Python. 

·        https://www.edx.org/course/machine-learning-with-pythonþ

§  Partial Derivatives. edX recommended Khan Academy.  I also traded a sushi dinner to my son for some tutoring, where I posted my notes here and here.

·         https://www.youtube.com/watch?v=AXqhWeUEtQU Introduction þ

·         https://www.youtube.com/watch?v=kdMep5GUOBw Formal definition þ

·         https://www.youtube.com/watch?v=dfvnCHqzK54 And Graphs þ

·         oDeep Learning with Python and PyTorch

o   Prerequisites.  Here I elect to use Pluralsight courses I deemed similar to the edX courses.

§  Python & Jupyter notebooks

·         Python

o   https://app.pluralsight.com/library/courses/python-big-picture/table-of-contents  þ

o   https://app.pluralsight.com/library/courses/python-getting-started/table-of-contents    þ

o   https://app.pluralsight.com/library/courses/python-fundamentals/table-of-contents þ

·         Jupyter notebook

o   https://app.pluralsight.com/library/courses/jupyter-notebook-python/table-of-contents þ

§  Machine Learning concepts

·         https://app.pluralsight.com/library/courses/understanding-machine-learning/table-of-contents þ

§  Deep Learning concepts

·         https://app.pluralsight.com/library/courses/deep-learning-big-picture/table-of-contents þ

·         oDeep Learning with Tensorflow  

o   Prerequisites:

§  Same prerequisites as previous course.

·        oUsing GPUs to Scale and Speed-up Deep Learning  

o   Prerequisites:

§  No prerequisites.

·         oApplied Deep Learning Capstone Project  

o   Prerequisites:

§  Completed all courses in the Deep Learning Professional Certification Program  o

Partial Derivatives

Although this applies to higher dimensions too, this example sticks with a three variable equation because it can be visualized in three dimensional space.  It will look like a sheet that is tilted and/or bent in various ways.

Example equation

Graphed at geogebra

Partial derivative

This is essentially picking a point along one of the planes to get the equation for the line that is formed out of the sheet right there with respect to the other two planes.

Process

Starting with the original equation graphed above 

Pick a point along the Y axis.  In the graph above, the “sheet” crosses the Y axis at position 0, so using that it gives us this equation for the line that forms where the sheet passes that axis.

Deriving that like any other two variable equation gives this:

This is the slope along the z and x axes, it can be helpful to express it as 2/1.  If the image is turned to view edgewise along the sheet with respect to the z and x axes, this slope becomes observable:

Derivatives

A two variable equation can be graphed on a simple two dimensional grid.  It will often be some sort of curved line.  The derivative is an equation that gives the slope of that line at a given point.

Example Equation

Can also be expressed as

Derivative

How was it derived?

The following power rule is applied to each term:

The way I think of it is that the “t” in the power rule is the entire term, which is usually a constant, a variable, and an exponent.

First term

The first term of the example equation is

Which when thought of as the “t” in the power rule with constant, variable, and exponent looks like this:

Applying the power rule means taking the exponent as the value “n”.  In this case 4.  Multiplying it by the front of the term is (4 x 1) or 4.  Changing the exponent to n-1 is (4-1) or 3.  So the first term derives to:

Second Term

Looking at the second term of the example equation and what it derives into shows the 3 being multiplied by the 2 and the exponent decreasing to 2.

Third Term

The third term is similar and becomes 8x.  The exponent was reduced to 1 so it need not be displayed anymore.

Fourth Term

The fourth term sees the exponent reducing down to zero, and since anything to the zeroth power is one, the x to the zeroth power becomes 1 and doesn’t need to be displayed anymore.  It just multiplies by the 5 and becomes the value 5.

Fifth Term

The fifth term is just a constant so like all constants, it derives to zero, so is effectively just dropped.  Why?  Because to get it into the “t” form for the power rule, it becomes:

With the starting exponent of zero, that means “n” is zero, and since the whole thing is going to be multiplied by “n” on the front, the whole term solves to zero.