autodiff.reverse.Node

class autodiff.reverse.Node(val)

Primary data structure for reverse mode automatic differentiation.

The process of evaluating derivatives in reverse mode consists of two passes, forward pass and reverse pass. During the forward pass, we calculate the primal values and the local gradient of child nodes with respect of each parent node in the computational graph. In the reverse pass, we recursively calculate the gradients.

Parameters

valfloat

The value of the Node.

childrenlist

List of tuples of (weight, child) where weight is the weight of the sensitivity of the child node with respect to the parent node and child is the child node.

derfloat

Initialiazed to None, represents the derivative of the last descendent with respect to self.

See also

Node.grad

Node.zero_grad

Node.constant

Node.from_array

Examples

Construct a Node for a univariate function:

>>> x = ad.Node(42)
>>> x
Node(42)

Construct multiple Nodes from array of scalars:

>>> x, y, z = ad.Node.from_array([1, 2, 4])
>>> x
Node(1)
>>> y
Node(2)

Create a function from multiple Nodes:

>>> x, y, z = ad.Node.from_array([1, 2, 4])
>>> f = (x * y)/z
>>> f.val
0.5
>>> x.grad()
0.5
>>> y.grad()
0.25
>>> z.grad()
-0.125

Methods

constant(val)	Create a Node representing a constant.
from_array(X)	Generate Nodes for a multivariable function.
grad()	Return the gradient of the last descendent with respect to self.
zero_grad(*args)	Reset Nodes to their default attributes.

grad()

Return the gradient of the last descendent with respect to self.

Parameters

self: Node

Returns

out: float

See also

Node.zero_grad

Examples

>>> x, y = ad.Node(1), ad.Node(1)
>>> x + 3 * y
Node(4)
>>> x.grad()
1.0
>>> y.grad()
3.0

static zero_grad(*args)

Reset Nodes to their default attributes.

Parameters

*args: arbitrary number of Nodes

Returns

None

See also

Node.grad

Examples

>>> x = ad.Node(3)
>>> f = ad.sin(x)
>>> x.grad()
-0.9899924966004454
>>> ad.Node.zero_grad(x)
>>> x.grad()
1.0

static constant(val)

Create a Node representing a constant.

Parameters

valint or float

Value of Node.

Returns

outNode

Node of value val with children set to None and der set to 1.0.

Examples

>>> ad.Node.constant(42)
Node(42)
>>> ad.Node.constant(1)
Node(1)

static from_array(X)

Generate Nodes for a multivariable function.

Parameters

Xarray

Array of numbers which will be values of Nodes.

Returns

outNode generator

Nodes of value X[i] with der set to None and an empty list of children.

Examples

>>> x, y, z = ad.Node.from_array([1,2,3])
>>> x
Node(1)
>>> y
Node(2)
>>> z
Node(3)

previous

Node

next

autodiff.reverse.Node.__add__