Documentation

Objects are lists of dimensions, the product of the spaces of those dimensions.

A differentiable map: at a point, its image and its reverse derivative there, which takes a change of the image to the change of the point it comes from.

The map of an arrow.

The reverse derivative of an arrow at a point.

The arrow with the given map and reverse derivative, for when the two share no work.

A point of a product of spaces, or a change of one: a vector of coordinates for each space. Taking a point of a product apart and putting it together handles the vectors without copying their coordinates.

A point of a single space.

The coordinates of a point, one space after the other.

The point with the given coordinates, one space after the other.

The origin, which is also the change that changes nothing.

The sum of two points, or two changes.

A point, or a change, multiplied by a number.

A point of a product as its two parts.

The point of a product with the given parts.

A linear map, given as a tensor network, its input and result each the vector of a tensor's entries. The change of the input is the change of the result through the dual of the map.

A function applied to each coordinate, given with its derivative.

The coordinatewise product of two vectors: bilinear with the merging spider, written out.

The dot product of two vectors: bilinear with the cap, which sums the products.

The product of an m by n matrix, given row by row, with a vector: matMat with a single column.

The product of an m by n matrix with an n by k matrix, both given row by row. With a batch of m vectors as the rows of the first, the second is a linear map applied to each of them. It is bilinear with the cap on the summed index, written out with matMul: the change of the first matrix is the change of the result times the transposed second, and the change of the second is the transposed first times the change of the result, which adds up the changes from the whole batch.

A vector copied to each of the b rows of a matrix, given row by row: for adding the same vector to each of a batch of b vectors. The change of the vector adds up the rows of the change of the matrix.

The softmax of each of the b rows of a matrix of scores, given row by row: the exponentials of a row divided by their sum, for each of a batch of b vectors a probability for each of k classes. The change of a row of scores is the probabilities times the change of the row minus its average under them.

The cross-entropy of the softmax of each of the b rows of a matrix of scores against the same row of a matrix of target probabilities, both given row by row, summed over the rows: the loss of a classifier on a batch. With the softmax computed along, the change of the scores is the probabilities minus the targets, for targets that sum to 1, and the change of the targets is minus the logarithms of the probabilities.

A bilinear map, given as a tensor network from the tensor product of its two inputs, each input and the result the vector of a tensor's entries: linear in each input. The change of one input is the change of the result through the dual of the map with the other input fixed.

The map to a fixed point, whose reverse derivative is the zero change.

An arrow as a lens: the map is the view, and the reverse derivative the update. Composing arrows and composing their lenses agree, which is the chain rule; AsLens is the functor. A lens views and updates separately, so the update runs the map again.

The underlying functions: a differentiable map as a function from the points of its source to those of its target, forgetting the reverse derivative.

Arrows as lenses, by toLens. The lens of an arrow goes the other way, so this is a functor into the opposite of the category of lenses, and that it is a functor is the chain rule.