Compute gradients

This option is only available in the python API for the Tabular solution.

The parameters \(w\) of the model are obtained via a call to the funcion get_weights.

If we consider that a NeurEco tabular model is a function \(F(x, w) = y\), where \(x\) is the inputs of the network, \(w\) are the weights and math:y is the output, the gradients of the model are obtained via the calls to:

• the forward gradient: forward_derivative(w, dw, x, dx=None). This function computes a forward derivative of the model with parameters \(w\) and inputs \(x\) that corresponds to the perturbations of the parameters \(dw\) and of the inputs math:dx:

  \[\frac{dy}{dx} + \frac{dy}{dw}\]

• the backward gradient: gradient(w, x, py). This function computes the gradients of the model with respect to parameters and inputs, given the output perturbation \(py\), the model parameters \(w\) and the inputs \(x\):

  \[\frac{pw}{py} , \frac{px}{py}\]

Thus, the NeurEco Tabular model can be used as a block inside user’s script involving the gradients flows (for example, an optimization problem).

The model parameters can be set to defined by user values \(w\) via set_weights(w)

There are four methods to use for the derivatives:

• get_weights: this method will retrieve the weights of a NeurEco Tabular model.

  neureco_tabular_model.get_weights()
  

  return

  a numpy (n, 1) array where n is the number of trainable parameters in the model

• set_weights: sets the new weights of a NeurEco Tabular model.

  neureco_tabular_model.set_weights(w)
  

  param w

  new weights array

  type w

  numpy array with a shape (n, 1) where n is a number of trainable parameters in the model

  return

  set_status: 0 if ok, other if not

• forward_derivative: computes the forward mode of the automatic differentiation.

  neureco_tabular_model.forward_derivative(w, dw, x, dx)
  

  param w

  weights array, the trainable parameters of the model will be set to these values

  type w

  numpy array with a shape (p, 1) where p is the number of trainable parameters of the model

  param dw

  weights perturbation amount

  type dw

  numpy array with a shape (p, 1) where p is the number of trainable parameters of the model

  param x

  input data array

  type x

  numpy array with a shape (n, m) where n is the number of samples and m is the number of input count.

  param dx

  inputs perturbation amount (if None, inputs are static)

  type dx

  numpy array with the same shape as \(x\)

  return

  \(dy/dx + dy/dw\), where \(y\) is the output of the model

• gradient: computes the reverse mode of the automatic differentiation.

  neureco_tabular_model.gradient(w, x, py)
  

  param w

  weight array (shape = (n_trainable_parameters, 1))

  param x

  input array

  param py

  output perturbation amount (should have the same shape of the outputs of the model)

  return

  tuple \((pw/py, px/py)\)

Tutorial: compute gradients gives an example of the usage of these methods.