Tutorial: control the size of a network

Impose the maximum number of links

Extract MetamaterialAntenna Example, see Metamaterial Antennas, provided with NeurEco installation. The created directory should contain the following files:

• x_test.csv

• y_test.csv

• x_train.csv

• y_train.csv

Once that’s done, we will create import the needed libraries:

from NeurEco import NeurEcoTabular as Tabular
import numpy as np

the next step is to load the training data:

x_train = np.genfromtxt("x_train.csv", delimiter=";", skip_header=True)
y_train = np.genfromtxt("y_train.csv", delimiter=";", skip_header=True)

At this stage, we will build a regression model without controlling the size of the network (this will be our reference point for comparison):

builder = Tabular.Regressor()
''' Building the NeurEco Model '''
builder.build(input_data=x_train, output_data=y_train,
              # the rest of these parameters are optional
              write_model_to="./MetamaterialsAntennas/MetamaterialsAntennas",
              checkpoint_address="./MetamaterialsAntennas/MetamaterialsAntennas.checkpoint",
              valid_percentage=33.33,
              outputs_normalize_per_feature=True,
              links_maximum_number=0)

Once that’s done, we will create a regression model where the size of the network is limited (for embedding). To control the size, we just need to change the parameter links_maximum_number. This is an integer that controls the maximum number of trainable parameters.

builder.build(input_data=x_train, output_data=y_train,
              # the rest of these parameters are optional
              write_model_to="./MetamaterialsAntennas/MetamaterialsAntennas_Mbed",
              checkpoint_address="./MetamaterialsAntennas/MetamaterialsAntennas_Mbed.checkpoint",
              valid_percentage=33.33,
              outputs_normalize_per_feature=True,
              links_maximum_number=150)

''' Delete the builder from memory'''
print("Deleting the NeurEco builder".center(60, "*"))
builder.delete()

Once the two models are created, we will compare their sizes and performances. We will load the testing data, and check the relative l2 error of each network on the unseen set of data. After which, we will compare the sizes of these two networks.

print("Loading the training data".center(60, "*"))
x_test = np.genfromtxt("x_test.csv", delimiter=";", skip_header=True)
y_test = np.genfromtxt("y_test.csv", delimiter=";", skip_header=True)

neureco_model = Tabular.Regressor()
''' Loading the normal size model '''
neureco_model.load('./MetamaterialsAntennas/MetamaterialsAntennas')
print("Regular model n° of trainable parameters: {0}".format(neureco_model.vec.size))
out_ = neureco_model.evaluate(x_test)
l2_error_regular = neureco_model.compute_error(out_, y_test)
print("Regular model l2 error on testing data: {0}".format(l2_error_regular))
neureco_model.plot_network()

''' Loading the Mbed  model '''
neureco_model.load('./MetamaterialsAntennas/MetamaterialsAntennas_Mbed')
print("Embedded model n° of trainable parameters: {0}".format(neureco_model.vec.size))
out_ = neureco_model.evaluate(x_test)
l2_error_mbed = neureco_model.compute_error(out_, y_test)
print("Embedded model l2 error on testing data: {0}".format(l2_error_mbed))
neureco_model.plot_network()
neureco_model.delete()

The output of the previous code is as follows:

Regular model n° of trainable parameters: 1361
Regular model l2 error on testing data: 0.010744763022509518

Embedded model n° of trainable parameters: 298
Embedded model l2 error on testing data: 0.019175265003600478

The two networks architectures are given in the following figures

Regular network

Embedded network

Select a model from a checkpoint

We can also control the size of the network, by selecting any model from the checkpoint file of a built model, and restart the build from that stage. We can choose to enrich the model by adding some trainable parameters, or not (meaning that the weights will be optimized but without adding trainable parameters). In this case, there are 3 building parameters to use:

• checkpoint_to_start_build_from

• start_build_from_model_number

• freeze_structure

For example, using the regular network built above we can choose the checkpoint n°5, and restart the build from that point.

new_model = Tabular.Regressor()
index_of_model_to_improve = 4

new_model.build(input_data=x_train, output_data=y_train,
                # the rest of these parameters are optional
                checkpoint_to_start_build_from="./MetamaterialsAntennas/MetamaterialsAntennas.checkpoint",
                start_build_from_model_number=index_of_model_to_improve,
                freeze_structure=True,
                write_model_to = "./MetamaterialsAntennas/MetamaterialsAntennas_improved",
                checkpoint_address = "./MetamaterialsAntennas/MetamaterialsAntennas_improved.checkpoint",
                valid_percentage = 33.33,
                outputs_normalize_per_feature = True)

print("Imroved Model n° of trainable parameters: {0}".format(new_model.vec.size))
new_model.plot_network()

The new model will have the following architecture:

Controlling the size of a network by restarting the build from a checkpoint