Information Architecture¶

The neet.information provides a collection of functions which compute various information measures over the dynamics of discrete-state network models. Each measure computes a time series of a desired length from each initial state of the network and uses the time series to populate probability distributions over the state transitions of each node. From there any number of information or entropy measures may be applied. All of this is bookkeeping is taken care of by the function; all you have to do is provide a network and any necessary parameters for the calculation

Example (Active Information for Fission Yeast)

>>> active_information(s_pombe, k=5, timesteps=20)
array([0.        , 0.4083436 , 0.62956679, 0.62956679, 0.37915718,
       0.40046165, 0.67019615, 0.67019615, 0.39189127])

The core information-theoretic computations are supported by the PyInform package.

API Documentation¶

The neet.information module is broken into two parts: Information Measures and the Architecture Class.

The Information Measures are a collection of module-level functions which independently compute a variety of information measures over a network of interest:

Active Information Storage (active_information())
Entropy Rate (entropy_rate())
Transfer Entropy (transfer_entropy())
Mutual Information (mutual_information()).

However, since they do not share any data between them, if you plan to compute many different measures, it is much more efficient to cache the computed time series. This is done by the Architecture class which provides all of the same measures, only computes the time series once.

Information Measures¶

neet.information.active_information(net, k, timesteps, size=None, local=False)[source]¶

Compute the active information storage for each node in a network.

>>> active_information(s_pombe, k=5, timesteps=20)
array([0.        , 0.4083436 , 0.62956679, 0.62956679, 0.37915718,
       0.40046165, 0.67019615, 0.67019615, 0.39189127])
>>> lais = active_information(s_pombe, k=5, timesteps=20, local=True)
>>> lais[1]
array([[0.13079175, 0.13079175, 0.13079175, ..., 0.13079175, 0.13079175,
        0.13079175],
       [0.13079175, 0.13079175, 0.13079175, ..., 0.13079175, 0.13079175,
        0.13079175],
       [0.13079175, 0.13079175, 0.13079175, ..., 0.13079175, 0.13079175,
        0.13079175],
       ...,
       [0.13079175, 0.13079175, 0.13079175, ..., 0.13079175, 0.13079175,
        0.13079175],
       [0.13079175, 0.13079175, 0.13079175, ..., 0.13079175, 0.13079175,
        0.13079175],
       [0.13079175, 0.13079175, 0.13079175, ..., 0.13079175, 0.13079175,
        0.13079175]])
>>> np.mean(lais[1])
0.4083435963963132

Parameters:	net – a NEET network k – the history length timesteps – the number of timesteps to evaluate the network size – the size of variable-sized network (or `None`) local – whether or not to compute the local active information
Returns:	a numpy array of active information values

neet.information.entropy_rate(net, k, timesteps, size=None, local=False)[source]¶

Compute the entropy rate for each node in a network.

>>> entropy_rate(s_pombe, k=5, timesteps=20)
array([0.        , 0.01691208, 0.07280268, 0.07280268, 0.05841994,
       0.02479402, 0.03217332, 0.03217332, 0.08966941])
>>> ler = entropy_rate(s_pombe, k=5, timesteps=20, local=True)
>>> ler[4]
array([[0.        , 0.        , 0.        , ..., 0.00507099, 0.00507099,
        0.00507099],
       [0.        , 0.        , 0.        , ..., 0.00507099, 0.00507099,
        0.00507099],
       [0.        , 0.        , 0.        , ..., 0.00507099, 0.00507099,
        0.00507099],
       ...,
       [0.        , 0.29604946, 0.00507099, ..., 0.00507099, 0.00507099,
        0.00507099],
       [0.        , 0.29604946, 0.00507099, ..., 0.00507099, 0.00507099,
        0.00507099],
       [0.        , 0.29604946, 0.00507099, ..., 0.00507099, 0.00507099,
        0.00507099]])
>>> np.mean(ler[4])
0.0584199434476326

Parameters:	net – a NEET network k – the history length timesteps – the number of timesteps to evaluate the network size – the size of variable-sized network (or `None`) local – whether or not to compute the local entropy rate
Returns:	a numpy array of entropy rate values

neet.information.transfer_entropy(net, k, timesteps, size=None, local=False)[source]¶

Compute the transfer entropy matrix for a network.

>>> transfer_entropy(s_pombe, k=5, timesteps=20)
array([[ 0.00000000e+00,  0.00000000e+00, -1.11022302e-16,
        -1.11022302e-16,  0.00000000e+00,  0.00000000e+00,
        -1.11022302e-16,  0.00000000e+00,  0.00000000e+00],
       [-4.44089210e-16, -4.44089210e-16, -4.44089210e-16,
        -4.44089210e-16,  1.69120759e-02, -4.44089210e-16,
        -4.44089210e-16, -4.44089210e-16, -4.44089210e-16],
       [ 4.44089210e-16,  5.13704599e-02,  4.44089210e-16,
         1.22248438e-02,  1.99473023e-02,  5.13704599e-02,
         6.03879253e-03,  6.03879253e-03,  7.28026801e-02],
       [ 4.44089210e-16,  5.13704599e-02,  1.22248438e-02,
         4.44089210e-16,  1.99473023e-02,  5.13704599e-02,
         6.03879253e-03,  6.03879253e-03,  7.28026801e-02],
       [ 0.00000000e+00,  5.84199434e-02,  4.76020591e-02,
         4.76020591e-02,  0.00000000e+00,  5.84199434e-02,
         4.76020591e-02,  4.76020591e-02,  0.00000000e+00],
       [ 2.22044605e-16,  2.22044605e-16,  2.47940243e-02,
         2.47940243e-02,  2.22044605e-16,  2.22044605e-16,
         2.47940243e-02,  2.47940243e-02,  2.22044605e-16],
       [-4.44089210e-16,  1.66898258e-02,  4.52634832e-03,
         4.52634832e-03,  1.19161772e-02,  1.66898258e-02,
        -4.44089210e-16,  2.98276692e-03,  3.21733224e-02],
       [-4.44089210e-16,  1.66898258e-02,  4.52634832e-03,
         4.52634832e-03,  1.19161772e-02,  1.66898258e-02,
         2.98276692e-03, -4.44089210e-16,  3.21733224e-02],
       [-4.44089210e-16,  6.03036989e-02,  4.82889077e-02,
         4.82889077e-02,  8.96694146e-02,  6.03036989e-02,
         4.89270931e-02,  4.89270931e-02, -4.44089210e-16]])

>>> lte = transfer_entropy(s_pombe, k=5, timesteps=20, local=True)
>>> lte[4,3]
array([[0.        , 0.        , 0.        , ..., 0.00507099, 0.00507099,
        0.00507099],
       [0.        , 0.        , 0.        , ..., 0.00507099, 0.00507099,
        0.00507099],
       [0.        , 0.        , 0.        , ..., 0.00507099, 0.00507099,
        0.00507099],
       ...,
       [0.        , 0.29604946, 0.00507099, ..., 0.00507099, 0.00507099,
        0.00507099],
       [0.        , 0.29604946, 0.00507099, ..., 0.00507099, 0.00507099,
        0.00507099],
       [0.        , 0.29604946, 0.00507099, ..., 0.00507099, 0.00507099,
        0.00507099]])
>>> np.mean(lte[4,3])
0.047602059103704124

Parameters:	net – a NEET network k – the history length timesteps – the number of timesteps to evaluate the network size – the size of variable-sized network (or `None`) local – whether or not to compute the local transfer entropy
Returns:	a numpy matrix of transfer entropy values

neet.information.mutual_information(net, timesteps, size=None, local=False)[source]¶

Compute the mutual information matrix for a network.

>>> mutual_information(s_pombe, timesteps=20)
array([[0.16232618, 0.01374672, 0.00428548, 0.00428548, 0.01340937,
        0.01586238, 0.00516987, 0.00516987, 0.01102766],
       [0.01374672, 0.56660996, 0.00745714, 0.00745714, 0.00639113,
        0.32790848, 0.0067609 , 0.0067609 , 0.00468342],
       [0.00428548, 0.00745714, 0.83837294, 0.475582  , 0.21157695,
        0.00432855, 0.4590254 , 0.4590254 , 0.12755745],
       [0.00428548, 0.00745714, 0.475582  , 0.83837294, 0.21157695,
        0.00432855, 0.4590254 , 0.4590254 , 0.12755745],
       [0.01340937, 0.00639113, 0.21157695, 0.21157695, 0.57459066,
        0.00703145, 0.17560769, 0.17560769, 0.01233356],
       [0.01586238, 0.32790848, 0.00432855, 0.00432855, 0.00703145,
        0.51905053, 0.00621124, 0.00621124, 0.00260667],
       [0.00516987, 0.0067609 , 0.4590254 , 0.4590254 , 0.17560769,
        0.00621124, 0.80831657, 0.49349527, 0.10390475],
       [0.00516987, 0.0067609 , 0.4590254 , 0.4590254 , 0.17560769,
        0.00621124, 0.49349527, 0.80831657, 0.10390475],
       [0.01102766, 0.00468342, 0.12755745, 0.12755745, 0.01233356,
        0.00260667, 0.10390475, 0.10390475, 0.63423835]])
>>> lmi = mutual_information(s_pombe, timesteps=20, local=True)
>>> lmi[4,3]
array([[-0.67489772, -0.67489772, -0.67489772, ...,  0.18484073,
         0.18484073,  0.18484073],
       [-0.67489772, -0.67489772, -0.67489772, ...,  0.18484073,
         0.18484073,  0.18484073],
       [-0.67489772, -0.67489772, -0.67489772, ...,  0.18484073,
         0.18484073,  0.18484073],
       ...,
       [-2.89794147,  1.7513014 ,  0.18484073, ...,  0.18484073,
         0.18484073,  0.18484073],
       [-2.89794147,  1.7513014 ,  0.18484073, ...,  0.18484073,
         0.18484073,  0.18484073],
       [-2.89794147,  1.7513014 ,  0.18484073, ...,  0.18484073,
         0.18484073,  0.18484073]])
>>> np.mean(lmi[4,3])
0.21157695279993294

Parameters:	net – a NEET network k – the history length timesteps – the number of timesteps to evaluate the network size – the size of variable-sized network (or `None`) local – whether or not to compute the local mutual information
Returns:	a numpy matrix of mutual information values

Architecture Class¶

class neet.information.Architecture(net, k, timesteps, size=None)[source]¶

A class to represent the k-history informational architecture of a network.

Note

The class:Architecture computes the average information measures a bit differently than the associated module-level functions. Specifically, it first computes the local measures and then averages them; this leads different numerical results on account of the subtlties of floating-point mathematics. However, they will always be “close”. In particular

>>> arch = Architecture(s_pombe, k=5, timesteps=20)
>>> function = transfer_entropy(s_pombe, k=5, timesteps=20)
>>> method = arch.transfer_entropy()
>>> np.testing.assert_almost_equal(method, function)

__init__(net, k, timesteps, size=None)[source]¶

Initialize the architecture given a network and enough information to compute a time series.

During initialization the following measures are computed and cached: * Local and Average Active Information Storage * Local and Average Entropy Rate * Local and Average Transfer Entropy * Local and Average Mutual Information

>>> arch = Architecture(s_pombe, k=5, timesteps=20)
>>> arch.active_information()
array([0.        , 0.4083436 , 0.62956679, 0.62956679, 0.37915718,
       0.40046165, 0.67019615, 0.67019615, 0.39189127])

Parameters:	net – a NEET network k – the history length timesteps – the number of timesteps to evaluate the network size – the size of variable-sized network (or `None`)

active_information(local=False)[source]¶

Get the local or average active information

>>> arch = Architecture(s_pombe, k=5, timesteps=20)
>>> arch.active_information()
array([0.        , 0.4083436 , 0.62956679, 0.62956679, 0.37915718,
       0.40046165, 0.67019615, 0.67019615, 0.39189127])
>>> lais = arch.active_information(local=True)
>>> lais[1]
array([[0.13079175, 0.13079175, 0.13079175, ..., 0.13079175, 0.13079175,
        0.13079175],
       [0.13079175, 0.13079175, 0.13079175, ..., 0.13079175, 0.13079175,
        0.13079175],
       [0.13079175, 0.13079175, 0.13079175, ..., 0.13079175, 0.13079175,
        0.13079175],
       ...,
       [0.13079175, 0.13079175, 0.13079175, ..., 0.13079175, 0.13079175,
        0.13079175],
       [0.13079175, 0.13079175, 0.13079175, ..., 0.13079175, 0.13079175,
        0.13079175],
       [0.13079175, 0.13079175, 0.13079175, ..., 0.13079175, 0.13079175,
        0.13079175]])
>>> np.mean(lais[1])
0.4083435963963132

Parameters:	local (bool) – whether to return local (True) or global active information
Returns:	a numpy array containing the (local) active information

entropy_rate(local=False)[source]¶

Get the local or average entropy rate

>>> arch = Architecture(s_pombe, k=5, timesteps=20)
>>> arch.entropy_rate()
array([0.        , 0.01691208, 0.07280268, 0.07280268, 0.05841994,
       0.02479402, 0.03217332, 0.03217332, 0.08966941])
>>> ler = arch.entropy_rate(local=True)
>>> ler[4]
array([[0.        , 0.        , 0.        , ..., 0.00507099, 0.00507099,
        0.00507099],
       [0.        , 0.        , 0.        , ..., 0.00507099, 0.00507099,
        0.00507099],
       [0.        , 0.        , 0.        , ..., 0.00507099, 0.00507099,
        0.00507099],
       ...,
       [0.        , 0.29604946, 0.00507099, ..., 0.00507099, 0.00507099,
        0.00507099],
       [0.        , 0.29604946, 0.00507099, ..., 0.00507099, 0.00507099,
        0.00507099],
       [0.        , 0.29604946, 0.00507099, ..., 0.00507099, 0.00507099,
        0.00507099]])

Parameters:	local (bool) – whether to return local (True) or global entropy rate
Returns:	a numpy array containing the (local) entropy rate

mutual_information(local=False)[source]¶

Get the local or average mutual information

>>> arch = Architecture(s_pombe, k=5, timesteps=20)
>>> arch.mutual_information()
array([[0.16232618, 0.01374672, 0.00428548, 0.00428548, 0.01340937,
        0.01586238, 0.00516987, 0.00516987, 0.01102766],
       [0.01374672, 0.56660996, 0.00745714, 0.00745714, 0.00639113,
        0.32790848, 0.0067609 , 0.0067609 , 0.00468342],
       [0.00428548, 0.00745714, 0.83837294, 0.475582  , 0.21157695,
        0.00432855, 0.4590254 , 0.4590254 , 0.12755745],
       [0.00428548, 0.00745714, 0.475582  , 0.83837294, 0.21157695,
        0.00432855, 0.4590254 , 0.4590254 , 0.12755745],
       [0.01340937, 0.00639113, 0.21157695, 0.21157695, 0.57459066,
        0.00703145, 0.17560769, 0.17560769, 0.01233356],
       [0.01586238, 0.32790848, 0.00432855, 0.00432855, 0.00703145,
        0.51905053, 0.00621124, 0.00621124, 0.00260667],
       [0.00516987, 0.0067609 , 0.4590254 , 0.4590254 , 0.17560769,
        0.00621124, 0.80831657, 0.49349527, 0.10390475],
       [0.00516987, 0.0067609 , 0.4590254 , 0.4590254 , 0.17560769,
        0.00621124, 0.49349527, 0.80831657, 0.10390475],
       [0.01102766, 0.00468342, 0.12755745, 0.12755745, 0.01233356,
        0.00260667, 0.10390475, 0.10390475, 0.63423835]])
>>> lmi = arch.mutual_information(local=True)
>>> lmi[4,3]
array([[-0.67489772, -0.67489772, -0.67489772, ...,  0.18484073,
         0.18484073,  0.18484073],
       [-0.67489772, -0.67489772, -0.67489772, ...,  0.18484073,
         0.18484073,  0.18484073],
       [-0.67489772, -0.67489772, -0.67489772, ...,  0.18484073,
         0.18484073,  0.18484073],
       ...,
       [-2.89794147,  1.7513014 ,  0.18484073, ...,  0.18484073,
         0.18484073,  0.18484073],
       [-2.89794147,  1.7513014 ,  0.18484073, ...,  0.18484073,
         0.18484073,  0.18484073],
       [-2.89794147,  1.7513014 ,  0.18484073, ...,  0.18484073,
         0.18484073,  0.18484073]])

Parameters:	local (bool) – whether to return local (True) or global mutual information
Returns:	a numpy array containing the (local) mutual information

transfer_entropy(local=False)[source]¶

Get the local or average transfer entropy

>>> arch = Architecture(s_pombe, k=5, timesteps=20)
>>> arch.transfer_entropy()
array([[ 0.00000000e+00,  3.37457926e-18, -2.18195687e-18,
        -2.18195687e-18, -5.39390581e-18,  3.37457926e-18,
         5.10930274e-18,  5.10930274e-18,  1.80248611e-18],
       [-3.25260652e-18, -3.25260652e-18, -3.05745013e-17,
        -3.05745013e-17,  1.69120759e-02, -3.25260652e-18,
        -3.25260652e-18, -3.25260652e-18, -3.25260652e-18],
       [-4.20670443e-17,  5.13704599e-02, -4.20670443e-17,
         1.22248438e-02,  1.99473023e-02,  5.13704599e-02,
         6.03879253e-03,  6.03879253e-03,  7.28026801e-02],
       [-4.20670443e-17,  5.13704599e-02,  1.22248438e-02,
        -4.20670443e-17,  1.99473023e-02,  5.13704599e-02,
         6.03879253e-03,  6.03879253e-03,  7.28026801e-02],
       [-1.09531524e-16,  5.84199434e-02,  4.76020591e-02,
         4.76020591e-02, -1.09531524e-16,  5.84199434e-02,
         4.76020591e-02,  4.76020591e-02, -1.09531524e-16],
       [-5.20417043e-18, -5.20417043e-18,  2.47940243e-02,
         2.47940243e-02, -5.20417043e-18, -5.20417043e-18,
         2.47940243e-02,  2.47940243e-02, -5.20417043e-18],
       [ 3.08997619e-17,  1.66898258e-02,  4.52634832e-03,
         4.52634832e-03,  1.19161772e-02,  1.66898258e-02,
         3.08997619e-17,  2.98276692e-03,  3.21733224e-02],
       [ 3.08997619e-17,  1.66898258e-02,  4.52634832e-03,
         4.52634832e-03,  1.19161772e-02,  1.66898258e-02,
         2.98276692e-03,  3.08997619e-17,  3.21733224e-02],
       [ 1.37693676e-17,  6.03036989e-02,  4.82889077e-02,
         4.82889077e-02,  8.96694146e-02,  6.03036989e-02,
         4.89270931e-02,  4.89270931e-02,  1.37693676e-17]])

>>> lte = arch.transfer_entropy(local=True)
>>> lte[4,3]
array([[0.        , 0.        , 0.        , ..., 0.00507099, 0.00507099,
        0.00507099],
       [0.        , 0.        , 0.        , ..., 0.00507099, 0.00507099,
        0.00507099],
       [0.        , 0.        , 0.        , ..., 0.00507099, 0.00507099,
        0.00507099],
       ...,
       [0.        , 0.29604946, 0.00507099, ..., 0.00507099, 0.00507099,
        0.00507099],
       [0.        , 0.29604946, 0.00507099, ..., 0.00507099, 0.00507099,
        0.00507099],
       [0.        , 0.29604946, 0.00507099, ..., 0.00507099, 0.00507099,
        0.00507099]])

Parameters:	local (bool) – whether to return local (True) or global transfer entropy
Returns:	a numpy array containing the (local) transfer entropy