Algorithms¶
This section contains the API documentation for solution algorithms and optimization methods used to solve economic models.
Value Function Iteration¶
The Value Function Iteration (VBI) algorithm implements backwards induction to derive value functions from model blocks.
Use backwards induction to derive the arrival value function from a continuation value function and stage dynamics.
- skagent.algos.vbi.ar_from_data(da)¶
Produce a function from any inputs to a given value. This is useful for constructing decision rules with fixed actions.
- skagent.algos.vbi.get_action_rule(action)¶
Produce a function from any inputs to a given value. This is useful for constructing decision rules with fixed actions.
- skagent.algos.vbi.grid_to_data_array(grid={})¶
Construct a zero-valued DataArray with the coordinates based on the Grid passed in.
- skagent.algos.vbi.solve(block, continuation, state_grid, disc_params={}, calibration={})¶
Solve a DBlock using backwards induction on the value function.
Core VBI Functions¶
- skagent.algos.vbi.solve(block, continuation, state_grid, disc_params={}, calibration={})
Solve a DBlock using backwards induction on the value function.
- skagent.algos.vbi.get_action_rule(action)
Produce a function from any inputs to a given value. This is useful for constructing decision rules with fixed actions.
- skagent.algos.vbi.ar_from_data(da)
Produce a function from any inputs to a given value. This is useful for constructing decision rules with fixed actions.
- skagent.algos.vbi.grid_to_data_array(grid={})
Construct a zero-valued DataArray with the coordinates based on the Grid passed in.
Maliar-Style Algorithms¶
Neural network-based solution methods following Maliar et al.
Tools for the implementation of the Maliar, Maliar, and Winant (JME ‘21) method.
This method relies on a simpler problem representation than that elaborated by the skagent Block system.
Note
generate_givens_from_states currently accesses bellman_period.block
directly rather than working through the BellmanPeriod interface. A future
refactoring could route shock generation through BellmanPeriod itself.
Similarly, shock draws are currently Monte Carlo only; structured draws
(e.g. exact discretizations) could be supported via BellmanPeriod.
- skagent.algos.maliar.generate_givens_from_states(states, model_block, shock_copies)¶
Generate omega_i values of the MMW JME ‘21 method.
- Parameters:
- Returns:
Grid containing states augmented with shock copies.
- Return type:
- skagent.algos.maliar.maliar_training_loop(bellman_period, loss_function, states_0_n, parameters, shock_copies=2, max_iterations=5, tolerance=1e-06, random_seed=None, simulation_steps=1, network_width=16, epochs_per_iteration=250)¶
Run the Maliar, Maliar, and Winant (JME ‘21) training loop.
This implements the machine learning method for solving dynamic economic models by training a neural network policy to minimize empirical risk (loss). The network architecture (width, training epochs per iteration) is configurable via optional parameters.
- Parameters:
bellman_period (
BellmanPeriod) – A model definition containing block dynamics and transitions.loss_function (
Callable) – The empirical risk function Xi^n from MMW JME’21. This function is passed to the neural network training routine asloss_function(decision_function, input_grid) -> loss_tensor.states_0_n (
Grid) – A panel of starting states for training. Must contain at least one state.parameters (
dict) – Given parameters for the model.shock_copies (
int) – Number of copies of shocks to include in the training set omega. Must match expected number of shock copies in the loss function. Must be >= 1. Default is 2.max_iterations (
int) – Maximum number of training loop iterations before stopping. Must be >= 1. Default is 5.tolerance (
float) – Convergence tolerance. Training stops when either the L2 norm of parameter changes or the absolute difference in loss is below this threshold. Satisfying either criterion alone is sufficient. Must be > 0. Default is 1e-6.random_seed (
Optional[int]) – Random seed for reproducibility. Default is None.simulation_steps (
int) – Number of time steps to simulate forward when determining the next omega set for training. Higher values let the training states explore more of the state space at higher computational cost. Must be >= 1. Default is 1.network_width (
int) – Width of hidden layers in the policy neural network. Must be >= 1. Default is 16.epochs_per_iteration (
int) – Number of training epochs per iteration. Must be >= 1. Default is 250.
- Returns:
(trained_policy_network, training_states) where trained_policy_network is the BlockPolicyNet and training_states is the Grid of states from the last training iteration (or the convergence point if reached early).
- Return type:
- Raises:
ValueError – If max_iterations < 1, tolerance <= 0, shock_copies < 1, simulation_steps < 1, network_width < 1, epochs_per_iteration < 1, or states_0_n contains no states.
TypeError – If bellman_period is None or loss_function is not callable.
- skagent.algos.maliar.simulate_forward(states_t, bellman_period, decision_function, parameters, big_t)¶
Simulate the model forward for a specified number of periods.
- Parameters:
bellman_period (
BellmanPeriod) – The Bellman period containing model dynamics.decision_function (
Callable) – Function mapping (states, shocks, parameters) to controls.parameters (
dict) – Model parameters.big_t (
int) – Number of time periods to simulate forward. If 0, returns the initial states unchanged.
- Returns:
Final state values after big_t periods.
- Return type:
- Raises:
ValueError – If big_t < 0 or if states_t is an empty dict.
Neural Network Components¶
Net¶
Base neural network class with device management.
- class skagent.ann.Net(n_inputs, n_outputs, width=32, n_layers=2, activation='silu', transform=None, init_seed=None, copy_weights_from=None)¶
Bases:
ModuleA flexible feedforward neural network with configurable architecture.
- Parameters:
n_inputs (int) – Number of input features
n_outputs (int) – Number of output features
width (int, optional) – Width of hidden layers. Default is 32.
n_layers (int, optional) – Number of hidden layers (1-10). Default is 2.
activation (str, list, callable, or None, optional) –
Activation function(s) to use. Options: - str: Apply same activation to all layers (‘silu’, ‘relu’, ‘tanh’, ‘sigmoid’) - list: Apply different activations to each layer, e.g., [‘relu’, ‘tanh’, ‘silu’] - callable: Custom activation function - None: No activation (identity function)
Available activations: ‘silu’, ‘relu’, ‘tanh’, ‘sigmoid’, ‘identity’ Default is ‘silu’.
transform (str, list, callable, or None, optional) –
Transformation to apply to outputs. Options: - str: Apply same transform to all outputs (‘sigmoid’, ‘exp’, ‘tanh’, etc.) - list: Apply different transforms to each output, e.g., [‘sigmoid’, ‘exp’] - callable: Custom transformation function - None: No transformation
Available transforms: ‘sigmoid’, ‘exp’, ‘tanh’, ‘relu’, ‘softplus’, ‘softmax’, ‘abs’, ‘square’, ‘identity’ Default is None.
- property device¶
Device property for backward compatibility.
- forward(x)¶
Define the computation performed at every call.
Should be overridden by all subclasses.
Note
Although the recipe for forward pass needs to be defined within this function, one should call the
Moduleinstance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.
BlockPolicyNet¶
Specialized neural network for policy functions in economic models.
- class skagent.ann.BlockPolicyNet(bellman_period, control_sym=None, apply_open_bounds=True, width=32, **kwargs)¶
Bases:
BellmanPeriodMixin,NetA neural network for policy functions in dynamic programming problems.
This network inherits from Net and provides economic model integration. It automatically determines input/output dimensions from the model block specification and handles control variable bounds.
- Parameters:
bellman_period (BellmanPeriod) – The model Bellman Period
apply_open_bounds (bool, optional) – If True, then the network forward output is normalized by the upper and/or lower bounds, computed as a function of the input tensor. These bounds are “open” because output can be arbitrarily close to, but not equal to, the bounds. Default is True.
control_sym (string, optional) – The symbol for the control variable.
width (int, optional) – Width of hidden layers. Default is 32.
**kwargs – Additional keyword arguments passed to Net. See Net class documentation for all available options including activation, transform, n_layers, init_seed, copy_weights_from, etc.
- decision_function(states_t, shocks_t, parameters)¶
A decision function, from states, shocks, and parameters, to control variable values.
- forward(x)¶
Note that this uses the same architecture of the superclass but adds on a normalization layer appropriate to the bounds of the decision rule.
- get_core_function(length=None)¶
- get_decision_function()¶
- get_decision_rule(length=None)¶
Returns the decision rule corresponding to this neural network.
Training Functions¶
Grid and Computational Tools¶
Grid Class¶
- class skagent.grid.Grid(labels, values, torched=True)¶
Bases:
objectA class representing a labeled grid of numerical values.
- Parameters:
(dict) (config) – dictionary with the following keys: “min” (float): The minimum value for the variable.; “max” (float): The maximum value for the variable; “count” (int): The number of points to generate for the variable.
- classmethod from_config(config={}, torched=True)¶
- classmethod from_dict(kv={}, torched=False)¶
- len()¶
Returns the number of columns, similar to a dict.
- n()¶
Returns the number of values for each symbol
- shape()¶
Returns the shape of the grid values.
- to_dict()¶
Returns a data structure, key: column, similar to tensordict or structured array.
- torch()¶
- update_from_dict(kv)¶
Grid Utility Functions¶
- skagent.grid.make_grid(config)¶
Make a ‘grid’ of values based on the provided configuration.
- Parameters:
(dict) (config) – dictionary with the following keys: “min” (float): The minimum value for the variable; “max” (float): The maximum value for the variable; “count” (int): The number of points to generate for the variable.
- Returns:
numpy.ndarray (A NumPy array of shape (product_of_counts, num_variables), where) – product_of_counts is the product of all count values in the config dictionary, and num_variables is the number of keys in the config.
- skagent.grid.cartesian_product(*arrays)¶
Create a Cartesian product of input arrays.
- Parameters:
*arrays – Variable length arrays to compute product
- Returns:
Array of shape (product_of_lengths, num_arrays)
where product_of_lengths is the product of the lengths of the input arrays,
and num_arrays is the number of input arrays. Each row contains one element
of the Cartesian product.
This page is under construction. Content will be added as the API develops.