ixmp package

ixmp provides three classes:

Platform([dbprops, dbtype, jvmargs]) Database-backed instance of the ixmp.
TimeSeries(mp, model, scenario[, version, …]) Generic collection of data in time series format.
Scenario(mp, model, scenario[, version, …]) Collection of model-related input and output data.

as well as some utility classes and methods:

ixmp.config.Config([read]) Configuration for ixmp.
ixmp.model_settings.register_model(name, config) Register a new model with ixmp.
ixmp.testing.make_dantzig(mp[, solve]) Return ixmp.Scenario of Dantzig’s canning/transport problem.

Core classes

class ixmp.Platform(dbprops=None, dbtype=None, jvmargs=None)

Database-backed instance of the ixmp.

Each Platform connects three components:

  1. A database for storing model inputs and outputs. This may either be a local file (dbtype='HSQLDB') or a database server accessed via a network connection. In the latter case, connection information is read from a properties file.
  2. A Java Virtual Machine (JVM) to run core ixmp logic and access the database.
  3. One or more model(s), implemented in GAMS or another language or framework.

The constructor parameters control these components. TimeSeries and Scenario objects are specific to one Platform; to move data between platforms, see Scenario.clone().

Parameters:
  • dbprops (path-like, optional) –

    If dbtype is None, the name of a database properties file (default: ‘default.properties’) in the properties file directory (default: ???) or the path of a properties file.

    If dbtype == ‘HSQLDB’, the path of a local database, (default: “$HOME/.local/ixmp/localdb/default”) or name of a database file in the local database directory (default: “$HOME/.local/ixmp/localdb/”).

  • dbtype ('HSQLDB', optional) – Database type to use. If None, a remote database is accessed. If ‘HSQLDB’, a local database is created and used at the path given by dbprops.
  • jvmargs (str, optional) –

    Options for launching the Java Virtual Machine, e.g., the maximum heap space: “-Xmx4G”. See the JVM documentation for a list of options.
Scenario(model, scen, version=None, scheme=None, annotation=None, cache=False)

Initialize a new Scenario.

Deprecated since version 1.1.0: Instead, use:

>>> mp = ixmp.Platform(…)
>>> ixmp.Scenario(mp, …)
add_region(region, hierarchy, parent='World')

Define a region including a hierarchy level and a ‘parent’ region.

Tip

On a Platform backed by a shared database, a region may already exist with a different spelling. Use regions() first to check, and consider calling add_region_synonym() instead.

Parameters:
  • region (str) – Name of the region.
  • hierarchy (str) – Hierarchy level of the region (e.g., country, R11, basin)
  • parent (str, default 'World') – Assign a ‘parent’ region.
add_region_synonym(region, mapped_to)

Define a synonym for a region.

When adding timeseries data using the synonym in the region column, it will be converted to mapped_to.

Parameters:
  • region (str) – Name of the region synonym.
  • mapped_to (str) – Name of the region to which the synonym should be mapped.
add_unit(unit, comment='None')

Define a unit.

Parameters:
  • unit (str) – Name of the unit.
  • comment (str, optional) – Annotation describing the unit or why it was added. The current database user and timestamp are appended automatically.
check_access(user, models, access='view')

Check access to specific model

Parameters:
  • user (str) – Registered user name
  • models (str or list of str) – Model(s) name
  • access (str, optional) – Access type - view or edit
close_db()

Close the database connection.

A HSQL database can only be used by one Platform instance at a time. Any existing connection must be closed before a new one can be opened.

open_db()

(Re-)open the database connection.

The database connection is opened automatically for many operations. After calling close_db(), it must be re-opened.

regions()

Return all regions defined for the IAMC-style timeseries format including known synonyms.

Returns:
Return type:pandas.DataFrame
scenario_list(default=True, model=None, scen=None)

Return information on TimeSeries and Scenarios in the database.

Parameters:
  • default (bool, optional) – Return only the default version of each TimeSeries/Scenario (see TimeSeries.set_as_default()). Any (model, scenario) without a default version is omitted. If False, return all versions.
  • model (str, optional) – A model name. If given, only return information for model.
  • scen (str, optional) – A scenario name. If given, only return information for scen.
Returns:

Scenario information, with the columns:

  • model, scenario, version, and scheme—Scenario identifiers; see Scenario.
  • is_default—True if the version is the default version for the (model, scenario).
  • is_locked—True if the Scenario has been locked for use.
  • cre_user and cre_date—database user that created the Scenario, and creation time.
  • upd_user and upd_date—user and time for last modification of the Scenario.
  • lock_user and lock_date—user that locked the Scenario and lock time.
  • annotation: description of the Scenario or changelog.
Return type:

pandas.DataFrame
set_log_level(level)

Set global logger level (for both Python and Java)

Parameters:level (str) – set the logger level if specified, see https://docs.python.org/3/library/logging.html#logging-levels
units()

Return all units described in the database.

Returns:
Return type:list
class ixmp.TimeSeries(mp, model, scenario, version=None, annotation=None)

Generic collection of data in time series format.

TimeSeries is the parent/super-class of Scenario.

A TimeSeries is uniquely identified on its Platform by three values:

  1. model: the name of a model used to perform calculations between input and output data.

  2. scenario: the name of a specific, coherent description of the real- world system being modeled. Any model may be used to represent mutiple alternate, or ‘counter-factual’, scenarios.

  3. version: an integer identifying a specific iteration of a (model, scenario). A new version is created by:

    • Instantiating a new TimeSeries with the same model and scenario as an existing TimeSeries.
    • Calling Scenario.clone().

    Optionally, one version may be set as a default version. See set_as_default().

Parameters:
  • mp (Platform) – ixmp instance in which to store data.
  • model (str) – Model name.
  • scenario (str) – Scenario name.
  • version (int or str, optional) – If omitted and a default version of the (model, scenario) has been designated (see set_as_default()), load that version. If int, load a specific version. If 'new', create a new TimeSeries.
  • annotation (str, optional) – A short annotation/comment used when version='new'.
add_timeseries(df, meta=False)

Add data to the TimeSeries.

Parameters:
  • df (pandas.DataFrame) –

    Data to add. df must have the following columns:

    • region or node
    • variable
    • unit

    Additional column names may be either of:

    • year and value—long, or ‘tabular’, format.
    • one or more specific years—wide, or ‘IAMC’ format.
  • meta (bool, optional) – If True, store df as metadata. Metadata is treated specially when Scenario.clone() is called for Scenarios created with scheme='MESSAGE'.
check_out(timeseries_only=False)

check out from the ixmp database instance for making changes

commit(comment)

Commit all changed data to the database.

version is not incremented.

discard_changes()

Discard all changes and reload from the database.

is_default()

Return True if the version is the default version.

last_update()

get the timestamp of the last update/edit of this TimeSeries

preload_timeseries()

Preload timeseries data to in-memory cache. Useful for bulk updates.

remove_timeseries(df)

Remove timeseries data from the TimeSeries instance.

Parameters:df (pandas.DataFrame) –

Data to remove. df must have the following columns:

  • region or node
  • variable
  • unit
  • year
run_id()

get the run id of this TimeSeries

set_as_default()

Set the current version as the default.

timeseries(iamc=False, region=None, variable=None, level=None, unit=None, year=None, **kwargs)

Retrieve TimeSeries data.

Parameters:
  • iamc (bool, default: False) – Return data in wide/’IAMC’ format. If False, return data in long/’tabular’ format; see add_timeseries().
  • region (str or list of strings) – Regions to include in returned data.
  • variable (str or list of strings) – Variables to include in returned data.
  • unit (str or list of strings) – Units to include in returned data.
  • year (str, int or list of strings or integers) – Years to include in returned data.
Returns:

Specified data.
Return type:

pandas.DataFrame
version()

get the version number of this TimeSeries

class ixmp.Scenario(mp, model, scenario, version=None, scheme=None, annotation=None, cache=False)

Collection of model-related input and output data.

A Scenario is a TimeSeries associated with a particular model that can be run on the current Platform by calling solve(). The Scenario also stores the output, or ‘solution’ of a model run; this includes the ‘level’ and ‘marginal’ values of GAMS equations and variables.

Data in a Scenario are closely related to different types in the GAMS data model:

  • A set is a named collection of labels. See init_set(), add_set(), and set(). There are two types of sets:
    1. Sets that are lists of labels.
    2. Sets that are ‘indexed’ by one or more other set(s). For this type of set, each member is an ordered tuple of the labels in the index sets.
  • A scalar is a named, single, numerical value. See init_scalar(), change_scalar(), and scalar().
  • Parameters, variables, and equations are multi-dimensional arrays of values that are indexed by one or more sets (i.e. with dimension 1 or greater). The Scenario methods for handling these types are very similar; they mainly differ in how they are used within GAMS models registered with ixmp:
    • Parameters are generic data that can be defined before a model run. They may be altered by the model solution. See init_par(), remove_par(), par_list(), add_par(), and par().
    • Variables are calculated during or after a model run by GAMS code, so they cannot be modified by a Scenario. See init_var(), var_list(), and var().
    • Equations describe fundamental relationships between other types (parameters, variables, and scalars) in a model. They are defined in GAMS code, so cannot be modified by a Scenario. See init_equ(), equ_list(), and equ().
Parameters:
  • mp (Platform) – ixmp instance in which to store data.
  • model (str) – Model name; must be a registered model.
  • scenario (str) – Scenario name.
  • version (str or int or at.ac.iiasa.ixmp.objects.Scenario, optional) – If omitted, load the default version of the (model, scenario). If int, load the specified version. If 'new', initialize a new Scenario.
  • scheme (str, optional) – Use an explicit scheme for initializing a new scenario.
  • annotation (str, optional) – A short annotation/comment used when version='new'.
  • cache (bool, optional) – Store data in memory and return cached values instead of repeatedly querying the database.
add_par(name, key, val=None, unit=None, comment=None)

Set the values of a parameter.

Parameters:
  • name (str) – Name of the parameter.
  • key (str, list/range of strings/values, dictionary, dataframe) – element(s) to be added
  • val (values, list/range of values, optional) – element values (only used if ‘key’ is a string or list/range)
  • unit (str, list/range of strings, optional) – element units (only used if ‘key’ is a string or list/range)
  • comment (str or list/range of strings, optional) – comment (optional, only used if ‘key’ is a string or list/range)
add_set(name, key, comment=None)

Add elements to an existing set.

Parameters:
  • name (str) – Name of the set.
  • key (str or iterable of str or dict or pandas.DataFrame) – Element(s) to be added. If name exists, the elements are appended to existing elements.
  • comment (str or iterable of str, optional) – Comment describing the element(s). Only used if key is a string or list/range.
Raises:

jpype.JavaException – If the set name does not exist. init_set() must be called before add_set().
change_scalar(name, val, unit, comment=None)

Set the value and unit of a scalar.

Parameters:
  • name (str) – Name of the scalar.
  • val (number) – New value of the scalar.
  • unit (str) – New unit of the scalar.
  • comment (str, optional) – Description of the change.
clear_cache(name=None, ix_type=None)

clear the Python cache of item elements

Parameters:
  • name (str, optional) – item name (None clears entire Python cache)
  • ix_type (str, optional) – type of item (if provided, cache clearing is faster)
clone(model=None, scenario=None, annotation=None, keep_solution=True, first_model_year=None, platform=None, **kwargs)

Clone the current scenario and return the clone.

If the (model, scenario) given already exist on the Platform, the version for the cloned Scenario follows the last existing version. Otherwise, the version for the cloned Scenario is 1.

Note

clone() does not set or alter default versions. This means that a clone to new (model, scenario) names has no default version, and will not be returned by Platform.scenario_list() unless default=False is given.

Parameters:
  • model (str, optional) – New model name. If not given, use the existing model name.
  • scenario (str, optional) – New scenario name. If not given, use the existing scenario name.
  • annotation (str, optional) – Explanatory comment for the clone commit message to the database.
  • keep_solution (bool, optional) – If True, include all timeseries data and the solution (vars and equs) from the source scenario in the clone. If False, only include timeseries data marked meta=True (see TimeSeries.add_timeseries()).
  • first_model_year (int, optional) – If given, all timeseries data in the Scenario is omitted from the clone for years from first_model_year onwards. Timeseries data with the meta flag (see TimeSeries.add_timeseries()) are cloned for all years.
  • platform (Platform, optional) – Platform to clone to (default: current platform)
equ(name, filters=None, **kwargs)

return a dataframe of (filtered) elements for a specific equation

Parameters:
  • name (str) – name of the equation
  • filters (dict) – index names mapped list of index set elements
equ_list()

List all defined equations.

get_meta(name=None)

get scenario metadata

Parameters:name (str, optional) – metadata attribute name
has_equ(name)

check whether the scenario has an equation with that name

has_par(name)

check whether the scenario has a parameter with that name

has_set(name)

check whether the scenario has a set with that name

has_solution()

Return True if the Scenario has been solved.

If has_solution() == True, model solution data exists in the db.

has_var(name)

check whether the scenario has a variable with that name

idx_names(name)

return the list of index names for an item (set, par, var, equ)

Parameters:name (str) – name of the item
idx_sets(name)

Return the list of index sets for an item (set, par, var, equ)

Parameters:name (str) – name of the item
init_equ(name, idx_sets=None, idx_names=None)

Initialize a new equation.

Parameters:
  • name (str) – name of the item
  • idx_sets (list of str) – index set list
  • idx_names (list of str, optional) – index name list
init_par(name, idx_sets, idx_names=None)

Initialize a new parameter.

Parameters:
  • name (str) – Name of the parameter.
  • idx_sets (list of str) – Names of sets that index this parameter.
  • idx_names (list of str, optional) – Names of the dimensions indexed by idx_sets.
init_scalar(name, val, unit, comment=None)

Initialize a new scalar.

Parameters:
  • name (str) – Name of the scalar
  • val (number) – Initial value of the scalar.
  • unit (str) – Unit of the scalar.
  • comment (str, optional) – Description of the scalar.
init_set(name, idx_sets=None, idx_names=None)

Initialize a new set.

Parameters:
  • name (str) – Name of the set.
  • idx_sets (list of str, optional) – Names of other sets that index this set.
  • idx_names (list of str, optional) – Names of the dimensions indexed by idx_sets.
Raises:

jpype.JavaException – If the set (or another object with the same name) already exists.
init_var(name, idx_sets=None, idx_names=None)

initialize a new variable in the scenario

Parameters:
  • name (str) – name of the item
  • idx_sets (list of str) – index set list
  • idx_names (list of str, optional) – index name list
load_scenario_data()

Load all Scenario data into memory.

Raises:ValueError – If the Scenario was instantiated with cache=False.
par(name, filters=None, **kwargs)

return a dataframe of (filtered) elements for a specific parameter

Parameters:
  • name (str) – name of the parameter
  • filters (dict) – index names mapped list of index set elements
par_list()

List all defined parameters.

read_sol_from_gdx(path, filename, comment=None, var_list=None, equ_list=None, check_solution=True)

read solution from GAMS gdx and import it to the scenario

Parameters:
  • path (str) – path to the folder
  • filename (str) – name of the gdx file
  • comment (str) – comment to be added to the changelog
  • var_list (list of str) – variables (levels and marginals) to be imported from gdx
  • equ_list (list of str) – equations (levels and marginals) to be imported from gdx
  • check_solution (boolean, optional) – raise an error if GAMS did not solve to optimality (only applicable for a MESSAGE-scheme scenario)
remove_par(name, key=None)

Remove parameter values or an entire parameter.

Parameters:
  • name (str) – Name of the parameter.
  • key (dataframe or key list or concatenated string, optional) – elements to be removed
remove_set(name, key=None)

delete a set from the scenario or remove an element from a set (if key is specified)

Parameters:
  • name (str) – name of the set
  • key (dataframe or key list or concatenated string) – elements to be removed
remove_solution(first_model_year=None)

Remove the solution from the scenario

This function removes the solution (variables and equations) and timeseries data marked as meta=False from the scenario (see TimeSeries.add_timeseries()).

Parameters:first_model_year (int, optional) – If given, timeseries data marked as meta=False is removed only for years from first_model_year onwards.
Raises:ValueError – If Scenario has no solution or if first_model_year is not int.
scalar(name)

Return the value and unit of a scalar.

Parameters:name (str) – Name of the scalar.
Returns:{‘value’
Return type:value, ‘unit’: unit}
set(name, filters=None, **kwargs)

Return the (filtered) elements of a set.

Parameters:
  • name (str) – Name of the set.
  • filters (dict) – Mapping of dimension_name → elements, where dimension_name is one of the idx_names given when the set was initialized (see init_set()), and elements is an iterable of labels to include in the return value.
Returns:
Return type:

pandas.DataFrame
set_list()

List all defined sets.

set_meta(name, value)

set scenario metadata

Parameters:
  • name (str) – metadata attribute name
  • value (str or number or bool) – metadata attribute value
solve(model, case=None, model_file=None, in_file=None, out_file=None, solve_args=None, comment=None, var_list=None, equ_list=None, check_solution=True, callback=None, gams_args=['LogOption=4'], cb_kwargs={})

Solve the model and store output.

ixmp ‘solves’ a model using the following steps:

  1. Write all Scenario data to a GDX model input file.
  2. Run GAMS for the specified model to perform calculations.
  3. Read the model output, or ‘solution’, into the database.

If the optional argument callback is given, then additional steps are performed:

  1. Execute the callback with the Scenario as an argument. The Scenario has an iteration attribute that stores the number of times the underlying model has been solved (#2).
  2. If the callback returns False or similar, go to #1; otherwise exit.
Parameters:
  • model (str) – model (e.g., MESSAGE) or GAMS file name (excluding ‘.gms’)
  • case (str) – identifier of gdx file names, defaults to ‘model_name_scen_name’
  • model_file (str, optional) – path to GAMS file (including ‘.gms’ extension)
  • in_file (str, optional) – path to GAMS gdx input file (including ‘.gdx’ extension)
  • out_file (str, optional) – path to GAMS gdx output file (including ‘.gdx’ extension)
  • solve_args (str, optional) – arguments to be passed to GAMS (input/output file names, etc.)
  • comment (str, optional) – additional comment added to changelog when importing the solution
  • var_list (list of str, optional) – variables to be imported from the solution
  • equ_list (list of str, optional) – equations to be imported from the solution
  • check_solution (boolean, optional) – flag whether a non-optimal solution raises an exception (only applies to MESSAGE runs)
  • callback (callable, optional) – Method to execute arbitrary non-model code. Must accept a single argument, the Scenario. Must return a non-False value to indicate convergence.
  • gams_args (list of str, optional) –

    Additional arguments for the CLI call to GAMS. See, e.g., https://www.gams.com/latest/docs/UG_GamsCall.html#UG_GamsCall_ListOfCommandLineParameters

    • LogOption=4 prints output to stdout (not console) and the log file.
  • cb_kwargs (dict, optional) – Keyword arguments to pass to callback.
Warns:

UserWarning – If callback is given and returns None. This may indicate that the user has forgotten a return statement, in which case the iteration will continue indefinitely.
Raises:

ValueError – If the Scenario has already been solved.
to_gdx(path, filename, include_var_equ=False)

export the scenario data to GAMS gdx

Parameters:
  • path (str) – path to the folder
  • filename (str) – name of the gdx file
  • include_var_equ (boolean, optional) – indicator whether to include variables/equations in gdx
var(name, filters=None, **kwargs)

return a dataframe of (filtered) elements for a specific variable

Parameters:
  • name (str) – name of the variable
  • filters (dict) – index names mapped list of index set elements
var_list()

List all defined variables.

years_active(node, tec, yr_vtg)

return a list of years in which a technology of certain vintage at a specific node can be active

Parameters:
  • node (str) – node name
  • tec (str) – name of the technology
  • yr_vtg (str) – vintage year

Utility methods

class ixmp.config.Config(read=True)

Configuration for ixmp.

When imported, ixmp reads a configuration file config.json in the first of the following directories:

  1. IXMP_DATA, if defined.
  2. ${XDG_DATA_HOME}/ixmp, if defined.
  3. $HOME/.local/share/ixmp.
  4. $HOME/.local/ixmp (used by ixmp <= 1.1).

The file may define either or both of the following configuration keys, in JSON format:

  • DB_CONFIG_PATH: location for database properties files. A ixmp.Platform instantiated with a relative path name for the dbprops argument will locate the file first in the current working directory, then in DB_CONFIG_PATH, then in the four directories above.
  • DEFAULT_DBPROPS_FILE: path to a default database properties file. A ixmp.Platform instantiated with no arguments will use this file.
  • DEFAULT_LOCAL_DB_PATH: path to a directory where a local directory should be created. A ixmp.Platform instantiated with dbtype=’HSQLDB’ will create or reuse a database in this path.
Parameters:read (bool) – Read config.json on startup.
clear()

Clear all configuration keys by setting their values to None.

find_dbprops(fname)

Return the absolute path to a database properties file.

Searches for a file named fname, first in the current working directory (.), then in the ixmp default location.

Parameters:fname (str) – Name of a database properties file to locate.
Returns:Absolute path to fname.
Return type:str
Raises:FileNotFoundError – fname is not found in any of the search paths.
get(key)

Return the value of a configuration key.

read()

Try to read configuration keys from file.

If successful, the configuration key ‘CONFIG_PATH’ is set to the path of the file.

save()

Write configuration keys to file.

config.json is created in the first of the ixmp configuration directories that exists. Only non-null values are written.

set(key, value)

Set configuration key to value.

ixmp.model_settings.register_model(name, config)

Register a new model with ixmp.

Parameters:
  • name (str) – Model name.
  • config (dict) –

    Configuration settings for the model, with the following keys. Each key is a format string that may use certain named values:

    • model_file (str): GAMS source file (.gms) containing the model. E.g. “{model}_run.gms”. Available values: model, case.
    • inp (str): input path; location where the model expects a GDX file containing input data. E.g. “{model}_{case}_in.gdx”. Available values: model, case.
    • outp (str): output path; location where the model will create a GDX file containing output data. E.g. “{model}_{case}_out.gdx”. Available values: model, case.
    • args (list of str): additional GAMS command-line args to be passed when invoking the model. Available values: model, case, inp, outp.

    The model and case formatting values are generated from ixmp.Scenario.model and ixmp.Scenario.scenario, respectively, with spaces (“ ”) converted to underscores (“_”).

Utilities for testing ixmp.

These include:

  • pytest hooks, and 3 fixtures: tmp_env, test_mp, and test_mp_props.
  • Methods for setting up and populating test ixmp databases: create_local_testdb() and dantzig_transport().
  • Methods to run and retrieve values from Jupyter notebooks: run_notebook() and get_cell_output().
ixmp.testing.make_dantzig(mp, solve=False)

Return ixmp.Scenario of Dantzig’s canning/transport problem.

Parameters:
  • mp (ixmp.Platform) – Platform on which to create the scenario.
  • solve (bool or os.PathLike) – If not False, then solve is interpreted as a path to a directory, and the model transport_ixmp.gms in the directory is run for the scenario.