bluepymm.prepare_combos package

Submodules

bluepymm.prepare_combos.create_mm_sqlite module

Create sqlite database

bluepymm.prepare_combos.create_mm_sqlite.check_morphology_existence(morph_name, morph_type, morph_path)[source]

Check if a morphology exists based on its path.

Parameters:

  • morph_name – a string representing the name of the morphology. Used for makeing a pretty error string.

  • morph_type – a string representing the type of the morphology. Used for making a pretty error string.

  • morph_path – the path to the morphology file

Returns:

True if the morphology exists.

Raises:

ValueError – The morphology does not exist.

bluepymm.prepare_combos.create_mm_sqlite.create_exemplar_rows(final_dict, rep_fullmtype_morph_map, emodel_etype_map, emodels, emodel_dirs, rep_morph_dir, unrep_morph_dir, skip_repaired_exemplar=False)[source]

Create exemplar rows.

Parameters:

  • final_dict – final e-model map

  • rep_fullmtype_morph_map – pandas.DataFrame with morphology database

  • emodel_etype_map – e-model e-type map

  • emodel_dirs – a dict mapping e-models to prepared e-model directories

  • rep_morph_dir – directory with repaired morphologies

  • skip_repaired_exemplar – indicates whether repaired exemplar should be skipped. Default value is False.

Returns:

pandas.DataFrame with one row for each exemplar. Keys are ‘layer’, ‘fullmtype’, ‘mtype’, ‘msubtype’, ‘etype’, ‘morph_name’, ‘emodel’, ‘original_emodel’, ‘morph_dir’, ‘scores’, ‘opt_scores’, ‘exception’, ‘to_run’, ‘is_exemplar’, ‘is_repaired’, and ‘is_original’.

bluepymm.prepare_combos.create_mm_sqlite.create_mm_sqlite(output_filename, recipe_filename, morph_dir, rep_morph_dir, unrep_morph_dir, original_emodel_etype_map, final_dict, emodel_dirs, skip_repaired_exemplar=False)[source]

Create SQLite database with all possible me-combinations.

Parameters:

  • output_filename –

  • recipe_filename –

  • morph_dir – directory with morphology release, contains neuronDB.xml file

  • original_emodel_etype_map –

  • final_dict – e-model parameters

  • emodel_dirs – prepared e-model directories

  • skip_repaired_exemplar – indicates whether repaired exemplar should be skipped. Default value is False.

bluepymm.prepare_combos.create_mm_sqlite.create_mm_sqlite_circuitmvd3(output_filename, circuitmvd3_path, morph_dir, rep_morph_dir, unrep_morph_dir, original_emodel_etype_map, final_dict, emodel_dirs, skip_repaired_exemplar=False)[source]

Create SQLite database using circuit.mvd3.

Parameters:

  • output_filename –

  • circuitmvd3_filename –

  • morph_dir – directory with morphology release, contains neuronDB.xml file

  • original_emodel_etype_map –

  • final_dict – e-model parameters

  • emodel_dirs – prepared e-model directories

  • skip_repaired_exemplar – indicates whether repaired exemplar should be skipped. Default value is False.

bluepymm.prepare_combos.create_mm_sqlite.remove_morph_regex_failures(full_map)[source]

Remove all rows where morph_name doesn’t match morph_regex.

Parameters:

full_map – pandas.DataFrame with keys ‘morph_name’ and ‘morph_regex’

Returns:

The processed pandas.DataFrame, with all rows of the input table where ‘morph_name’ matched ‘morph_regex’. The column ‘morph_regex’ is removed.

bluepymm.prepare_combos.main module

Create database of possible me-combinations.

bluepymm.prepare_combos.main.add_parser(action)[source]

Add option parser

bluepymm.prepare_combos.main.prepare_combos(conf_filename, continu, n_processes=None)[source]

Prepare combos

bluepymm.prepare_combos.main.prepare_emodels(conf_dict, continu, scores_db_path, n_processes)[source]

Prepare emodels

bluepymm.prepare_combos.parse_files module

Create sqlite database

bluepymm.prepare_combos.parse_files.convert_emodel_etype_map(emodel_etype_map, fullmtypes, etypes)[source]

Resolve regular expressions in an e-model e-type map and convert the result to a pandas.DataFrame. In the absence of the key “etype”, “mtype”, or “morph_name” in the e-model e-type map, the regular expression “.*” is assumed.

Parameters:

  • emodel_etype_map – A dict mapping e-models to a dict with keys “mm_recipe” and “layer”. Optional additional keys are “etype”, “mtype”, and “morph_name”, which may contain regular expressions. In absence of these keys, the regular expression “.*” is assumed.

  • fullmtypes – A set of unique full m-types

  • etypes – A set of unique e-types

Returns:

A pandas.DataFrame with fields ‘emodel’, ‘layer’, ‘fullmtype’, ‘etype’, ‘morph_regex’, and ‘original_emodel’. Each row corresponds to a unique e-model description.

bluepymm.prepare_combos.parse_files.fullmatch(regex, string)[source]

Make sure string matches regex fully

bluepymm.prepare_combos.parse_files.read_circuitmvd3(circuitmvd3_path)[source]

Read data from circuit.mvd3

bluepymm.prepare_combos.parse_files.read_mm_recipe(recipe_filename)[source]

Read a BBP builder recipe and return a pandas.DataFrame with all possible (layer, m-type, e-type)-combinations.

Parameters:

recipe_filename (str) – filename of recipe (XML/YAML)

Returns:

A pandas.DataFrame with fields “layer”, “fullmtype”, and “etype”.

bluepymm.prepare_combos.parse_files.read_mm_recipe_xml(recipe_filename)[source]

Read a BBP builder recipe and return a pandas.DataFrame with all possible (layer, m-type, e-type)-combinations.

Parameters:

recipe_filename (str) – filename of recipe (XML)

Returns:

A pandas.DataFrame with fields “layer”, “fullmtype”, and “etype”.

bluepymm.prepare_combos.parse_files.read_mm_recipe_yaml(recipe_filename)[source]

Read a BBP builder recipe and return a pandas.DataFrame with all possible (layer, m-type, e-type)-combinations.

Parameters:

recipe_filename (str) – filename of recipe (YAML)

Returns:

A pandas.DataFrame with fields “layer”, “fullmtype”, and “etype”.

bluepymm.prepare_combos.parse_files.read_morph_records(morph_tree)[source]

Parse morphology tree and yield (name, fullmtype, mtype, msubtype, layer)-tuples.

Parameters:

morph_tree – xml.etree.ElementTree.ElementTree or xml.etree.ElementTree.Element

Yields:

(name, fullmtype, mtype, msubtype, layer)-tuples

bluepymm.prepare_combos.parse_files.read_mtype_morph_map(neurondb_filename)[source]

Read morphology database and return a pandas.DataFrame with all morphology records. :param neurondb_filename: filename of morphology database (XML) :type neurondb_filename: str

Returns:

A pandas.DataFrame with field “morph_name”, “fullmtype”, “mtype”, “submtype”, “layer”.

bluepymm.prepare_combos.parse_files.read_recipe_records(recipe_tree)[source]

Parse recipe tree and yield (layer, m-type, e-type)-tuples.

Parameters:

recipe_tree – xml.etree.ElementTree.ElementTree or xml.etree.ElementTree.Element

Yields:

(layer, m-type, e-type)-tuples

bluepymm.prepare_combos.parse_files.verify_no_zero_percentage(tree_element_list)[source]

Verify that none of the elements of a given list have a zero value for the field ‘percentage’.

Parameters:

tree_element_list (list of xml.etree.ElementTree) – list of tree elements with ‘percentage’ field

Returns:

True if no percentage of zero is found.

Raises:

ValueError – if a percentage of zero is found.

bluepymm.prepare_combos.prepare_emodel_dirs module

Python Model Management

bluepymm.prepare_combos.prepare_emodel_dirs.check_emodels_in_repo(conf_dict)[source]

Check whether input e-models are organized in branches of a repository.

Parameters:

conf_dict – A dict with either the key “emodels_repo” or the key “emodels_dir”.

Returns:

True if the input e-models are organized in separate branches of a git repository, false if the e-models are organized into separate subdirectories.

Raises:

ValueError – if both or none of the keys “emodels_repo” and “emodels_dir” are present.

TODO: replace “emodels_repo” and “emodels_dir” by “emodels_input_type” and

“emodels_path” or similar.

bluepymm.prepare_combos.prepare_emodel_dirs.convert_emodel_input(emodels_in_repo, conf_dict, continu)[source]

Convert e-model input to BluePyMM file structure and return path to that structure.

Parameters:

  • emodels_in_repo – True if the input e-models are organized in separate branches of a git repository, false if the e-models are organized into separate subdirectories.

  • conf_dict – A dict with e-model input configuration.

  • continu – True if this BluePyMM run builds on a previous run, False otherwise

Returns:

Path to BluePyMM file structure.

bluepymm.prepare_combos.prepare_emodel_dirs.create_and_write_hoc_file(emodel, emodel_dir, hoc_dir, emodel_params, template, morph_path=None, model_name=None)[source]

Create .hoc code for a given e-model based on code from ‘<emodel_dir>/setup’, e-model parameters and a given template, and write out the result to a file named <hoc_dir>/<model_name or emodel>.hoc.

Parameters:

  • emodel – e-model name

  • emodel_dir – the directory containing a module ‘setup’, which describes the e-model

  • hoc_dir – the directory to which the resulting .hoc file will be written out.

  • emodel_params – a dict with e-model parameters

  • template – template file used for the creation of the .hoc file

  • morph_path – path to morphology file, used to overwrite the original morphology of an e-model. Default is None.

  • model_name – used to name the .hoc file. If None, the e-model name is used. Default is None.

bluepymm.prepare_combos.prepare_emodel_dirs.get_emodel_dicts(emodels_dir, final_json_path, emodel_etype_map_path)[source]

Read and return detailed e-model information.

Parameters:

  • emodels_dir – Path to BluePyMM file structure.

  • final_json_path – Path to final e-model map, relative to emodels_dir.

  • emodel_etype_map_path – Path to e-model e-type map, relative to emodels_dir.

Returns:

  • final e-model map,

  • e-model e-type map,

  • name of directory containing final e-model map.

Return type:

(string, dict, dict)-tuple with

bluepymm.prepare_combos.prepare_emodel_dirs.prepare_emodel_dir(input_args)[source]

Clone e-model input and prepare the e-model directory.

Parameters:

input_args – 9-tuple - original_emodel(str): e-model name - emodel(str): e-model name - emodel_dict: dict with all e-model parameters - emodels_dir: directory with all e-models - opt_dir: directory with all opt e-models (TODO: clarify) - hoc_dir: absolute path to the directory to which the .hoc files will be written out - hoc_template: path to the jinja hoc template - emodels_in_repo: True if the input e-models are organized in separate branches of a git repository, false if the e-models are organized into separate subdirectories. - continu: True if this BluePyMM run builds on a previous run, False otherwise

Returns:

A dict mapping the e-model and the original e-model to the e-model dir

bluepymm.prepare_combos.prepare_emodel_dirs.prepare_emodel_dirs(final_dict, emodel_etype_map, emodels_dir, opt_dir, emodels_hoc_dir, emodels_in_repo, hoc_template, continu=False, n_processes=None)[source]

Prepare the directories for the emodels.

Parameters:

  • final_dict – final e-model map

  • emodel_etype_map – e-model e-type map

  • emodels_dir – absolute path to the directory with all e-models. This directory is created by this function if it does not exist yet.

  • opt_dir – directory with all opt e-models (TODO: clarify)

  • emodels_hoc_dir – absolute path to the directory to which the .hoc files will be written out. Created by this function if it does not exist yet.

  • emodels_in_repo – True if the input e-models are organized in separate branches of a git repository, false if the e-models are organized into separate subdirectories.

  • hoc_template – path to the jinja hoc template.

  • continu – True if this BluePyMM run builds on a previous run, False otherwise. Default is False.

  • n_processes – the integer number of processes. If None,

  • used. (all processes are going to be) –

Returns:

A dict mapping e-models to prepared e-model directories.

Module contents

Init prepare combos