bluepymm.prepare_combos package¶
Submodules¶
bluepymm.prepare_combos.create_mm_sqlite module¶
Create sqlite database
- bluepymm.prepare_combos.create_mm_sqlite.check_morphology_existence(morph_name, morph_type, morph_path)[source]¶
Check if a morphology exists based on its path.
- Parameters:
morph_name – a string representing the name of the morphology. Used for makeing a pretty error string.
morph_type – a string representing the type of the morphology. Used for making a pretty error string.
morph_path – the path to the morphology file
- Returns:
True if the morphology exists.
- Raises:
ValueError – The morphology does not exist.
- bluepymm.prepare_combos.create_mm_sqlite.create_exemplar_rows(final_dict, rep_fullmtype_morph_map, emodel_etype_map, emodels, emodel_dirs, rep_morph_dir, unrep_morph_dir, skip_repaired_exemplar=False)[source]¶
Create exemplar rows.
- Parameters:
final_dict – final e-model map
rep_fullmtype_morph_map – pandas.DataFrame with morphology database
emodel_etype_map – e-model e-type map
emodel_dirs – a dict mapping e-models to prepared e-model directories
rep_morph_dir – directory with repaired morphologies
skip_repaired_exemplar – indicates whether repaired exemplar should be skipped. Default value is False.
- Returns:
pandas.DataFrame with one row for each exemplar. Keys are ‘layer’, ‘fullmtype’, ‘mtype’, ‘msubtype’, ‘etype’, ‘morph_name’, ‘emodel’, ‘original_emodel’, ‘morph_dir’, ‘scores’, ‘opt_scores’, ‘exception’, ‘to_run’, ‘is_exemplar’, ‘is_repaired’, and ‘is_original’.
- bluepymm.prepare_combos.create_mm_sqlite.create_mm_sqlite(output_filename, recipe_filename, morph_dir, rep_morph_dir, unrep_morph_dir, original_emodel_etype_map, final_dict, emodel_dirs, skip_repaired_exemplar=False)[source]¶
Create SQLite database with all possible me-combinations.
- Parameters:
output_filename –
recipe_filename –
morph_dir – directory with morphology release, contains neuronDB.xml file
original_emodel_etype_map –
final_dict – e-model parameters
emodel_dirs – prepared e-model directories
skip_repaired_exemplar – indicates whether repaired exemplar should be skipped. Default value is False.
- bluepymm.prepare_combos.create_mm_sqlite.create_mm_sqlite_circuitmvd3(output_filename, circuitmvd3_path, morph_dir, rep_morph_dir, unrep_morph_dir, original_emodel_etype_map, final_dict, emodel_dirs, skip_repaired_exemplar=False)[source]¶
Create SQLite database using circuit.mvd3.
- Parameters:
output_filename –
circuitmvd3_filename –
morph_dir – directory with morphology release, contains neuronDB.xml file
original_emodel_etype_map –
final_dict – e-model parameters
emodel_dirs – prepared e-model directories
skip_repaired_exemplar – indicates whether repaired exemplar should be skipped. Default value is False.
- bluepymm.prepare_combos.create_mm_sqlite.remove_morph_regex_failures(full_map)[source]¶
Remove all rows where morph_name doesn’t match morph_regex.
- Parameters:
full_map – pandas.DataFrame with keys ‘morph_name’ and ‘morph_regex’
- Returns:
The processed pandas.DataFrame, with all rows of the input table where ‘morph_name’ matched ‘morph_regex’. The column ‘morph_regex’ is removed.
bluepymm.prepare_combos.main module¶
bluepymm.prepare_combos.parse_files module¶
Create sqlite database
- bluepymm.prepare_combos.parse_files.convert_emodel_etype_map(emodel_etype_map, fullmtypes, etypes)[source]¶
Resolve regular expressions in an e-model e-type map and convert the result to a pandas.DataFrame. In the absence of the key “etype”, “mtype”, or “morph_name” in the e-model e-type map, the regular expression “.*” is assumed.
- Parameters:
emodel_etype_map – A dict mapping e-models to a dict with keys “mm_recipe” and “layer”. Optional additional keys are “etype”, “mtype”, and “morph_name”, which may contain regular expressions. In absence of these keys, the regular expression “.*” is assumed.
fullmtypes – A set of unique full m-types
etypes – A set of unique e-types
- Returns:
A pandas.DataFrame with fields ‘emodel’, ‘layer’, ‘fullmtype’, ‘etype’, ‘morph_regex’, and ‘original_emodel’. Each row corresponds to a unique e-model description.
- bluepymm.prepare_combos.parse_files.fullmatch(regex, string)[source]¶
Make sure string matches regex fully
- bluepymm.prepare_combos.parse_files.read_circuitmvd3(circuitmvd3_path)[source]¶
Read data from circuit.mvd3
- bluepymm.prepare_combos.parse_files.read_mm_recipe(recipe_filename)[source]¶
Read a BBP builder recipe and return a pandas.DataFrame with all possible (layer, m-type, e-type)-combinations.
- Parameters:
recipe_filename (str) – filename of recipe (XML/YAML)
- Returns:
A pandas.DataFrame with fields “layer”, “fullmtype”, and “etype”.
- bluepymm.prepare_combos.parse_files.read_mm_recipe_xml(recipe_filename)[source]¶
Read a BBP builder recipe and return a pandas.DataFrame with all possible (layer, m-type, e-type)-combinations.
- Parameters:
recipe_filename (str) – filename of recipe (XML)
- Returns:
A pandas.DataFrame with fields “layer”, “fullmtype”, and “etype”.
- bluepymm.prepare_combos.parse_files.read_mm_recipe_yaml(recipe_filename)[source]¶
Read a BBP builder recipe and return a pandas.DataFrame with all possible (layer, m-type, e-type)-combinations.
- Parameters:
recipe_filename (str) – filename of recipe (YAML)
- Returns:
A pandas.DataFrame with fields “layer”, “fullmtype”, and “etype”.
- bluepymm.prepare_combos.parse_files.read_morph_records(morph_tree)[source]¶
Parse morphology tree and yield (name, fullmtype, mtype, msubtype, layer)-tuples.
- Parameters:
morph_tree – xml.etree.ElementTree.ElementTree or xml.etree.ElementTree.Element
- Yields:
(name, fullmtype, mtype, msubtype, layer)-tuples
- bluepymm.prepare_combos.parse_files.read_mtype_morph_map(neurondb_filename)[source]¶
Read morphology database and return a pandas.DataFrame with all morphology records. :param neurondb_filename: filename of morphology database (XML) :type neurondb_filename: str
- Returns:
A pandas.DataFrame with field “morph_name”, “fullmtype”, “mtype”, “submtype”, “layer”.
- bluepymm.prepare_combos.parse_files.read_recipe_records(recipe_tree)[source]¶
Parse recipe tree and yield (layer, m-type, e-type)-tuples.
- Parameters:
recipe_tree – xml.etree.ElementTree.ElementTree or xml.etree.ElementTree.Element
- Yields:
(layer, m-type, e-type)-tuples
- bluepymm.prepare_combos.parse_files.verify_no_zero_percentage(tree_element_list)[source]¶
Verify that none of the elements of a given list have a zero value for the field ‘percentage’.
- Parameters:
tree_element_list (list of xml.etree.ElementTree) – list of tree elements with ‘percentage’ field
- Returns:
True if no percentage of zero is found.
- Raises:
ValueError – if a percentage of zero is found.
bluepymm.prepare_combos.prepare_emodel_dirs module¶
Python Model Management
- bluepymm.prepare_combos.prepare_emodel_dirs.check_emodels_in_repo(conf_dict)[source]¶
Check whether input e-models are organized in branches of a repository.
- Parameters:
conf_dict – A dict with either the key “emodels_repo” or the key “emodels_dir”.
- Returns:
True if the input e-models are organized in separate branches of a git repository, false if the e-models are organized into separate subdirectories.
- Raises:
ValueError – if both or none of the keys “emodels_repo” and “emodels_dir” are present.
- TODO: replace “emodels_repo” and “emodels_dir” by “emodels_input_type” and
“emodels_path” or similar.
- bluepymm.prepare_combos.prepare_emodel_dirs.convert_emodel_input(emodels_in_repo, conf_dict, continu)[source]¶
Convert e-model input to BluePyMM file structure and return path to that structure.
- Parameters:
emodels_in_repo – True if the input e-models are organized in separate branches of a git repository, false if the e-models are organized into separate subdirectories.
conf_dict – A dict with e-model input configuration.
continu – True if this BluePyMM run builds on a previous run, False otherwise
- Returns:
Path to BluePyMM file structure.
- bluepymm.prepare_combos.prepare_emodel_dirs.create_and_write_hoc_file(emodel, emodel_dir, hoc_dir, emodel_params, template, morph_path=None, model_name=None)[source]¶
Create .hoc code for a given e-model based on code from ‘<emodel_dir>/setup’, e-model parameters and a given template, and write out the result to a file named <hoc_dir>/<model_name or emodel>.hoc.
- Parameters:
emodel – e-model name
emodel_dir – the directory containing a module ‘setup’, which describes the e-model
hoc_dir – the directory to which the resulting .hoc file will be written out.
emodel_params – a dict with e-model parameters
template – template file used for the creation of the .hoc file
morph_path – path to morphology file, used to overwrite the original morphology of an e-model. Default is None.
model_name – used to name the .hoc file. If None, the e-model name is used. Default is None.
- bluepymm.prepare_combos.prepare_emodel_dirs.get_emodel_dicts(emodels_dir, final_json_path, emodel_etype_map_path)[source]¶
Read and return detailed e-model information.
- Parameters:
emodels_dir – Path to BluePyMM file structure.
final_json_path – Path to final e-model map, relative to emodels_dir.
emodel_etype_map_path – Path to e-model e-type map, relative to emodels_dir.
- Returns:
final e-model map,
e-model e-type map,
name of directory containing final e-model map.
- Return type:
(string, dict, dict)-tuple with
- bluepymm.prepare_combos.prepare_emodel_dirs.prepare_emodel_dir(input_args)[source]¶
Clone e-model input and prepare the e-model directory.
- Parameters:
input_args – 9-tuple - original_emodel(str): e-model name - emodel(str): e-model name - emodel_dict: dict with all e-model parameters - emodels_dir: directory with all e-models - opt_dir: directory with all opt e-models (TODO: clarify) - hoc_dir: absolute path to the directory to which the .hoc files will be written out - hoc_template: path to the jinja hoc template - emodels_in_repo: True if the input e-models are organized in separate branches of a git repository, false if the e-models are organized into separate subdirectories. - continu: True if this BluePyMM run builds on a previous run, False otherwise
- Returns:
A dict mapping the e-model and the original e-model to the e-model dir
- bluepymm.prepare_combos.prepare_emodel_dirs.prepare_emodel_dirs(final_dict, emodel_etype_map, emodels_dir, opt_dir, emodels_hoc_dir, emodels_in_repo, hoc_template, continu=False, n_processes=None)[source]¶
Prepare the directories for the emodels.
- Parameters:
final_dict – final e-model map
emodel_etype_map – e-model e-type map
emodels_dir – absolute path to the directory with all e-models. This directory is created by this function if it does not exist yet.
opt_dir – directory with all opt e-models (TODO: clarify)
emodels_hoc_dir – absolute path to the directory to which the .hoc files will be written out. Created by this function if it does not exist yet.
emodels_in_repo – True if the input e-models are organized in separate branches of a git repository, false if the e-models are organized into separate subdirectories.
hoc_template – path to the jinja hoc template.
continu – True if this BluePyMM run builds on a previous run, False otherwise. Default is False.
n_processes – the integer number of processes. If None,
used. (all processes are going to be) –
- Returns:
A dict mapping e-models to prepared e-model directories.
Module contents¶
Init prepare combos