Share experiments

On this page, see how you can easily share and/or release publicly the experiments you created with the carputils framework. In particular, learn how to attach metadata to your experiment and how to make sure that anyone can run your experiment from their computers.

In order to do so, you can use the bundle feature of the carputils framework, which allows you to easily bundle a specific simulation in a structured data format. This feature also allows you to share your experiments easily by releasing them on the research data repository RADAR4KIT.

The bundle feature

The carputils framework now includes an option to create a self-contained bundle for a specific simulation, including all necessary configuration information and data to reproduce the simulation.

Prerequisites

The self-contained bundle of a simulation must contain all files required to run the simulation. As a consequence, in order to create a bundle from a simulation, you have to indicate to carputils which external files are required to run your experiment.

In order to do so, you have to modify your script (let's call it run.py) in the following way:

  • every path to a required external file has to be encapsulated in the carputils.tools.simfile_path() function.
  • In addition, if the file is a mesh file, the 'mesh' flag in the simfile_path() function has to be set to True.

The simfile_path() function allows carputils to find the files required for running your experiment. These files will then automatically be added to the bundle, and the path will be adapted so that your experiment can be run from within the bundle.

Here is a typical example of modifications that have to be done before creating a bundle from an experiment:

Original script

import os
from datetime import date
from carputils import tools

EXAMPLE_DIR = os.path.dirname(__file__)

[...]

@tools.carpexample(parser, jobID)
def run(args, job):

    #####################################
    #Perform the simulation
    #####################################
    cmd  = tools.carp_cmd(os.path.join(EXAMPLE_DIR, 'sim.par'))

    cmd += ['-stimulus[0].vtx_file', os.path.join(EXAMPLE_DIR, 'stim')]
    cmd += ['-meshname', os.path.join(EXAMPLE_DIR, 'mesh')]

[...]

Ready-to-bundle script

import os
from datetime import date
from carputils import tools

EXAMPLE_DIR = os.path.dirname(__file__)

[...]

@tools.carpexample(parser, jobID)
def run(args, job):

    #####################################
    #Perform the simulation
    #####################################
    cmd  = tools.carp_cmd(tools.simfile_path(os.path.join(EXAMPLE_DIR, 'sim.par')))

    cmd += ['-stimulus[0].vtx_file', tools.simfile_path(os.path.join(EXAMPLE_DIR, 'stim'))]
    cmd += ['-meshname', tools.simfile_path(os.path.join(EXAMPLE_DIR, 'mesh'), mesh=True)]

[...]

Creation of the bundle

Once the prequisites are satisfied, the bundle of the experiment can be created. In order to do so, launch the simulation you want to bundle (with all desired arguments) and add the --bundle option:

./run.py --your-arguments --bundle experiment_name

where experiment_name is the name that will be given to the bundle.

After the execution, an archive named experiment_name_bundle.zip will be created in your working directory. You can test the bundle by unzipping it and launching the experiment it contains, using the same command line as for the bundle creation (without the --bundle argument). The command line is also stored in the METADATA.yml file of the bundle in the field named experiment_command_line.

Including metadata

When a bundle is created, a metadata file is automatically generated and included. This file contains autogenerated metadata such as:

  • the information about the creator found in the carputils settings file (name, email, ORCID)
  • the name given to the bundle
  • the command line to use in order to reproduce the bundled experiment
  • the python version used to create the bundle

Yet, this metadata file only contains few information about your experiment, and you should consider providing a metadata file with extended information manually, before generating the bundle.

You can generate a metadata file from a template by using the generateMetadata script located in the bin directory of carputils. If this directory is in your path, you can open a shell and run:

cd /path/to/experiment
generateMetadata

where /path/to/experiment is the path to your experiment's directory. You can now open the file named METADATA.yml and fill it with the information related to your experiment. When creating the bundle, this file will automatically be included if it is located in the same directory as run.py.

Including supplementary material

One should be aware that the external files included in the bundle are only those that are actually being used when running the experiment with the specific set of command line options chosen by the user. For example, if some files are only used when the --visualize option is used, then a bundle created without using this option could not contain these files.

For this reason (among others), it is possible to add additional files to a bundle manually:

  • if there is a folder named external in the same directory as run.py, its content will be added to the bundle automatically.
  • if there is a folder named Mesh in the same directory as run.py, its content will be added to the bundle automatically.
  • If the folder carputils.settings.config.MESH_DIR/experiment_name/ exists, then its content will be added to the bundle automatically.

Including simulation outputs

If a bundle is created for archiving a simulation, it can be useful to provide the output of the simulation in the bundle.

It is possible to do so by using the --bundle-output option in addition to the --bundle option.

This option creates a specific folder named sim_outputs in the bundle, in which the following folders are copied after the experiment ran (paths are given relatively to the experiment directory):

  • meshes (where meshes created on the fly are usually stored)
  • $jobID (where the outputs of the simulation are located)

Upload your bundle in a git repository

Instead of creating a .zip archive of your experiment, you can also create a git repository of the bundle. An easy way to create (public or private) git repositories is to create an account on gitlab.com. In the rest of this section, username will refer to your Gitlab.com username and the examples will be given for Gitlab repositories.

To create a git repository of your bundle, you have to use the --push-bundle option, followed by the link to the git repository you want to create:

./run.py --bundle exp_name --push-bundle https://gitlab.com/username/my-project.git

Note: the git repository has to be non-existent or empty, otherwise the upload of your bundle will fail.

Note: On some platforms (such as Github), the git repository has to be created before using the --push-bundle option, and it has to be empty (in particular it shouldn't contain a README file).

When running this command, default LICENSE and NOTICE files will be added to the bundle, applying the same license as the openCARP framework to your experiment: Apache 2.0. If you want to use another license, replace these files by the corresponding ones. Please be careful that the license you are using is compatible with the Apache 2.0 license used for openCARP.

Release your experiment

We offer the possibility to release your experiments publicly on the research data repository RADAR4KIT. On this repository, your experiment will be released in accordance with the FAIR principles, described with metadata and given a persistent identifier (DOI).

Before asking for a release of your experiment, please check that you provide sufficient metadata. In the METADATA.yml file, you should provide at least:

  • a title for your experiment
  • a short description of your experiment
  • Information about at least one creator (the author of the bundle): given name, family name, orcid (optional but recommended)

Using a git repository (recommended)

In order to ask for a release of your experiment on RADAR4KIT, you have to use the --push-bundle option (in order to upload your experiment on a public git repository), in association with the --release-bundle option:

./run.py --bundle exp_name --push-bundle https://gitlab.com/username/my-project.git --release-bundle

After running this command, you will get a link starting with

mailto:?to=...

By clicking on this link or opening it with your web browser, you will open a pre-filled email in your email client. You just have to send this email in order to ask for a release of your experiment on RADAR4KIT.

Using a .zip bundle

If you don't want to use git, you can also ask for a release of your experiment on RADAR4KIT by writing an email to opencarp-experiment-curationāˆ‚lists.kit.edu .

In this email, please provide:

  • the bundle of your experiment, generated with the --bundle option.
  • your name and the institution you belong to.

Please note that without any further indication from your side, the experiment will be released under the same license as openCARP, Apache 2.0.

Further steps

The bundle feature is still evolving. If you have questions or suggestions, please use openCARP Q&A.

Subscribe to our newsletter and we will keep you posted.

© Copyright 2020 openCARP project    Supported by DFG and EuroHPC    Contact    Imprint and data protection