Installation and startup

The following section provides a step-by-step guide to the installation process.

Two options are available for running the HMC Toolbox for Data Mining on your local machine. Although the step-by-step-instructions are given for Linux operating systems, similar ways exist for installation on Windows and MacOS:

Option 1: Run the HMC Toolbox for Data Mining as a stand-alone application using docker compose.
Option 2: Run the HMC Toolbox for Data Mining as a stand-alone CLI-application using poetry.
Option 3: Joint deployment of the HMC Toolbox for Data Mining together with the HMC FAIR Data Dashboard using docker compose.

The installation instructions for these options can be found below.

Option 1: Run the HMC Toolbox for Data Mining as a stand-alone application using docker compose.

The HMC Toolbox for Data Mining can be deployed as a standalone product using docker images.

Install docker and docker compose

This installation requires docker and docker compose to be installed on your machine. To do so, please follow the official installation instructions for Docker (opens in a new tab) as well as for Docker Compose V2 (opens in a new tab).

Clone the source code from the GitLab repository

Clone the HMC Toolbox for Data Mining (opens in a new tab) from the public GitLab repository:

 git clone https://codebase.helmholtz.cloud/hmc/hmc-public/FAIR-dashboard/hmc-toolbox-for-data-mining.git

 git clone git@codebase.helmholtz.cloud:hmc/hmc-public/FAIR-dashboard/hmc-toolbox-for-data-mining.git

Configuration of environment variables

Change path to the new directory:

cd hmc-toolbox-for-data-mining

Create .env file and fill it as .env.example

touch .env

or copy the .env.example and then fill it with your desire variables

cp .env.example .env

Run the HMC Toolbox for Data Mining using docker compose

docker compose up

Please note: A prerequisite for using the HMC FAIR Data Dashboard is a local instance of the F-UJI Server (opens in a new tab) as well as a preconfigured MariaDB-database. With the instructions given in docker-compose.yml, both the database as well as an instance of the F-UJI Server should be launched in parallel to the HMC Toolbox for Data Mining.

Option 2: Run the HMC Toolbox for Data Mining as a stand-alone CLI-application using poetry.

Alternatively you can also use the HMC Toolbox for Data Mining without Docker. In this case please make sure, that you have installed all dependencies.

Install Poetry

Please install poetry (opens in a new tab) first.

Clone the source code from the GitLab repository

Clone the HMC Toolbox for Data Mining (opens in a new tab) from the public GitLab repository:

 git clone https://codebase.helmholtz.cloud/hmc/hmc-public/FAIR-dashboard/hmc-toolbox-for-data-mining.git

 git clone git@codebase.helmholtz.cloud:hmc/hmc-public/FAIR-dashboard/hmc-toolbox-for-data-mining.git

Change into the project directory

cd hmc-toolbox-for-data-mining

Install project dependencies using Poetry

poetry install

Activate the virtual environment:

poetry shell

Run the Toolbox as a CLI application

The HMC Toolbox for Data Mining can now be run as a CLI-application using the following command on the project directory:

hmc-toolbox run

For more detaile information on available commands:

hmc-toolbox --help

It is also possible to execute only parts of the functionalities.

For a developer setup, install additional dependencies:

To run the project in a developer setup, dependencies used for development (e.g., pre-commit or pytest) can also be installed in addition using

poetry install --with dev, test

(Optional) To deactivate the virtual environment when finished:

exit

Option 3: Joint deployment of the HMC Toolbox for Data Mining together with the HMC FAIR Data Dashboard using docker compose (Linux)

The following instructions address a joint deployment of the HMC Toolbox for Data Mining together with the HMC FAIR Data Dashboard using docker compose on Linux machines. This allows you to show and explore the data harvested by the HMC Toolbox for Data Mining in the HMC FAIR Data Dashboard. For this purpose, we create another public GitLab repository Production setup HMC Toolbox and HMC Dashboard (opens in a new tab).

Install docker and docker compose

The Production setup HMC Toolbox and HMC Dashboard comes with an easy-to-use Docker container. First you need to install Docker (opens in a new tab) as well as Docker Compose V2 (opens in a new tab).

Clone the source code from the GitLab repository

Clone the Production setup HMC Toolbox and HMC Dashboard (opens in a new tab) public GitLab repository, by

 git clone https://codebase.helmholtz.cloud/hmc/hmc-public/FAIR-dashboard/production-setup-hmc-toolbox-and-hmc-dashboard.git

 git clone git@codebase.helmholtz.cloud:hmc/hmc-public/FAIR-dashboard/production-setup-hmc-toolbox-and-hmc-dashboard.git

Configuration of environment variables

Change path to the new directory:

cd production-setup-hmc-toolbox-and-hmc-dashboard

Create .env file and fill it as .env.example

touch .env

or copy the .env.example and then fill it with your desire variables

cp .env.example .env

An example of the environment variable file can be found here (opens in a new tab).

Run the dashboard using docker compose

docker compose up

Please Note: Upon completion of the pull and create process of the corresponding docker containers, the HMC FAIR Data Dashboard will show you the emply graphs. This is because the database is emplyed in a first step. HMC Toolbox for Data Mining container will harvest the data, enrich it, score it and finally store it in the database.

After finishing the process of HMC Toolbox for Data Mining jobs, HMC FAIR Data Dashboard will have the data to show. (at this stage it is necessary to restart HMC FAIR Data Dashboard)

Description Overview