niamoto.core.services package

Submodules

niamoto.core.services.exporter module

Service responsible for handling the data export process based on export.yml configuration.

It loads the export configuration, validates it using Pydantic models, loads the necessary exporter and widget plugins, retrieves transformed data, and orchestrates the execution of export plugins to generate output files.

class niamoto.core.services.exporter.ExporterService(db_path, config)

Bases: object

Manages the data export process based on export configuration.

Parameters:

  • db_path (str)

  • config (Config)

get_export_targets()

Get all export targets and their configurations.

Returns:

Dict mapping target names to their configuration info.

Return type:

Dict[str, Dict[str, Any]]

run_export(target_name=None, group_filter=None)

Executes the specified export target or all enabled targets.

Returns:

Dict mapping target names to their export results.

Parameters:

  • target_name (str | None)

  • group_filter (str | None)

Return type:

Dict[str, Dict[str, Any]]

niamoto.core.services.importer module

Generic import service using entity registry and typed configurations.

class niamoto.core.services.importer.ImporterService(db_path)

Bases: object

Service for importing entities using the generic import engine and registry.

Parameters:

db_path (str)

close()

Close database connections and dispose of engine.

Return type:

None

import_reference(name, config, reset_table=False)

Import a reference entity from generic configuration.

Parameters:

  • name (str) – Entity name

  • config (ReferenceEntityConfig) – Reference configuration

  • reset_table (bool) – If True, drop and recreate the table

Returns:

Status message with row count

Raises:
Return type:

str

import_dataset(name, config, reset_table=False)

Import a dataset entity from generic configuration.

Parameters:

  • name (str) – Entity name

  • config (DatasetEntityConfig) – Dataset configuration

  • reset_table (bool) – If True, drop and recreate the table

Returns:

Status message with row count

Raises:
Return type:

str

import_all(generic_config, reset_table=False)

Import all entities from a generic import configuration.

This imports in 3 phases: 1. Datasets (sources for derived references) 2. Derived references (depend on datasets) 3. Direct references (no dependencies)

Parameters:

  • generic_config (GenericImportConfig) – Generic import configuration

  • reset_table (bool) – If True, drop and recreate all tables

Returns:

Status message with import summary

Raises:
Return type:

str

niamoto.core.services.transformer module

Service for transforming data based on YAML configuration.

class niamoto.core.services.transformer.TransformerService(db_path, config, *, enable_cli_integration=None)

Bases: object

Service for transforming data based on YAML configuration.

Parameters:

  • db_path (str)

  • config (Config)

  • enable_cli_integration (bool | None)

classmethod for_preview(db, config_dir)

Lightweight factory for widget preview.

Properly initialises all fields and loads plugins via cascade, but reuses an existing Database connection and skips CLI setup.

Parameters:
Return type:

TransformerService

transform_single_widget(group_config, widget_name, group_id)

Transform a single widget for a given entity.

Replicates the per-widget logic from transform_data() so that both the full pipeline and the preview engine share one code path.

Parameters:

  • group_config (Dict[str, Any]) – Full group config dict from transform.yml.

  • widget_name (str) – Key in widgets_data.

  • group_id (Any) – The entity ID to process.

Returns:

Transformer result (typically a dict).

Raises:

DataTransformError – On missing widget, source or transform failure.

Return type:

Any

transform_data(group_by=None, csv_file=None, recreate_table=True, progress_callback=None)

Transform data according to the configuration.

Parameters:

  • group_by (str | None) – Optional filter by group

  • csv_file (str | None) – Optional CSV file to use instead of the database

  • recreate_table (bool) – Indicates whether to recreate the results table

  • progress_callback (Callable[[Dict[str, Any]], None] | None)

Returns:

Results of the transformation with metrics data

Return type:

Dict[str, Any]

Raises:
validate_configuration(config)

Validate transformation configuration.

Parameters:

config (Dict[str, Any]) – Configuration to validate

Raises:

ValidationError – If configuration is invalid

Return type:

None

Module contents