ETL+ Extract, Transform and Load (ETL) Page

Owned by Cortlandt _
Jun 24, 2023
7 min read

Quick links

Quick links

10. ETL Source panel

11. ETL Objects panel

12. ETL SQL Statement panel

  1. Data Warehouse info: The target database/data warehouse that is loaded with data extracted from the ETL Sources and ETL Objects. Name of the data warehouse connected to this ETL+ session. The info shows as [<MS SQL Instance Name>].[<Database name>].

  2. Find. Click to open the search box. (shortcut: Ctrl + F)

  3. Log. Click to open the ETL Log page. (shortcut Ctrl + G)

  4. Help. Click to open documentation for this ETL+ page in a web browser (shortcut F1)

  5. Settings. Click to open the Settings page.

  6. Version number: format: vYYYY.MM.DDBB.

    • Example: v2022.08.2901 means the 1st build on 08/29/2022.

  7. ETL+ User name and Entity for the current session.

    • User: the ETL+ username (email address) that is currently logged to ETL+.

    • Entity: the Entity identifies the mappings shown on this page.

  8. Close. Click to close the current ETL+ session.
    To close without saving changes click Close and click Yes to the ‘Leave without saving?' prompt.

  9. Save. Click to save changes (shortcut Ctrl + S).

    • An asterisk indicates there are unsaved changes.

    • Create Table, Delete Table, Load Now and other functions automatically save changes.

  10. ETL Source panel.

  11. ETL Objects panel.

  12. ETL SQL Statement panel.

NOTES: Hover over and/or right-click icons and objects on ETL+ to view their functions and/or metadata.

ETL Source Panel

Names and maintains connections to data sources.

ETL Source panel of ETL+ Extract, Transform and Load (ETL) window.

  1. Add Table: Select a table under a data source and click Add Table.

  2. Add Data Source: Click to add a new data source. More at https://dataself.atlassian.net/wiki/spaces/E2/pages/1968541015#Adding-Data-Sources

  3. Delete Source.
    Select a data source and click to delete.

  4. Refresh. Select a data source and click to refresh it's list of tables. (F5)

  5. Properties. Select a data source and click to change the settings.
    Similar to Add Data Source. More at ETL+ Data Sources | Configuring a Data Source


data sources. Select a data source and click Properties to configure. (Ctrl + I)

Data Sources

Data sources are source systems (data sources) available to extract data.

zData Warehouse -- Default Data Warehouse

zData Warehouse is the default, destination data warehouse. Read and write-back data to transform data already in the data ware. See also zData Warehouse (Reimports) in the ETL Objects panel.

  • Used to created calculated fields, apply formulas, run SQL functions and make other data manipulations to the data warehouse.

How to Expand the Data Source in the ETL Source panel

To show the names of the tables in zData Warehouse for example:

  1. Click zData Warehouse.

  2. Click Refresh.

  3. Click the expand button next to zData Warehouse.

Related Pages: https://dataself.atlassian.net/wiki/spaces/E2/pages/1968540238, https://dataself.atlassian.net/wiki/spaces/E2/pages/1968541015 , https://dataself.atlassian.net/wiki/spaces/DS/pages/1816887297

ETL Objects Panel

Configures the mapping of ETL Sources to the data warehouse zData Warehouse (Reimports). Allows data warehouse table maintenance and data load. Lists the tables and columns mapped to the data warehouse grouped by data source.

x

x

x

x

x

x

x

  1. Create Table. Select to create a new data warehouse table from scratch. Name it and then create SQL for it in the ETL SQL Statement panel.

  2. Delete Table. Select a table then select Delete Table to delete it, select Yes to confirm. (Ctrl + Del)

  3. Duplicate Table. Select a table, then select Duplicate Table to duplicate it. (Ctrl + 2)

  4. Rename Table: Select a table, then click Rename Table to rename it. (F2)

  5. Design: Select a table and click Design to design the columns format and extra data manipulation.
    More at: https://dataself.atlassian.net/wiki/spaces/E2/pages/1968539374

  6. Preview Table: Select a table, then lick Preview Table to preview data in the Preview page. (Ctrl + P)
    The Preview page display a sample of its records from the table’s data source (ETL+ executes the top records from the ETL SQL Statement to preview).

  7. Load Now: Select a table and click Load Now to load its source data into the data warehouse. (Ctrl + L)
    (The corresponding SQL and Load type options in the ETL SQL Statement panel determines how and what data is loaded.)

  8. More Actions.

    • Show Columns

    • Load All Columns

    • Collapse All

    • Download to CSV

  9. Job. Opens the Job page. (Ctrl + J)
    More at: https://dataself.atlassian.net/wiki/spaces/E2/pages/1968540318

NOTES

  • Select a table to view its mapping on the ETL SQL Statement panel.

  • Expand/collapse the list of tables by clicking the +/- icon by the Source Group (header).

  • Double-click a table to enable expand/collapse.

Source Groups

A source group is defined as a list of data warehouse tables mapped to a data source in the ETL Source panel.

zData Warehouse (Reimports) is a pre-defined source group linked to the data warehouse. This source group lists mapped tables from the data warehouse re-imported into the data warehouse (for data manipulations).

Tables

Tables represent tables in the data warehouse.

Account, AccountClass, etc. through Sub are tables in OData v4.

Table Columns

Every table consists of one or more table columns.

Double-click a table or right-click it → Show Columns → +/- to expand/collapse its list of columns.

Types of Table Columns

  • PK (Primary Key) icon: Columns that are part of the table’s primary key.

  • Black Column icon: The data warehouse column format is the same as from the source.

  • Green Column Icon: The data warehouse column format has been edited in the Design page.

How to edit column data types and PK

ETL SQL Statement Panel

Edits the SQL code (SQL statements) that controls the mapping of the source data connections in the ETL Source panel to the data warehouse tables in ETL Objects panel.

Each SQL statement corresponds to a source group and table from the ETL Objects panel. The output of the SQL SELECT statement loads into the destination data warehouse displayed just below the page title. See Data Warehouse info.

  1. Display of the names of the corresponding source group and table on the ETL Objects panel.

  2. ETL SQL Statements. Editable text of SQL code/statement.

    • To view or edit SQL for a table: Select a table on the ETL Objects panel. The corresponding SQL statement should appear in the ETL SQL Statement panel.

    • Use the SQL Statement syntax of the data source. Look at the SQL syntax for MS SQL Server sources in MS SQL Server, Providex syntax for for the Providex ODBC, etc.

  3. Load Type: Controls how the data load process for the selected table.
    More at https://dataself.atlassian.net/wiki/spaces/E2/pages/1968538781.

  4. Confirm. Click Confirm to accept the changes.
    To reject the changes click on another table and click No to the Confirm changes? prompt.

    • A asterisk indicates that the SQL has been edited.

More about SQL and the ETL SQL Statement panel.

v2022.08

/ex