Page Comparison

...

...

List of ETL+ Process Logged

• Load NowImage Added – process initiated by the Load NowImage Added button on the Target Object panel of ETL+ Extract, Transform and Load (ETL) Page

...

• .

• Refresh Image Added (refresh data source attributes) – on Source Objects panel of ETL+ Extract, Transform and Load (ETL) Page

• Job Page

...

• Image Added – Run JobImage Added, Jobs run by a scheduler (see Job Properties Image Added panel)

How to Access Logs

• Log PageImage Added displays the ETL Log and Table Stats logs

• Logs saved in Files – these log files should be available on the computer running ETL+.

• Logs saved in ETL+ Data Warehouse/SQL.

Anchor
logPage logPage

ETL+ Log Page

On the ETL+ Extract, Transform and Load (ETL) Page click ETL+ Extract, Transform and Load (ETL) Page Logto open this page.

...

1. ETL+ Log Name. ETL Log or Table Stats

2. Rows Retrieved. Limits the number of log rows displayed.

3. Refresh. Reloads the log rows. (F5)

4. Log Type Selection: Select ETL Log or Table Stats.

5. Close. Go back to the prior ETL+ page.

6. Help. Click to open documentation for this ETL+ page in a web browser (shortcut F1)

7. Version number: format: vYYYY.MM.DDBB.

8. Log Records.

   • Columns See ETL Log Records.

     • Date Time: DateTime stamp for each log line. By default, the logs is listed by this column in descending order (latest lines at the top). Click the header to sort it desc/ascending.

9. ETL+ User name and Entity for the current session.

...

• Click a column header to sort the log by the values in that column.

• See also https://dataself.atlassian.net/wiki/spaces/DS/pages/1570963835/ETL+Logging#Where-to-Find-Logs

  Anchor
  log_columns log_columns

...

ETL Log

...

Record Columns

...

• Index. Serially assigned value depending upon sort order.

• DateTime. Date-Time stamp for each log line. Default sort order.
  By default, the logs is listed by this column in descending order (latest lines at the top).
  Click the header to sort it desc/ascending.

• Steps. Log description.

• SourceError

• RefreshID

• ResultType. Log record type.

  • 0 = Job start log. Shows the job #, if applicable, for the entity. (“Job <n>”)

  • 1 = A task ETL+ executed. e.g. (“<name of target table> (<n> rows loaded)”)

  • 2 =The start of a Refresh Batch. (“REFRESH START”)

  • 3 = Total Refresh Time of an individual or set of ETL+ tasks.

  • 4 = Start of a Upload Now task.

• RefreshRunID. Unique key serially assigned to each ETL+ job or task. The key groups together log records from a ETL+ action such as loading a single table or running a job.

• Entity_Key. Internal key corresponding to a ETL+ Entity.

• DeviceUserName. Concatenation of a server name

• MachineUserName Only available for admin users. Identifies the Windows username running ETL+.

Anchor
tableStats tableStats

Table Stats

...

• Index.

• DTBegin. Date-time stamp process began.

• TableName. Name of data warehouse table loaded (refreshed).

  • See ETL Objects panel in ETL+ Extract, Transform and Load (ETL) Page

• Seconds. Time in seconds to execute the process.

• RowsLoaded.

• LoadType. Load, Replace, Upsert, Append. See ETL+ Table Load Types

  • 

• RowsBefore. Number of rows in TableName at DTBegin (before processing).

• RowsDeleted. Number of rows in TableName deleted.

• RowsKept. Number of rows in TableName not updated.

• RowsUpdated. Number of rows in TableName upserted. (LoadType = Upsert)

  • Value = 0 when LoadType not Upsert.

• RowsAfter. Number of rows in TableName after process complete.

• RefreshRunID. Unique key serially assigned to each ETL+ job or task. The key identifies one or more ETL+ tasks executed as part of a ETL+ action such as loading a single table or running a job.

  Anchor
  configure configure

...

Log from Data Load and Running Scripts On-Screen Page

ETL+ logs on screen (see image) when executing the following tasks directly on the user interface:

• Table load –

• Data source load – Refresh on https://dataself.atlassian.net/wiki/spaces/DS/pages/1570013215/ETL+Extract+Transform+and+Load+ETL+Page#Source-Objects-Panel

• Executing a Refresh Batch

...

1. Object Group: Blank if loading a single table. <Source Alias> when loading all tables from a data source. <Refresh Batch Name> when executing a Refresh Batch.

2. Progress Counter: Lists the current object number, the total objects to process, and percent progress.

3. Object currently processing: The name of the object currently processing. For some objects, it might provide additional information such as number of rows loaded.

4. List of Objects Executed: Lists all objects that have been executed and their status.

5. Data Warehouse Tables: Shows tables and their status such as rows loaded/replaced/updated/appended.

6. Tableau Extracts and other Scripts: Shows their names and status.

...

Anchor
configuring configuring

Configuring Logging

See ETL+ Settings Page

...

Where Log Files are Stored

The logs are also saved on files and in ETL+'s data warehouses.

Cloud Log Tables

ETL+ logs its loads and script runs on ETL+ cloud metadata tables. This allows users to retrieve their log anywhere and anytime ETL+ is connected to its cloud metadata (see section above).

...

Data Warehouse Log Tables

ETL+ also saves logs records on the data warehouse tables listed below. This is
These records store the same information as presented shown on the ETL+ log Log page:

• zLog_ETL

• zLog_TableStats

Data Load and Running Scripts On-Screen Log

ETL+ logs on screen (see image) when executing the following tasks directly on the user interface:
- Table load
- Data source load
- Executing a Refresh Batch

...

1. Object Group: Blank if loading a single table. <Source Alias> when loading all tables from a data source. <Refresh Batch Name> when executing a Refresh Batch.

2. Progress Counter: Lists the current object number, the total objects to process, and percent progress.

3. Object currently processing: The name of the object currently processing. For some objects, it might provide additional information such as number of rows loaded.

4. List of Objects Executed: Lists all objects that have been executed and their status.

5. Data Warehouse Tables: Shows tables and their status such as rows loaded/replaced/updated/appended.

...

Related Pages

• ETL+ Log Records Details on the formatting and interpretation of ETL+ log records.

• ETL+ Settings Page Set the logging level.

• ETL+ Troubleshooting

...