Data Warehouse Column Formatting: Double-click a data warehouse table to enable column formatting and primary key display mode (a + is enabled by the tables).
Data Warehouse Column Formatting: Click + / - by a table name to expand/collapse its column formatting.
Data Warehouse Column Formatting: 
icon for regular columns.
Data Warehouse Column Formatting: icon for transformed columns.
Data Warehouse Column Formatting: icon for primary key columns.
Data Warehouse Column Formatting: icon for primary key transformed columns.
Design: New page for column design: configure primary key columns, change column formats, apply column transformations.
Design - Column Transformations: Many sources don’t allow transformations during the extraction, or extraction transformations are limited and/or slow. The data is extracted and loaded into a temporary data warehouse table, and then transformed into to official table. These transformations happen in the data warehouse engine, therefore they are fast, efficient and provide a broad range of MS SQL transformation functions.
Design: Select a data warehouse table → Design to open the Design page.
Design: New page shows PK (primary key), Column Name, Source Data Type, Dw Data Type, Transform SQL Expression.
Design - PK: PK check boxes define a table’s primary key columns. Currently:
PKs are necessary to process Update delta refreshes.
No validation if the PK columns are the correct primary key.
No creation of the physical PKs in the data warehouse.
If physical PKs are needed, please create them directly in SQL after the first ETL+ import.
Design - PK: When importing a new MS SQL or Odata table that has PK in the source, ETL+ will automatically recognize and pre-configure its PK.
Design - PK: Check the PK check boxes to change/define the table’s primary key columns.
Design - PK: ETL+ creates the SQL PK columns set to Not Null.
Design - PK: For text-based sources like OData, varchar(max) columns are automatically changed to varchar(50). We recommend reviewing the varchar size.
Design - PK: ETL+ logs the following when MS SQL can’t create/recalculate a PK: "Primary Key failure. PK requires rows with unique values and no NULLs. Review the PK fields.". This means the selected PK columns are not the table’s correct primary key - review/fix the PK configuration.
Design - PK: When upgrading an ETL+ entity, Primary Index columns automatically become Primary Key columns. Oct version?
Design : To view the source’s Column format, click the Refresh icon on top of Source Data Type. Depending on your source, this can take time (even minutes from OData v4).
Design: To change a Column’s format, use its Dw Data Type dropdown menu, or double-click the dropdown and type in the new format.
Design: Column MS SQL transformations available on Transform MS SQL Expression. Ex.: LEFT(ColumnName,5).
Design: Click the column header for Column Name or Source Data Type to sort the list.
Source Properties: Renamed Dw Schema to Schema. This checkbox sets Alias as the schema for tables imported from this source.
Settings - PK: Ability to create data warehouse physical PKs. Reason: PKs can be taxing for data load and storage (CPU, RAM and disk space), but might dramatically improve query performance.
Settings - PK: New “Create PK” dropdown with options:
Update Only: Default option. Only creates physical data warehouse PKs for tables with Update delta feature.
Always: Creates physical data warehouse PKs for all tables with PKs.
Never: Doesn’t create physical data warehouse PKs.
ETL Page - Load Type: Load Type section is now colapsed.
ETL Page - Load Type: Selecting Replace or Update will expand its settings section.
ETL Page - Load Type: Hovering over the Load Type options shows its function.
ETL Page - Load Type: Replace All has been renamed to Load.
ETL Page - Load Type: Replace with Control has been renamed to Replace.
ETL Page - Load Type: Hover over Load: Full load. Delete & load.
ETL Page - Load Type: Hover over Replace: Delta Load. 1st Load. Ongoing: Replace filtered rows.
ETL Page - Load Type: Hover over Update: Delta Load. 1st Load. Ongoing: Add new, update modified rows.
ETL Page - Load Type: Hover over Append: Full load. 1st load. Ongoing: Append.
ETL Page - Replace: Filter column now is a drop down.
ETL Page - Replace: The filter expression is automatically set to Max(FilterColumn).
ETL Page - Replace: Users can customize the filter SQL pressions. For instance:
Integer constant: 1000
Date constant: ‘02/15/2021’ (use quotes around the MM/DD/YYYY date format)
SQL Expressions. Ex.: getdate()-30
ETL Page - Update: Renamed Primary Index to Primary Key.
ETL Page - Update: Primary Key lists the PK columns (from the Design page).
ETL Page - Update: To edit the PK columns, click the Edit icon by the Primary Key or the Design icon on the table.
ETL Page - Update: Use the dropdown to select the Column with Last Modified Value.
ETL Page - Update: The Update filter is automatically set to Max(LastModifiedValueColumn).
Load Engine - Load: The load happens on a temporary table to allow users to query the official table with minimum interruption. Once the load completes, the temporary table becomes the official one. The temporary table is named Schema.TableName_TEMP.
Load Engine - Load: The transition between temporary to official table happens quickly when there are no transformations performed (delete official, rename temporary to official, create <0_EntityID> when applicable).
Load Engine - Load: The transition between temporary to official table happens quickly when there are transformations performed (delete official, insert into official from temp table applying transformations).
Load Engine: Ability to apply transformations after the load (previously, only available during extraction when the source SQL syntax allowed).
Data Warehouse TablesPanel: All tables now show schema (including dbo). <Schema>.<TableName>.
Data Warehouse TablesPanel: Renaming a table only applies to its name, not the schema (schemas might be changed in Source’s Properties and/or Data Warehouse Settings).
Data Warehouse Source: The data warehouse automatically shows as a data source for additional data preparations and transformations.
Data Warehouse Source: This source provides access to all data warehouse tables and views for single-tenant data warehouses (DataSelf Analytics Pro and Enterprise Editions).
Data Warehouse Source: This source provides access to data warehouse tables and views where <SQL schema> = <EntityID> or <SQL schema> = <dbo> in multi-tenant data warehouses (DataSelf Analytics Advanced Edition).
Data Warehouse - Source: Source’s Properties disabled.
Data Warehouse - Source: Source 'sDelete disabled.
Data Warehouse - Source: Use Settings -> Data Warehouse for configuration.
Data Warehouse - Source: Source credentials are sync’ed up its data warehouse configuration.Data Warehouse - Center Panel: Tables imported from the data warehouse are automatically renamed to <TableName_DW>. Users can then rename them as needed.
Data Warehouse - Center Panel: Tables imported from the data warehouse keep the same SQL schema as from the original source. Ex.: Abc table imported from ERP source using alias as schema will be stored in <Schema>.<TableName>=<ERP>.<Abc>. Upon re-importing it from the data warehouse, this table will be automatically set to <ERP>.<Abc_DW>.
Data Warehouse - Center Panel: When importing a data warehouse table into itself (usually, a mistake), ETL+ will log: "Warning: Data warehouse source table imported into itself. This is usually a mistake. Fix it by renaming the table.". Oct version?
Sources: Addition to native extraction from cloud Microsoft Dynamics CRM. Alpha version.
Source Properties: Checkbox for 0_UnionID. For this version, this field will be populated with “001”.
Source Properties: Renamed Dw Schema to Schema. Sets Alias as the source tables' schema.
ETL Log icon now retrieves live cloud log.
Refresh Batch - WTS: Repeat Task checkbox unchecked by default.