This article describes the principal changes of Generic Templates.
The Template download section can be found at this page
Note:
Stambia DI is a flexible and agile solution. It can be quickly adapted to your needs.
If you have any question, any feature request or any issue, do not hesitate to contact us.
templates.generic.2019-08-09
LOAD Xml to Rdbms
New parameter 'Use Transaction On Work Tables DDL'
A new parameter "Use Transaction On Work Tables DDL" has been added to define if the execution of Drop and Create table SQL DDL statements performed by the Template should be done in a transaction.
This can be useful to perform everything in a transaction, drop and create table included, when working with databases which allows such statements to be executed in transaction.
Note that some databases perform an implicit commit on DDL statements. In this case it is necessary to let this option to false to avoid committing the transaction to early in the process.
Some other databases handle the DDL in a transaction. In this case it is necessary to set this option to true to make sure that the tables are visible on the insert statements.
Refer to parameter's documentation for further information.
Drop and create table statements performed with "if exists" clause when applicable.
All drop and create table statements performed by the Template will now be performed with the "if exists" clause when the database supports it, to avoid logging errors when it does not exist.
When used with Netezza as target, set catalog clause is added
When using the Template to load XML data into Netezza, a "set catalog" statement is performed before drop and create table ones.
This is required when dropping or creating tables from another database than the one specified in the JDBC URL.
Ability to disable creation and drop of tables
An internal mechanism has been introduced to be able to re-use this Template more easily.
This is used for instance behind the scenes by the new "Load XML to Netezza" Template.
templates.generic.2019-07-11
REPLICATOR Rdbms
Support loading data when replicating to Google BigQuery
The replicator now supports loading data when replicating into a target BigQuery datastore.
REPLICATOR Xml to RDBMS
Primary key columns now created with NOT NULL option
When replicating source XML files into a database through the XML Replicator, tables are created on the target database when the "Create Tables" option is enabled.
The primary key columns were created as nullable, which could cause issue in some databases.
Primary key columns are now created with the non-nullable keyword, such as "NOT NULL" for most databases.
templates.generic.2019-06-11
REPLICATOR Rdbms
Fix several issues for Oracle SQL*Loader data loading
Two issues happening when loading data into Oracle through SQL*Loader have been fixed.
There were some cases where generated CTL file was not containing the defining of columns, which was causing issues, and which is now fixed.
There was also an issue with the record delimiter used to export source data into files which could be different than the one used by SQL*Loader which could also lead to issue and bad data loading, and which has also been fixed.
New truncate table parameter
A new additional parameter for removing data on target table before the replication has been added: 'Truncate Tables'.
Refer to parameter's documentation for further information.
templates.generic.2019-04-17
REPLICATOR Rdbms
Snowflake loading updated
Snowflake specific loading has been updated to take into account the changes which have been implemented on Snowflake Connector from templates.snowflake.2019-04-17.
Data loading will now be more efficient and match connector updates.
Generated target table name was incorrect for Microsoft SQL Server when using prefix parameter
Replication on a target Microsoft SQL Server database using "Target Object Prefix" parameter was not working properly.
Specified prefix was not included into the delimiters which surround generated target table but outside, which was causing issues.
This has been fixed, prefix is now placed properly in generated table name.
Replicator Xml to Rdbms
Ability to use technology specific loaders
XML Replicator now supports using technology specific loaders.
A new parameter named "Load Method" has been added to enable specific loading.
Refer to parameter documentation for further information.
templates.generic.2019-02-08
INTEGRATION Rdbms and INTEGRATION Hsql
Recycling of previous rejects fixed
When using the option to recycle the rejects of previous execution an extra step is executed to add those previous rejects in the integration flow.
Possible duplicates while retrieving those rejects are now filtered using DISTINCT keyword.
templates.generic.2019-01-29
Unload Rdbms to File
Template has been updated to use automatically technology specific syntax when generating empty NULL columns.
This has been modified to address issues with technologies which requires a different syntax than using "NULL" keyword, such as IBM Informix database which requires the column to be dynamically datatyped using "NULL:integer" for instance.
Prerequisites:
Stambia DI Designer S19.0.15 or higher is required for the Template to use technology specific syntax for this case.
Notes:
When using prior Designer versions, or a technology on which the syntax has not been defined, this will continue to work as before, by using the 'NULL' keyword
templates.generic.2019-01-15
TOOL Ldap Integrator
Operation failure management
Each row of source file is now processed even if an issue occurs while executing one of the iterations.
Previously, as soon as one of the operations was failing, the tool stopped executing and returned the error, avoiding all other iterations to be processed.
From now, all rows will be processed and when an iteration fails it will simply log information about it if required and process the next one.
New parameter for exporting encountered errors in file
A new parameter called 'Export Errors' has been added.
When enabled encountered errors will be traced in a log file.
New statistics
New variables are now published by tool, allowing to consult easily the result of operations: how many succeeded and how many failed.
They are all listed in the tool's description.
Allow having multiple instance of the tool in same process
All scripting actions contained in the tool have been updated to have a dynamically generated scripting connection.
This allows using multiple times the tools in the same process and also in parallel.
templates.generic.2018-12-19
REPLICATOR Rdbms
The Replicator now supports specific loaders for SybaseIQ and Snowflake for data loading.
They will now be used by default for these technologies instead of the generic mode.
If required, generic mode can still be used by enabling 'Force Generic Rdbms Mode'.
templates.generic.2018-12-05
REPLICATOR Rdbms
A new parameter named 'Split By Pattern' has been added to offer the possibility to customize the xpath expression used to find the column which is used to split source data into multiple subsets when using the split by mechanism.
As a reminder this mechanism can be used to replicate source table data into target with multiple parallel tasks.
Each task contains a number of rows calculated from the numeric column which is used to split data.
templates.generic.2018-10-22
INTEGRATION Rdbms to Rdbms (no load)
The 'INTEGRATION Rdbms to Rdbms (no load)', which has the specificity to integrates source data directly in the target without creating any temporary work table, is now available in more situations.
It was previously displayed in Mappings when integrating data on Excel and Google Spreadsheet as they are technologies which requires not using temporary objects and has now been expended to more cases.
It will now be available when integrating data in a target database as soon as the source is also a database, offering the possibility to perform direct integration in more situations.
Note that this applies only for technologies working through JDBC like databases, or files (which uses or own JDBC driver).
templates.generic.2018-10-08
REJECT Hsql
- Generated queries for Primary Keys and Alternate Keys were incorrect for composite keys
TOOL Ldap Extractor (search)
- Fix undefined variable error which is thrown when using the tool