Question / Problem:
After upgrading to KTM 6.3 database look up in KTM validation is very slow takes about 10 to 40 seconds to get results. Previous version of KTM used to be instant
Answer / Solution:
The root cause is a new algorithm implemented in KTM 6.3 for TXT Unicode files. The new algorithm is used when opening the database file to read the records returned to the DB lookup. The reason of this algorithm is to find the file encoding to read it correctly when this one do not have the BOM information set. The problem is that without BOM we need to find if the encoding is ASCII or UTF8, and for that we have to parse the file to find symbols that could be only Unicode. This cost of course time, and in addition we are doing it for each record so sometimes 100 times for a DB lookup. We will have to improve that part.
The good news is that there is an easy fix if you are using TXT files for database lookup. The only thing that needs to done is to convert the database source file which is currently UTF8 to UTF8 with BOM. This gives information about the encoding and the algorithm does not need to spend time anymore to find it.
This can be achieved for example with notepad++ text tool: Encoding tab -> Convert to UTF-8-BOM.
Once the TXT files are converted to UTF-8-BOM then the current databases needs to be replace in the source folder and then reimport the database in project builder.
Enhancement Request 1350574 was created to add this feature to future release of KTM