Service Desk Configuration fails with the following error "pdm_sql 4468 ERROR encrypt.cpp 136 unable to initialise encryption operation"

Document ID : KB000041327
Last Modified Date : 17/01/2019
Show Technical Document Details

While attempting to upgrade Service Desk Manager the following error is recorded in the configure.log file

pdm_sql 4468 ERROR encrypt.cpp 136 unable to initialise encryption operation

In addition the file capki_install.log located in the %temp% folder contains the following error:

"Unable to open installed files log file: C:\Program Files (x86)\CA\SC\eTPKI\CAPKI\Windows/x86/32\files.dat
CAPKI files seem to have been deleted without cleaning the Registry....Installation will not proceed.
Existing Version is higher than installing Version."

In the checkdb.log you may see this error:

sql_ckeckdb           368 ERROR        encrypt.cpp            520 etpki_lib_init return -1
sql_ckeckdb           368 ERROR        encrypt.cpp            139 unable to initialise encryption operation
sql_ckeckdb           368 ERROR        encrypt.cpp            260 unable to initialise encryption operation

SDM 12.x/14.1
Service Desk Manager 17.1
When higher version of SDM is installed (i.e. SDM 14.1) in the server previously and if this version was not uninstalled properly, i.e. leaving the CA directory, files and registry entries still on the server, it will prevent to complete a new installation of an early release (i.e. SDM 12.9) and therefore the configuration will fail.
1. Make a backup of your registry first.
2. Delete the CAPKI entries from the registry - make sure all are deleted (Search for CAPKI). Check this key - and delete it:
3. Delete the C:\Program Files (x86)\CA\SC\CAPKI folder.
4. On the SDM 1DVD go to the folder \casd.nt\etpki and Run the following from the command line:
setup.exe install caller=SDM verbose

Once this completes, check the capki_install.log to see whether it has been installed successfully. If it's all OK - you should be able to run pdm_configure successfully.

With the 17.1 install media, the capki that gets installed creates a registry key called "CAPKI5", instead of the default from earlier versions. In order to resolve this you can reimport the key that was deleted in step 2 after the install is complete, and then attempt the install again.
Additional Information:
Uninstall CA Service Desk Manager

Similar problem might also happen with webengine.exe. We can also use Microsoft Process Explorer to see what environment variables webengine.exe is using.  If it is using a variable called ETPKIHOME that it is picking up from the Windows environment variables, because another CA product was using it, then we can follow below approach too:
a)  Ensure that we obtain correct path for CAPKI5, example: c:/progra~2/CA/SC/CAPKI/CAPKI5 
b)  Define a Windows System variable (Right click on My PC -> Properties -> Advanced System Settings -> Environment Variables,  create a new System Variable called CAPKIHOME  with a value of c:/progra~2/CA/SC/CAPKI/CAPKI5 
NOTE:  in certain environments, it was noted that the presence of this environment variable was causing the problem.  Removal of the variable is needed in such a case.

c) Restart webengine or SDM and verify the crashes no longer happen.