Compare and Synchronize eTrust Directory 8.1 Databases within SSO 8.1

Document ID : KB000051341
Last Modified Date : 14/02/2018
Show Technical Document Details

Description:

This document was created to define the correct steps to compare CA Directory Databases that have differences between them. Our goal is to ensure you know the exact differences between each servers' Directory database to make informed and intelligent decisions before synching the other servers in the farm. It specifically speaks about the default PS database (ps-ldap) for SSO 8.1 within the embedded CA Directory 8.1, but its general concepts hold true for other MultiWrite Directory farms.

Solution:

IMPORTANT: Before you start... If you already know which server is out of sync, and can be sure the master servers' DB is indeed the most up to date, then you do NOT need to use this document. You can load the master servers' database export into the secondary server/s using the dxdumpdb and dxloaddb commands. See the SSO and/or Directory Documentation describing how to dump and load a database from one server to another with the dxdumpdb and dxloaddb commands.

  • SSO Implementation Guide - Chapter "Post Installation Configuration Options" OR "Migrating SSO Data Stores"

  • eTrust Directory Administrators Guide.

To compare and synchronize the eTrust Directory 8.1 Databases within SSO 8.1 you can take 2 different approaches for your environment.

STRICT: If you need to ensure your Directory datastores are 100% in sync, you will need to ensure there are no updates happening when you are exporting the current database information. This will require you stop the SSO and Directory services. If you believe this is the method you will need please start at step 1 below and expect to schedule the outage.

RELAXED: In many cases you are only looking to make sure there are no major differences in your databases. In this case you can accept that a few differences may happen in the time you take one export on the master server until the time you take the secondary server export. Try to dump the online databases at the exact same time to be as close as possible. Understand that updates can occur within a millisecond. If you are ok with this method you will not need to bring down the SSO and Directory services and can start with Step 3 below.

IF you need to determine the differences between the servers ps-ldap (PS) Directory database, you should use the procedures here. Then utilize the dxmodify command provided later in the document to only load the differences into the out of sync server/s.

The following environments were used for this document:

  • Windows 2003 SP2 - eTrust SSO 8.1.4289 Servers running eTrust Directory 8.1 (build 1278).

  • HPUX 11.23 - eTrust SSO 8.1.4281 Server running eTrust Directory 8.1 (build 1253).


For the purpose of this document we will be referring to the "master" files as being from the Primary/Master server database which we want the secondary server/database to synchronize with. This master servers database will be the master/good database. The "secondary" files we are referring to are for the secondary server. The only database we will be modifying is the secondary server/s database. Changes will be made to the secondary server's database, to match it to the master/primary server's database. If you do not know which server is the master you may need to run the "Determine" and "Review" steps in the Delta section later in the document with the Secondary and Master files in different order to find the correct changes needed in the environment.

The following instructions are based off the eTrust Directory 8.1 Documentation, practical knowledge, and testing for the best method.

  • eTrust Directory 8.1 Administrators Guide.

  • Practical knowledge from various CA groups.

Definitions

  • DSA - Directory System Agent.

  • LDIF - Lightweight Directory Interchange Format.

  • Delta - Difference.

  • LDAP - Lightweight Directory Access Protocol.

  • JRE - Java Runtime Environment.

Responsibilities

The Single Sign-On, database, or server administrator would be responsible for completing these procedures.

Export the eTrust Directory PS Database to LDIF format

To back up and export the eTrust Directory data, follow this procedure for all servers/databases.

NOTE: This will need to be scheduled in most cases as the SSO and Directory services will be unavailable during the time the services are down.

  1. Stop the "eTrust SSO Watchdog" and "eTrust SSO Server" services.

    • Windows: Stop in Windows "Services" panel

    • UNIX: (As root)

      • /opt/CA/eTrustSSO/Server/bin/PolicyServer -stop

  2. Shut down the eTrust Directory DSAs by typing the following from a command prompt.

    • Windows:

      • dxserver stop all

        OR

      • Stop in Windows "Services" panel

    • UNIX: (Change to the dsa user first)

      • su - dsa

      • dxserver stop all

  3. Go to a command line and change to the directory you want to create your backup file in. Then backup all the PS data by typing the following command which relates to your SSO version.

    • Windows

      • dxdumpdb -p "o=PS" -S PS_ ServerNameHere -f PS_Master_ ServerNameHere .ldif ps

    • UNIX: you will need to change to the dsa user first.

      • su - dsa

      • dxdumpdb -p "o=PS" -S PS_ ServerNameHere -f PS_Master_ ServerNameHere .ldif ps

    EXAMPLE:

    • dxdumpdb -p "o=PS" -S PS_SSOSVR1 -f PS_Backup_SSOSVR1.ldif ps

    You should now have the PS_Backup_ServerNameHere.ldif fi.ldifle which is an LDIF formatted backup of the PS eTrust Directory information.

    Notes:

    The PS_ServerNameHere is actually the name of your PS DSA. If you look in your Windows Services list you should see your PS DSA name listed there after the "eTrust Directory -" for the PS service. You can also type "dxserver status all" from a command line and you should see the PS DSA name listed there.

    The -f is to specify the filename to write the backup to. You can use the example above but replace the ServerNameHere with a name that identifies the Server the database is from. Also remember where you saved the file and what you named it. In my example below I named my files PS_Master_SSOSVR1.ldif for the Primary server and PS_Secondary_SSOSVR2.ldif for the Secondary server.

    TIP: It is a good idea to copy these database backup files to a safe location; preferably on a remote server or machine.

    Example:

    On the Primary server I ran the following command.

    dxdumpdb -p "o=PS" -S PS_SSOSVR1 -f PS_Master_SSOSVR1.ldif ps

    On the Secondary server I ran the following command.

    dxdumpdb -p "o=PS" -S PS_SSOSVR2 -f PS_Secondary_SSOSVR2.ldif ps

Sort the Records in the Database export LDIF files

  1. Copy the 2 files created (PS_Master_ServerNameHere.ldif and PS_Secondary_ServerNameHere.ldif) to a new folder on the secondary server. I named my new folder delta.

  2. Open a command prompt to the new folder location with the files above.

  3. Sort the records in the exported Database LDIF files by running the following command.

    FORMAT:
    ldifsort FileNameHere.ldif SortedFileNameHere.ldif

    Example:
    For the master file I ran the following command.
    ldifsort PS_Master_SSOSVR1.ldif PS_Master_SSOSVR1_sorted.ldif
    For the secondary file I ran the following command.
    ldifsort PS_Secondary_SSOSVR2.ldif PS_Secondary_SSOSVR2_sorted.ldif

Determine the Delta (difference) between the Master, and Secondary Databases.

  1. Open a command line to the location of the files you created above.

  2. Find the difference between the Master, and Secondary sorted database LDIF files with the ldifdelta command.
    Note: The order of the files in the command is very important to make sure you find the correct set of differences.

    FORMAT:
    ldifdelta -S PS_ServerNameHere OutOfSyncSortedFileHere CurrentSortedFileHere > DifferencesFileHere

    Example:^
    I ran the following command to determine the differences.
    ldifdelta -S PS_SSOSVR2 PS_Secondary_SSOSVR2_sorted.ldif PS_Master_SSOSVR1_sorted.ldif > PS_Delta.ldif
  • NOTE: The PS_Delta.ldif file contains the differences that will be loaded into the Secondary server/s to synchronize it with the Master.

  • MULTIPLE SERVERS: If you are running this process for multiple servers and trying to see if they are in sync, you may not really know who is the master. In this case pick a server which has seemed to be the most reliable and up consistently. Once you have picked the master you will need to do a ldifdelta with each other servers dump and see if there are any which are severely out of sync. If they are all severely out of sync it may be the master itself if the one out of sync and you should try the process using another servers export as the master to see which ones are actually out of sync. If needed you can review the delta output and see what changes it thinks needs to be made (add, delete, modify). If there is any confusion as to which one if correct have your HR team review the changes and approve any deletes or modify records against their records of employees being added or removed.

Review the Delta (differences) to see if a database sync is needed

  1. Open the Delta (PS_Delta.ldif) file and look at the commands within. You will see commands have different formats for different purposes like Modify, add, and delete. Below I will provide examples of each type to help you determine the changes which will be made if the file is loaded

    Modify command format (has the replace: line)
    EXAMPLE: (Change in password)

    dn: cn=Notepad,ou=Mike:cn,ou=ps-ldap,ou=LoginInfos,o=PS
    changetype: modify
    replace: eTssoCurrPwd
    eTssoCurrPwd: SSO_AC35B2A2695D90D

    Add command format (has changetype: line with add)
    EXAMPLE: (Add a user to the ps-ldap database)

    dn: cn=test2,o=PS
    changetype: add
    displayName: test2
    eTssoAuthnMethod: Method5
    eTssoIsPwdAutoGen: False
    eTssoIsPwdSync: False
    eTssoPwdInterval: 0
    oc: top
    oc: person
    oc: eTssoUser
    oc: inetOrgPerson
    oc: organizationalPerson
    sn: 1

    Delete command format (has the changetype: line of delete)
    EXAMPLE: (Delete a user and associated Applications/Passwords)

    dn: cn=test1,o=PS
    changetype: delete

    dn: cn=__SSO__,ou=test1:cn,ou=ps-ldap,ou=LoginInfos,o=PS
    changetype: delete

    dn: cn=Notepad,ou=test1:cn,ou=ps-ldap,ou=LoginInfos,o=PS
    changetype: delete

    dn: ou=test1:cn,ou=ps-ldap,ou=LoginInfos,o=PS
    changetype: delete

    Use these record definitions in the "Decide" step below to determine the changes which the Delta file will make if you proceed.

Decide if the Delta (differences) file needs to be loaded into the Secondary databases.

  • Based on the changes in the delta file you may need to verify the changes are good for the databases OR if you need to run the steps again but swap the Master and Secondary file positions in the "Determine the Delta" phase. Once you have run this both ways you can find out which set of changes is correct for your environment. This may require you check with Human Resources, Security, or other departments involved in the addition, change and removal of personnel and their access.
  • Once you have found the delta file that has the correct changes for your environment then you may proceed to the next step to cleanup the delta file (PS_Delta.ldif) to prepare for loading it into the other servers you want to sync up with the Master.

Modify the Delta file to remove the "-" to prepare for loading.

You will need to remove dashes "-" from the Delta file (PS_Delta.ldif) so that the changes will load properly into the other servers. You may use any text editor you like, as long as you ONLY replace the - when found on its own line. We do NOT want to remove dashes within usernames or other relevant data. I provided examples on how to make the change using VI (Text editor found on Unix machines) and Microsoft WordPad (Text editor on Windows Machines).

  1. First make a copy of the PS_Delta.ldif file called PS_Delta_load.ldif

  2. Modify the PS_Delta_load.ldif file as per the steps below.

    UNIX:
    For the VI on most Unix platforms the commands below should work, verify the command on your platform and version.

    • vi PS_Delta_load.ldif

    • hit Esc key

    • :1,$s/^-/

    • hit enter key (make sure the changes took effect)

    • hit Esc key

    • :x!

    • hit enter key

    Windows:
  • In Microsoft WordPad the change was made using the following steps.
    Find what: -
    Replace with:
    You MUST Check off "Match whole word only" box otherwise it will find user names with dashes in it.

Load the Differences into the Secondary Database to Synchronize with the Master Database.

  1. Make sure eTrust Directory services are running. Run the following from a command prompt to check the services.

    dxserver status

    1. If the services are not running start them with the following from the command prompt.

    dxserver start all

  2. Open a command line and navigate to the location of the Delta LDIF file (PS_Delta_load.ldif).

  3. Execute the command below to modify and synchronize the PS database.

    dxmodify -a -c -h localhost:13389 -D "cn=ldap-pers,o=PS" -w "UserPasswordHere" -f DifferencesFileHere
    The default port is 13389
    This default Directory_User is ldap-pers. Its full context is cn=ldap-pers,o=PS.
    The UserPasswordHere is what you entered for Directory_User during the installation of the Policy Server.
    The DifferencesFileHere is the file you created in the ldifdelta step above and then modified. It is referred to as delta_load.ldif in the below example.

    Note: Replace the PasswordHere with your ldap-pers personalities password in the below Example commands.
    dxmodify -a -c -h localhost:13389 -D "cn=ldap-pers,o=PS" -w "PasswordHere" -f delta_load.ldif

    EXAMPLE command and sample output of dxmodify on HP UNIX:

    [35] % dxmodify -c -h localhost -p 13389 -D "cn=ldap-admin,o=PS" -w "PasswordHere " -f PS_Delta_2.ldif

    deleting entry cn=test1,o=PS

    modifying entry cn=Notepad,ou=Mike:cn,ou=ps-ldap,ou=LoginInfos,o=PS

    deleting entry cn=__SSO__,ou=test1:cn,ou=ps-ldap,ou=LoginInfos,o=PS

    deleting entry cn=Notepad,ou=test1:cn,ou=ps-ldap,ou=LoginInfos,o=PS

    deleting entry ou=test1:cn,ou=ps-ldap,ou=LoginInfos,o=PS

    EXAMPLE command and sample output on Windows:

    C:\Documents and Settings\Administrator>dxmodify -c -h localhost -p 13389 -D "cn=ldap-admin,o=PS" -w " PasswordHere" -f PS_delta2.ldif

    adding new entry cn=ps-bgc,o=PS
    dap_add: [-14] Unknown attribute dn:<cn=ps-bgc,o=PS>

    deleting entry cn=ldap-admin2,o=PS

    deleting entry cn=AutoGenTest,ou=test 1:CN,ou=SSOUsers:OU,ou=ad-acme,ou=LoginInf os,o=PS

    deleting entry ou=test 1:CN,ou=SSOUsers:OU,ou=ad-acme,ou=LoginInfos,o=PS

    deleting entry ou=SSOUsers:OU,ou=ad-acme,ou=LoginInfos,o=PS

    modifying entry cn=__SSO__,ou=ps-bgc:cn,ou=ps-ldap,ou=LoginInfos,o=PS dap_modify: [-14] Unknown attribute dn:<cn=__SSO__,ou=ps-bgc:cn,ou=ps-ldap,ou=Lo ginInfos,o=PS>

    modifying entry cn=__SSO__,ou=ldap-admin:cn,ou=ps-ldap,ou=LoginInfos,o=PS dap_modify: [-14] Unknown attribute dn:<cn=__SSO__,ou=ldap-admin:cn,ou=ps-ldap,ou=LoginInfos,o=PS>

  4. Restart the eTrust Policy Server Service from Windows Services or type net start ssod from a command prompt.

Verify you can connect to the PS-LDAP Datastore with an LDAP browser to verify the PS Databases are running and synchronized

  1. Install and run open source JXplorer (or use an LDAP browser of your choice).
    Obtain the current version of JXplorer by going to www.JXplorer.org and downloading the appropriate install for your OS.

    You then need to install Java 2 JRE (Java Runtime Environment).
    It is recommended you download the current Java 2 JRE if possible.
    www.java.com
    After Java is installed you may then install JXplorer as per the steps provided on the JXplorer.org site.

  2. Open JXplorer and connect to your PS (ps-ldap) database.

    Start JXplorer.
    Click "File" in the top left hand cornet and then click "connect" on the dropdown menu.
    Please replace the variables below, with your values that represent your environment.

    Host: Your_Machine_Name_Here
    Protocol: LDAP V3
    Port: 13389

    Base DN:o=PS
    Level: User + Password
    User DN: cn=ldap-admin,o=PS
    Password: "The Admin Password which was entered when you installed SSO"
    EXAMPLE JXplorer connection settings:
    Below are my settings for JXplorer to connect to my SSO Servers ps-ldap database.

    Figure 1
  3. Verify that you can view your users, and that the database looks as expected on all servers which were synchronized.
    Check on the records that we expect to be updated (added or removed) in the Secondary database.

  4. Congratulations!, your Directory synchronization should now be complete.