The default MaxObjects setting allowed by Siteminder/SSO is too low.
** Example: Suppose that a realm has 60 sub-realms as nested.
Increase the MaxObjects allowed in the Policy Server registry:
1. Open your registry, on Windows type Start > type regedit.exe in the search field and click Run, or on unix OS's open the "<policy server install>/registry/sm.registry" file
2. Navigate to: HKEY_LOCAL_MACHINE\SOFTWARE\Netegrity\Siteminder\CurrentVersion\ObjectStore
3. Modify MaxObjects to a higher value than it is currently set to, depending on your release... such as 0x64 (100 in decimal) or 0x96 (150 in decimal). The actual number is not as important as making it larger than it currently is set to.
4. Restart the policy server, and try connecting again with the Agent for SharePoint connection wizard.
** Increase this value as needed but 0x64 should be enough to resolve this issue