Slow Identity Manager performance when there are large number of groups and nested groups

Document ID : KB000008251
Last Modified Date : 14/02/2018
Show Technical Document Details
Issue:

You have a large number of user groups (i.e. thousands of groups) and you experience extreme slowness in task processing of tasks like Modify My Profile, Change My Password, etc.

Cause:

Environments configured with large numbers of groups and nested groups required extensive processing by Identity Manager as it evaluates the directory policies. If, for example, there are a few dozen directory policies that need to be evaluated as part of the IDM security model, and this evaluation is triggered for the Modify My Profile task, the security model needs to determine  if the admin has adequate privileges to run the task – so different types of policies are evaluated to figure this out.

The directory policies comprise member policies that include directory attributes. One such member policy (out of the dozens) could be like this one:

[<AttributeExpression attribute="%GROUP_TYPE%" comparator="EQUALS" value="abc"/>, <AttributeExpression attribute="objectclass" comparator="EQUALS" value="myGroup"/>]

The search unit IDM will build based on this member policy is as follows:

Search from:

                DN: dc=dir,dc=answers,dc=mycorp,dc=com + children

                Group DN: uniqueIdentifier=005430870a38559dc9bcaea17776de0,ou=Groups,dc=abc,dc=dir,dc=answers,dc=mycorp,dc=com

                Group DN: uniqueIdentifier=0029004d8e13d8d7759da787617776de0,ou=Groups,dc=abc,dc=dir,dc=answers,dc=mycorp,dc=com...  etc.

I have omitted all the groups from this list, but you get the picture.

This is still fairly quick – the real time consumption is when IDM attempts to probe for nested groups within these groups – that takes the bulk of the time.

So in this environment, we can ask IDM to not search through the nested groups – We can do this by setting <GroupTypes type="NONE"/> instead of <GroupTypes type="ALL"/> in the directory xml.

Once We do that, the Modify My Profile task returns in about 2 or 3 seconds instead of the 2 mins+ with the original setting. However, the above directory.xml setting is global for all tasks and will prevent any nested group searches from occurring at all. The solution is to use the DisableNestedGroupEvaluation=TRUE property/value only on tasks that do not require nested group searches.

This can still leave long search times when nested group searches are required (and large numbers of groups are configured) therefore you sure ensure that you've also implemented the appropriate well-known group attributes in the directory.xml.

 

Resolution:

This is the property that needs to be set on a per task basis:
 
The property is DisableNestedGroupEvaluation.
 
It is case-sensitive like all task properties – if it’s not present then execution will proceed normally. It has to be set to “true” to take effect.

For each task that want to disable nested group searches, go to Roles and Tasks, Modify Admin Task, (select your task) and add the property/value  DisableNestedGroupEvaluation/true.

 

Additional performance improvements can be realized by implementing the well-known attributes %MEMBER_OF% and %NESTED_GROUP_MEMBERSHIP%. Refer to the documentation:

https://docops.ca.com/ca-identity-manager/12-6-8/EN/configuring/ldap-user-store-management/well-known-attributes-for-an-ldap-user-store 

 

Also,  it may benefit if the policy rules are more granular and return fewer numbers of groups. For example, a large number of groups returned by member policy rules of this type: 

[<AttributeExpression attribute="%GROUP_TYPE%" comparator="EQUALS" value="abc"/>, <AttributeExpression attribute="objectclass" comparator="EQUALS" value="myGroup"/>] 

Let’s assume the above expression returns 10,000 groups. 

If %GROUP_TYPE% had more group types, like abc1, abc2 to abc10, then the returned results could be 1000 instead of 10,000 which would speed things up tremendously. Wherever possible, dividing the data set (and therefore the associated policy expression) would help in better managing the processing overhead. This ahe last option as it may or may not be possible to break down the data further for your use case, but if it is possible then it would benefit greatly.