How to arrive at an RSK recommended threshold with respect to UBP Model Score

Document ID : KB000111835
Last Modified Date : 23/08/2018
Show Technical Document Details
Introduction:
in CA Risk Authentication, User Behavior Profiling introduces a new modeling technique that learns individual user behavior pattern and allows customers to do a step-up authentication when their end user’s behavioral patterns deviate from their norm.
Background:
The User Behavior Profiling model populates the variable MODEL_SCORE.  In the rules configuration one compares this to a threshold to decide whether this event warrants authentication. 
Environment:
RiskFort Server
Instructions:
The model score forms a distribution when plotted as a histogram. Refer to an idealized plot below.  This plot is rating the normality or abnormality of the user’s actions as compared to their history.  Typically you will want to challenge the most abnormal behavior.


User-added image

The recommended method for defining this threshold is:

1. Run the model without defining a rule. Next refer to the value of the Model Score in the Transaction Summary Report. Then export a day's worth of report and look at the distribution of the scores.  Set the threshold such that 5% of the scores are above the threshold. 

One can use an Excel Spreadsheet for computing the threshold. Open the exported Transaction Summary Report via Excel. Sort the data on Model Score column. from highest score to the lowest. The transaction 5% down in the sorted Model Score column will be the the Model Score that is optimal for a 5% threshold. 

2. Create a Riskfort rule with score threshold from the analysis in #1. Prioritize the rule lower in the list of Riskfort rules. Generally all rules that are configured for hard policies such as blacklisting certain countries and Exception User Check should to begore the User Behavior Profiling Rule(s).

Note, one may wish to create two rules, as shown in the illustration below, one for when a end user is accessing from a device they has already been used successfully in the past, and one for a new device that has a more sensitive threshold.  The UBP behavior model excludes consideration of whether the device is associated with the user to give one this control.

User-added image







 
Additional Information:
None.