Before proceeding, you must ensure that your ADFS(STS) and your O365 is integrated successfully and your enterprise users are able to login via browser.
Then you can configure the following in the PAM GUI.
Security Token Service (STS) Endpoint URL
This is your ADFS server in your enterprise.
STS is a component in ADFS that you would have configured.
The https://<FQHN>/adfs/services/trust/2005/usernamemixed is one of the applications already deployed on ADFS (without involving PAM) and this is where the O365 exchange some information with STS.
If your STS is hosted on https://dummy.com then you should enter https://dummy.com/adfs/services/trust/2005/usernamemixed as the "Security Token Service (STS) Endpoint URL.
It is hardcoded and you just need to replace the <FQHN> with whatever hostname you are using for the STS.
Security Token Service (STS) Endpoint Reference URI
This would have been more clearer if it just said, "Enter O365 EntityID"
This is Microsoft decided EntityID value to be used for O365 so you just need to enter "urn:federation:MicrosoftOnline"
Microsoft Online Portal URL
This is the login page and also where MicrosoftOnline accepts SAML token. This is also decided by Microsoft so just enter "https://login.microsoftonline.com/login.srf".
Microsoft Online Portal Context Data
If you have a fiddler(from telerik) or any HTTP header tracing tools you would be able to get the request going to "https://login.microsoftonline.com/login.srf" URL.
The value of "wctx" parameter VALUE would be what you need to copy and past in to this field.