* Allow the Portal server to access IDM management console directly, not through the SSO authentication. It doesn't mean you can't protect the management console with SSO, as long as it allows the Portal server to pass without authentication.
* Don't protect the management console with SSO (you can use the IDM native authentication mechanism for protecting the management console).
* Use a static roles and tasks file - you can configure the IM connector to use a local file from the server files system instead of fetching the roles and tasks xml file from the IDM server. Export the Roles and Tasks xml file manually from the management console and save it on the Portal server. This way, the connector won't try to connect to the IDM management console, instead it will simply read the roles and tasks file from its local files system. However, this means that every time something changes in the roles and tasks definition of the IDM environment you will have to manually update the file on the Portal server as well.
In newer versions, 14.0 and up, the connector doesn't need to fetch the Roles and Tasks xml file from IDM management console, so this problem won't occur in these versions.