First published on TECHNET on Apr 09, 2018
Scenario: Pure Online (O365) environment, SFB user is homed Online, ADFS is Configured, MA (Modern Auth) is Disabled in O365
NOTE:
I have tried my best to ensure the information below is accurate. Some of the terms I use to describe things like Modern Auth provider, O365 AD, Org ID etc. may not be standard terminology, I use them solely to make the understanding simpler. My intention here is to explain what happens in the background when a SFB client signs in so that it helps engineers and customers troubleshooting issues related to Sign in and Authentication.
How Does it Work?
Below is a High level explanation on how the SFB online Client Sign in process works
SIP URI of the user - NJ@JohnsonDataSystems.com
- SFB client Queries DNS for Lyncdiscover.domain.com. This should point to Webdir.online.lync.com
- SFB Client then sends a unauthenticated GET request to Lyncdiscover.domain.com
- The Client is then redirected to Autodiscover https://webdir2a.online.lync.com/Autodiscover/AutodiscoverService.svc/root/user?originalDomain=johnsondatasystems.com
- SFB Client then sends a Request to Autodiscover to discover its pool for sign in.
- The Client is then challenged and is provided the URL for Webticket service ( https://webdir2a.online.lync.com/WebTicket/WebTicketService.svc ) where it can request a Webticket
- The Client then sends a POST request to Webticket Service which requires the client to provide a Token from Org ID (login.microsoftonline.com)
- Now in order to authenticate the client reaches out to Ord ID and requests a Token
- Since the tenant is enabled for ADFS the client is redirected to the ON Premise ADFS server https://sts.cloudsfb.com
- SFB client will then send a request to ADFS server and request a token
- The Client may receive a Password prompt (or previously saved password from credential manager is passed) and once the correct password is provided, ADFS will issue a Token to the client
- The Client then submits this token to Org ID
- ORG ID will now issue its own Token to the client
- The Client then submits this token that it received from ORG ID to Webticket Service
- Webticket service now will grant a Webticket to the Client
- The client then submits this webticket to Autodiscover
- In Response Autodiscover will provide the Pool names (sipfed2a.online.lync.com" port="443) where the client can send Register to Sign in
- The SFB client now sends a SIP register to the Online Edge pool (sipfed2a.online.lync.com" port="443)
- It is then challenged for authentication again, here the ONLY supported method of authentication is TLS-DSK, The client is provided a Cert provisioning URL ( https://webdir2a.online.lync.com:443/CertProv/CertProvisioningService.svc ) in the 401 unauthenticated response
- The SFB client then sends a request to Certprov
- Here again the Client is challenged for authentication and is redirected to webticket service to get Webticket
- The Client had already Obtained a webticket in step 14 above
- The client will submit the same webticket obtained in step 14 to the Cert provisioning service
- The Client then receives a certificate
- The SFB client can now send a Register again and use the certificate it downloaded for authentication
Below is a graphical representation of the SFB online Client Sign in process
Detailed Explanation of SFB online Client Sign in process with LOG Snippets:
SIP URI of the user - NJ@JohnsonDataSystems.com
When a SFB client wants to Sign in, It needs to know where it can send its request to be able to Sign in. Whenever a user enters his SIP URI to sign in the SFB client forms an autodiscover URL using the domain name that it extracts from the users SIP URI to start the discovery process and then it sends an Unauthenticated Get request to the URL, lyncdiscover.domain.com. The response code for this request will be '200 ok' and in the response we should receive the external webservices URL for autodiscover.
You can see the request and Response below
The SFB Client learns that it needs to Contact https://webdir2a.online.lync.com/
It then tries to Do a TCP handshake with webdir2a.online.lync.com, Followed by a TLS handshake. (I haven't included the TCP and TLS handshake screen shots here, you can see those if you collect a Network trace while signing in)
The client then sends a request to the Autodiscover URL for its own domain (in my case @ JohnsonDataSystems.com ) and in Response it receives the Autodiscover URL's specific to the users Tenant. You can see the request and Response below
The client then sends a request to the user URL. We are here trying to discover a specific users home pool, hence the request will go to the “User” URL.
In the response, the Client receives a Web ticket URL, which provides the location of the WebTicketService.
You can see the request and Response below
The Client then needs to send a Request to the Web ticket service URL in order to obtain a Web ticket. The client will send this request in a POST message to the web ticket Service and in response it receives the actual individual Web ticket service URL's
The Client has to submit a Request to this web ticket URL now in order to obtain a web ticket. But if it does that then it will need to authenticate first, unless the Client authenticates itself it will not be issued a web ticket. Since this user is Homed in SFB online the Client needs to reach out to O35 AD (Org ID) to get authenticated first
SO right now the Client has to First reach out to Org ID in order to authenticate and Get a Token.
The process of reaching out to Org ID is initiated with the help of Microsoft Online Sign in Assistant (also known as IDCRL) that is installed on the Computer.
You may not always see this in the fiddler trace, this attempt to get Token from AD is called Org ID auth and is captured in IDCRL logs on the client PC
The Client will try to reach Org ID (O365 AD) to get a token, but since the Tenant is Enabled for ADFS the O365 AD (org ID) will redirect the client to the ADFS Server URL and the client will have to request a Token from ADFS
The way this works is
- The client first tries to reach Org ID (O365 AD) to request a Token
- Here it learns that the tenant is enabled for ADFS so it has to now go to the ADFS URL first and ask for a token from ADFS
- The client reaches out to ADFS and requests a Token
- ADFS will challenge for authentication which will cause a Password prompt to appear, or the user credential stored in the credential manager will be passed to ADFS in which case the user will not be prompted for password
- Once the password is provided ADFS will provide the client a Token
- The client will then submit this token back to Org ID (O365 AD)
- Org ID will in turn provide the client the Org ID token
You can view all the above if you open the IDCRL log (MSOTrace folder) in the IDCRL parser, A successful token retrieval from ADFS and Org ID should look like below in the Parser (A more detailed explanation can be found in the bottom of this document if needed for reference)
Once the Client receives a Token from O365 AD (Org ID) it then submits this token to the Web Ticket Service https://webdir2a.online.lync.com/WebTicket/WebTicketAdvancedService.svc/WsFed_bearer
In Response the Web Ticket Service will now Issue the Client a Web Ticket. You can see this in the Trace below
The Client will Then Submit this web ticket back to the AutoDiscover User URL - https://webdir2a.online.lync.com/Autodiscover/AutodiscoverService.svc/root/user?originalDomain=johnsondatasystems.com&sipuri=nj@johnsondatasystems.com
In response it will now receive the Internal and External addresses of the Pool names where the user is Homed.
You can see this in the trace below
Once the Client receives the pool names it will then Send a SIP REGISTER message to the SFB pool in order to sign in. . You can see that in the Client UCCAPI log file. This is shown in the snippet below
In response the Client will now receive a 401 Unauthorized message again and the server will again ask the client to authenticate itself. Here the ONLY method of authentication that is available is TLS-DSK (Cert based authentication)
The SFB online server will provide the Client a Cert provisioning URL in the 401 you can see that in the snippet below
This means that the Client now needs to present a Certificate that can then be used to authenticate the client. Since this is the first time the client is signing in it will NOT have the certificate installed. This certificate is ideally downloaded after the client signs in for the first time and is valid for about 8 hours.
Since the client does not have a valid certificate it now has to Re-Authenticate to the Cert provisioning service.
The Process for this will again be the same, the client has to first get a Web ticket, to get a web ticket it needs to get a Token from O365 AD, but we know that the client has already done these steps earlier. SO it already has a Web Ticket from the Web services URL. The Client needs to submit this same web ticket that it had obtained now to the Cert provisioning Service and once it submits the web ticket it will serve as a proof of authentication.
The Client learns about this by first sending a Mex request to the Cert provisioning URL. You can see that in the Trace below
The Client then submits the Web Ticket that it had received to the Cert provisioning URL it received above, after this it receives a 200 OK and sign in is now complete
Sign in is now complete!!
__________________________________________________________________________________________________________________________________________
More Information:
Detailed log snippets showing the client requesting and receiving Token from Org ID and ADFS.
We Discussed above that in order to obtain a webticket the Client has to get authenticated with O365 AD. We described this process as below
- The client first tries to reach Org ID (O365 AD) to request a Token
- Here it learns that the tenant is enabled for ADFS so it has to now go to the ADFS URL first and ask for a token from ADFS
- The client reaches out to ADFS and requests a Token
- ADFS will challenge for authentication which will cause a Password prompt to appear, or the user credential stored in the credential manager will be passed to ADFS in which case the user will not be prompted for password
- Once the password is provided ADFS will provide the client a Token
- The client will then submit this token back to Org ID (O365 AD)
- Org ID will in turn provide the client the Org ID token
We can see this in the IDCRL Logs which are located on the Client PC in the MSOTrace folder on the C:\Drive.
Below are snippets from IDCRL logs showing all the above steps in detail for reference.
Below is the Client finding out that it needs to go to ADFS URL to authenticate
<?xml version="1.0" encoding="UTF-8"?><RealmInfo Success="true">
<Login>nj@johnsondatasystems.com</Login>
<NameSpaceType>Federated</NameSpaceType>
<DomainName>JOHNSONDATASYSTEMS.COM</DomainName>
<FederationGlobalVersion>-1</FederationGlobalVersion><AuthURL>https://sts.johnsondatasystems.com/adfs/ls/</AuthURL>
<IsFederatedNS>true</IsFederatedNS>
<STSAuthURL>https://sts.johnsondatasystems.com/adfs/services/trust/2005/usernamemixed</STSAuthURL>
<FederationTier>0</FederationTier><FederationBrandName>JOHNSONDATASYSTEMS.COM</FederationBrandName>
<AllowFedUsersWLIDSignIn>false</AllowFedUsersWLIDSignIn>
<MEXURL>https://sts.johnsondatasystems.com/adfs/services/trust/mex</MEXURL>
<SAML_AuthURL></SAML_AuthURL><PreferredProtocol>1</PreferredProtocol>
The Client then sends a request to the ADFS URL to get a Token
<s:Header>
<wsa:Action s:mustUnderstand="1">http://schemas.xmlsoap.org/ws/2005/02/trust/RST/Issue</wsa:Action>
<wsa:To s:mustUnderstand="1">https://sts.johnsondatasystems.com:443/adfs/services/trust/2005/usernamemixed</wsa:To>
<wsa:MessageID>1507666614</wsa:MessageID>
<wsse:Security><wsse:UsernameToken wsu:Id="user">
<wsse:Username>nj@johnsondatasystems.com</wsse:Username>
<wsse:Password>*********</wsse:Password>
</wsse:UsernameToken><wsu:Timestamp Id="Timestamp"><wsu:Created>2017-10-10T20:16:53Z</wsu:Created><wsu:Expires>2017-10-10T20:21:53Z</wsu:Expires>
</wsu:Timestamp></wsse:Security>
</s:Header>
<wst:RequestSecurityToken Id="RST0">
<wst:RequestType>http://schemas.xmlsoap.org/ws/2005/02/trust/Issue</wst:RequestType>
<wsp:AppliesTo><wsa:EndpointReference><wsa:Address>urn:federation:MicrosoftOnline</wsa:Address></wsa:EndpointReference></wsp:AppliesTo>
<wst:KeyType>http://schemas.xmlsoap.org/ws/2005/05/identity/NoProofKey</wst:KeyType></wst:RequestSecurityToken></s:Body></s:Envelope>
It will then receive the Token from ADFS
<s:Body>
<t:RequestSecurityTokenResponse xmlns:t="http://schemas.xmlsoap.org/ws/2005/02/trust">
<t:Lifetime>
<wsu:Created xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">2017-10-10T20:16:55.065Z</wsu:Created>
<wsu:Expires xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">2017-10-10T21:16:55.065Z</wsu:Expires>
</t:Lifetime>
<wsp:AppliesTo xmlns:wsp="http://schemas.xmlsoap.org/ws/2004/09/policy">
<wsa:EndpointReference xmlns:wsa="http://www.w3.org/2005/08/addressing"><wsa:Address>urn:federation:MicrosoftOnline</wsa:Address></wsa:EndpointReference>
</wsp:AppliesTo>
<t:RequestedSecurityToken>**********</t:RequestedSecurityToken>
Once the Client receives the Token from ADFS it then submits this token to O365 AD (Org ID) and requests a token from Org ID
Below is the request that the client sends to Org ID asking for a token
<s:Header>
<wsa:Action s:mustUnderstand="1">http://schemas.xmlsoap.org/ws/2005/02/trust/RST/Issue</wsa:Action>
<wsa:To s:mustUnderstand="1">https://login.microsoftonline.com:443/rst2.srf</wsa:To>
<wsa:MessageID>1507666615</wsa:MessageID>
<ps:AuthInfo xmlns:ps="http://schemas.microsoft.com/Passport/SoapServices/PPCRL" Id="PPAuthInfo">
<ps:HostingApp>{0000003F-002B-0000-4C0A-AE614E000000}</ps:HostingApp>
<ps:BinaryVersion>7</ps:BinaryVersion>
<ps:UIVersion>1</ps:UIVersion>
<ps:Cookies></ps:Cookies>
<ps:RequestParams>AQAAAAIAAABsYwQAAAAxMDMz</ps:RequestParams>
</ps:AuthInfo><wsse:Security>*********</wsse:Security>
</s:Header>
<s:Body>
<ps:RequestMultipleSecurityTokens xmlns:ps="http://schemas.microsoft.com/Passport/SoapServices/PPCRL" Id="RSTS">
<wst:RequestSecurityToken Id="RST0">
<wst:RequestType>http://schemas.xmlsoap.org/ws/2005/02/trust/Issue</wst:RequestType>
<wsp:AppliesTo>
<wsa:EndpointReference><wsa:Address>http://Passport.NET/tb</wsa:Address></wsa:EndpointReference>
</wsp:AppliesTo>
</wst:RequestSecurityToken><wst:RequestSecurityToken Id="RST1">
<wst:RequestType>http://schemas.xmlsoap.org/ws/2005/02/trust/Issue</wst:RequestType>
<wsp:AppliesTo>
<wsa:EndpointReference><wsa:Address>https://webpooldm12a05.infra.lync.com/WebTicket/WebTicketAdvancedService.svc/WsFed_bearer</wsa:Address></wsa:EndpointReference></wsp:AppliesTo>
</wst:RequestSecurityToken></ps:RequestMultipleSecurityTokens>
</s:Body>
Org ID then issues a Token to the client which we can see below
<wst:RequestedProofToken><wst:BinarySecret>x5hTCJZbYuSHUl0mKtoAab/zN4DE+YLW</wst:BinarySecret></wst:RequestedProofToken>
</wst:RequestSecurityTokenResponse><wst:RequestSecurityTokenResponse>
<wst:TokenType>urn:oasis:names:tc:SAML:1.0</wst:TokenType>
<wsp:AppliesTo xmlns:wsa="http://www.w3.org/2005/08/addressing">
<wsa:EndpointReference><wsa:Address>https://webpooldm12a05.infra.lync.com/WebTicket/WebTicketAdvancedService.svc/WsFed_bearer</wsa:Address></wsa:EndpointReference>
</wsp:AppliesTo>
<wst:Lifetime><wsu:Created>2017-10-10T20:16:56Z</wsu:Created><wsu:Expires>2017-10-11T04:16:56Z</wsu:Expires></wst:Lifetime>
<wst:RequestedSecurityToken>**********</wst:RequestedSecurityToken>