Recently we have seen quite a few customer issues where using RDP cookies (mstshash cookies see – http://www.snakelegs.org for more details) seems to result in multiple active sessions over several RDP servers as shown below (Notice the user Rob on both TS Servers).
So we decided to investigate this and find out why……..
Where do we start?, well our first thought was to check if there were any issues with the load balancing application or its configuration. we found that everything there looked to be fine except that not every user seemed to be in the stick table and that testing the exact same environment with Windows XP or a Linux client we were unable to replicate the issue.
OK so what do we do now. we know the load balancer and its configuration seam to be working as it should as the XP client etc. So what is windows 7 and above doing that is different and why is there not always all the users in the stick table?…
After some research we found the TechNet documentation on the x.224 Connection Request PDU and the following key text.
<13> Section 3.2.5.3.1: Microsoft RDP 5.1, 5.2, 6.0, 6.1, 7.0, 7.1, and 8.0 clients always include the cookie field in the X.224 Connection Request PDU if a nonempty username can be retrieved for the current user and the routingToken field is not present (the IDENTIFIER used in the cookie string is the login name of the user truncated to nine characters).
Ok so that makes sense, when making a connection request the client should send a cookie made from the users username so the loadbalancer knows who the user is. We fired up Wireshark and what do you know we found that the new client does not always send this cookie. it seems it will not send it or the incorrect cookie in the following conditions:
- Starting the client and using the cached/saved credentials, then no hash is sent.
- Mistyping the username and then correcting, mstsc still uses the old username as the hash and not the updated username.
- Under certain conditions if a user entered the wrong password the cookie would default to domain/user and not the expected user@domain format that was entered in the username field. This is an issue for users with domains over 9 characters long as the cookie is then allowed to be duplicated for many users or not match the previously used cookie.
Ok so what happens with the XP client. Well this seems to follow the documentation and explains why that worked.
how do we fix this? Well the TechNet Docs say this is how it works, yet the client does not do this…
Sounds like a bug so lets log it with Microsoft……First problem. how the hell do you log a bug? nothing stands out from their site so instead we paid and logged a support call with them (which is still ongoing 4 months later).
So where does this all lead. Well so far Microsoft have stated this is and i quote (including spelling mistake) from them:
“The behavior that you are experiencing is a documentation bug and we are currently in the process of publishing a Knowledge Base article for it”
Err ok, so a Microsoft Product does not follow their own documentation and they say it is a Documentation bug? So their answer is if a product does not fit the design spec, just change the spec. I have now seen the new version of the documentation and it now says…
“cookie (variable): An optional and variable-length ANSI character string terminated by a 0x0D0A two-byte sequence. This text string MUST be “Cookie: mstshash=IDENTIFIER”, where IDENTIFIER is an ANSI character string (an example cookie string is shown in section 4.1.1). The length of the entire cookie string and CR+LF sequence is included in the X.224 Connection Request Length Indicator field. This field MUST NOT be present if the routingToken field is present.”
So now its an optional item,and not always sent. Well thats no good to anyone using RDP cookies.
They also say there is a perfectly valid workaround….. but failed to advise what it was, well after chasing for it we are told….
“This problem occurs only when the same MSTSC process is used to connect to a server with 2 different credentials. The scenario can easily workaround by starting a new MSTSC process before connecting to the server with different credentials.”
Really? Well to us this indicates an issue with the MSTSC client where it is unable to update its own cookies without having to restart the application, and after many emails back and forth we get the very latest response.
“Based on my internal discussions, we conclude that for load balancing terminal servers, using Session Directory is the recommended approach. This whitepaper talks more about the implementation http://download.microsoft.com/
download/8/6/2/8624174c-8587- 4a37-8722-00139613a5bc/TS_ Session_Directory.doc. In the business impact statement that you had provided, I did see that you have mentioned that you cannot use Session Directory.”
Well thats lovely for anyone with Windows 2003 Enterprise and above, what about users of 2003 Standard who do not have the Session Directory included? hell their document even says this.
“Note:
Terminal servers must be running Windows Server 2003 Enterprise Edition or Windows Server 2003 Datacenter Edition to participate in a Session Directory-enabled farm.”
Well we are still waiting for an answer to this from Microsoft but it looks like the RDP cookie maybe being dropped by Microsoft and no fix in sight… Guess we will just have to wait and see what they say next, however we would love to hear from anyone else who is experiencing the same issues as us so we can advise Microsoft of other users who are being hampered by this issue.
You Can Learn More About the LoadBalancer.org’s Product Line By Going to www.LoadBalancerSolutions.com/LoadBalancer-org
The original article/video can be found at Microsoft drops support for mstshash cookies?