10 Replies Latest reply on Aug 9, 2017 2:52 PM by Kirill Maximov

    HostnameVerifier.verify not called with actual hostname

    Anthony Sorvari

      I have an XMPP client using Smack 4.0 which connects to a server using TLS and uses EXTERNAL SASL authentication to log in using a client certificate.  This works fine, but as a client, I want to verify the identity of the server that I am connecting to, much like a web browser verifies a website's host name against the server certificate when connecting over SSL/TLS.  Smack 4.0 provides the method ConnectionConfiguration.setHostnameVerifier, which allows me to provide my own class for validating the host name. However, it seems that I am not given sufficient information to validate the host name.

       

      Suppose my XMPP server is at goodhost.com, with a server certificate whose Common Name is goodhost.com. The HostnameVerifier should protect against a Man-in-the-Middle attack where the attacker has obtained a valid certificate for a different host name and is trying to use that to impersonate my XMPP server. This is easily tested in reverse: add a hosts file entry for wronghost.com pointing to the IP address of my XMPP server, and try to connect to wronghost.com. I will connect to my XMPP server, which presents a certificate with a CN of goodhost.com, and I should be able to see that this does not match my expected host name of wronghost.com.

       

      To my surprise, I see that my HostnameVerifier's verify method is called with a hostname of goodhost.com. Naturally, this matches the server certificate. It seems that "hostname" in this context is actually the Service Name which is set during the connection process.

       

      In XMPPTCPConnection.login:

       

        // Set the user.
        String response = bindResourceAndEstablishSession(resource);
        if (response != null) {
         this.user = response;
         // Update the serviceName with the one returned by the server
         setServiceName(StringUtils.parseServer(response));
        }

       

      And in XMPPTCPConnection.proceedTLSReceived:

       

        final HostnameVerifier verifier = getConfiguration().getHostnameVerifier();
        if (verifier != null && !verifier.verify(getServiceName(), sslSocket.getSession())) {
         throw new CertificateException("Hostname verification of certificate failed. Certificate does not authenticate " + getServiceName());
        }

       

      Note that the login code has changed in 4.1, but the result appears to be the same, to take the Service Name from the server instead of using the configured host name.

       

              // Set the connections user to the result of resource binding. It is important that we don't infer the user
              // from the login() arguments and the configurations service name, as, for example, when SASL External is used,
              // the username is not given to login but taken from the 'external' certificate.
              user = response.getJid();
              serviceName = XmppStringUtils.parseDomain(user);

       

      This is all well and good if I use a custom HostnameVerifier that knows the "hostname" is actually the XMPP Service Name and verifies it against the hostname that I actually wanted, but a standard implementation of HostnameVerifier that follows the HostnameVerifier interface documentation, which specifically states "hostname" throughout, won't work here.

      What about the peer host name on the SSLSession? HostnameVerifier can check that, too. It turns out that is actually the IP address.

       

      In XMPPTCPConnection.proceedTLSReceived:

       

              Socket plain = socket;
              // Secure the plain connection
              socket = context.getSocketFactory().createSocket(plain,
                      plain.getInetAddress().getHostAddress(), plain.getPort(), true);

       

      Socket.getInetAddress returns an InetAddress representing the IP address of the plain socket, and getHostAddress converts this to a String. This is not the host name that was used to open the plain socket. That information has been lost - it's as if I had connected to the server directly by IP address. (Fun fact - SSLParameters.setEndpointIdentificationAlgorithm lets you enforce HTTPS-style hostname validation on the SSLSocket itself, but if you give Smack an SSLContext which generates SSLSockets with this property set, the connection is rejected because the "host name" (IP address) does not match.)

       

      My workaround is to validate the "hostname" passed to HostnameVerifier against my actual desired hostname, which is not a bad idea anyway. But, if you intended this "hostname" to be the actual hostname, as per the HostnameVerifier interface, then this needs to be fixed. If you do want to pass in the XMPP Service Name as the "hostname", then I strongly think this should be documented, for example in the Javadoc for ConnectionConfiguration.setHostnameVerifier. I understand not wanting to change this functionality due to the possibility of breaking existing HostnameVerifiers, especially since doing so has critical security implications. Fixing the SSLSocket hostname, on the other hand, should hopefully not break anything. In connectUsingConfiguration, sockets are opened by FQDN. It would make sense to set the SSLSocket's hostname to the same FQDN used to open the underlying socket so that this information can be available to a HostnameVerifier.


        • Re: HostnameVerifier.verify not called with actual hostname
          Flow
          Suppose my XMPP server is at goodhost.com, with a server certificate whose Common Name is goodhost.com. The HostnameVerifier should protect against a Man-in-the-Middle attack where the attacker has obtained a valid certificate for a different host name and is trying to use that to impersonate my XMPP server. This is easily tested in reverse: add a hosts file entry for wronghost.com pointing to the IP address of my XMPP server, and try to connect to wronghost.com. I will connect to my XMPP server, which presents a certificate with a CN of goodhost.com, and I should be able to see that this does not match my expected host name of wronghost.com.

          How would wronghost obtain a CA certified cert with a CN of goodhost.com?

           

          It's unfortunate that Java's HostnameVerifier is named this way, you are right that a Host denotes a specific machine, whereas a Service denotes an abstract entity. The fact that a single service can be provided by multiple hosts, makes TLS certificate validation complicated. It's no longer a single host that serves a service, but maybe multiple.

           

          The details on how TLS should be handled with regard to XMPP are specified in

          - RFC 6120 § 13.7.1.4

          - RFC 6125 § 4.1

          - RFC 4985

          - RFC 5280

          - https://datatracker.ietf.org/doc/draft-ietf-uta-xmpp/

           

          So much for the specification, now let's look how it's done in practice:

           

          - Prosody recommends setting CN to your XMPP service name when generating the certificate

          - A certificate request generated with "prosodyctl cert request example.com" does generate a cert which CN, DNSName and XMPPAddr set to the service name

           

          So it's the XMPP service name that is announced usually in the certificate. And this is the one you want to verify. Which is the reason Smack does use that as 'hostname' argument for HostnameVerifier.verify(String, SSLSession).

           

          But you are right, the SSLSession should be given the DNS name of the host Smack connected to, so that this information could be used in the verification process, which, as you may became aware now, is not as easy as it was when: ∀host, hostname(host) == servicename. I propose the following change: https://github.com/Flowdalic/Smack/commit/5f4374ec260809f934093bfd26885c370dc3e7 13

           

          So at least verification implementations have now at least the two important strings available: the service name and the host name. How do you verify a XMPP service cert reliable and secure? To be honest, I'm not sure. XMPPAddr is specificed in RFC 6120, but reading RFC 6125 makes it sound that it's deprecated:

           

          Support for the XmppAddr identifier type is encouraged in XMPP

            client and server software implementations for the sake of

            backward-compatibility, but is no longer encouraged in

            certificates issued by certification authorities or requested by

            XMPP service providers.

           

          And of course, the verification process get's even more complicated once POSH is used.

           

                  // Set the connections user to the result of resource binding. It is important that we don't infer the user

                  // from the login() arguments and the configurations service name, as, for example, when SASL External is used,

                  // the username is not given to login but taken from the 'external' certificate.

                  user = response.getJid();

                  serviceName = XmppStringUtils.parseDomain(user);

          Note that this is done after TLS, and is therefore not relevant in this discussion.

           

          but a standard implementation of HostnameVerifier that follows the HostnameVerifier interface documentation, which specifically states "hostname" throughout, won't work here.

          That's not the whole truth: It depends on the information found in the certificate.

            • Re: HostnameVerifier.verify not called with actual hostname
              Anthony Sorvari

              Flow wrote:

               

              Suppose my XMPP server is at goodhost.com, with a server certificate whose Common Name is goodhost.com. The HostnameVerifier should protect against a Man-in-the-Middle attack where the attacker has obtained a valid certificate for a different host name and is trying to use that to impersonate my XMPP server. This is easily tested in reverse: add a hosts file entry for wronghost.com pointing to the IP address of my XMPP server, and try to connect to wronghost.com. I will connect to my XMPP server, which presents a certificate with a CN of goodhost.com, and I should be able to see that this does not match my expected host name of wronghost.com.

              How would wronghost obtain a CA certified cert with a CN of goodhost.com?

              It wouldn't. It would obtain a CA certified cert with a CN of wronghost.com.  In my test, I just added an entry to my hosts file so I could access my XMPP server via a different hostname.  The server itself still reported the same service name and certificate, of course, and those fields are passed to the HostnameVerifier.  So if I validate only those fields, then the connection is accepted incorrectly.

               

              Of course, I should validate the service name somehow. If I connect to im.goodhost.com and it reports a service name of goodhost.com and certificate CN of goodhost.com, I may want to accept that. If it reports a service name of wronghost.com, I should reject the connection on that basis alone. So I see your reasoning for providing the service name during validation. I guess I just think it violates the principle of least surprise. I can't drop in a standard HostnameVerifier like Apache HttpComponent's StrictHostnameVerifier and expect it to validate the actual hostname. If it were up to me, I would make service name validation independent of hostname/certificate or whether TLS is even used. Thankfully, I have full control over both client and server in my application, and I can make simplifying assumptions when writing my own HostnameVerfier such as "host name and service name will be exactly equal" and "certificate CN will not have wildcards". I'll let someone else worry about how to correctly perform verification in systems where those assumptions do not apply.

               

              But you are right, the SSLSession should be given the DNS name of the host Smack connected to, so that this information could be used in the verification process, which, as you may became aware now, is not as easy as it was when: ∀host, hostname(host) == servicename. I propose the following change: https://github.com/Flowdalic/Smack/commit/5f4374ec260809f934093bfd26885c370dc3e7 13

              Sounds good to me!

                • Re: HostnameVerifier.verify not called with actual hostname
                  Flow
                  It wouldn't. It would obtain a CA certified cert with a CN of wronghost.com.

                  Then a HostnameVerifier used in Smack should not verify, because the given hostname (first parameter of HostnameVerifier.verify()) is "goodhost.com" and the verifier is supposed to compare those with the CNs and DNS subjectAlt names found in the certificate.

                   

                  The server itself still reported the same service name and certificate, of course, and those fields are passed to the HostnameVerifier.  So if I validate only those fields, then the connection is accepted incorrectly.

                  No, it's not, when e.g. org.apache.http.conn.ssl.StrictHostnameVerifier is used.

                   

                  If I connect to im.goodhost.com and it reports a service name of goodhost.com and certificate CN of goodhost.com, I may want to accept that.

                  Not only *may*, that is, as far as I can tell, the only combination, besides DNS subjectAlt, where verify should return 'true'.

                   

                  If I connect to im.goodhost.com and it reports a service name of goodhost.com and certificate CN of goodhost.com, I may want to accept that.

                  Yes you can. You just need to make sure that the certificate of your XMPP servers for you XMPP domain have the domain(part) as set as CN.

                    • Re: HostnameVerifier.verify not called with actual hostname
                      Anthony Sorvari

                      Flow wrote:

                       

                      It wouldn't. It would obtain a CA certified cert with a CN of wronghost.com.

                      Then a HostnameVerifier used in Smack should not verify, because the given hostname (first parameter of HostnameVerifier.verify()) is "goodhost.com" and the verifier is supposed to compare those with the CNs and DNS subjectAlt names found in the certificate.

                       

                      This is the very thing that I was complaining about.  The given hostname, the first parameter of HostnameVerify.verify(), is not the actual hostname, but instead the XMPP service name, a string that is reported by the server itself.  Hence, it is not "goodhost.com".  This is something I specifically tested in Smack 4.0: if I connect to the server using a different hostname that resolves to the same IP, the given hostname does not change. Hence a class such as org.apache.http.conn.ssl.StrictHostnameVerifier, which has no state and only validates the arguments to its verify method, cannot tell that I tried to connect to a different hostname.  (Not without your proposed change to the SSLSocket in XMPPTCPConnection.proceedTLSReceived, anyway.)

                       

                      You said that in Smack 4.1, the code I highlighted for setting the service name based on the JID received from the server is executed after the TLS session has been established. I haven't upgraded to Smack 4.1 because it is not officially released yet, so unfortunately I haven't tested with it.  If the given hostname passed to HostnameVerifier.verify in Smack 4.1 is the original hostname instead of the service name, then I have no complaint.

                       

                      I admit I'm not clear on all the rules on TLS host name validation in general and XMPP in particular. I definitely wouldn't feel confident implementing host name validation by spec for general use, at least not without doing more research. Requiring exact matches across the board with a custom HostnameVerifier probably wouldn't work for many configurations out there, but for my purpose, it's better to reject connections for configurations I'm not using anyway.