19 Replies Latest reply on Jun 30, 2011 5:56 PM by Antony

    XEP-147 (URI scheme) not working with 8 bit chars

    Antony Bronze

      Wolf_P,

       

      Thanks for making those changes for this.  We've tried this, but it seems it does not handle 8 bit characters properly

       

      We have a roster URI, which is 

      ...roster;name=%C3%84%C3%A4kk%C3%B6nen%20%C3%96ky

       

      This represents the name "Ääkkönen Öky" and it is utf-8 encoded and then pct-encoded.  This follows step 2 of section 3.1 of RFC 3987, which states

       

      Step 2.  For each character in 'ucschar' or 'iprivate', apply steps

                  2.1 through 2.3 below.

       

             2.1.  Convert the character to a sequence of one or more octets

                   using UTF-8 [RFC3629].

       

             2.2.  Convert each octet to %HH, where HH is the hexadecimal

                   notation of the octet value.  Note that this is identical

                   to the percent-encoding mechanism in section 2.1 of
                   [RFC3986]
      .  To reduce variability, the hexadecimal notation

                   SHOULD use uppercase letters.

       

             2.3.  Replace the original character with the resulting character

                   sequence (i.e., a sequence of %HH triplets).

       

      If the data is sent with no utf-8 encoding, i.e. the 8 bit characters are sent as is, then the name will appear in the Nickname.

       

      The %20 is decoded to a space.  Looking at UriIManager, it does not do any decoding of the String apart from the call to replace for the %20 encoding. 

       

      I am not sure of the fix for this as I don't know how the command line parameters are handled, but from my reading of the RFC, all the URI mapping handlers should actually be treating the Java uriMapping String as a pct-encoded utf-8 byte stream, so they should be doing something like

       

      a) Decode pct-encoding to byte

      b) Create byte array from command line including decoded bytes

      c) Create java String from byte array with new String (byte[], Charset.forName("UTF-8"));

       

      Do you agree?

      Antony

        • Re: XEP-147 (URI scheme) not working with 8 bit chars
          Wolf Gold

          you should post a patch

           

           

          also it shouldnt work for the other commands, because they dont convert 8bit chars either

            • Re: XEP-147 (URI scheme) not working with 8 bit chars
              Antony Bronze

              Set up Eclipse to checkout project from SVN, but it does not set up library build path in Eclipse project.  I can build with ant build.xml, but cannot debug as Eclipse complains about unresolved compilation problems.  I followed the doc here http://community.igniterealtime.org/docs/DOC-1040  but that still ends up with the same issue.  I'm using Helios with Subversive.

               

              I can add jars from build/lib I guess until the problems go away, but is there an update to that doc that deals with the Eclipse side a bit better?

              • Re: XEP-147 (URI scheme) not working with 8 bit chars
                Antony Bronze

                Wolf_P. wrote:

                 

                you should post a patch

                I've changed ChatManager and URIManager, attached here.  I couldn't test it properly because I can't get it built properly and make the exe, but at least I can partially debug it. Don't know how you do your testing and what test cases exist, but I tested it in the debugger and it seems to do the right thing, e.g. URI of

                 

                xmpp://%C3%A4%C3%A4%C3%A4@atacama/spark?subscribe;name=Haha

                 

                will convert to äää@atacama/spark/?subscribe;name=Haha

                 

                Just uses java.net.URI class.  Tested with JDK 1.6.  Not sure what the required JDK is for Spark, so don't know if URI class has changed in previous versions.

                 

                Antony

                  • Re: XEP-147 (URI scheme) not working with 8 bit chars
                    Antony Bronze

                    Hi Wolf_P, were these patches any use and was this the right place to put them?

                      • Re: XEP-147 (URI scheme) not working with 8 bit chars
                        Walter Ebeling KeyContributor

                        I'll ask Wolf to take a look into it. We have tentatively scheduled XEP 147 into the next release 2.6.3 that will go this week (or will be delayed for 8 -12 weeks).

                          • Re: XEP-147 (URI scheme) not working with 8 bit chars
                            Antony Bronze

                            Thanks!  Would be great if Wolf can take a look.  I'm working with a Finnish organisation to integrate our clients with Spark/Openfire.  Wolf's original XEP 147 implementation is working nicely, but the 8 bit chars is an issue in Finnish as they use the äåöÄÅÖ characters.  Am happy to do any testing.

                              • Re: XEP-147 (URI scheme) not working with 8 bit chars
                                Wolf Gold

                                Completed: At revision: 12546 

                                 

                                thank you for submitting

                                 

                                Can you please check the nightly at http://bamboo.igniterealtime.org/browse/SPARK and report the result?

                                • Re: XEP-147 (URI scheme) not working with 8 bit chars
                                  Walter Ebeling KeyContributor

                                  Please report test results.

                                    • Re: XEP-147 (URI scheme) not working with 8 bit chars
                                      Antony Bronze

                                      The encoding is working, but there is a problem somewhere.  The following URL

                                       

                                      xmpp://riitta@atacama?subscribe;name=Riitta

                                       

                                      sends the following presence packet to the server

                                       

                                      <presence id="z4haN-30" to="riitta@atacama/" type="subscribe"></presence><presence id="z4haN-30" to="riitta@atacama/" type="subscribe"></presence>

                                       

                                      which causes an Exception in Openfire

                                       

                                      2011.06.30 10:27:05 Rejecting packet. JID malformed

                                      java.lang.IllegalArgumentException: Existing slash at the very end of the string indicates that an empty resource part is provided. This is illegal.

                                              at org.xmpp.packet.JID.getParts(JID.java:584)

                                       

                                      If I debug the latest sources in Eclipse giving that URL as a startup argument, it is fine, it is only if I embed that URL in a web page.  Unfortunately I cannot find a way to get clicked URLs to launch my debug copy of Spark, so I can't see what is really being passed as a command line arg to Spark.

                                       

                                      There's one small typo in UriManager.java - changed version attached.

                                       

                                      Is there a dummies guide to getting Spark set up with Eclipse so it can build - I've got the SVN project, but have lots of build errors due to missing build path things, e.g.

                                       

                                      com.jivesoftware.spark.plugin.apple.AppleBounce.java - can't find

                                       

                                      import com.apple.eawt.Application;

                                       

                                      Any hints on how to run Spark so it can be remote debugged by Eclipse? 

                                        • Re: XEP-147 (URI scheme) not working with 8 bit chars
                                          Wolf Gold

                                          when starting eclipse from windows you can remove the following sources:

                                          - apple

                                          - growl

                                          - linux

                                           

                                          same goes when starting from apple you will need to remove:

                                          - linux

                                           

                                          on linux remove:

                                          - apple

                                          - growl

                                           

                                           

                                           

                                          ----------------

                                           

                                          please post patches not whole source files

                                            • Re: XEP-147 (URI scheme) not working with 8 bit chars
                                              Antony Bronze

                                              Sorry for the silly question...

                                              How to make patches? 

                                                • Re: XEP-147 (URI scheme) not working with 8 bit chars
                                                  Wolf Gold

                                                  from commandline:

                                                  svn diff sparkfolder > somename.patch

                                                   

                                                  from whatever your client is:

                                                  google

                                                    • Re: XEP-147 (URI scheme) not working with 8 bit chars
                                                      Antony Bronze

                                                      Thx

                                                      Index: src/java/org/jivesoftware/spark/uri/UriManager.java
                                                      ===================================================================
                                                      --- src/java/org/jivesoftware/spark/uri/UriManager.java    (revision 12554)
                                                      +++ src/java/org/jivesoftware/spark/uri/UriManager.java    (working copy)
                                                      @@ -233,7 +233,7 @@
                                                           sb.append(uri.getHost());
                                                           // Resource contains the leading /
                                                           String resource = uri.getPath();
                                                      -    if (resource != null && resource != "") {
                                                      +    if (resource != null && resource.length() > 0) {
                                                               sb.append(resource);
                                                           }
                                                           return sb.toString();
                                                      
                                                        • Re: XEP-147 (URI scheme) not working with 8 bit chars
                                                          Wolf Gold

                                                          try this in URIManager.java line #128

                                                           

                                                          jid= jid.contains("/")&&jid.lastIndexOf("/")==jid.length()-1 ? jid.substring(0,jid.length()-1):jid;

                                                            • Re: XEP-147 (URI scheme) not working with 8 bit chars
                                                              Antony Bronze

                                                              I check that path is not / in retrieveJid().  Sensible change, but still not clear where that / is coming from in the failure case.

                                                               

                                                              Index: src/java/org/jivesoftware/spark/uri/UriManager.java
                                                              ===================================================================
                                                              --- src/java/org/jivesoftware/spark/uri/UriManager.java    (revision 12554)
                                                              +++ src/java/org/jivesoftware/spark/uri/UriManager.java    (working copy)
                                                              @@ -233,7 +233,8 @@
                                                                   sb.append(uri.getHost());
                                                                   // Resource contains the leading /
                                                                   String resource = uri.getPath();
                                                              -    if (resource != null && resource != "") {
                                                              +    
                                                              +    if (resource != null && resource.length() > 0 && !resource.equals("/")) {
                                                                       sb.append(resource);
                                                                   }
                                                                   return sb.toString();
                                                              

                                                               

                                                              Ha!  Now I can make patches

                                                                • Re: XEP-147 (URI scheme) not working with 8 bit chars
                                                                  Wolf Gold

                                                                  Antony schrieb:

                                                                   

                                                                  I check that path is not / in retrieveJid().  Sensible change, but still not clear where that / is coming from in the failure case.

                                                                   

                                                                  Index: src/java/org/jivesoftware/spark/uri/UriManager.java
                                                                  ===================================================================
                                                                  --- src/java/org/jivesoftware/spark/uri/UriManager.java    (revision 12554)
                                                                  +++ src/java/org/jivesoftware/spark/uri/UriManager.java    (working copy)
                                                                  @@ -233,7 +233,8 @@
                                                                       sb.append(uri.getHost());
                                                                       // Resource contains the leading /
                                                                       String resource = uri.getPath();
                                                                  -    if (resource != null && resource != "") {
                                                                  +    
                                                                  +    if (resource != null && resource.length() > 0 && !resource.equals("/")) {
                                                                           sb.append(resource);
                                                                       }
                                                                       return sb.toString();
                                                                  

                                                                   

                                                                  Ha!  Now I can make patches

                                                                   

                                                                  Does this work now? cannot test at this moment

                                        • XEP-147 (URI scheme) not working with 8 bit chars
                                          Wolf Gold

                                          ant -f sparksourcefolder/build/build.xml jar

                                           

                                          cp sparksourcefolder/target/build/lib/spark.jar yousparkinstallfolder/lib/spark.jar

                                           

                                          you dont need to build an exe, the spark.jar file is enough

                                            • Re: XEP-147 (URI scheme) not working with 8 bit chars
                                              Antony Bronze

                                              Thanks, OK, all works now.

                                              One comment about the subscribe implementation:  Currently it just sends the presence packet.  The XEP for subscribe says http://xmpp.org/extensions/xep-0147.html#actions-sub that

                                               

                                              the invoked XMPP application SHOULD first send a roster add stanza as shown below (this is consistent with the recommendations in RFC 3921)

                                               

                                              Also, it would be nice if Spark could control whether the 'Add contact' window was displayed when receiving these types of URL.  As it stands, you don't know if anything has happened because the user's not added to the roster in the client display and there's no indication the presence packet is sent.

                                               

                                              I think the solution should be that the Add Contact window is shown by default, allowing nickname and group to be set before doing the protocal stuff.

                                               

                                              Can you give me an idea where to look so I can try to achieve this?

                                              Thanks

                                              Antony