Skip navigation
This discussion is archived
47173 Views 87 Replies Latest reply: Aug 2, 2006 9:40 AM by evan swendsen RSS
Calculating status... 35 posts since
Apr 15, 2005
Currently Being Moderated

May 25, 2005 11:23 AM

Logging Conversations

I have setup the server to log conversations between users in a corporate environment. But the log file has a lot of stuff that I really don''t need and it is difficult to navigate when looking for a specific item. Is there an app I can use to read the file easier or a plugin I can use to save them as a plain text conversation?

 

ItsNewToYou

  • Bronze 31 posts since
    Nov 30, 2004
    Currently Being Moderated
    May 26, 2005 8:51 AM (in response to Jeffery A. Stout)
    Re: Logging Conversations

    I''d also be interested in a feature like this if it isn''t available.  I''m doing some searches for an XML parser app that I can use in the meantime.

  • Ryan Graham KeyContributor 1,940 posts since
    Jan 17, 2003
    Currently Being Moderated
    May 26, 2005 9:15 AM (in response to Jeffery A. Stout)
    Re: Logging Conversations

    There''s been some discussion before about how to improve logging and auditing (here''s one url=http://www.jivesoftware.org/forums/thread.jspa?threadID=14432thread[/url] in particular) but I don''t believe anyone is actively working on anything.

     

    Hope that helps,

    Ryan

  • Gaston Dombiak Jiver 3,863 posts since
    Sep 26, 2001
    Currently Being Moderated
    May 26, 2005 12:53 PM (in response to Jeffery A. Stout)
    Re: Logging Conversations

    Hey ItsNewToYou,

     

    There aren''t any tools for it yet, but we do hope to add some ways to view/search it in the admin console in a future release. Meanwhile, if you are logging stuff that you don''t need you can only check to audit Message packets and leave unchecked IQ and Presence packets in the Admin Console.

     

    Regards,

     

      -- Gato

  • Calculating status... 69 posts since
    Nov 29, 2004
    Currently Being Moderated
    Jun 2, 2005 1:29 AM (in response to Jeffery A. Stout)
    Re: Logging Conversations

    I''ve been doing a little work on this.

     

    I''ve put together a small Perl script that parses the jive.audit log files and creates dated log files from them (default is the current date but an offset in days can be passed in). This could be run as a cron job (or scheduled task) early in the morning, specifying an offset of -1 to grab all the previous day''s traffic.  All log files are parsed as there is no accounting for preferred rollover file sizes and also depending on heavy usage, a single day''s traffic could span several files.  The original log files remain intact.

     

    I have submitted a feature request to have the facility to specify log rollover by time rather than by file size and Matt has kindly filed it as JM-298, so the Perl script would eventually become redundant.

     

    I then have a program in C#  - .NET but I''m sure it can be ported to Mono,  (I''m all over the place with languages , except Java ) that parses the dated log file, although it can quite happily parse the original audit file as the only difference being the dated log files only contain message packets for a specific date. I''ve not tested very large files.

     

    Messages are presented in the form of a list, giving date, sender, message type (chat or message), subject and first 20 words of message body.  Double clicking will provide the whole message.  This list can then be filtered by senders and message type, i.e. being able to focus in on a chat, or just view all messages sent by a particular user.

     

    As message packets are logged twice, ie. from the sender''s stream and also to the recipient''s stream, these are ignored based on sender, subject (if one exists), and message body.  Recipient is ignored to allow for broadcast messages.

     

    I could post the code if anyone is interested, but bear in mind that I tend to take extreme programming to extremes and just omit design completely . Currently there are no configuration options so lots of stuff is hard coded, but I could fix that before posting.  Also none this is internationalised in terms of date/time formats, but again, date formatting could be set as an option.

     

    Things to do:

    Stuff this all into a database to allow for easy searching, and add a liberal sprinkling of nice icons.

     

    Cheers,

     

    Jason.

    • Matt Tucker Jiver 3,190 posts since
      Jun 28, 2001
      Currently Being Moderated
      Jun 2, 2005 9:43 AM (in response to Jason McClean)
      Re: Logging Conversations

      Jason,

       

      Sounds cool. I''ve given you permission to post attachments in case you want to attach your app or screenshots.

       

      Regards,

      Matt

      • Calculating status... 69 posts since
        Nov 29, 2004
        Currently Being Moderated
        Jun 3, 2005 3:08 AM (in response to Matt Tucker)
        Re: Logging Conversations

        Hi,

         

        I''ve attached a png image of the log viewer itself to whet your appetites.  It shows the main points of the application.  Appearance is a little bland at the moment.  No screen shot for the Perl script - no output on stdout (hopefully none on stderr either!).

         

        The context menu - Focus this type lets you easily focus in on a chat or a series of messages between two users, without having to set the filter via the filter window.  Focus all types selects all types of message between the two users.  The last one is handy here as lots of users still haven''t got to grips with the difference between chat and message.  The Filter window allows you to select from all available users (that are listed in the log file) and view various types of message between them.

         

        For the list of users, the application strips off the domain section of the JID and assumes that if the first portion has a . in it then it is Firstname.Surname and uses proper case on the username (only for display - the original message is stored intact).

         

        Currently, the application only reads the dated log file, however, I will get around to enabling this to be entered into a database.

         

        I''ll do a little work on putting some of the hard coded settings in the log viewer into some sort of config file to make things easier.  Hopefully being able to post the code in the next few days.

         

        Sorry, no smilies in the message window , not yet at least.  I''m using a RichText control and displaying images is a little involved if you want to avoid using the clipboard.

         

        I have also attached the Perl script that splits the audit logs into dated files.  This should work on both Windows and Linux, the path separator really being the only difference.

         

        The Perl script can take 4 options:

        -o specify an offset in days previous to the current day.  I.e -o "-1" for yesterday

        Default is 0 i.e. today.  I suppose that means you can supply and offset for a future date, but that won''t get you anywhere

         

        -O output location for the dated log file

        Default is ./logs (I think this works on Windows regardless of the / separator)

         

        -l location of the jive audit files

        Default is /opt/jive_messenger/logs (no idea where the default is on Windows)

         

        -f a template for the audit file

        Default is jive.audit-*.log.  The script then ''globs'' all matching files.

         

        The output log file is named yyyy-mm-dd.log and is in the same format as the original, just containing the message packets from the specified date.

         

        The current BIG assumption in the Perl script is that the message packet dates are  in the form: Fri Jun 02 09:35:00 BST 2005, with BST and GMT being used depending on daylight savings being in effect.  I''m sure someone can point out the error of my ways here.

         

        Actually, the Log viewer makes this assumption as well (as that is the format our logs take).

         

        Hope this helps a few people.

         

        Cheers,

         

        Jason.

         

        Message was edited by:

                jasonmcclean

        Attachments:
        • Calculating status... 69 posts since
          Nov 29, 2004
          Currently Being Moderated
          Jul 19, 2005 1:39 AM (in response to Jason McClean)
          Re: Logging Conversations

          Ok, turned out there weren''t that many hard coded settings

           

          I''ve attached the C# source code.  You will need, at least, the .NET v1.1 SDK to compile.  Much easier if you have something like Visual Studio .Net, or what I use, a GPL .NET IDE called SharpDevelop (http://www.icsharpcode.net/OpenSource/SD/Default.aspx).

           

          Create a new project, add in all the source files.  The file called Log Viewer.exe.config should be placed in the output folder and renamed to match the executable that you produce, i.e. yourApp.exe.config.  This file contains the broadcast address, your domain (not used at moment), date/time format for display and the window caption.

           

          I can post the executable if you wish, but you will need the .NET v1.1 runtime to run it.

           

          The whole thing needs a good polish, but it works.  Let me know when you find all the  bugs.

           

          EDIT

          Just checking and noticed something.  If you are going to use it on the actual log files produced by Jive messenger, use a copy.  This is because in the current log file, the closing </jive> tag as not yet been added.  The log viewer checks for this and adds it if it can''t be found.  This avoids errors being thrown from the Xml classes being used.  Not sure what impact that would have if Jive Messenger found the closing tag when it wasn''t supposed to be there, unless it just blindly appends to the file.  However, you would then still end up with an invalid XML file at the end due to two closing tags being present.

           

          Sorry

           

          End of edit

           

          Cheers,

           

          Jason.

           

          Message was edited by:

                  jasonmcclean

           

          edit

          source added to message towards end

          • Calculating status... 69 posts since
            Nov 29, 2004
            Currently Being Moderated
            Jun 8, 2005 5:50 AM (in response to Jason McClean)
            Re: Logging Conversations

            Any feedback?

             

            I''ve added support for emoticons and you can easily recreate psuedo chats from either chats or related messages.

             

            I''ll update the previous post with new code if anyone interested.  If not, nevermind, it''s doing what I need

             

            Jason.

            • Calculating status... 69 posts since
              Nov 29, 2004
              Currently Being Moderated
              Jun 8, 2005 8:08 AM (in response to Jason McClean)
              Re: Logging Conversations

              Added image of chat window created from the logs. (couldn''t add it to previous post as it didn''t originally have an attachment).

               

              I''ll stop now.

               

              Jason.

              Attachments:
              • Matt Tucker Jiver 3,190 posts since
                Jun 28, 2001
                Currently Being Moderated
                Jun 8, 2005 8:28 AM (in response to Jason McClean)
                Re: Logging Conversations

                Jason,

                 

                It''s starting to look very cool! Would you be interested in doing binary releases so that people can more easily use the software?

                 

                Regards,

                Matt

                • Calculating status... 69 posts since
                  Nov 29, 2004
                  Currently Being Moderated
                  Jul 19, 2005 1:49 AM (in response to Matt Tucker)
                  Re: Logging Conversations

                  Would you be

                  interested in doing binary releases so that people

                  can more easily use the software?

                   

                  Not a problem.  As long as .NET v1.1 runtime is installed, there shouldn''t be a problem.

                   

                  Have attached binary.

                   

                  Emoticon support was added by using the RichEdit control by Khendys Gordon (http://www.codeproject.com/cs/miscctrl/csexrichtextbox.asp)

                   

                  The emoticons are taken from the Psi client (http://psi.affinix.com) as that is the client we use - using the same icondef.xml as Psi uses.  That means any icon set with a corresponding icondef.xml can be used.

                   

                  Just extract and run.

                   

                  edit

                  Oh, edit the Log Viewer.exe.config file first with your broadcast address to pick up broadcast messages.  Will work without, but you won''t see broadcasts marked as such, just the first recipient message.

                   

                  Jason.

                   

                  Message was edited by:

                          jasonmcclean

                  edit

                  Seem to have lost the binary.  Reattached to a message towards the end.

                  • Calculating status... 7 posts since
                    Jun 16, 2005
                    Currently Being Moderated
                    Jun 16, 2005 1:39 PM (in response to Jason McClean)
                    Re: Logging Conversations

                    Just wanted to thank you all for your efforts in doing this work... especially the spiffy log reader!

                  • Calculating status... 13 posts since
                    Jul 12, 2005
                    Currently Being Moderated
                    Jul 13, 2005 5:29 AM (in response to Jason McClean)
                    Re: Logging Conversations

                    Hi!

                     

                    I am trying to use the LogViewer, but in both scenarios (source and bin) I get the same error when i open the log file:

                     

                    ==========================================

                    See the end of this message for details on invoking

                    just-in-time (JIT) debugging instead of this dialog box.

                     

                                              • Exception Text **************

                    System.NullReferenceException: Object reference not set to an instance of an object.

                       at Listawood.Xmpp.LogViewer.XmppLog.Add(XmlNode value) in c:\Documents and Settings\nromero\My Documents\SharpDevelop Projects\LogViewer\XmppLog.cs:line 55

                       at Listawood.Xmpp.LogViewer.MainForm.LoadFile(String fileName) in c:\Documents and Settings\nromero\My Documents\SharpDevelop Projects\LogViewer\MainForm.cs:line 413

                       at Listawood.Xmpp.LogViewer.MainForm.OpenLogFile() in c:\Documents and Settings\nromero\My Documents\SharpDevelop Projects\LogViewer\MainForm.cs:line 281

                       at Listawood.Xmpp.LogViewer.MainForm.MainToolBarButtonClick(Object sender, ToolBarButtonClickEventArgs e) in c:\Documents and Settings\nromero\My Documents\SharpDevelop Projects\LogViewer\MainForm.cs:line 235

                       at System.Windows.Forms.ToolBar.OnButtonClick(ToolBarButtonClickEventArgs e)

                       at System.Windows.Forms.ToolBar.WmReflectCommand(Message& m)

                       at System.Windows.Forms.ToolBar.WndProc(Message& m)

                       at System.Windows.Forms.ControlNativeWindow.OnMessage(Message& m)

                       at System.Windows.Forms.ControlNativeWindow.WndProc(Message& m)

                       at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

                     

                     

                                              • Loaded Assemblies **************

                    mscorlib

                        Assembly Version: 1.0.5000.0

                        Win32 Version: 1.1.4322.573

                        CodeBase: file:///c:/windows/microsoft.net/framework/v1.1.4322/mscorlib.dll

                    -


                    LogViewer

                        Assembly Version: 1.0.2020.24043

                        Win32 Version: 1.0.2020.24043

                        CodeBase: file:///C:/Documents%20and%20Settings/nromero/My%20Documents/SharpDevelop%20Pro jects/LogViewer/bin/Debug/LogViewer.exe

                    -


                    System.Windows.Forms

                        Assembly Version: 1.0.5000.0

                        Win32 Version: 1.1.4322.573

                        CodeBase: file:///c:/windows/assembly/gac/system.windows.forms/1.0.5000.0__b77a5c561934e0 89/system.windows.forms.dll

                    -


                    System

                        Assembly Version: 1.0.5000.0

                        Win32 Version: 1.1.4322.573

                        CodeBase: file:///c:/windows/assembly/gac/system/1.0.5000.0__b77a5c561934e089/system.dll

                    -


                    System.Drawing

                        Assembly Version: 1.0.5000.0

                        Win32 Version: 1.1.4322.573

                        CodeBase: file:///c:/windows/assembly/gac/system.drawing/1.0.5000.0__b03f5f7f11d50a3a/sys tem.drawing.dll

                    -


                    System.Xml

                        Assembly Version: 1.0.5000.0

                        Win32 Version: 1.1.4322.573

                        CodeBase: file:///c:/windows/assembly/gac/system.xml/1.0.5000.0__b77a5c561934e089/system. xml.dll

                    -


                     

                                              • JIT Debugging **************

                    To enable just in time (JIT) debugging, the config file for this

                    application or machine (machine.config) must have the

                    jitDebugging value set in the system.windows.forms section.

                    The application must also be compiled with debugging

                    enabled.

                     

                    For example:

                     

                     

                     

                    When JIT debugging is enabled, any unhandled exception

                    will be sent to the JIT debugger registered on the machine

                    rather than being handled by this dialog.

                    =================================================

                     

                    What am I missing?

                     

                    thanks!

                    • Calculating status... 69 posts since
                      Nov 29, 2004
                      Currently Being Moderated
                      Jul 13, 2005 8:54 AM (in response to Nata)
                      Re: Logging Conversations

                      I''m guessing that there is a problem with the XML, so the XML classes are not happy.  Are you accessing the raw log file or using the Perl script to extract specific dates from the raw log file, then accessing the file that the Perl script creates?

                       

                      If it is the raw log file, then maybe the bit of code I shoved in to add the missing </jive> tag isn''t working. I''ll have a play about tomorrow and get back to you.  Unfortunately no time at the moment.

                       

                      Jason.

                      • Calculating status... 69 posts since
                        Nov 29, 2004
                        Currently Being Moderated
                        Jul 13, 2005 9:01 AM (in response to Jason McClean)
                        Re: Logging Conversations

                        Nope, it''s not that.  Just taken a file and removed the </jive> tag from it and everything still worked.  opened the file up in notepad after and the </jive> was back again, so that is definitely working.

                         

                        Hmm, if you are using the raw log file, do you have write access to it?

                         

                        edit

                        Unfortunately, as I just threw this together for internal use I didn''t put in a lot of error checking code - darn, remember that next time I release code on the unsuspecting world.

                         

                        Message was edited by:

                                jasonmcclean

                        • Calculating status... 13 posts since
                          Jul 12, 2005
                          Currently Being Moderated
                          Jul 13, 2005 10:27 AM (in response to Jason McClean)
                          Re: Logging Conversations

                          Thanks for your fast response.  In fact I''m using the raw data.... first I thought I had to add </jive> to the copy I made...but since i got errors i tried without it as well and same result.  Permissions are okey... though now that i thing I might have been trying when the audit-file was still opened (editor)... Now I can''t check that .. but I will do first thing 2morrow morning...

                          Any other suggestion please tell me!

                          I will also try to use the perl script to see if that makes a different.

                          • Calculating status... 7 posts since
                            Jul 13, 2005
                            Currently Being Moderated
                            Jul 13, 2005 12:46 PM (in response to Nata)
                            Re: Logging Conversations

                            I am pulling up the same exact error. Any clue on how to fix this? I''m looking to implement this fantastic Jive Messenger in a corp. environment where everything can be controlled.. unlike using AIM or MSN or the other crap.

                             

                            Thanks for your help you guys are very bright.

                             

                            Thanks,

                            Steve

                            • Calculating status... 7 posts since
                              Jul 13, 2005
                              Currently Being Moderated
                              Jul 13, 2005 12:51 PM (in response to Steve)
                              Re: Logging Conversations

                              Just an FYI - I''m looking to log ALL conversations to reference at any time - without having to install anything locally on each machine.

                               

                              This is what I am looking for right?

                               

                              Thanks again,

                              Steve

                              • Calculating status... 69 posts since
                                Nov 29, 2004
                                Currently Being Moderated
                                Jul 14, 2005 12:40 AM (in response to Steve)
                                Re: Logging Conversations

                                Just an FYI - I''m looking to log ALL conversations to

                                reference at any time - without having to install

                                anything locally on each machine.

                                 

                                This is what I am looking for right?

                                 

                                All this program does is organise the logs that Jive messenger outputs into a more useable form.  The only plave you need to use this program in on the machine that your want to view the logs from.

                                 

                                As for the error, still trying to reproduce it.

                                 

                                edit

                                If any use, I''ve updated the source code on the earlier post, however this should give the binary that I posted. You get the error with that as well.

                                 

                                From the debug output, the problem seems to be happening on the initial parse of the logfile.  I''ve just tried a raw log and still didn''t get any errors.

                                 

                                Question to devs: Are the xml tags in the logs the same regardless of localisation?  What about time stamps?

                                 

                                I''ll update the sources again once I''ve added some error checking around the log file loading code.

                                 

                                Another edit

                                Source and binary attachments updated.  Error checking just reports whether the log file can be loaded into an XmlDocument.  If it can''t, that points to invalid XML. It''s a start.

                                 

                                Jason.

                                 

                                Message was edited by:

                                        jasonmcclean

                                • Calculating status... 7 posts since
                                  Jul 13, 2005
                                  Currently Being Moderated
                                  Jul 14, 2005 9:22 AM (in response to Jason McClean)
                                  Re: Logging Conversations

                                  What settings do I have to enable under administration for it to log all conversations had?

                                  • Calculating status... 7 posts since
                                    Jul 13, 2005
                                    Currently Being Moderated
                                    Jul 14, 2005 9:54 AM (in response to Steve)
                                    Re: Logging Conversations

                                    Including private messages

                                    • Calculating status... 7 posts since
                                      Jul 13, 2005
                                      Currently Being Moderated
                                      Jul 14, 2005 10:00 AM (in response to Steve)
                                      Re: Logging Conversations

                                      Just an FYI - I downloaded the new binary and am getting these errors when trying to view the log file:

                                       

                                      First message box: Error parsing child nodes of root node jive

                                      Second message box: Error loading log file

                                       

                                       

                                      Thanks,

                                      Steve

                                      • Calculating status... 69 posts since
                                        Nov 29, 2004
                                        Currently Being Moderated
                                        Jul 15, 2005 12:33 AM (in response to Steve)
                                        Re: Logging Conversations

                                        Nothing was "fixed" with the new binary, just some error tracking.  I need to find out where it is going wrong first.

                                         

                                        Ok, we''re zooming in on the problem.

                                         

                                        btw, to log all conversations, go to the Server tab (the default one after entering the admin console), then click on Message Audit policy on the left.  Click on ''Enable Message Auditing'', then if it is only messages you are after, just click on ''Audit Message Packets''.

                                         

                                        This will produce files like jive.audit-0.log etc.  These files will contain all the message packets, and it is these files that the log viewer (and associated Perl script) process.

                                         

                                        If you don''t have message auditing switched on, what are you trying to view with the log viewer??

                                        • Calculating status... 13 posts since
                                          Jul 12, 2005
                                          Currently Being Moderated
                                          Jul 15, 2005 1:12 AM (in response to Jason McClean)
                                          Re: Logging Conversations

                                          I get the same errors as Steve.

                                          I did also check the audit file after trying to open it in LogViewer and now it does have a </jive> at the end.

                                           

                                          Btw, I used the binary.

                                        • Calculating status... 7 posts since
                                          Jul 13, 2005
                                          Currently Being Moderated
                                          Jul 15, 2005 5:43 AM (in response to Jason McClean)
                                          Re: Logging Conversations

                                          oh, I was just double checking that I had the settings correct - which I did.

                                           

                                          This utility would be AWESOME if you programmer guru''s can zoom in on this error

                                          • Calculating status... 69 posts since
                                            Nov 29, 2004
                                            Currently Being Moderated
                                            Jul 15, 2005 5:58 AM (in response to Steve)
                                            Re: Logging Conversations

                                            I''ve sent you a private message with my email address.  Could you send me a copy of your log so that I can try and reproduce the error?

                                             

                                            Jason.

                                            • Calculating status... 69 posts since
                                              Nov 29, 2004
                                              Currently Being Moderated
                                              Jul 18, 2005 2:01 AM (in response to Jason McClean)
                                              Re: Logging Conversations

                                              Ok,

                                               

                                              I''ve received a copy of Natata''s log file and the problem is now blindingly obvious.  I wrote the log viewer only to be expecting message packets, not everything!  This is a bit of major boob on my part   It is the status packets, log on/off packets etc. etc. that is completely throwing the viewer.

                                               

                                              Hopefully I''ll have some time today to modify the code to filter out the rest of the other stuff and concentrate only on message packets.

                                               

                                              Once again, oops!

                                               

                                              Jason.

                                              • Calculating status... 69 posts since
                                                Nov 29, 2004
                                                Currently Being Moderated
                                                Jul 18, 2005 3:17 AM (in response to Jason McClean)
                                                Re: Logging Conversations

                                                Hi Natata & Steve,

                                                 

                                                I think I''ve almost sorted the problem.  If you are logging all packets, only message packets are used, everything else is ignored.  Also fixed a time stamp problem, but as you didn''t get that far, you wouldn''t have noticed it

                                                 

                                                Another thing that I didn''t consider was groupchats, doh!  As we don''t currently use them, support for them is limited in the viewer at the moment.  You can view individiual messages by double clicking the entry in the list, but you cannot recreate a chat window ... yet.  I''ve added the type so that the viewer correctly reports the message type (groupchat), but currently that is as far as it goes at the moment.

                                                 

                                                Hopefully I''ll get a little more time again later today to try and address the remaining problems, then I will post updated source and binary.

                                                • Calculating status... 7 posts since
                                                  Jul 13, 2005
                                                  Currently Being Moderated
                                                  Jul 18, 2005 7:25 AM (in response to Jason McClean)
                                                  Re: Logging Conversations

                                                  so the Viewer will work if I stop logging everything and simply only log message packets?

                                                  • Calculating status... 69 posts since
                                                    Nov 29, 2004
                                                    Currently Being Moderated
                                                    Jul 19, 2005 1:03 AM (in response to Steve)
                                                    Re: Logging Conversations

                                                    so the Viewer will work if I stop logging everything

                                                    and simply only log message packets?

                                                     

                                                    Most likely, however, I have removed this restriction. You shouldn''t have to stop logging everything just to use this utility.  That''s not the way it should go. 

                                                     

                                                    Natata''s log file has highlighted a few other little problems like extracting the date from the message, I was assuming only a 3 character time zone code (like BST), but Natata''s was CEST, which threw it a bit.  Fixed that.

                                                     

                                                    Group chat.  We don''t use that here yet, so the logger doesn''t cope well with that.  When viewing a chat, the log viewer assumes just two participants, however when it can be many (including the room itself), it becomes a little more involved to reconstruct the conversations.  Working on that now.

                                                     

                                                    Hopefully I''ll get something posted today.

                                                     

                                                    Jason.

                                                    • Calculating status... 69 posts since
                                                      Nov 29, 2004
                                                      Currently Being Moderated
                                                      Sep 1, 2005 12:53 AM (in response to Jason McClean)
                                                      Re: Logging Conversations

                                                      I seem to have fluffed updating the attachments further back in the thread and managed to lose them.  So, here they are again.

                                                       

                                                      If logging everything is turned on, only message, chat and groupchat packets are processed.  Everything else is ignored.

                                                       

                                                      Reconstruction of a group chat sort of works, but I have by no means thoroughly tested it   Just right click and choose "View as chat".

                                                       

                                                      In order to recognise when the sender or recipient is actually the chat room, the key "Conference" in the config file must be set to the subdomain of your conference, i.e. conference.yourdomain.com.  There will be repetitions of messages to and from chat rooms in the main window, but when viewing as a chat, the messages sent from the room are ignored so all you get are the messages sent by the participants of the group chat.

                                                       

                                                      I suppose that I could combine all of these so that you can just specify your domain then just the subdomain part for broadcasting and groupchats.  Maybe in the next update.

                                                       

                                                      BTW I''ve not checked how the Perl script copes with everything.  I think it just picks out packet nodes so it shouldn''t be affected by what is being logged.

                                                       

                                                      Let me know how you get on.

                                                       

                                                      Jason.

                                                       

                                                      edit

                                                      Attachments have been updated - 01-09-2005

                                                       

                                                      Message was edited by:

                                                              jasonmcclean

                                                      Attachments:
                                                      • Calculating status... 13 posts since
                                                        Jul 12, 2005
                                                        Currently Being Moderated
                                                        Jul 19, 2005 7:07 AM (in response to Jason McClean)
                                                        Re: Logging Conversations

                                                        Many many thanks Jason!

                                                        I am able now to load a log file. Somehow when I use focus or filter in GROUPCHAT sender and receiver get a bit messed up.  I can perfectly "view as chat" normal chats but with groupchat I got the following undhandled exception:

                                                        - 71 is not a valid message item

                                                          Parameter name: index

                                                         

                                                        and in details:

                                                        See the end of this message for details on invoking

                                                        just-in-time (JIT) debugging instead of this dialog box.

                                                         

                                                                                  • Exception Text **************

                                                        System.ArgumentOutOfRangeException: 71 is not a valid message item

                                                        Parameter name: index

                                                           at Listawood.Xmpp.LogViewer.Sender.get_Item(Int32 index)

                                                           at Listawood.Xmpp.LogViewer.ChatWindow.PopulateGroupChatWindow(Boolean useEmoticons)

                                                           at Listawood.Xmpp.LogViewer.ChatWindow.Setup(XmppLog log, XmppMsgType msgType, String room)

                                                           at Listawood.Xmpp.LogViewer.ChatWindow..ctor(XmppLog log, XmppMsgType msgType, String room)

                                                           at Listawood.Xmpp.LogViewer.MainForm.GroupChatWindow(XmppMsg msg)

                                                           at Listawood.Xmpp.LogViewer.MainForm.MenuItem3Click(Object sender, EventArgs e)

                                                           at System.Windows.Forms.MenuItem.OnClick(EventArgs e)

                                                           at System.Windows.Forms.MenuItemData.Execute()

                                                           at System.Windows.Forms.Command.Invoke()

                                                           at System.Windows.Forms.Control.WmCommand(Message& m)

                                                           at System.Windows.Forms.Control.WndProc(Message& m)

                                                           at System.Windows.Forms.ListView.WndProc(Message& m)

                                                           at System.Windows.Forms.ControlNativeWindow.OnMessage(Message& m)

                                                           at System.Windows.Forms.ControlNativeWindow.WndProc(Message& m)

                                                           at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

                                                         

                                                         

                                                                                  • Loaded Assemblies **************

                                                        mscorlib

                                                            Assembly Version: 1.0.5000.0

                                                            Win32 Version: 1.1.4322.573

                                                            CodeBase: file:///c:/windows/microsoft.net/framework/v1.1.4322/mscorlib.dll

                                                        -


                                                        Log Viewer

                                                            Assembly Version: 1.0.2026.15186

                                                            Win32 Version: 1.0.2026.15186

                                                            CodeBase: file:///C:/Documents%20and%20Settings/nromero/My%20Documents/TUe/Development/ji ve_messenger_src/LogViewer-bin/Log%20Viewer.exe

                                                        -


                                                        System.Windows.Forms

                                                            Assembly Version: 1.0.5000.0

                                                            Win32 Version: 1.1.4322.573

                                                            CodeBase: file:///c:/windows/assembly/gac/system.windows.forms/1.0.5000.0__b77a5c561934e0 89/system.windows.forms.dll

                                                        -


                                                        System

                                                            Assembly Version: 1.0.5000.0

                                                            Win32 Version: 1.1.4322.573

                                                            CodeBase: file:///c:/windows/assembly/gac/system/1.0.5000.0__b77a5c561934e089/system.dll

                                                        -


                                                        System.Drawing

                                                            Assembly Version: 1.0.5000.0

                                                            Win32 Version: 1.1.4322.573

                                                            CodeBase: file:///c:/windows/assembly/gac/system.drawing/1.0.5000.0__b03f5f7f11d50a3a/sys tem.drawing.dll

                                                        -


                                                        System.Xml

                                                            Assembly Version: 1.0.5000.0

                                                            Win32 Version: 1.1.4322.573

                                                            CodeBase: file:///c:/windows/assembly/gac/system.xml/1.0.5000.0__b77a5c561934e089/system. xml.dll

                                                        -


                                                        Khendys.Controls.ExRichTextBox

                                                            Assembly Version: 1.0.1985.28025

                                                            Win32 Version: 1.0.1985.28025

                                                            CodeBase: file:///C:/Documents%20and%20Settings/nromero/My%20Documents/TUe/Development/ji ve_messenger_src/LogViewer-bin/Khendys.Controls.ExRichTextBox.DLL

                                                        -


                                                         

                                                                                  • JIT Debugging **************

                                                        To enable just in time (JIT) debugging, the config file for this

                                                        application or machine (machine.config) must have the

                                                        jitDebugging value set in the system.windows.forms section.

                                                        The application must also be compiled with debugging

                                                        enabled.

                                                         

                                                        For example:

                                                         

                                                         

                                                         

                                                        When JIT debugging is enabled, any unhandled exception

                                                        will be sent to the JIT debugger registered on the machine

                                                        rather than being handled by this dialog.

                                                         

                                                        ==========

                                                         

                                                        The tool looks very promising!! thanks once more for all your effort Jason... I''m trying to go through the code now to understand how it works so I can help with the errors

                                                        • Calculating status... 69 posts since
                                                          Nov 29, 2004
                                                          Currently Being Moderated
                                                          Jul 20, 2005 12:13 AM (in response to Nata)
                                                          Re: Logging Conversations

                                                          Hi Natata,

                                                           

                                                          Glad to hear that you can load a log file.  Group chat, hmm, that was only added yesterday and is still up in the air in terms of working.  I need to test that further.  The problem is that we don''t really use groupchat here yet, so I''ve not really had anything to test the code with.  I''ll probably be setting up a testing group chat later today with a few colleagues so that I can get some log entries to test with.

                                                           

                                                          BTW it''s just the first few lines of the error that are most useful as that is closest to where the error actually is. I.e. the exception text.

                                                           

                                                          Thanks,

                                                           

                                                          Jason.

                                      • Calculating status... 4 posts since
                                        Aug 1, 2006
                                        Currently Being Moderated
                                        Aug 1, 2006 2:39 PM (in response to Steve)
                                        Re: Logging Conversations

                                        I had the error with the root node jive also.

                                        I was compiling the source.   it turned out that if my exe was named anything other than log viewer.exe it errored.

              • Bronze 71 posts since
                Feb 1, 2005
                Currently Being Moderated
                Jun 8, 2005 8:37 AM (in response to Jason McClean)
                Re: Logging Conversations

                Looks really nice, yes!

                Just tried to compile it on Mono, but in spite of a (citing the website) 98% complete implementation of System.Windows.Forms it''s still missing the RichTextBox.. sigh

                "Major areas still needing work are DataGrid, RichText, MDI and Printing."

                Would be very cool to have your tool on a cross-platform framework..

                 

                Regards,

                Ben

                • Calculating status... 69 posts since
                  Nov 29, 2004
                  Currently Being Moderated
                  Jun 8, 2005 8:59 AM (in response to Benjamin Podszun)
                  Re: Logging Conversations

                  Looks really nice, yes!

                  Just tried to compile it on Mono, but in spite of a

                  (citing the website) 98% complete implementation of

                  System.Windows.Forms it''s still missing the

                  RichTextBox.. sigh

                  "Major areas still needing work are DataGrid,

                  RichText, MDI and Printing."

                  Would be very cool to have your tool on a

                  cross-platform framework..

                   

                  Haven''t played with Mono yet (forgot about Windows.Forms Doh!).  Maybe soon

                   

                  edit

                  don''t see a reason why it can''t be re-implemented using something other than C#.  That just happens to be my language of the moment.

                   

                  Jason

                   

                  Message was edited by:

                          jasonmcclean

                  • wroot KeyContributor 6,838 posts since
                    Jan 24, 2005
                    Currently Being Moderated
                    Jun 8, 2005 12:49 PM (in response to Jason McClean)
                    Re: Logging Conversations

                    what about *nix?

                     

                    i mean bin

                    • Calculating status... 69 posts since
                      Nov 29, 2004
                      Currently Being Moderated
                      Jun 8, 2005 1:44 PM (in response to wroot)
                      Re: Logging Conversations

                      what about *nix?

                       

                      i mean bin

                       

                      Do you mean a *nix binary?  Sorry, not until this could be compiled with Mono...

                      I''ve only got as far a scripting (Perl & PHP) on *nix.

                      • Bronze 71 posts since
                        Feb 1, 2005
                        Currently Being Moderated
                        Jun 8, 2005 2:35 PM (in response to Jason McClean)
                        Re: Logging Conversations

                        Well - a mono assembly is not exactly what I associate with a unix binary (since it runs on MS .NET as well and is nothing native).

                         

                        I''ve some experiences on *nix, so I''m not whining here.. It''s just that I tried (curiosity..) to compile the app with mono.

                        • Calculating status... 69 posts since
                          Nov 29, 2004
                          Currently Being Moderated
                          Jun 9, 2005 12:31 AM (in response to Benjamin Podszun)
                          Re: Logging Conversations

                          Well - a mono assembly is not exactly what I

                          associate with a unix binary (since it runs on MS

                          .NET as well and is nothing native).

                           

                          What I meant was, Mono is the only way this particular app will most likely ever end up running on a *nix system.  It''s a long time since I played with c/c++. Pascal (whatever version), not sure.

                          • Matt Tucker Jiver 3,190 posts since
                            Jun 28, 2001
                            Currently Being Moderated
                            Jun 9, 2005 8:37 AM (in response to Jason McClean)
                            Re: Logging Conversations

                            Jason,

                             

                            Any interest in playing with desktop Java? It''s actually pretty great these days. It''s fast and looks native. It would certainly fit well with JM itself...

                             

                            -Matt

                            • Calculating status... 69 posts since
                              Nov 29, 2004
                              Currently Being Moderated
                              Jun 9, 2005 12:51 PM (in response to Matt Tucker)
                              Re: Logging Conversations

                              Jason,

                               

                              Any interest in playing with desktop Java? It''s

                              actually pretty great these days. It''s fast and looks

                              native. It would certainly fit well with JM

                              itself...

                               

                              Definitely an idea.  I''ve not really tried Java before, but I guess this could be the perfect opportunity to start

                               

                              edit s/not really/never/

                               

                              Message was edited by:

                                      jasonmcclean

                            • Calculating status... 69 posts since
                              Nov 29, 2004
                              Currently Being Moderated
                              Jun 17, 2005 12:08 AM (in response to Matt Tucker)
                              Re: Logging Conversations

                              Work has started on porting the log viewer to Java.  Not as bad as I thought as C# is quite close syntactically (since java was a big influence).  GUI will come last.

                               

                              Any views as to whether this should still be a stand alone application, an applet or a messenger plugin (no idea how to tackle that one )?

                               

                              Jason.

                              • Matt Tucker Jiver 3,190 posts since
                                Jun 28, 2001
                                Currently Being Moderated
                                Jun 17, 2005 12:44 AM (in response to Jason McClean)
                                Re: Logging Conversations

                                That''s good news to hear. One thing that would be very interesting is indexing the logs using Lucene (http://lucene.apache.org/). That would let people do full-text searches to locate conversations.

                                 

                                As for the medium (app, applet, plugin). Eventually, we''ll definitely need a robust tool inside the Jive Messenger admin console for viewing log information. However, there may definitely be a place for a desktop application as well. So, I would start where you want to and see where it goes.

                                 

                                -Matt

                    • Calculating status... 4 posts since
                      Jun 16, 2005
                      Currently Being Moderated
                      Jun 17, 2005 1:27 PM (in response to wroot)
                      Re: Logging Conversations

                      I have Jive 2.1.5 running on a fresh install of Solaris 9.  I was able to take the audit log from the Solaris box, ftp the log over to a Windows 2000 machine, run the binary that was posted and viewed the chats perfectly.

                       

                      Works very well ... unless you have 0 Windows machines at your location. 

                      • wroot KeyContributor 6,838 posts since
                        Jan 24, 2005
                        Currently Being Moderated
                        Jun 18, 2005 10:16 AM (in response to startours)
                        Re: Logging Conversations

                        windows binary?? Do i''ve missed something?

                        • Calculating status... 4 posts since
                          Jun 16, 2005
                          Currently Being Moderated
                          Jun 20, 2005 12:41 PM (in response to wroot)
                          Re: Logging Conversations

                          LogViewer-bin.zip (55.1 K) 

                           

                           

                          Would you be

                          interested in doing binary releases so that people

                          can more easily use the software?

                           

                          Not a problem. As long as .NET v1.1 runtime is installed, there shouldn''t be a problem.

                           

                          Check the first page of posts ... the one from Jason with the ZIP attached has an EXE file to view logs in a very easy to use format ... I''ve already busted a few people with it. 

                      • Calculating status... 69 posts since
                        Nov 29, 2004
                        Currently Being Moderated
                        Jun 20, 2005 12:31 AM (in response to startours)
                        Re: Logging Conversations

                        I''ve got Jive running on Linux, so I''m just using a restricted Samba share to access the dated log files produced by the Perl script.  No copying required.

                  • Bronze 71 posts since
                    Feb 1, 2005
                    Currently Being Moderated
                    Jun 8, 2005 12:54 PM (in response to Jason McClean)
                    Re: Logging Conversations

                    Haven''t played with Mono yet (forgot about

                    Windows.Forms Doh!).  Maybe soon

                     

                    It''s currently "in the pipeline" (Talking about RichTextBox here). But the next problem would be, that you are using the Codecentral approach of inserting icons, right? With P/Invoke on gdi/windows dlls..

                     

                    edit

                    don''t see a reason why it can''t be re-implemented

                    using something other than C#.  That just happens to

                    be my language of the moment.

                     

                    I do like C# as well (currently even better than Java sometimes). It certainly could be implemented cross-platform with C# / gtk# or lots of other languages of course (Well - Java would be quite a good idea for a java IM server.. ponder)

                     

                    wroot: No idea what you mean, sorry?

                    • wroot KeyContributor 6,838 posts since
                      Jan 24, 2005
                      Currently Being Moderated
                      Jun 8, 2005 1:02 PM (in response to Benjamin Podszun)
                      Re: Logging Conversations

                      wroot: No idea what you mean, sorry?

                       

                      nevermind. i guess i''ll go to bed right now, this is the best solution so far for me:)

                    • Calculating status... 69 posts since
                      Nov 29, 2004
                      Currently Being Moderated
                      Jun 8, 2005 1:41 PM (in response to Benjamin Podszun)
                      Re: Logging Conversations

                      It''s currently "in the pipeline" (Talking about

                      RichTextBox here). But the next problem would be,

                      that you are using the Codecentral approach of

                      inserting icons, right? With P/Invoke on gdi/windows

                      dlls..

                       

                      The RichEdit control that I''m using has extended the standard one by adding in functions to wrap up an image in a Windows MetaFile (WMF) and inserting directly into the control wrapped in RTF codes.  If you looked at one of the icons in this state you would see the RTF control codes and the image represented in Hex.

                       

                      The way I''m using it is I''ve created a class to search for the emoticons using regexes then generate the RTF codes required and replacing in the text of the message.  The message is then inserted into the RichText control as standard RTF text.

                       

                      As RTF is a well known standard, as soon as Mono sort out the control, this should work (I think)

                      • Bronze 71 posts since
                        Feb 1, 2005
                        Currently Being Moderated
                        Jun 8, 2005 3:10 PM (in response to Jason McClean)
                        Re: Logging Conversations

                        It''s currently "in the pipeline" (Talking about

                        RichTextBox here). But the next problem would be,

                        that you are using the Codecentral approach of

                        inserting icons, right? With P/Invoke on

                        gdi/windows

                        dlls..

                         

                        The RichEdit control that I''m using has extended the

                        standard one by adding in functions to wrap up an

                        image in a Windows MetaFile (WMF) and inserting

                        directly into the control wrapped in RTF codes.  If

                        you looked at one of the icons in this state you

                        would see the RTF control codes and the image

                        represented in Hex.

                         

                        The way I''m using it is I''ve created a class to

                        search for the emoticons using regexes then generate

                        the RTF codes required and replacing in the text of

                        the message.  The message is then inserted into the

                        RichText control as standard RTF text.

                         

                        If you are using the Codecentral version (http://www.codeproject.com/cs/miscctrl/csexrichtextbox.asp): That one uses Platform/Invoke and uses windows-only functions. Nothing wrong with that, but since it uses for example

                        [DllImportAttribute("gdiplus.dll")]

                          private static extern uint GdipEmfToWmfBits (IntPtr _hEmf,

                            uint _bufferSize, byte[] _buffer,

                            int _mappingMode, EmfToWmfBitsFlags _flags);

                        this won''t ever work on mono on a non-windows system.. Well - it would work, if you have the same function and can map the dll.. But  - GDI is quite windows-specific, I guess. So is this function..

                         

                        As RTF is a well known standard, as soon as Mono sort

                        out the control, this should work (I think)

                         

                        Yes, what ends up in the box is RTF and fine. The way it gets there is what I was talking about.. But I may be wrong, of course. And it''s seems a great tool anyway, independent of that "cosmetic" thing.

                        • Calculating status... 69 posts since
                          Nov 29, 2004
                          Currently Being Moderated
                          Jun 9, 2005 12:37 AM (in response to Benjamin Podszun)
                          Re: Logging Conversations

                          If you are using the Codecentral version

                          (http://www.codeproject.com/cs/miscctrl/csexrichtextbo

                          x.asp): That one uses Platform/Invoke and uses

                          windows-only functions. Nothing wrong with that, but

                          since it uses for example

                          [DllImportAttribute("gdiplus.dll")]

                          private static extern uint GdipEmfToWmfBits (IntPtr

                          tr _hEmf,

                              uint _bufferSize, byte[] _buffer,

                              int _mappingMode, EmfToWmfBitsFlags _flags);

                          this won''t ever work on mono on a non-windows

                          system.. Well - it would work, if you have the same

                          function and can map the dll.. But  - GDI is quite

                          windows-specific, I guess. So is this function..

                           

                          Just goes to show I didn''t look closely at the code

                           

                          edit

                          I wonder if a browser type component could be used instead?

                           

                          Message was edited by:

                                  jasonmcclean

  • Calculating status... 9 posts since
    Aug 12, 2005
    Currently Being Moderated
    Aug 12, 2005 8:08 AM (in response to Jeffery A. Stout)
    Re: Logging Conversations

    How do i use this?

    I''m not used about Perl...

    I''ve installed Jive messenger and it''s working ok!

    How to run Log Viewer? Need to install Perl?

     

    Thank''s in Advance

    • Calculating status... 69 posts since
      Nov 29, 2004
      Currently Being Moderated
      Aug 12, 2005 9:32 AM (in response to Leandro)
      Re: Logging Conversations

      How do i use this?

      I''m not used about Perl...

      I''ve installed Jive messenger and it''s working ok!

      How to run Log Viewer? Need to install Perl?

       

      Thank''s in Advance

       

      Hi there,

       

      No you don''t necessarily need Perl.  All the Perl script does is to enable splitting the audit log files into dated files, so it is easier to keep track.  At some point I think the option to roll over the log files based on time will be included, just not yet

       

      To run the Log viewer, however, you will need the Microsoft .NET 1.1 runtime installed (available from the Microsoft web site).

       

      All the log viewer does is to present the Jive audit logs in a more readable form.  It is still work in progress.  You need to have auditing turned on (In the admin console - in the logs section).

       

      Hope that helps.

       

      Jason.

      • Bronze 49 posts since
        Aug 16, 2005
        Currently Being Moderated
        Aug 16, 2005 6:43 AM (in response to Jason McClean)
        Re: Logging Conversations

        Jason,

         

        Just wanted to say Excellent work on the chat logger!!

        Makes searching a lot easier then using Hurrican Search or WinGrep to find stuff

         

        Thanks again!

         

        Edit

        I see no option to export from filtered searchs, this would be a nice thing to see added

         

        Message was edited by:

                lcameron

        • Calculating status... 69 posts since
          Nov 29, 2004
          Currently Being Moderated
          Aug 18, 2005 1:04 AM (in response to lcameron)
          Re: Logging Conversations

          I see no option to export from filtered searchs, this

          would be a nice thing to see added

           

          Good idea.  I''d also like to add this stuff to a database aswell, unfortunately, I just don''t have much time at the moment.  I''m also in the process of familiarising myself with Java so I can port this to Java and be a little more intune with Jive and a little more cross platform.

           

          Things to do:

          - Port to Java

          - enable phrase/word searching withing the viewer

          - export filtered results

          - DB storage (so you don''t have to keep opening older files)

          - Maybe turn the viewer into a plugin... That would be nice.

           

          The source is posted in this thread so anyone is more than welcome to have a go too

           

          Cheers,

           

          Jason.

  • Calculating status... 9 posts since
    Aug 12, 2005
    Currently Being Moderated
    Aug 24, 2005 5:57 AM (in response to Jeffery A. Stout)
    Re: Logging Conversations

    Log Viewer shows me all dates as 01/01/1980, does anybody know why?!

     

    Thank''s!

    • Bronze 49 posts since
      Aug 16, 2005
      Currently Being Moderated
      Aug 26, 2005 5:30 PM (in response to Leandro)
      Re: Logging Conversations

      the_lok,

      The log viewer grab''s the date from the log files, so you may want to check your system date and make sure it is correct.

    • Calculating status... 69 posts since
      Nov 29, 2004
      Currently Being Moderated
      Aug 29, 2005 1:06 PM (in response to Leandro)
      Re: Logging Conversations

      That is the default date given if for some reason the date can''t be obtained from the log.  It may be a format thing.  I thought that would be a better idea than throwing an error.

       

      There are lots of ways this app can be improved, internationalising it and coping with different date and time formats is one of them.

       

      How is the date formatted in your log?

       

      Jason

      • Calculating status... 9 posts since
        Aug 12, 2005
        Currently Being Moderated
        Aug 31, 2005 12:40 PM (in response to Jason McClean)
        Re: Logging Conversations

        My date format:

        timestamp="Mon Aug 29 08:04:44 GMT-03:00 2005"

        • Calculating status... 69 posts since
          Nov 29, 2004
          Currently Being Moderated
          Sep 1, 2005 12:59 AM (in response to Leandro)
          Re: Logging Conversations

          My date format:

          timestamp="Mon Aug 29 08:04:44 GMT-03:00 2005"

           

          That''s the kiddy . The regex that pulls out the date and time is being thrown off by the -03:00 bit. 

           

          I''ll have a fiddle with that and upload new source and binary packages on the message a little further back on this thread.

           

          Will try and get that sorted today

           

          Jason

           

          edit

          Done. I don''t actually do anything with the -03:00 part, just that it doesn''t cause the regex to fail anymore.

           

          Attachments further back in the thread have been updated.

           

          Message was edited by:

                  jasonmcclean

      • Calculating status... 69 posts since
        Nov 29, 2004
        Currently Being Moderated
        Sep 1, 2005 1:01 AM (in response to Jason McClean)
        Re: Logging Conversations

        Question to the Devs.

         

        Do you think that I should move this off the forums, just leaving a pointer to where it has gone, or are you quite happy for this thread to keep getting bigger?

         

        Cheers,

         

        Jason.

  • Calculating status... 7 posts since
    Jun 16, 2005
    Currently Being Moderated
    Aug 31, 2005 3:41 PM (in response to Jeffery A. Stout)
    Re: Logging Conversations

    Again wanted to thank Jason for his dedication to getting this thing up and running... I wish I knew something about programming so I could be of assistance to you on this. But I am sure that everyone who has found this has rejoiced at finding something other than opening the log in a text editor.

     

    Brett

  • Calculating status... 2 posts since
    Sep 29, 2005
    Currently Being Moderated
    Oct 9, 2005 8:41 PM (in response to Jeffery A. Stout)
    Re: Logging Conversations

    I started using this perl script on my corporate server recently and I noticed a bug in the code.

     

    At the bottom of the script, the code reads as follows:

    Re: Logging Conversations  #short date

      #add leading zero to month, if required

      #first make it non-zero based

      $mon++;

      if ($mon<10) {

        $month = "0$mon";

        }

      my $shortDate = "$year-$month-$day";[/i]

     

    It needs to be modified in one of several ways (I have tested multiple which work, though I''ll leave it up to each person to execute it in their own way as I have zero perl experience and I know there is a much much more elegant solution, but I simply don''t know enough about the language syntax to code it myself.)

     

    The problem is this: if the value variable "mon" is 10 or greater, the variable "month" (which is used to create "shortDate") has a value of null.  This causes the resulting filename of the log file to be "2005--04.log" instead of "2005-10-04.log"

     

    The fixes I have tried:

    Re: Logging Conversations1)[/b]  Add an else statement to the if statement that sets "month" to "mon"

    Re: Logging Conversations  #short date

      #add leading zero to month, if required

      #first make it non-zero based

      $mon++;

      if ($mon<10) {

        $month = "0$mon";

        }

    Re: Logging Conversations else {

        $month = "$mon";

        }[/b]

      my $shortDate = "$year-$month-$day";[/i]

     

    Re: Logging Conversations2)[/b] Re: Logging ConversationsModify the line assigning a value to "shortDate" to use "mon" instead of "month" and modify the if statement to assign the resulting value back to "mon"

     

      #short date

      #add leading zero to month, if required

      #first make it non-zero based

      $mon++;

      if ($mon<10) {

        Re: Logging Conversations$mon = "0$mon";[/b]

        }

     

      Re: Logging Conversationsmy $shortDate = "$year-$mon-$day";[/b][/i]

     

    Hope this helps others that are using this perl script who may or may not have noticed this bug.

     

    Peace,

    -Bartman007

    • Calculating status... 69 posts since
      Nov 29, 2004
      Currently Being Moderated
      Oct 10, 2005 12:26 AM (in response to Bartman007)
      Re: Logging Conversations

      Thanks for the fix. 

       

      I''m by no means an expert with Perl either but that was just simply stupid of me!  Just shows also that I''ve not actually checked my logs for the last week or so, otherwise I would have seen that too - Doh!

       

      Undoubtedly there are many more ''issues'' to be found and I welcome people pointing them out.

       

      Cheers,

       

      Jason.

       

      edit

      p.s. I think I favour the shorter solution.  Why introduce variables when you don''t have too.  Good point! Why did I?

      • Calculating status... 26 posts since
        Oct 19, 2005
        Currently Being Moderated
        Oct 19, 2005 1:39 PM (in response to Jason McClean)
        Re: Logging Conversations

        Hi Jason!

         

        Just checked that software and liek it so far, bhut:

        - i second the option to export or print results (export is probably better for formatting purposes -> Export to CSV)

         

        then one thing i miss a lot:

        The filter seems to only filter the ''from'' address, and not the too address'', so if i want to check a complete conversation between user1 and all other users it should also show replies user1 got from user 2, so everythign where ''to'' is user2 too.

         

        also a selction by date (grouped) woudl be nice

        • Calculating status... 69 posts since
          Nov 29, 2004
          Currently Being Moderated
          Oct 20, 2005 1:45 AM (in response to SlowIce)
          Re: Logging Conversations

          I think the filter thing broke a little when I had to rejig some things a little while ago.  Exporting would be good, I totally agree, however, I''m a really bogged down in other projects at the moment

           

          I think the thing to do would be to move this onto another forum where anyone wishing to contribute can do so more easily.

           

          Jason.

          • Calculating status... 2 posts since
            Dec 28, 2005
            Currently Being Moderated
            Dec 28, 2005 12:25 PM (in response to Jason McClean)
            Re: Logging Conversations

            When pointing the viewer to the log file, I get an message "Error parsing child nodes of root node jive" then a "Error loading log file."  I''m trying to run the binary and view Wildfire 2.4.0 logs.  Did the logs change with the change to Wildfire by chance...they look the same to me. 

             

            Thanks,

            Chad

            • Calculating status... 69 posts since
              Nov 29, 2004
              Currently Being Moderated
              Dec 28, 2005 1:23 PM (in response to Chad Barbry)
              Re: Logging Conversations

              Not sure about the log files. I''m not currently using the latest version :-o

               

              I''ll look into it anf get back to you.

               

              Jason.

            • Bronze 8 posts since
              Dec 28, 2005
              Currently Being Moderated
              Dec 28, 2005 9:12 PM (in response to Chad Barbry)
              Re: Logging Conversations

              ignore this, ironing out the kinks with it working with 2.4

              • Calculating status... 7 posts since
                Jun 16, 2005
                Currently Being Moderated
                Feb 7, 2006 8:33 AM (in response to Michael)
                Re: Logging Conversations

                Hello everyone - has this project been abandoned? if so could someone please point me to the right place to look?

                I have an issue with the current binaries where when i pull my log into the viewer every chat has a date / time of 1/1/1980 00:00:00 - this really screws up attempting to make sense of a conversation.

                When I open the .log file itself i can definately see that the dates are in there correctly. Please help!!

                Thank you

                • Calculating status... 69 posts since
                  Nov 29, 2004
                  Currently Being Moderated
                  Feb 7, 2006 8:57 AM (in response to Brett A)
                  Re: Logging Conversations

                  Hello everyone - has this project been abandoned?

                   

                  It''s not been abandoned per se, however I just don''t have any time to spend on it.  Shucks, I didn''t even notice the name change to Wildfire until I got notification of this post. Is that bad? Do i really need to be upgrading my Jive Messenger installation (2.2.0 and working wonderfully - especially since I got my new server ). Maybe I need to look at that anyway.

                   

                  >if so could someone please point me to the right place

                  to look?

                  I have an issue with the current binaries where when

                  i pull my log into the viewer every chat has a date /

                  time of 1/1/1980 00:00:00 - this really screws up

                  attempting to make sense of a conversation.

                   

                  This will be due to the date handling and the huge assumptions that I made when creating the regex to extract them.  what is happening is that it is not properly recognising the dates, therefore a default date is being used. Regular expressions are not my strongest subject, but I''m getting better at it.

                   

                  When I open the .log file itself i can definately

                  see that the dates are in there correctly. Please

                  help!!

                   

                  Oh yes, the dates will be fine in the log file.  It is the log viewer that is having the problem.

                   

                  If you can post an extract of the date format in your log, that will help me refine the date extraction regex.  Really needs overhauling anyway - bit too specific.

                   

                  Please bear in mind that this project started of as an internal project so that I, and a few others in management could easily review what was happening on our system.  The viewer still works fine for us - this is where the big assumptions came in.  I had no access to the formats of other installations.  I threw the code into this forum so others could take advantage.  Please, please feel free to review the source and see if anyone can help take this project forward.

                   

                  Hopefully, soon I will have more time to spend on this and take it to the next level, fix all the broken bits and finally port it to Java (I have started learning Java, honest ).

                   

                  Fingers crossed, I may soon have some splutter free time.

                   

                  Jason.

                  • Calculating status... 69 posts since
                    Nov 29, 2004
                    Currently Being Moderated
                    Feb 7, 2006 9:02 AM (in response to Jason McClean)
                    Re: Logging Conversations

                    Just had a thought.  As I''m several versions behind, has any progress been made with the internal log viewer?  There was talk of this some time ago.

                     

                    Just wondering.  Not had a chance to trawl through the forums and stuff.

                     

                    Jason.

                    • Calculating status... 2 posts since
                      Dec 28, 2005
                      Currently Being Moderated
                      Feb 7, 2006 11:18 AM (in response to Jason McClean)
                      Re: Logging Conversations

                      Just had a thought.  As I''m several versions behind,

                      has any progress been made with the internal log

                      viewer?  There was talk of this some time ago.

                       

                      It seems to me that this would be the best way to handle it.  IMHO, either the core or at least a plugin needs to be developed that will put the logs in a readable fomat.

                       

                      cb

                    • Calculating status... 2 posts since
                      Sep 29, 2005
                      Currently Being Moderated
                      Feb 7, 2006 11:26 AM (in response to Jason McClean)
                      Re: Logging Conversations

                      There was talk of it, but that appears to have fall by the wayside.  There was also discussion of date-based log files, http://www.jivesoftware.org/issues/browse/ JM-298 , but the version it was to be implemented in has been bumped back several times and now sits at 2.5.0

                       

                      khoehn, a forum member has created a admin console plugin (http://sourceforge.net/projects/iball-auditor) that creates logs - separate from the built-in Message Auditing - which are specifically for chat and MUC.  This plugin integrates a viewer into the Admin console.  A request has been made to integrate this plugin into Jive/Wildfire itself, http://www.jivesoftware.org/issues/browse/ JM-523  Based on posts khoehn has made on the forums, he plans to continue developing the plugin and has plans for the ability to search through the logs in various methods.

                       

                      I plan to test this plugin but probably won''t be able to for several months as other projects take priority.

                       

                      Message was edited by: Bartman007

  • Calculating status... 3 posts since
    Oct 6, 2005
    Currently Being Moderated
    Mar 7, 2006 9:16 PM (in response to Jeffery A. Stout)
    Re: Logging Conversations

    I am almost done with a related project for my coursework here in graduate school. I have written an app that listens for message packets and writes them to an .xml file. I currently filter to only write MUC messages, but it could be modified. These files are then mirrored into a WEBDAV enabled native-xml database (using eXist). The files are then accessible for query via an xquery webapp.

     

    This stuff has a lot of design tweaks that are specifically aimed at meeting the needs of my problem, but the basic principles of logging to an XML doc, importing into an XMLdb, and then web-enabled query access is working out pretty slick and isn''t too tough to implement either.

  • Calculating status... 1 posts since
    Mar 20, 2006
    Currently Being Moderated
    Mar 20, 2006 2:31 PM (in response to Jeffery A. Stout)
    Re: Logging Conversations

    I am having the same problem as ChiefofTheClass is having with the 0 dates. The date format in the log shows up as:

     

    Mar 17, 2006 6:03:19 PM

    Mar 18, 2006 12:00:40 AM

     

    Hopefully this can be corrected quickly, I know your pressed for time but we use the chat for internal communication and a small HR issue has arisen in which we need a manager who isn''t quite as ready to read raw logs.

    • LG KeyContributor 6,128 posts since
      Dec 13, 2005
      Currently Being Moderated
      Mar 20, 2006 2:50 PM (in response to David Williams)
      Re: Logging Conversations

      Hi Traciatim,

       

      well you know how it is. If one falls off a roof and can not fly it has only little time to learn flying *) **), if one falls in the water and can not swim it has only little time to learn swimming. So you have a manager which can not read raw logs, so it may teach itself very fast to read them unless one can provide a solution.

       

       

      ( Someone mentioned: "If you disappeared from the roof falling brake before you down land." *)

      ( some lyrics: "The important thing is not the fall but the landing" **)

       

      LG

    • Win Myo Htet Silver 333 posts since
      Aug 4, 2005
      Currently Being Moderated
      Mar 20, 2006 3:02 PM (in response to David Williams)
      Re: Logging Conversations

      Traciatim

       

      May be, it will get the fix you want in WildFire 2.6.0 ? JM-298 Try voting there and please comment the date format you want there so that the developer can implement that too.

       

      Regards,

      wmhtet

      • Calculating status... 69 posts since
        Nov 29, 2004
        Currently Being Moderated
        Mar 22, 2006 3:11 AM (in response to Win Myo Htet)
        Re: Logging Conversations

        As far as I understand it, JM-298 just allows raw log files to be rolled over based on date rather than size.  This is a feature that I requested and removes the need to run the Perl script (see earlier in the thread) to split the log files into dated files.

         

        This doesn''t get around the date parsing problem.

         

        What we have is a multitude of date formats.  If the log viewer ran on the same platform that generates the logs, we would probably not have this problem as it would probably utilise the same date format as Wildfire.

         

        However, the log viewer is currently restricted to Windows and .NET (Windows Forms and the Rich Edit control prevent compilation under Mono at this time).

         

        There are several possible solutions to this problem:

         

        1. Having some sort of option to define the timestamp format when the logs are actually generated - This really isn''t a fair solution as that just offloads the problem onto the Wildfire devs and they have got better things to do.

         

        2. Port the log viewer to Java so it can run on the same platform and automatically utilise the same sort of timestamp format - This was mooted some time ago, but I haven''t got time at the moment to do a port (whilst learning Java in the process), however this is something I want to do.

         

        2.5 Related to above - transform the viewer into a console plugin.  The only problem I can think about that one is providing access to other people to the admin console to view the logs.  It may not always be the tech guys that are reviewing these.

         

        3. Have an option in the current implementation of the Log Viewer to define the log timestamp format.  Obviously this is the quickest solution, but how to implement it?

         

        The Log Viewer user could probably define a regex in the config file that would be used during parsing, but that means the user needs to have some understanding of regular expressions.  If some people balk at just looking at XML, maybe that isn''t a good way to go.

         

        Use standard dd/mm/yyyy type definitions in the config file - probably the best way and is probably what I will try and do.

         

        I know that there are other things that need doing with this application, like fixing the filtering options, making sure group chats are rendered correctly and introducing indexing and exporting, but the date thing seems to be the most critical.

         

        I know that I have left the project languishing for a while, but I will try and get new source and binary files posted as soon as I can.

         

        Jason.

         

        edit added clarification of restriction to .NET

         

        another edit Has anyone had a go at hacking the source?

         

             

        Message was edited by: jasonmcclean

        • Calculating status... 4 posts since
          Aug 1, 2006
          Currently Being Moderated
          Aug 2, 2006 9:40 AM (in response to Jason McClean)
          Re: Logging Conversations

          I took a crack at the source code since my computer is logging the dates in the same format as described above..

          &lt;jive xmlns="http://www.jivesoftware.org"&gt;&lt;packet xmlns="http://www.jivesoftware.org" streamID="7915039" status="auth" timestamp="Aug 1, 2006 2:13:17 PM"&gt;

           

          I modified the code in XmppMsg.cs

          I also configured my date time format in the config file to be yyyy/MM/dd HH:mm:ss because i like my dates and times to be chronologically sorted. i at first used a smal mm for the months, but that caused it to put the minutes in the month''s spot. The capital MM works for months properly, my bad

           

          The XmppMsg.cs section modified i will paste here.  The RED code is what i put in/changed  I have a binary that works for this.  if anyone is interested email me and I''ll send it to you.

          evan.swendsen@champ-tech.com

           

          public void SetTimeStamp(DateTime timeStamp)

          {

          _timeStamp = timeStamp;

          }

           

          public void SetTimeStamp(string dateTime)

          {

          //Msg time stamp in form of

          //day month date time zone year

          //e.g Mon Aug 29 08:04:44 GMT-03:00 2005

          {color:#ff0000}dateTime = string.Format("",DateTime.Parse(dateTime));

           

          Regex r = new Regex(@"(\w?)\s(\w?)\s(\d)\s(\d:\d:\d)\s\w{3,4}[0-9\-:|0-9/-:]*?\s(\d)");

          Match m = r.Match(dateTime);

           

          if(m.Success)

          {

          _timeStamp = GetDateTime(m);

          }

          else

          {

          //this is what you get if, for some reason, the regex fails to extract the date and time

          {color:#ff0000}//_timeStamp = DateTime.Parse("01/Jan/1900 00:00:00");

          _timeStamp = DateTime.Parse(dateTime); }

          }

           

           

          I am not good at per Regular expressions so i just formatted the string containing the date to match the format of the 01/Jan/1900 and forced it to pass my variable instead. If there is a better format that would be best. Please contact me if you have any questions or would like to help me with writing this section better.

           

           

More Like This

  • Retrieving data ...

Bookmarked By (0)