Wildfire stopped when running as service

Once i have some problem running Wildfire 3.0.1 as service.

After upgrade to 3.1.0 i set LdapConnectionPool to TRUE - http://www.jivesoftware.org/community/message.jspa?messageID=132105.

And now i have problem: after two days or something i see event in Windows Event Log - “The Wildfire service entered the stopped state.”

But i don`t see any errors before this message, wildfire logs not contain any serious errors too.

Now i switch Wildfire to application mode again.

Should i try to disable connection pool?

I`ve just found file wildfire\bin\hs_err_pid2352.log


  1. An unexpected error has been detected by HotSpot Virtual Machine:

  1. EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x7c8327f9, pid=2352, tid=4580

  1. Java VM: Java HotSpot™ Server VM (1.5.0_08-b03 mixed mode)

  2. Problematic frame:

  3. C http://ntdll.dll+0x327f9


T H R E A D


Current thread (0x0d471e18): JavaThread “Client SR - 8988616” daemon _thread_in_native, id=4580

siginfo: ExceptionCode=0xc0000005, reading address 0x00000000

Registers:

EAX=0x17be1690, EBX=0x01100000, ECX=0x00000000, EDX=0x00000000

ESP=0x1574f80c, EBP=0x1574f818, ESI=0x17be1688, EDI=0x17be2000

EIP=0x7c8327f9, EFLAGS=0x00010246

Top of Stack: (sp=0x1574f80c)

0x1574f80c: 01100000 00000001 01100002 1574f850

0x1574f81c: 7c81e85e 00000000 17be2000 1574f844

0x1574f82c: 00000000 0000012f 01100178 01100000

0x1574f83c: 0d471e02 0000004d 00000200 17b50000

0x1574f84c: 00000050 1574fa7c 7c81e4ef 01100000

0x1574f85c: 00000978 0d471e18 00000970 08a1e498

0x1574f86c: a3e780ec 1f3cacd2 04a01918 0d471e18

0x1574f87c: 6db2e8f2 00c02c38 000005e8 00000001

Instructions: (pc=0x7c8327f9)

0x7c8327e9: 85 99 cf 02 00 8b 4e 0c 8d 46 08 8b 10 89 4d 08

0x7c8327f9: 8b 09 3b 4a 04 89 55 0c 0f 85 b2 d0 02 00 3b c8

Stack: [0x15550000,0x15750000), sp=0x1574f80c, free space=2046k

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)

C http://ntdll.dll+0x327f9

C http://ntdll.dll+0x1e85e

C http://ntdll.dll+0x1e4ef

C http://SaslSspi.dll+0x1eb2

j net.za.darkskies.security.sasl.SSPIImpl.<init>(Ljava/lang/String;Ljava/la ng/String;Ljava/lang/String;Ljava/util/Map;Ljavax/security/auth/callback/Callbac kHandler;)V+12

j net.za.darkskies.security.sasl.ServerFactoryImpl.createSaslServer(Ljava/lang/St ring;Ljava/lang/String;Ljava/lang/String;Ljava/util/Map;Ljavax/security/auth/cal lback/CallbackHandler;)Ljavax/security/sasl/SaslServer;+11

j javax.security.sasl.Sasl.createSaslServer(Ljava/lang/String;Ljava/lang/String;L java/lang/String;Ljava/util/Map;Ljavax/security/auth/callback/CallbackHandler;)L javax/security/sasl/SaslServer;+143

j org.jivesoftware.wildfire.net.SASLAuthentication.handle(Lorg/jivesoftware/wildf ire/Session;Lorg/dom4j/Element;)Lorg/jivesoftware/wildfire/net/SASLAuthenticatio n$Status;+229

j org.jivesoftware.wildfire.net.SocketReadingMode.authenticateClient(Lorg/dom4j/E lement;)Z+54

j org.jivesoftware.wildfire.net.BlockingReadingMode.readStream()V+99

j org.jivesoftware.wildfire.net.BlockingReadingMode.run()V+63

j org.jivesoftware.wildfire.net.SocketReader.run()V+4

j java.lang.Thread.run()V+11

v ~StubRoutines::call_stub

V http://jvm.dll+0xf96e0

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)

j net.za.darkskies.security.sasl.SSPIImpl.serverInit(Ljava/lang/String;Ljava/lang /String;Ljava/lang/String;Ljava/util/Map;Ljavax/security/auth/callback/CallbackH andler;)V+0

j net.za.darkskies.security.sasl.SSPIImpl.<init>(Ljava/lang/String;Ljava/la ng/String;Ljava/lang/String;Ljava/util/Map;Ljavax/security/auth/callback/Callbac kHandler;)V+12

j net.za.darkskies.security.sasl.ServerFactoryImpl.createSaslServer(Ljava/lang/St ring;Ljava/lang/String;Ljava/lang/String;Ljava/util/Map;Ljavax/security/auth/cal lback/CallbackHandler;)Ljavax/security/sasl/SaslServer;+11

j javax.security.sasl.Sasl.createSaslServer(Ljava/lang/String;Ljava/lang/String;L java/lang/String;Ljava/util/Map;Ljavax/security/auth/callback/CallbackHandler;)L javax/security/sasl/SaslServer;+143

j org.jivesoftware.wildfire.net.SASLAuthentication.handle(Lorg/jivesoftware/wildf ire/Session;Lorg/dom4j/Element;)Lorg/jivesoftware/wildfire/net/SASLAuthenticatio n$Status;+229

j org.jivesoftware.wildfire.net.SocketReadingMode.authenticateClient(Lorg/dom4j/E lement;)Z+54

j org.jivesoftware.wildfire.net.BlockingReadingMode.readStream()V+99

j org.jivesoftware.wildfire.net.BlockingReadingMode.run()V+63

j org.jivesoftware.wildfire.net.SocketReader.run()V+4

j java.lang.Thread.run()V+11

v ~StubRoutines::call_stub


P R O C E S S


Java Threads: ( => current thread )

=>0x0d471e18 JavaThread “Client SR - 8988616” daemon _thread_in_native, id=4580

0x0df59d10 JavaThread “Client SR - 4302978” daemon _thread_in_native, id=5484

0x0e111ca8 JavaThread “Thread-320” daemon _thread_in_native, id=5556

0x0dc60748 JavaThread “Outgoing Server Reader” daemon _thread_in_native, id=7936

0x0dfae008 JavaThread “Client SR - 6446153” daemon _thread_in_native, id=1736

0x0e0b55b0 JavaThread “Component SR - 31616993” daemon _thread_in_native, id=7316

0x0d5c95c0 JavaThread “pool-3-thread-6” _thread_blocked, id=6948

0x0d741828 JavaThread “Client SR - 10969862” daemon _thread_in_native, id=5772

0x0dcb6e20 JavaThread “MultiThreadedHttpConnectionManager cleanup” daemon _thread_blocked, id=5820

0x0e061ca0 JavaThread “pool-3-thread-4” _thread_blocked, id=1556

0x0e2c0e40 JavaThread “pool-3-thread-3” _thread_blocked, id=3740

0x0e25d780 JavaThread “Thread-20” daemon _thread_in_native, id=1592

0x0e25d418 JavaThread “pool-3-thread-2” _thread_blocked, id=532

0x0e348808 JavaThread “pool-3-thread-1” _thread_blocked, id=268

0x0dd929b0 JavaThread “Queued Packets Processor” daemon _thread_blocked, id=4616

0x100b8b40 JavaThread “SunJsseListener1-1” _thread_blocked, id=2336

0x0e249a68 JavaThread “SunJsseListener1-0” _thread_blocked, id=3300

0x0e2498e0 JavaThread “Acceptor [SSL: ServerSocket[http://addr=0.0.0.0/0.0.0.0,port=0,localport=9091]]” _thread_in_native, id=5564

0x0dd21a00 JavaThread “SocketListener0-1” _thread_blocked, id=4796

0x1024d400 JavaThread “SocketListener0-0” _thread_blocked, id=5492

0x10151e68 JavaThread “Acceptor ServerSocket[http://addr=0.0.0.0/0.0.0.0,port=0,localport=9090]” _thread_in_native, id=6124

0x0d499310 JavaThread “SessionScavenger” daemon _thread_blocked, id=5348

0x0d6c1350 JavaThread “Rollover” daemon _thread_blocked, id=3588

0x0d60e788 JavaThread “pool-2-thread-1” _thread_blocked, id=1852

0x0dc0d148 JavaThread “SocketSendingTracker” daemon _thread_blocked, id=624

0x0d62dcb0 JavaThread “Secure Socket Listener” daemon _thread_in_native, id=5784

0x0d6d9008 JavaThread “Socket Listener at port 5222” daemon _thread_in_native, id=6016

0x0dbdc9c8 JavaThread “Socket Listener at port 5555” daemon _thread_in_native, id=6008

0x0dbdbfc8 JavaThread “Socket Listener at port 5269” daemon _thread_in_native, id=5112

0x0dbdce40 JavaThread “pool-1-thread-1” _thread_in_native, id=5156

0x0dbdccb8 JavaThread “Update Manager” daemon _thread_blocked, id=3636

0x0d66b958 JavaThread “PubSub maintenance” _thread_blocked, id=2292

0x0d58f6c8 JavaThread “Auditor” _thread_blocked, id=3432

0x0d67bdc0 JavaThread “MUC cleanup” _thread_blocked, id=5980

0x0d592618 JavaThread “Sessions cleanup” _thread_blocked, id=3492

0x0d459310 JavaThread “Thread-0” daemon _thread_blocked, id=3484

0x0cfb6598 JavaThread “Exe4JStartupThread” daemon _thread_blocked, id=3936

0x00c8c740 JavaThread “Low Memory Detector” daemon _thread_blocked, id=5900

0x00c8b398 JavaThread “CompilerThread1” daemon _thread_blocked, id=3524

0x00c8a528 JavaThread “CompilerThread0” daemon _thread_blocked, id=1500

0x00c897f8 JavaThread “AdapterThread” daemon _thread_blocked, id=3560

0x00c80b20 JavaThread “Finalizer” daemon _thread_blocked, id=3556

0x00c7feb0 JavaThread “Reference Handler” daemon _thread_blocked, id=5904

0x00b92e70 JavaThread “main” _thread_in_native, id=2604

Other Threads:

0x00c7c4f8 VMThread id=4804

0x00c8da58 WatcherThread id=6000

VM state:not at safepoint (normal execution)

VM Mutex/Monitor currently owned by a thread: None

Heap

def new generation total 4672K, used 1045K [0x04180000, 0x04690000, 0x04890000)

eden space 4160K, 12% used [0x04180000, 0x04205770, 0x04590000)

from space 512K, 100% used [0x04610000, 0x04690000, 0x04690000)

to space 512K, 0% used [0x04590000, 0x04590000, 0x04610000)

tenured generation total 40524K, used 34456K [0x04890000, 0x07023000, 0x08180000)

the space 40524K, 85% used [0x04890000, 0x06a360a0, 0x06a36200, 0x07023000)

compacting perm gen total 18688K, used 18380K [0x08180000, 0x093c0000, 0x0c180000)

the space 18688K, 98% used [0x08180000, 0x093733f8, 0x09373400, 0x093c0000)

No shared spaces configured.

Dynamic libraries:

0x00400000 - 0x0043b000 e:\wildfire\bin\wildfi~1.exe

0x7c800000 - 0x7c8c0000 C:\WINDOWS\system32\ntdll.dll

0x77e40000 - 0x77f42000 C:\WINDOWS\system32\kernel32.dll

0x77f50000 - 0x77fec000 C:\WINDOWS\system32\ADVAPI32.DLL

0x77c50000 - 0x77cef000 C:\WINDOWS\system32\RPCRT4.dll

0x77c00000 - 0x77c48000 C:\WINDOWS\system32\GDI32.dll

0x77380000 - 0x77412000 C:\WINDOWS\system32\USER32.dll

0x77ba0000 - 0x77bfa000 C:\WINDOWS\system32\msvcrt.dll

0x7c8d0000 - 0x7d0d4000 C:\WINDOWS\system32\SHELL32.DLL

0x77da0000 - 0x77df2000 C:\WINDOWS\system32\SHLWAPI.dll

0x77420000 - 0x77523000 C:\WINDOWS\WinSxS\x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6 .0.3790.1830_x-ww_7AE38CCF\comctl32.dll

0x6d8d0000 - 0x6dc6f000 e:\wildfire\jre\bin\server\jvm.dll

0x76aa0000 - 0x76acd000 C:\WINDOWS\system32\WINMM.dll

0x6d280000 - 0x6d288000 e:\wildfire\jre\bin\hpi.dll

0x76b70000 - 0x76b7b000 C:\WINDOWS\system32\PSAPI.DLL

0x76f50000 - 0x76f63000 C:\WINDOWS\system32\Secur32.dll

0x6d690000 - 0x6d69c000 e:\wildfire\jre\bin\verify.dll

0x6d300000 - 0x6d31d000 e:\wildfire\jre\bin\java.dll

0x6d6b0000 - 0x6d6bf000 e:\wildfire\jre\bin\zip.dll

0x6d4c0000 - 0x6d4d3000 E:\Wildfire\jre\bin\net.dll

0x71c00000 - 0x71c17000 C:\WINDOWS\system32\WS2_32.dll

0x71bf0000 - 0x71bf8000 C:\WINDOWS\system32\WS2HELP.dll

0x71b20000 - 0x71b61000 C:\WINDOWS\System32\mswsock.dll

0x76ed0000 - 0x76eff000 C:\WINDOWS\system32\DNSAPI.dll

0x76f70000 - 0x76f77000 C:\WINDOWS\System32\winrnr.dll

0x76f10000 - 0x76f3e000 C:\WINDOWS\system32\WLDAP32.dll

0x76f80000 - 0x76f88000 C:\WINDOWS\system32\rasadhlp.dll

0x5f270000 - 0x5f2c9000 C:\WINDOWS\system32\hnetcfg.dll

0x71ae0000 - 0x71ae8000 C:\WINDOWS\System32\wshtcpip.dll

0x68000000 - 0x6802f000 C:\WINDOWS\system32\rsaenh.dll

0x10000000 - 0x1000f000 E:\Wildfire\bin\SaslSspi.dll

0x71f60000 - 0x71f64000 C:\WINDOWS\system32\security.dll

0x76c90000 - 0x76cb7000 C:\WINDOWS\system32\msv1_0.dll

0x76cf0000 - 0x76d0a000 C:\WINDOWS\system32\iphlpapi.dll

VM Arguments:

jvm_args: -Dexe4j.isInstall4j=true -Dexe4j.moduleName=e:\wildfire\bin\wildfi~1.exe -Dexe4j.processCommFile=C:\WINDOWS\TEMP\e4j_p2352.tmp -Dexe4j.tempDir= -Xrs

java_command: <unknown>

Launcher Type: generic

Environment Variables:

PATH=C:\Program Files\Windows Resource Kits\Tools;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program Files\Executive Software\Diskeeper;C:\Program Files\Microsoft SQL Server\80\Tools\BINN;C:\PROGRA1\CA\SHARED1\SCANEN~1

OS=Windows_NT

PROCESSOR_IDENTIFIER=x86 Family 15 Model 4 Stepping 1, GenuineIntel


S Y S T E M


OS: Windows Server 2003 family Build 3790 Service Pack 1

CPU:total 2 (cores per cpu 1, threads per core 2) family 15 model 4 stepping 1, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ht

Memory: 4k page, physical 523184k(73952k free), swap 1283596k(353996k free)

vm_info: Java HotSpot™ Server VM (1.5.0_08-b03) for windows-x86, built on Jul 26 2006 00:51:44 by “java_re” with MS VC++ 6.0

Hi,

it seems to have nothing to do with the connection pool.

ntdll.dll is a Windows DLL so one may wonder if MS offers an update for it.

And I really wonder where one can get a SaslSspi.dll - my Wildfire installation does not include this file.

LG

Ok, i see.

I have updated a JRE to “update_09”, waiting for result now.

SaslSspi.dll included in Norman patch for WF 3.*, used for transparent auth with Pandion. Did you think that this dll may cause trouble?

Hi,

the interesting lines in the hs file are:

  1. Problematic frame:

  2. C http://ntdll.dll+0x327f9

C http://ntdll.dll+0x327f9

C http://ntdll.dll+0x1e85e

C http://ntdll.dll+0x1e4ef

C http://SaslSspi.dll+0x1eb2

So it seems to be either the ntdll.dll itself which causes the problem or SaslSspi.dll by calling a method in ntdll.dll.

It’'s also recommended to update the JVM to the latest version which you already did but this may not help as there were only six bugfixes for 1.5.0_09.

LG

Which version are you using v4 (dll built on aug 20) or v5 (dll built on aug 30)? (v5 got an overhaul to handle exceptions better - it was causing java to segfault - like you describe)

btw: to anyone who sees za.net.darkskies or saslsspi mentioned, please don’‘t hesitate to poke me via email, or these two threads. (i.e. I don’'t ‘‘watch’’ all the threads, all the time.)

this is not problem of service.