Smack 4.2.1-beta2-SNAPSHOT: No response received within reply timeout even when reply is received well within the specified PacketReplyTimeout

Paul_Schaub · August 13, 2017, 10:45am

Hm, good to know, thank you .

In three weeks or so I’m available to OMEMO again, until then I really appreciate how you keep searching for the issue .

cmeng · August 14, 2017, 12:54am

Hi Paul,

I compare my current source checkout from the branch 4.2 and smack-4.2.1-beta2_SNAPSHOT source, I do not see much differences between the two. Am I testing source compiled from the wrong branch 4.2 (f7ae216138ce6588377a165f484a20ed5bfe7c53)

As for your mentioned branch “There is already a fix in the current 4.2 branch (2e61e0a4d333688187aa257a65bafe6783f80519)”, is this available to the public? What is the correct link?

cmeng · August 14, 2017, 6:01am

With reference to the muc reply timeout at 100% failures. I traced into the MultiUserChat#enter(MucEnterConfiguration). Everything seems Ok including the responseFilter setup. Appreciate if someone can clarify the following:

responseFilter checks for “status=110”, however the reply is with multiple “status codes”.

Does “status” and “status codes” are irrelevant during comparison? It is just the way the log message is being displayed.
Does the collector iterate over all the received “status code” or just the first encountered?

Also I found that within the MultiUserChat constructor line #191

“final EntityFullJid from = presence.getFrom().asEntityFullJidIfPossible()”

here the presence.getFrom() may return null hence throw exception if executed null.asEntityFullJidIfPossible()

// Create a listener for all presence updates.
presenceListener = new StanzaListener() {
@Override
public void processStanza(Stanza packet) {
Presence presence = (Presence) packet;
final EntityFullJid from = presence.getFrom().asEntityFullJidIfPossible();
if (from == null) {
LOGGER.warning("Presence not from a full JID: " + presence.getFrom());
return;
}

// ==================================//

org.jivesoftware.smack.SmackException$NoResponseException: No response received within reply timeout. Timeout was 10000ms (~10s). Waited for response using: AndFilter: (StanzaTypeFilter: Presence, OrFilter: (AndFilter: (FromMatchesFilter (ignoreResourcepart): chatroom-9z2e@conference.atalk.org, MUCUserStatusCodeFilter: status=110), AndFilter: (FromMatchesFilter (full): chatroom-9z2e@conference.atalk.org/leopard, StanzaIdFilter: id=s311k-261, PresenceTypeFilter: type=error))).

at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:2 53)

Paul_Schaub · August 14, 2017, 8:28am

Sorry, I haven’t noticed that you linked the latest snapshot in the title. The fix is already in that build (unfortunately it seems to not work), so the problem is probably caused by something else.

cmeng · August 14, 2017, 3:17pm

The following is the error log that I have captured. However I am not sure how genuine is the debug log; as I have been experiencing AS sometimes just throws exception randomly

(I have reported to AS in the link below. https://issuetracker.google.com/issues/64425255 )

====== Log capture with reply time out on joining chatroom =============

08-14 23:00:48.753 A/art: art/runtime/thread.cc:1116] Throwing new exception ‘Attempt to invoke virtual method 'org.jxmpp.jid.Jid org.jivesoftware.smack.packet.Presence.getFrom()’ on a null object reference’ with unexpected pending exception: org.jivesoftware.smack.SmackException$NoResponseException: No response received within reply timeout. Timeout was 10000ms (~10s). Waited for response using: AndFilter: (StanzaTypeFilter: Presence, OrFilter: (AndFilter: (FromMatchesFilter (ignoreResourcepart): chatroom1234@conference.icrypto.com, MUCUserStatusCodeFilter: status=110), AndFilter: (FromMatchesFilter (full): chatroom1234@conference.icrypto.com/leopard, StanzaIdFilter: id=STdAm-230, PresenceTypeFilter: type=error))).

08-14 23:00:48.753 A/art: art/runtime/thread.cc:1116] at org.jivesoftware.smack.packet.Stanza org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(long) (StanzaCollector.java:253)

08-14 23:00:48.753 A/art: art/runtime/thread.cc:1116] at org.jivesoftware.smack.packet.Presence org.jivesoftware.smackx.muc.MultiUserChat.enter(org.jivesoftware.smackx.muc.Muc EnterConfiguration) (MultiUserChat.java:355)

08-14 23:00:48.753 A/art: art/runtime/thread.cc:1116] at void org.jivesoftware.smackx.muc.MultiUserChat.join(org.jivesoftware.smackx.muc.MucE nterConfiguration) (MultiUserChat.java:712)

08-14 23:00:48.753 A/art: art/runtime/thread.cc:1116] at void org.jivesoftware.smackx.muc.MultiUserChat.join(org.jxmpp.jid.parts.Resourcepart ) (MultiUserChat.java:604)

08-14 23:00:48.753 A/art: art/runtime/thread.cc:1116] at void net.java.sip.communicator.impl.protocol.jabber.ChatRoomJabberImpl.joinAs(java.l ang.String, byte[]) (ChatRoomJabberImpl.java:571)

cmeng · August 16, 2017, 4:50am

I have inserted some debug codes in both smack AbstractXMPPConnection.java and AbstractFromToMatchesFilter.java to better understand the sequence flow.

With reference to smack MultiUserChat.java#enter(MucEnterConfiguration) and the below captured log for aTalk debug messages while joining chat room.

Following are my observations and understandings. Hopefully these help the smack team to find out the root cause of the reply timeout problem during muc setup.

The initial send and receive of the disco#info are working properly without problem. The Stanza collector is able to report correctly the received reply.
Executing the “presence = connection.createStanzaCollectorAndSend(responseFilter, joinPresence).nextResultOrThrow(conf.getTimeout());”

I have checked and verified that createStanzaCollector() registered correctly and added the responseFilter to the collectors.

The #presenceInterceptor correctly receives and processes the SENT.
However the AbstractXMPPConnecor#invokeStanzaCollectorsAndNotifyRecvListeners(final Stanza) never get triggered even server has responded to the SENT .

The #presenceListener is also not being called to process the stanza (not sure if this is also expected or overridden by the collector filter setup).

Exception “Attempt to invoke virtual method 'org.jxmpp.jid.Jid org.jivesoftware.smack.packet.Presence.getFrom()” from log

If the above exception is in fact reflects the actual situation, look like StanzaCollector has triggered the exception somehow, hence failed to process further the RECV .

leading to reply timeout.

I am still unable to find the location where and how the received stanza is checked against the responseFilter before invokeStanzaCollectorsAndNotifyRecvListeners. Any help?

============ aTalk log when joining chatRoom =============

08-16 12:07:29.240 D/SMACK: SENT (0):

08-16 12:07:29.250 D/SMACK: RECV (0): http://jabber.org/network/serverinfo

08-16 12:07:29.260 W/αTalk: [6] org.jivesoftware.smack.AbstractXMPPConnection.invokeStanzaCollectorsAndNotifyRe cvListeners() Collector: IQReplyFilter: iqAndIdFilter (AndFilter: (OrFilter: (IQTypeFilter: type=error, IQTypeFilter: type=result), StanzaIdFilter: id=Vg0iy-216)), : fromFilter (OrFilter: (FromMatchesFilter (full): conference.icrypto.com))

08-16 12:07:29.260 W/αTalk: [6] org.jivesoftware.smack.filter.AbstractFromToMatchesFilter.accept() Accept Stanza: IQ Stanza (query http://jabber.org/protocol/disco#info) [to=leopard@icrypto.com/atalk,from=conference.icrypto.com,id=Vg0iy-216,type=result,]

08-16 12:07:29.270 W/αTalk: [17] org.jivesoftware.smack.filter.AbstractFromToMatchesFilter.accept() Accept Stanza: Presence Stanza [to=chatroom1234@conference.icrypto.com/leopard,id=Vg0iy-215,type=available,] ### cmeng - for presenceInterceptor

08-16 12:07:29.270 D/SMACK: SENT (0): <presence to=‘chatroom1234@conference.icrypto.com/leopard’ id='Vg0iy-215’>

08-16 12:07:29.310 D/SMACK: RECV (0):

08-16 12:07:29.310 D/SMACK: RECV (0):

08-16 12:07:39.430 A/art: art/runtime/thread.cc:1116] Throwing new exception ‘Attempt to invoke virtual method 'org.jxmpp.jid.Jid org.jivesoftware.smack.packet.Presence.getFrom()’ on a null object reference’ with unexpected pending exception: org.jivesoftware.smack.SmackException$NoResponseException: No response received within reply timeout. Timeout was 10000ms (~10s). Waited for response using: AndFilter: (StanzaTypeFilter: Presence, OrFilter: (AndFilter: (FromMatchesFilter (ignoreResourcepart): chatroom1234@conference.icrypto.com, MUCUserStatusCodeFilter: status=110), AndFilter: (FromMatchesFilter (full): chatroom1234@conference.icrypto.com/leopard, StanzaIdFilter: id=Vg0iy-215, PresenceTypeFilter: type=error))).

08-16 12:07:39.430 A/art: art/runtime/thread.cc:1116] at org.jivesoftware.smack.packet.Stanza org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(long) (StanzaCollector.java:253)

cmeng · August 16, 2017, 11:33pm

I decided to trace back previous aTalk releases to see when the reply timeout problem occurred

aTalk 0.8.1 to 0.8.4 - (smack 4.2.1-SNAPSHOT) muc setup work correctly without problem
aTalk 0.8.5 - (smack 4.2.1-beta2-SNAPSHOT) muc setup work correctly without problem
aTalk 0.8.6 - (smack 4.2.1-beta2-SNAPSHOT) muc reply timeout started

Checking on the change log, I really cannot see anything changes that can cause this problem. However I also realize that this is also the time at which I have updated the AS SDK which also shown up in debug log when aTalk starts (v0.8.6). Since the new SDK, I begin to see weird behavior during aTalk debug as reported in

https://issuetracker.google.com/issues/64425255

I now wonder the problem that I see are actually due to new AS SDK built.

===== aTalk start up with new SDK compiled code =========

08-17 07:14:45.490 I/MultiDex: VM with version 1.6.0 does not have multidex support

08-17 07:14:45.490 I/MultiDex: install

08-17 07:14:45.490 I/MultiDex: MultiDexExtractor.load(/data/app/org.atalk.android-2.apk, false)

08-17 07:14:45.500 I/MultiDex: Detected that extraction must be performed.

========================

Version: 0.8.6 - change log

Author: cmeng

Upload Date: 08/05/2017

Fix implementation error in denying OTR session setup request from buddy while OMEMO session is in progress
Implement XEP-0084 User Avatar publishing with reflection patch for Smack PepManager.publish to support old server
Harmonize operations between XEP-0084 User Avatar and XEP-0153 VCardAvatar for co-existence
When XEP-0084 is enabled, omit avatar hash in presence stanza VCardTempXUpdate x-extension
Sync main menu avatar display with user selected picture
Delay the auto software update version check on application start up
Remove all unused smiley java files from built
Remove large screen landscape contactList and chatFragment side-by-side support

Flow · August 17, 2017, 10:15am

What you really want to do is taking a thread dump while the situation is happening: How to make Java Thread Dump in Android? - Stack Overflow

That would most likely show the culprit.

cmeng · August 17, 2017, 10:42am

Finally, I have found the root cause.

aTalk has implemented XEP-0084: User Avatar; if the option is enabled, then the vcard-temp:x:update" (XEP-0153) will be sent as

.in all the stanza including the muc join room stanza i.e. without element.

However reply from server will be in short-form, and looks like smack does not like to see a single element without the attribute. This results to a reply timeout exception thrown by smack.

// ============ aTalk with XEP-0084 option enabled ==============

08-17 06:43:45.790 D/SMACK: SENT (0):

08-17 06:43:45.880 D/SMACK: RECV (0):

================== //

If I enabled the full vcard-temp:x:update" (XEP-0153) together with elements in stanza when joining room, then smack works correctly as captured in the log below.

On seeing muc with hash, aTalk tries to retrieve the avatar from MUC. However smack throws an exception again as shown in the log below.

Setting vCards / Avatars for MUC rooms | ejabberd Docs

aTalk has implemented catch, bus seems it is not captured???

try {
mVCard = vCardMgr.loadVCard((EntityBareJid) userId);
}
catch (SmackException.NoResponseException | XMPPException.XMPPErrorException
| SmackException.NotConnectedException | InterruptedException e) {
LOGGER.log(Level.WARNING, “Error while downloading VCard for: '” + userId

"’. Exception: " + e.getMessage());
}

============= aTalk muc setup ================

08-17 17:35:51.418 D/SMACK: SENT (0): 186f39da130310dbc59002608c56d1bd26abd72d</ph oto>

08-17 17:35:51.468 D/SMACK: RECV (0): 186f39da130310dbc59002608c56d1bd26abd72d</ph oto>

08-17 17:35:51.468 D/SMACK: RECV (0):

08-17 17:35:51.478 D/SMACK: SENT (0):

08-17 17:35:51.488 D/SMACK: SENT (0): Group Chat

08-17 17:35:51.908 D/SMACK: RECV (0):

08-17 17:35:51.938 D/SMACK: RECV (0):

08-17 17:35:51.958 D/SMACK: RECV (0): Group Chat

08-17 17:35:52.008 E/αTalk: [15] util.UtilActivator.uncaughtException().95 An uncaught exception occurred in thread=Thread[Smack-Cached Executor 1 (0),5,main] and message was: XMPP error reply received from chatroom1234@conference.icrypto.com: XMPPError: item-not-found - cancel

                        org.jivesoftware.smack.XMPPException$XMPPErrorException: XMPP error reply received from chatroom1234@conference.icrypto.com: XMPPError: item-not-found - cancel

at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:2 56)

at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:2 08)

at org.jivesoftware.smackx.vcardtemp.VCardManager.loadVCard(VCardManager.java:133)

at org.jivesoftware.smackx.avatar.vcardavatar.VCardAvatarManager.downloadVCard(VCa rdAvatarManager.java:186)

at org.jivesoftware.smackx.avatar.vcardavatar.VCardAvatarManager.parseContactPhoto Presence(VCardAvatarManager.java:294)

at org.jivesoftware.smackx.avatar.vcardavatar.VCardAvatarManager.access$300(VCardA vatarManager.java:45)

at org.jivesoftware.smackx.avatar.vcardavatar.VCardAvatarManager$3.processStanza(V CardAvatarManager.java:163)

at org.jivesoftware.smack.AbstractXMPPConnection$4.run(AbstractXMPPConnection.java :1202)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1120)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)

at java.lang.Thread.run(Thread.java:820)

08-17 17:35:52.028 E/αTalk: [16] util.UtilActivator.uncaughtException().95 An uncaught exception occurred in thread=Thread[Smack-Single Threaded Executor 0 (0),5,main] and message was: Attempt to invoke interface method ‘boolean java.util.Iterator.hasNext()’ on a null object reference

java.lang.NullPointerException: Attempt to invoke interface method ‘boolean java.util.Iterator.hasNext()’ on a null object reference

at org.jivesoftware.smackx.muc.MultiUserChat$3.processStanza(MultiUserChat.java:25 2)

at org.jivesoftware.smack.AbstractXMPPConnection$5.run(AbstractXMPPConnection.java :1228)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1120)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)

at java.lang.Thread.run(Thread.java:820)

cmeng · August 17, 2017, 2:18pm

aTalk implements both the XEP-0084: User Avatar and XEP-0054: vcard-temp. aTalk still keeps XEP-0054 support so as to be able to receive avatar from client that only support XEP-0054. However I am not so sure what should I do with “vcard-temp:x:update” (cf. XEP-0153); so I just follow conversations implementation.

With the observed problem, I have changed aTalk to send when XEP-0084 is enabled. Not sure if this the correct implementation, or just completely remove XEP-0153 element when XEP-0084 is enabled. Any recommendation?

with the new implementation, no more reply timeout, but I still seeing smack throws exceptions as below. In aTalk version 0.8.5 I still received the event listener.joined(from); from MultiUserChat#presenceListener (need further investigation.)

08-17 21:51:42.849 E/αTalk: [15] util.UtilActivator.uncaughtException().95 An uncaught exception occurred in thread=Thread[Smack-Single Threaded Executor 0 (0),5,main] and message was: Attempt to invoke interface method ‘boolean java.util.Iterator.hasNext()’ on a null object reference

java.lang.NullPointerException: Attempt to invoke interface method ‘boolean java.util.Iterator.hasNext()’ on a null object reference

at org.jivesoftware.smackx.muc.MultiUserChat$3.processStanza(MultiUserChat.java:25 2)

at org.jivesoftware.smack.AbstractXMPPConnection$5.run(AbstractXMPPConnection.java :1228)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1120)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588)

at java.lang.Thread.run(Thread.java:820)

cmeng · August 18, 2017, 12:28am

After some thought, I decided to completely remove and not to send when XEP-0084 is enabled in aTalk implementation. Otherwise sending an empty element can cause some xmpp clients to remove the existing avatar; and may interfere with the normal XEP-0084 operation.

Just found out that ejabberd server will always send on user behalf in stanza if the user is not defined in XEP-0054. This causes jitsi-window to remove the avatar. So need to look again the optimized implementation. Look like has to go for full XEP-0153 implementation even when XEP-0084 is enabled.

cmeng · August 25, 2017, 5:14am

Another instance where reply timeout happen even with correct reply was received within time.

While Sony XTS was in standby, it disconnected the xmpp connection.

Upon wake up and start up a chatSession. The chat session check for omemo support …, the logcat is catpured below:

Large amount of omemo data being processed before exception.

======= aTalk logcat on Sony Xeperia Tablet S (Sony XTS) ===========

08-25 12:39:42.460 D/SMACK: SENT (0):

08-25 12:39:42.470 W/αTalk: [16] org.jivesoftware.smack.AbstractXMPPConnection.callConnectionClosedOnErrorListen er()** Connection** XMPPTCPConnection[smith476@atalk.org/atalk] (0) closed with error

javax.net.ssl.SSLException: Write error: ssl=0x1f52ac0: I/O error during system call, Broken pipe

at org.apache.harmony.xnet.provider.jsse.NativeCrypto.SSL_write(Native Method)

at org.apache.harmony.xnet.provider.jsse.OpenSSLSocketImpl$SSLOutputStream.write(O penSSLSocketImpl.java:713)

08-25 12:39:51.510 D/SMACK: XMPPConnection (XMPPTCPConnection[smith476@atalk.org/atalk] (0)) will reconnect in 0

08-25 12:39:52.460 W/System.err: org.jivesoftware.smack.SmackException$NotConnectedException: The connection XMPPTCPConnection[smith476@atalk.org/atalk] (0) is no longer connected while waiting for response with IQReplyFilter: iqAndIdFilter (AndFilter: (OrFilter: (IQTypeFilter: type=error, IQTypeFilter: type=result), StanzaIdFilter: id=cZEjs-133)), : fromFilter (OrFilter: (FromMatchesFilter (full): cmeng@atalk.org))

08-25 12:40:12.980 D/SMACK: User logged (0): smith476@atalk.org:5222/atalk

08-25 12:40:13.020 D/SMACK: SENT (0): 08-25 12:40:13.010 D/SMACK: RECV (0):

08-25 12:40:13.020 D/SMACK: SENT (0): <iq id='cZEjs-144’ type=‘set’>

08-25 12:40:13.220 D/SMACK: RECV (0):

08-25 12:40:21.330 D/SMACK: SENT (0): BckyQ4ns69pHmGYCVndH3uG41d1r4l/sm8xngtt6GTtr</signedPreKeyPu blic>bxomkKsXqZstMjOSepmsgUp4iulcaCNDg0K5Y40NJtZLMi5fnCou mst3T4a5HMuV22DrtuTcePsQMLSSBQLojA==BaYfEh9 Rp5HJDsSROw4MIW84Ub1tkyEhEf1BFZ1pARtUBXg+g/MM8wAm/nf3JtRxz2i8Z29jfJFJFLUxgOpW++d2<preKey Public

…

preKeyId=‘42’>BYMYj8B0UQic6xwPqlEPdKgan3gRFmS6Yxvy+rrjMA4ABfBOWPLFT62aTCNlKE

08-25 12:40:23.020 E/αTalk: [19] impl.protocol.jabber.OperationSetBasicInstantMessagingJabberImpl.enableDisableC arbon().760 Failed to set carbon state for: smith476@atalk.org/atalk to true

org.jivesoftware.smack.SmackException$NoResponseException: No response received within reply timeout. Timeout was 10000ms (~10s). Waited for response using: IQReplyFilter: iqAndIdFilter (AndFilter: (OrFilter: (IQTypeFilter: type=error, IQTypeFilter: type=result), StanzaIdFilter: id=cZEjs-144)), : fromFilter (OrFilter: (FromMatchesFilter (full): null, FromMatchesFilter (ignoreResourcepart): smith476@atalk.org, FromMatchesFilter (full): atalk.org)).

at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:2 53)

at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:2 08)

cmeng · January 11, 2018, 10:22pm

Smack version 4.2.2 still throwing response timeout exception even the reply stanza was received within timeout period. The observation was on Note3 happen for aTalk while doing IBR account registration. It happens for both the carbon and omemo devicelist stanzas in a single test instance. As the test is not carried out at initial apk launch, I believe android inherited JIT compilation time does not contribute to the observed problem; also unlikely due to slow device long process time since the problem happen on Note3.

Another observation is carbon throws at 10s timeout (the default aTalk global setting) but message shows 50s timeout which was intentionly set for omemo devicelist request.

===== an extraction of the debug log showing only relevant stanza ===============
01-12 05:37:04.553 D/SMACK: SENT (3): <iq id='8NKO8-376' type='set'><enable xmlns='urn:xmpp:carbons:2'/></iq>
01-12 05:37:04.693 D/SMACK: RECV (3): <iq xml:lang='en' to='test2@atalk.org/atalk' from='test2@atalk.org' type='result' id='8NKO8-376'/>
01-12 05:37:06.253 D/SMACK: SENT (3): <iq to='test2@atalk.org' id='8NKO8-388' type='get'><query xmlns='http://jabber.org/protocol/disco#info' node='eu.siacs.conversations.axolotl.devicelist'></query></iq><r xmlns='urn:xmpp:sm:3'/>
01-12 05:37:06.273 D/SMACK: RECV (3): <iq xml:lang='en' to='test2@atalk.org/atalk' from='test2@atalk.org' type='result' id='8NKO8-388'><query node='eu.siacs.conversations.axolotl.devicelist' xmlns='http://jabber.org/protocol/disco#info'><identity type='registered' category='account'/></query></iq>
01-12 05:37:14.563 E/aTalk: [111] impl.protocol.jabber.OperationSetBasicInstantMessagingJabberImpl.enableDisableCarbon().796 Failed to set carbon state for: test2@atalk.org/atalk to true
                            org.jivesoftware.smack.SmackException$NoResponseException: No response received within reply timeout. Timeout was 50000ms (~50s). Waited for response using: IQReplyFilter: iqAndIdFilter (AndFilter: (OrFilter: (IQTypeFilter: type=error, IQTypeFilter: type=result), StanzaIdFilter: id=8NKO8-376)), : fromFilter (OrFilter: (FromMatchesFilter (full): null, FromMatchesFilter (ignoreResourcepart): test2@atalk.org, FromMatchesFilter (full): atalk.org)).
                                at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:253)
                                at org.jivesoftware.smack.StanzaCollector.nextResultOrThrow(StanzaCollector.java:208)
                                at org.jivesoftware.smackx.carbons.CarbonManager.setCarbonsEnabled(CarbonManager.java:311)
01-12 05:37:56.263 E/aTalk: [112] org.jivesoftware.smackx.omemo.OmemoManager.run() connectionListener.authenticated() failed to initialize OmemoManager: No response received within reply timeout. Timeout was 50000ms (~50s). Waited for response using: IQReplyFilter: iqAndIdFilter (AndFilter: (OrFilter: (IQTypeFilter: type=error, IQTypeFilter: type=result), StanzaIdFilter: id=8NKO8-388)), : fromFilter (OrFilter: (FromMatchesFilter (full): test2@atalk.org, FromMatchesFilter (full): null)).

Paul_Schaub · January 12, 2018, 10:49am

Hm… I’ll try to take a look. What makes me wonder is, that I haven’t yet been able to reproduce this in the integration tests.

The rework of smack-omemo will soon be done. Maybe you want to give it a try? The API is unfortunately no longer the same, so you have to make some changes, but maybe it pays out?
In case you want to try it, you’ll have to compile Smack yourself from https://github.com/igniterealtime/Smack/pull/177 .

Maybe the new version solves your issue

cmeng · January 12, 2018, 11:51pm

Yes, I would certainly integrated the new omemo library into aTalk but
will wait a week or two.
I just released aTalk into google playstore and need to do some final tidy
up.
By the way, do you have any omemo migration guide?

Below is a link to aTalk apk.
http://atalk.sytes.net

cmeng · January 13, 2018, 12:05am

I am still fairy new in using git. By the way how do I pull in your source
into my local storage for compilation?

Paul_Schaub · January 13, 2018, 12:02pm

I’m working on it.

I’d recommend you to do the following in a working directory of your choice:

$ git clone https://github.com/igniterealtime/Smack.git
$ git remote add vanitasvitae https://github.com/vanitasvitae/Smack

That way you cloned the main smack repo and added my for as a remote.
Then you can checkout my branch and install it into your local maven repository:

$ cd Smack
$ git fetch vanitasvitae
$ git checkout vanitasvitae/storerework
$ gradle install

I’m not sure, which build system is used for building aTalk, but in gradle I included the mavenLocal repo like this:

repositories {
     mavenLocal()
}

If you need jar files, you can find those for example in ~/.m2/repository/org/igniterealtime/smack/smack-omemo/4.2.3-SNAPSHOT/

cmeng · January 13, 2018, 12:22pm

Thanks. Will take a look next week.

cmeng · January 15, 2018, 4:49am

I have completed the initial porting to use the new omemo library and start
system testing.

Would appreciate for advice on the following:

I see that you have obsoleted OmemoManager.regenerate(). Any reason?
aTalk provides this option to user. User has used this to recovere from
incomplete omemo database setup when connect to a slow server has timeout.
Smack class FileBasedOmemoStore extends OmemoStore, whereas aTalk class
SQLiteOmemoStore extends SignalOmemoStore
It has been this way for aTalk. Do you see any problem with this?
In aTalk earlier implementation, it uses the following method when
encountered CorruptedOmemoKey in database so as to create new.
OmemoService.getInstance().buildSessionFromOmemoBundle(omemoManager,
omemoDevice, false);

I see that this is no more available. Do I need to do this with the new
implementation?

I initi the OmemoManage with the following; is this correct?
OmemoManager omemoManager = OmemoManager.getInstanceFor(connection);
When a buddy added a new device, the previous OmemoManager seems to
ignore published devicelist from this buddy. So any new message send to
this buddy will not copy to the buddy new device? When and how OmemoManager
will start cc omemo message to this new device?

aTalk has the following in AndroidOmemoService.java class to add new buddy
device. No sure if this correct?

private PEPListener buddyDeviceListUpdateListener = new PEPListener() {};

How do you want me to feedback the test comment if any?
Continue to use email or via forum - how do I name the new omemo if via
forrum?

Regards,
CM ENg

Paul_Schaub · January 15, 2018, 10:22am

Hi @cmeng!
Great to hear, that you are already testing

I find that the regenerate method is not really useful. There is no real use-case for it. Instead of throwing the whole identity away, I’d rather find a way to avoid sessions from breaking. If you really need a regenerate function, you can just manually delete all data associated to the old identity from the OmemoStore and then call OmemoManager.getInstanceFor(OmemoManager.randomDeviceId()).
I dont fully understand the question, but it should be fine. Since you use a selfmade SQL store, extending SignalOmemoStore is he right choice to go. I advise you to wrap your SQL store in a SignalCachingOmemoStore in order to improve performance though (just call signalOmemoServiceInstance.setOmemoStoreBackend(new SignalCachingOmemoStore(yourSQLStoreInstance));.
Yeah, I just realized, that a client dev doesn’t have access to such a function anymore. I’ll fix that by adding a method to the OmemoManager. In a perfect world, you wouldn’t need such function, but I agree that it is probably a good idea to expose this to the dev.
By calling that code snippet you just get an instance of the OmemoManager. To initialize it, you’ll want to call either omemoManager.initialize(), or more suitable for older devices omemoManager.initializeAsync(initCallback). See the updated documentation.
That should not have happened and shouldn’t happen anymore. As soon as a buddy adds a new device, you should get a deviceList update. If you then try to send a message, smack-omemo should instantly throw an UndecidedOmemoIdentityException due to that new device. If that doesn’t happen, please let me know!
The code you have in your AndroidOmemoService should not be needed, as the OmemoManager already has a PEPListener (see these lines).
I highly appreciate any form of feedback. The fastest way would be via XMPP (for example in the open_chat muc). If we have to exchange longer messages (eg. stanza logs), I think the forum is most suitable. You can use the naming scheme “smack-omemo (#177)” for issues