Openfire.lobs file size

Our small Openfire + Spark setup with 30 users and embedded db produces this huge openfire.lobs file.

I’ve tried searching for lobs file on the forum but cannot fine any suggestions what is it for and why is it so big comparing to the openfire.script file.

Can anyone shed some light on this matter ?

C:\Program Files (x86)\Openfire\embedded-db>dir

Volume in drive C is System

Volume Serial Number is C4E5-A493

Directory of C:\Program Files (x86)\Openfire\embedded-db

19/04/2017 00:03 .

19/04/2017 00:03 …

19/04/2017 00:35 16 openfire.lck

18/04/2017 22:03 19,970,326,528 openfire.lobs

19/04/2017 12:45 465,552 openfire.log

19/04/2017 00:03 105 openfire.properties

19/04/2017 00:00 56,809,981 openfire.script

19/04/2017 00:02 openfire.tmp

5 File(s) 20,027,602,182 bytes

3 Dir(s) 7,667,118,080 bytes free

No reply after 10 days ? What is the purpose of this forum ?!

This is not a commercial software and there is no official support. Everyone here is volunteer helping the community if possible. I have seen your question, but i have no clue what this file is. Maybe you can stop Openfire and try moving it out and then see if Openfire works correctly (maybe do full backup first).

2 Likes

I’m guessing that this file contains binary data of sorts, or perhaps the RRD tool database that’s used in the monitoring plugin. I’m not aware of any problems related to this - keep an eye on the file size, but unless it exponentially starts growing, there’s nothing to worry about.

Hi,

I’m sorry for sounding rude. I’ve searched for any info about this file format and its only mentioned on Oracle’s website that its just another format of database and that it can hold anything, including multimedia files.

10 days later and the files are still pretty much the same - LOBS file still sitting at 20 Gigabytes while main DB is 59 Megabytes.

C:\Program Files (x86)\Openfire\embedded-db>dir

Volume in drive C is System

Volume Serial Number is C4E5-A493

Directory of C:\Program Files (x86)\Openfire\embedded-db

28/04/2017 00:03 .

28/04/2017 00:03 …

28/04/2017 00:35 16 openfire.lck

18/04/2017 22:03 19,970,326,528 openfire.lobs

28/04/2017 11:13 273,676 openfire.log

28/04/2017 00:03 105 openfire.properties

28/04/2017 00:00 59,324,404 openfire.script

28/04/2017 00:02 openfire.tmp

5 File(s) 20,029,924,729 bytes

3 Dir(s) 23,633,252,352 bytes free

I have Monitoring plugin installed and enabled logging on my test server, which is used very rarely to send a few test messages. In the embedded-db folder i don’t have openfire.lobs, but i do have openfire.log which seems to contain messages. When server is offline it starts with

COMMIT

/C8/SET SCHEMA PUBLIC

DELETE FROM OFRRDS WHERE ID=‘sessions’

INSERT INTO OFRRDS VALUES(‘sessions’,1

So it does look like RRD table to save conversations. I have installed Monitoring plugin not so long ago, so maybe lobs was some extension in the past. Mine logs file is at 10 MB with a very rare usage. So i can see how this can grow to gigs on a production server (that’s why we don’t have it enabled on our production server).

Getting back to this, it seems that lobs file is created when adding blobs objects to database, but it should be reused later. Have found a thread that it doesn’t shrink Delete or shrink lobs file in a HSQLDB database - Stack Overflow
Maybe it is fixed in the latest HSQLDB versions, or maybe we can change how embedded-db is created.

We are using latest 2.4.1 version at the moment. 2.5.0 is only planned at some point in 2019 maybe.

As i haven’t seen this file on my old production server, maybe this started with recent HSQLB versions and only when new database is created.

Filed [OF-1756] - Ignite Realtime Jira

A few more bits from HyperSQL Performance

HSQLDB is the only SQL open source database that supports a dedicated LOB store. Blobs and clobs can be very large and benefit from a separate store that avoids mixing their data with row data which is not too large. Internal database tables are used for the LOB catalog. Therefore each access to a LOB has the overhead of catalog lookup. This overhead is justified when the stored LOBs are large. HSQLDB supports long VARCHAR and VARBINARY columns that can be used instead of CLOB and BLOB especially when the average lob size is below 32 KB. These types do not have the LOB catalog overhead.

Related ticket on HSQLDB bug tracker https://sourceforge.net/p/hsqldb/feature-requests/343/

Reply from HSQLDB developer:

Check the .script file of the database. Any CREATE TABLE statement in this file that contains CLOB or BLOB will result in the creation of the .lobs file. You need to change the CREATE TABLE statements in your app to use VARCHAR(max_size) and VARBINARY(max_size) instead of the type you are currently using for those CLOB and BLOB columns.

To change an existing database, use

ALTER TABLE thetable ALTER COLUMN lobcolumn SET DATA TYPE VARCHAR(max_size) [or VARBINARY(max_size) ]

Hi There,

My openfire is filling up my server root and not Openfire is not starting anymore. No space left.
After investigating a bit, I found that my embedded DB is now a abit over 2.6GB… Openfire 4.6.0 running happily since 3 weeks with 8 users and not much traffic. Monitoring plugin installed.

How can this be ?
After looking closely, I found as well a openfire.lobs file with size of 2.6GB

Is this normal ? I don’t think so.
As Openfire is not starting what & how can I change/reduce things ?
System is running on Debian.

Any help VERY much appreciated.
Thank you

It could be that Openfire’s embedded db schema need to be changed to avoid using LOB/CLOB types. But i am not sure about this and as embedded db is not meant to be used in production developers probably don’t see this as a priority. So i can’t suggest any real solution other than:

Switch to some other db instead
Reserve more space for Openfire. Even with external db it will grow if you want to use Monitoring (not just messages, but stats round robin db).

The Round Robin Databases are used because they have a fixed size. They should not grow over time. If they do, then something might be wrong

I haven’t used Monitoring in production (i think i tried, it grew huge fast and i got rid of it as we didn’t really need to save messages and stats were not that worthy). And my test server was not enough to see, but i don’t remember db starting like 3 GB in the beginning and then not changing. It is always growing a bit by bit from zero. I blew up my test server recently, so can’t check now. I remember seeing tons and tons of gibberish data (rrd probably) in openfire.script file along with a small amount of real data (tables, users, messages).

Thank you for your replies.
I found my problem in regards to the issue that the partion was full. It was unrelated to Openfire. But in regards to the size of the database, I am still astonished of the size of 2.6GB in 4 weeks with just 8 users and not much traffic. Will keep n eye on that.

Besides that, some questions to the monitoring plugin:
@guus : the RRD will have a fixed size after some time ? Do I understand that correct ? Otherwise it would be good if one could set the time to collect data.

2nd question: embedded database, good for production or as Wroot mentioned better an external db ?

The size of the RRD should be fixed at the time of creation, I think. When I check the content of the corresponding database tables in our instance of Openfire (using Postgres), I get this:

SELECT id, updatedDate, length(bytes) FROM ofRRDs;

id	updateddate	length
sessions	1616083727247	18596
server_bytes	1616083727255	36848
muc_occupants	1616083727261	18596
proxyTransferRate	1616083727267	18596
conversations	1616083727272	18596
muc_traffic	1616083727279	36848
packet_count	1616083727285	18596
muc_rooms	1616083727290	18596
server_sessions	1616083727301	18596
muc_users	1616083727307	18596

That seems to indicate that they are indeed of a fixed size (this server has been running for years).

What you are looking at is the size on disc of how the embedded database operates. Maybe this is significantly different from the size as stored in the database, for example because it keeps track of changes in a transaction log or something like that?

My advice would be to query the database (the Openfire dbaccess plugin can help) to see if the data size in the tables corresponds with the file size of the DBMS on disc.