We have Poltergeists!

Ask questions about creating worlds, using worlds, etc.
User avatar
Ilan Tochner
Posts: 6503
Joined: Sun Dec 23, 2012 8:44 am
Has thanked: 4942 times
Been thanked: 4454 times
Contact:

Re: We have Poltergeists!

Post by Ilan Tochner »

If they belong to two different user accounts then not without using the Export World function, which filters out content based on exportability. What you could normally do is to use the Copy World function in the My Worlds page which requires less permissions when copying objects. See: https://www.kitely.com/virtual-world-ne ... rmissions/.

The problem is that you can't create a new Mega World now as we've discontinued that offering so any duplication would need to be done by us on a world we run in our test environment. That's why we need a specific case where there are differences between two consecutive world activations. We can then compare the different world state files between them to find the missing objects/assets and then hunt down information about them in our server logs. Even a single object that disappears between two consecutive activations (where none of you have made any manual changes) could help.
User avatar
Angelic Helling-Masala
Posts: 10
Joined: Sat Oct 24, 2020 10:54 pm
Has thanked: 1 time
Been thanked: 1 time

Re: We have Poltergeists!

Post by Angelic Helling-Masala »

We purchased a second world before the promotion ran out with the expectation of using it for another build after we finished with Tattered Sails. So no additional creation of a world is needed ... we already have it. It is blank at the moment. We are willing to offer that as a temporary test world.

Considering that you guys literally own the code, I'd think that moving or loading anything is not only possible but also doable. Harkening back to the times that Linden Labs had to move the content of entire sims to a new region when the old region hard broke. I'd think that capability is still within Open Sim.
User avatar
Ilan Tochner
Posts: 6503
Joined: Sun Dec 23, 2012 8:44 am
Has thanked: 4942 times
Been thanked: 4454 times
Contact:

Re: We have Poltergeists!

Post by Ilan Tochner »

The Tattered Sails account contains a single Mega World so we can't just copy this world into another existing Mega World inside the same Kitely account.

Copying worlds between different accounts requires copy+transfer permissions for the relevant objects. It's not a question of whether code can be changed to ignore that, it's a question of whether the people who duplicate the objects by copying a world have the permissions to do so (permissions meaning license from the relevant content creators).

In any case, I've discussed this with Oren and we'll create an additional temporary Mega World in your Tattered Sails account and manually duplicate the existing world into it so nothing is lost and no permissions are changed. We'll do so tomorrow and let you know once that's done. We'll then need you to provide us with the information I've asked for so that we can pin down the cause of the issue. Until we tell you otherwise, we'll need you to not take / take copy / delete / modify or move anything from the test world or change anything else inside it (including rezzing new objects) in order to ensure that no user action is responsible for the objects going missing. If the world still changes between world activations then we'll need to know exactly what changed and where it is/was located. I recommend only a single user account run this test in order to reduce noise in the experiment.
These users thanked the author Ilan Tochner for the post (total 2):
Selby EvansChris Namaste
User avatar
Angelic Helling-Masala
Posts: 10
Joined: Sat Oct 24, 2020 10:54 pm
Has thanked: 1 time
Been thanked: 1 time

Re: We have Poltergeists!

Post by Angelic Helling-Masala »

Sounds like we have a plan then. We'll keep everyone out of the test region unless requested by your team.

When you have a moment, please IM or note me what information you need from us and I'll work on it.
These users thanked the author Angelic Helling-Masala for the post:
Chris Namaste
User avatar
Oren Hurvitz
Posts: 361
Joined: Sun Dec 23, 2012 8:42 am
Has thanked: 19 times
Been thanked: 499 times
Contact:

Re: We have Poltergeists!

Post by Oren Hurvitz »

Hi Angelic,

I am currently creating a copy of your world. I'll let you know when it's ready.

However, I am now thinking that there's a better way to go about solving this problem. Since the mangled objects were uploaded using the viewer, my current theory is that something in the upload process is going wrong. One of Kitely, OpenSim or Firestorm are causing problems when uploading objects using the viewer. I can debug all three of these code bases at the same time, so I'm confident that I can figure out what's happening -- I just need to be able to reproduce the problem.

I think that the upload process results in objects that appear correctly in the viewer, but are stored incorrectly in the OpenSim database. That would explain why you only see the problems later: they would only be visible once the viewer loads the objects from the database. You can test this theory by uploading an object; stopping the world (click "Stop" in the World Page); and then re-entering it.

I would like to try this myself. Can you send me one (or several) of the objects that you're trying to upload using the viewer? I'll send you an email so that you don't have to post them here.
These users thanked the author Oren Hurvitz for the post (total 3):
Dot MatrixSelby EvansChris Namaste
User avatar
Oren Hurvitz
Posts: 361
Joined: Sun Dec 23, 2012 8:42 am
Has thanked: 19 times
Been thanked: 499 times
Contact:

Re: We have Poltergeists!

Post by Oren Hurvitz »

One more thing: I've noticed that your team has been visiting Tattered Sails using three versions of Firestorm: 6.3.9, 6.4.9 and 6.4.12. I wonder if the problem only occurs when objects are uploaded in some of these versions of Firestorm, but not in others?
These users thanked the author Oren Hurvitz for the post (total 3):
Dot MatrixSelby EvansChris Namaste
User avatar
Angelic Helling-Masala
Posts: 10
Joined: Sat Oct 24, 2020 10:54 pm
Has thanked: 1 time
Been thanked: 1 time

Re: We have Poltergeists!

Post by Angelic Helling-Masala »

We were speculating last night about just that thing ... some of our uploads were going wrong somehow and causing issues when the region spins back up after sleeping. Unfortunately, we do not know exactly what objects broke with the current roll back, nor which objects were messed up even before the rollback since the object names were changed to "object".

It might be a good idea to load a backup from 9 or 10 dec on the test world to look and data mine the broken objects directly. Maybe potentially we can get the UUIDs of those objects and backtrace to earlier backups?
User avatar
Ilan Tochner
Posts: 6503
Joined: Sun Dec 23, 2012 8:44 am
Has thanked: 4942 times
Been thanked: 4454 times
Contact:

Re: We have Poltergeists!

Post by Ilan Tochner »

As I've explained in my email to you, when we restore a world to a previous automatic backup we lose all the backups that we had stored for that world from later dates. As a result, when we restored your world to the backup from December 7 we lost all the backups we had saved after that date, which means that we now only have backups from November 30 until December 7 and from December 13 (i.e. backups that were created after the world was restored to the backup from December 7).

That said, I think going back further in time will not help track down the issue. But you telling us that affected objects change their name back to "object" is very important information as that can help us track down the issue. This leads us to a new line of investigation.

Please do the following:
1) Upload an object that previously broke using the Firestorm version you used when you last uploaded it.
2) Check that it looks okay then restart the world AND your viewer and see if that object changed.

Repeat this for different objects that broke and keep notes of which ones reproduce the problem. Then try this test again with Firestorm 6.0.2. Does this problem happen with those object when uploading them with that viewer version as well?

Once you've completed those tests please repeat them with the latest Firestorm release. Does it happen with it as well?

Please email us all that information, including the files you uploaded in order to reproduce that problem.
These users thanked the author Ilan Tochner for the post (total 2):
Graham MillsChris Namaste
User avatar
Angelic Helling-Masala
Posts: 10
Joined: Sat Oct 24, 2020 10:54 pm
Has thanked: 1 time
Been thanked: 1 time

Re: We have Poltergeists!

Post by Angelic Helling-Masala »

Thats kinda what I was trying to say ... we do not know which objects were broken because they were renamed to "objects". So far, nothing has broke since the last rollback, which in itself could leads to all sorts of guesses as to why stuff broke in the first place. Too much has changed and too many items uploaded in that time span to remember which ones we actually uploaded.

Hmm ... I wonder though if uploading actions are logged locally on our machines. I'll check and see ...

EDIT: Duh. Silly me. Uploaded objects should still be in our inventories. We'll check.
User avatar
Ilan Tochner
Posts: 6503
Joined: Sun Dec 23, 2012 8:44 am
Has thanked: 4942 times
Been thanked: 4454 times
Contact:

Re: We have Poltergeists!

Post by Ilan Tochner »

Please do, go inworld, try to find things that are broken now and reupload them. The goal is to try to reproduce the problem from the current world state, not to investigate what broke it previously. Also, please remember to restart both the viewer and the world after you've uploaded those assets as we're trying to determine if some viewer-side or server-side caching could be involved with the problem by causing things to look right on first upload but broken after they are no longer cached.
Post Reply