Slow loading region

Ask questions about creating worlds, using worlds, etc.
User avatar
Tess Juel
Posts: 267
Joined: Sun Sep 11, 2016 4:24 pm
Has thanked: 249 times
Been thanked: 438 times

Slow loading region

Post by Tess Juel »

I've had some annoying issues with load time recently. So I emptied my region (there was just some boxes I was going to take into inventory anyway so no big deal), waited about an hour and tried to log on. I took a minute and a half for the region to start. And yes, that's for a completely empty (4x4) region with a perfectly flat terrain.

That's not normal of course, it shouldn't take more than 10-15 seconds, but I don't know if it's just my region or if it's a grid wide issue. Does anybody else have the same problem?
Graham Mills
Posts: 1314
Joined: Sun Dec 23, 2012 2:26 pm
Has thanked: 1134 times
Been thanked: 1141 times

Re: Slow loading region

Post by Graham Mills »

Tess Juel wrote:
Sun Sep 25, 2022 8:14 pm
I've had some annoying issues with load time recently. So I emptied my region (there was just some boxes I was going to take into inventory anyway so no big deal), waited about an hour and tried to log on. I took a minute and a half for the region to start. And yes, that's for a completely empty (4x4) region with a perfectly flat terrain.

That's not normal of course, it shouldn't take more than 10-15 seconds, but I don't know if it's just my region or if it's a grid wide issue. Does anybody else have the same problem?
No. Not strictly comparable but load times much as usual for my almost empty single region (Premium) world "Security Test".
User avatar
Tess Juel
Posts: 267
Joined: Sun Sep 11, 2016 4:24 pm
Has thanked: 249 times
Been thanked: 438 times

Re: Slow loading region

Post by Tess Juel »

Graham Mills wrote:
Sun Sep 25, 2022 8:32 pm
No. Not strictly comparable but load times much as usual for my almost empty single region (Premium) world "Security Test".
Thank you! I got response from others in-world too and it seems it's only my sim that is misbehaving. Should I try to contact Ilan or just wait for him to see this post on the forum?
These users thanked the author Tess Juel for the post:
Graham Mills
Graham Mills
Posts: 1314
Joined: Sun Dec 23, 2012 2:26 pm
Has thanked: 1134 times
Been thanked: 1141 times

Re: Slow loading region

Post by Graham Mills »

Tess Juel wrote:
Sun Sep 25, 2022 8:39 pm
Graham Mills wrote:
Sun Sep 25, 2022 8:32 pm
No. Not strictly comparable but load times much as usual for my almost empty single region (Premium) world "Security Test".
Thank you! I got response from others in-world too and it seems it's only my sim that is misbehaving. Should I try to contact Ilan or just wait for him to see this post on the forum?
He's normally pretty quick but it's late where he is.
User avatar
Tess Juel
Posts: 267
Joined: Sun Sep 11, 2016 4:24 pm
Has thanked: 249 times
Been thanked: 438 times

Re: Slow loading region

Post by Tess Juel »

Graham Mills wrote:
Sun Sep 25, 2022 8:58 pm
He's normally pretty quick but it's late where he is.
Yes and I also noticed he wasn't at the community meeting. That's why I haven't sent him an IM. Even gridowners deserve a break every now and then and it's not as if it's urgent if it doesn't affect the whole grid.
These users thanked the author Tess Juel for the post (total 2):
Graham MillsIlan Tochner
User avatar
Ilan Tochner
Posts: 6504
Joined: Sun Dec 23, 2012 8:44 am
Has thanked: 4943 times
Been thanked: 4455 times
Contact:

Re: Slow loading region

Post by Ilan Tochner »

Tess, I'm traveling abroad until October 3 and only have very limited internet access. Please use area search in your world and see what is actually there. Then toggle scripts in Region debug to stop any running scripts that may still be active.

Once that's done, shut down your world. Wait 5 minutes and see how long it takes to restart.
These users thanked the author Ilan Tochner for the post:
Chris Namaste
User avatar
Tess Juel
Posts: 267
Joined: Sun Sep 11, 2016 4:24 pm
Has thanked: 249 times
Been thanked: 438 times

Re: Slow loading region

Post by Tess Juel »

Ilan Tochner wrote:
Mon Sep 26, 2022 6:55 am
Tess, I'm traveling abroad until October 3 and only have very limited internet access.
Hope you have a lovely trip! :-)

And as I said, there's no rush. I'm only using the region as a sandbox at the moment and I'm not building, just sorting my inventory.
Please use area search in your world and see what is actually there. Then toggle scripts in Region debug to stop any running scripts that may still be active.

Once that's done, shut down your world. Wait 5 minutes and see how long it takes to restart.
I've double checked the content, there really are no objects there and not a single script running. However, the performance has been steadily improving since I posted my first message here. I got a friend to try logging in there right after I posted and he got there in 50 seconds. Later yesterday I timed it to ten and when I logged on today to check as you told me, it was down to only two seconds. Hopefully that means it was just a temporary glitch although start time has been rather variable for a while.

One question: does the content in the inventories of objects affect start time? As I said, I've been sorting my inventory recently so I had several boxes rezzed. There were never more than a few hundred of them rezzed but some of them were quite full. Can that be the explanation?

If so, how long does it take an overloaded simulator to recover? It had been empty for over an hour before I did the minute-and-a-half timed test. Was that too short a time?
User avatar
Ilan Tochner
Posts: 6504
Joined: Sun Dec 23, 2012 8:44 am
Has thanked: 4943 times
Been thanked: 4455 times
Contact:

Re: Slow loading region

Post by Ilan Tochner »

Kitely has made a lot of changes and optimizations to the world startup process but there are still various tasks that need to complete before our system can allow avatars to start entering the world. One of those takes is making sure that all the world's contents are present on the server which was assigned to host the world. If that server doesn't already have that data cached then it has to download it from our storage system first.

The contents of objects are stored as XML and downloading those can take as much time as downloading the contents if they are rezzed. This means that a world with a single rezzed box that contains thousands of objects can slow down that world's startup time when the world is started on a new server.

Worlds are shut down when they are empty. Once that process is complete the next world startup time will depend on the amount of content in your world, the number of scripts, whether or not you changed world size or world physics engine from the last time the world was shut down, and on the cache state of the server that was assigned to host your world.
These users thanked the author Ilan Tochner for the post (total 2):
Ada RadiusChris Namaste
User avatar
Tess Juel
Posts: 267
Joined: Sun Sep 11, 2016 4:24 pm
Has thanked: 249 times
Been thanked: 438 times

Re: Slow loading region

Post by Tess Juel »

Ilan Tochner wrote:
Mon Sep 26, 2022 12:02 pm
The contents of objects are stored as XML and downloading those can take as much time as downloading the contents if they are rezzed. This means that a world with a single rezzed box that contains thousands of objects can slow down that world's startup time when the world is started on a new server.
I see. That's good to know and it explains everything except why the simulator remained slow to start long after I had removed all the content. Do you know why that happened?

Two more important question too, I don't think I'm the only one who needs to know this:

+ The XML file not only lists content inside the rezzed objects but also content inside content inside the rezzed objects, right? Most of the actual content were in nested boxes, buried at least three or four - sometimes eight or more - levels deep.

+ What data does the XML file include? Only the essential names and UUIDs or everything? I mean, imagine a world with only a few thousand innocent looking prims but each of them filled up with a hundred big linksets of high poly, highly textured and scripted meshes. What would happen then?
User avatar
Oren Hurvitz
Posts: 361
Joined: Sun Dec 23, 2012 8:42 am
Has thanked: 19 times
Been thanked: 499 times
Contact:

Re: Slow loading region

Post by Oren Hurvitz »

Hi Tess,

Ilan's internet access is spotty right now so he asked me to help.

In order to make worlds start and run faster, we maintain a list of all the assets used in the world, including in boxes and nested boxes. When the world is started, we download all these assets into the server that is hosting the world, so that if OpenSim requests them then we'll be able to provide them instantly and OpenSim wouldn't have to wait. We've measured, and this way of getting assets is much faster than if we'd waited until OpenSim requested the assets in order to download them.

If a world has a lot of assets then this download can take a while. I'm not sure when exactly you measured your startup time, but I looked at one world start on Sep 25 and it took 1:02 minutes, of which 45 seconds was spent downloading assets. At that time, the world was using 142,570 assets totaling 13 GB (that's a lot...).

When you start a world several times in a row there's a good chance that it will start faster after the first time, because often it will run on the same server as before, and that server already has the assets cached so it doesn't need to download them again. But if the world happens to end up on a different server then we have to download the assets again.

The reason that you didn't see an immediately change in startup time after deleting assets is that we only update the list of assets in a world once a day. We do this at the same time that we update the World Map, which we also do once a day. In your case, on Sep. 26 the list of assets in your world dropped to 3 assets totaling 198 KB.

So yeah, we've got lots of systems going on to improve performance, and many of them work in ways that users can't see in the World Page, but I hope that this was helpful in understanding how assets affect world startup time.
These users thanked the author Oren Hurvitz for the post (total 3):
Ilan TochnerTess JuelChris Namaste
Post Reply