Interesting script behavior with huge implications

Creating scripts
User avatar
RezMela Apps
Posts: 18
Joined: Sun May 05, 2019 8:07 pm
Has thanked: 35 times
Been thanked: 29 times

Interesting script behavior with huge implications

Post by RezMela Apps »

:o Hello everyone,
This is an intriguing puzzle. Can you help us figure out the underlying likely cause for the difference in behavior of the script below (it's only a few lines)?
The implications are deep for applications using lists (which is a basic data structure).


How to reproduce?:
1. Put the mysterious script below in a prim
2. Touch the prim to populate the list l
3. and Restart region under two different conditions
condition 1: size = 10 //size of list
condition 2: size = 10000 //size of list
4. Note time of region restarted.

Observation:
The set text in state_entry shows the time the script started.
With a size of 10000, the time displayed is that of the current region restart (that is the script got reset),
but with a size of 10, time of region restart doesn't change (that is the script was not reset) even if the region was restarted.

In short, the script resets depending on the size of the list. This is a critical issue for data driven applications that run on Kitely. We are trying to hard to push the envelope a little bit further. I know there are many talented scripters on this forum, your help and insights will be deeply appreciated.


Intriguing script follows

Code: Select all

list l = [];

default
{
    state_entry()
    {
        llSetText("Started at " + llGetSubString(llGetTimestamp(), 11, 18), <1, 0, 0>, 1);
    }
    touch_start(integer n) {
        integer size = 10000; //10000 resets script, 10 does not
        integer i;
        for (i = 0; i < size; i++) {
            l += [ "sdjkahfgfdaskjghfadsgkhj", "kljhsfdahkjldfshlkjhfdslhlfdsh", 234987432 ];
        }
        llOwnerSay("done");
    }
}

Ramesh
Last edited by RezMela Apps on Wed Sep 02, 2020 6:50 pm, edited 2 times in total.
These users thanked the author RezMela Apps for the post (total 2):
John MelaIlan Tochner
User avatar
John Mela
Posts: 91
Joined: Tue Feb 04, 2014 9:50 pm
Has thanked: 139 times
Been thanked: 127 times
Contact:

Re: Interesting script behavior with huge implications

Post by John Mela »

As far as I've been able to determine, the issue is that scripts are reset on region restart if their data occupies above a certain amount of RAM. Exactly how much that is would be time-consuming to determine, and would be of little use in practice because llGetUsedMemory() etc are not currently implemented in OpenSim. Obviously, real scripts that are using this amount of memory are likely to be doing so in a variety of different variables rather than a single list, and to measure the size of used memory at run-time in such an application would be a messy job.

It goes without saying that this issue limits greatly the capabilities of the affected scripts.
These users thanked the author John Mela for the post (total 2):
RezMela AppsChris Namaste
User avatar
Oren Hurvitz
Posts: 361
Joined: Sun Dec 23, 2012 8:42 am
Has thanked: 19 times
Been thanked: 499 times
Contact:

Re: Interesting script behavior with huge implications

Post by Oren Hurvitz »

OpenSim has a limit on the maximum of script state: it's 0.5 MB. This particular script, when it creates 10,000 strings, uses 3 MB.

It looks like OpenSim doesn't enforce this limit at runtime (a potential problem), but does enforce it when the region is restarted.
User avatar
Oren Hurvitz
Posts: 361
Joined: Sun Dec 23, 2012 8:42 am
Has thanked: 19 times
Been thanked: 499 times
Contact:

Re: Interesting script behavior with huge implications

Post by Oren Hurvitz »

If you need to store a lot of data then perhaps using Notecards would work.
These users thanked the author Oren Hurvitz for the post (total 3):
RezMela AppsJohn MelaGraham Mills
User avatar
John Mela
Posts: 91
Joined: Tue Feb 04, 2014 9:50 pm
Has thanked: 139 times
Been thanked: 127 times
Contact:

Re: Interesting script behavior with huge implications

Post by John Mela »

Thanks, Oren.

Unfortunately, the data is very dynamic, not at all suitable for serialised notecard storage. And besides, I don't believe it's possible to create a notecard without first storing that data in RAM in its entirety.

Would there be any possibility of increasing that limit? 500KB is really not much data these days for any kind of serious application.

Also, would it be possible for a script to monitor its RAM usage, such as with llGetUsedMemory() on SL?
These users thanked the author John Mela for the post (total 2):
Ramesh RamlollChris Namaste
User avatar
RezMela Apps
Posts: 18
Joined: Sun May 05, 2019 8:07 pm
Has thanked: 35 times
Been thanked: 29 times

Re: Interesting script behavior with huge implications

Post by RezMela Apps »

@Oren, is there a document somewhere where this limitation is described
Oren: 'OpenSim has a limit on the maximum of script state: it's 0.5 MB'
I want to understand the why of this constraint.

I tried googling for info but could not find anything.

Ramesh
Graham Mills
Posts: 1314
Joined: Sun Dec 23, 2012 2:26 pm
Has thanked: 1134 times
Been thanked: 1142 times

Re: Interesting script behavior with huge implications

Post by Graham Mills »

John Mela wrote:
Thu Sep 03, 2020 3:52 pm
Thanks, Oren.

Unfortunately, the data is very dynamic, not at all suitable for serialised notecard storage. And besides, I don't believe it's possible to create a notecard without first storing that data in RAM in its entirety.
Presumably it's script-dependent but I'm reasonably sure you can have the data for a single notecard in RAM before writing it to the (finite-sized and hence possibly one of several) notecard?
User avatar
Oren Hurvitz
Posts: 361
Joined: Sun Dec 23, 2012 8:42 am
Has thanked: 19 times
Been thanked: 499 times
Contact:

Re: Interesting script behavior with huge implications

Post by Oren Hurvitz »

John Mela wrote:
Thu Sep 03, 2020 3:52 pm
Would there be any possibility of increasing that limit? 500KB is really not much data these days for any kind of serious application.

Also, would it be possible for a script to monitor its RAM usage, such as with llGetUsedMemory() on SL?
I've talked this over with Ilan, and we decided to increase the limit to 5 MB. That's going to happen in the next few days.

Please note that if you take advantage of this increased capacity then you also increase the risk of encountering script problems:
1) If you have many scripts that use a lot of memory then you might run out of memory in OpenSim (I'm talking about the 1 GB to 4 GB of RAM that OpenSim gets).
2) If your system (RezMeLa) actually uses this much memory then it probably also creates a very large number of prims, and it might time out when creating or deleting them.

Regarding implementing llGetUsedMemory(): that's not something that we're going to do, sorry. It's a lot of work for something that's rarely used, and there are other priorities that we want to work on.
These users thanked the author Oren Hurvitz for the post (total 4):
Ramesh RamlollShandon LoringJohn MelaChris Namaste
User avatar
Oren Hurvitz
Posts: 361
Joined: Sun Dec 23, 2012 8:42 am
Has thanked: 19 times
Been thanked: 499 times
Contact:

Re: Interesting script behavior with huge implications

Post by Oren Hurvitz »

RezMela Apps wrote:
Thu Sep 03, 2020 5:43 pm
@Oren, is there a document somewhere where this limitation is described
Oren: 'OpenSim has a limit on the maximum of script state: it's 0.5 MB'
It's not documented, I just saw this in the code. It didn't even have a constant...

The limit is actually not in memory, but instead it's checked when the script's state is loaded from disk. I.e., it's the size of the disk file; not the size of something in RAM.
These users thanked the author Oren Hurvitz for the post (total 3):
Ramesh RamlollSelby EvansChris Namaste
User avatar
Ramesh Ramloll
Posts: 168
Joined: Sun Jan 13, 2013 5:16 pm
Has thanked: 81 times
Been thanked: 164 times

Re: Interesting script behavior with huge implications

Post by Ramesh Ramloll »

@Oren @Ilan, you guys are awesome. Thanks for responding so fast. We will keep you posted after testing. We are really happy about this.
These users thanked the author Ramesh Ramloll for the post (total 2):
Ilan TochnerChris Namaste
Post Reply