Downloaded RAW terrain files are corrupt?

[Dale Innis]

I tried downloading the RAW file for the terrain of my Kitely sim, and got a file that's 851,968 bytes in size, and no program I've pointed at it has been able to open. Looking at it in a hex editor, it has sequences of 00 00 00 00 FF FF FF FF inserted very often, like every five to nine bytes, variably.

I downloaded one of the standard RAW files from the site, and it was as expected 262,144 bytes (256 x 256 x 4), and in the hex editor it's clearly just a series of 4-byte integers, as expected.

I asked a friend to download the RAW file one of her Kitely regions and send it to me, and it's also 851,968 bytes, with constant 00 00 00 00 FF FF FF FF sequences in it.

Is it possible that there's something wrong with terrain RAW file download in the current Kitely? Does anyone else get a different result, or alternately understand what the result means?

Thanks!

(P.S. I want to download the RAW terrain so that I can do clever things to it using tools and programs and things, and then re-upload it as modified.)

[Ilan Tochner]

Hi Dale,

All RAW file operations in OpenSim are very sensitive to network noise as they are transferred to and from the viewer over an unreliable UDP protocol. This can often result in corrupt RAW file transfers.

Have you tried getting the RAW file by exporing your world to an OAR file then unzipping the OAR to get to the files it contains?

[Dale Innis]

Ah, an interesting thought! Funny both downloads would be exactly the same size, if that's the problem, but who knows!

I will try the OAR idea.

Is RAW terrain file upload also over UDP with no reliable protocol on top of it? That could be disastrous!

[Dale Innis]

Yes, RAW file uploads are done over UDP as well. That is why they can easily break if you try to upload a RAW file to a sim that is currently experiencing network load.

Have you tried this with different viewers? Viewers sometime differ in how they implement various network-related operations.

[Graham Mills]

Don't forget you can code basic terrain operations using OSSL so you could save the data as notecard text inworld and reconstitute it offline.

Scratch code: viewtopic.php?f=26&t=2408#p22150

[Dale Innis]

Yes, RAW file uploads are done over UDP as well. That is why they can easily break if you try to upload a RAW file to a sim that is currently experiencing network load.

Have you tried this with different viewers? Viewers sometime differ in how they implement various network-related operations.

Is it at least CRC checked on upload, or something? I mean, file transfer over UDP without some kind of error-checking is madness!

I think we've now tried it (the download) with both FS and Singularity, on two different sims, and from two different continents. So it seems oddly consistent!

[Dale Innis]

Don't forget you can code basic terrain operations using OSSL so you could save the data as notecard text inworld and reconstitute it offline.

Scratch code: viewtopic.php?f=26&t=2408#p22150

Woot! I was not aware of the OsXetTerrainHeight() functions; that is wonderful! I assume they do not have the wonderful "do completely random things to lots of nearby other terrain points" side-effects that the llModifyTerrain ones do?

I will begin playing with that at once; thanks much!

[Graham Mills]

It was adequate for my purposes but Kayaker waxes eloquently on the limitations in a notecard that comes with one of his terrain products so I doubt it's 100% glitch-immune.

[Dale Innis]

Don't forget you can code basic terrain operations using OSSL so you could save the data as notecard text inworld and reconstitute it offline.

Scratch code: viewtopic.php?f=26&t=2408#p22150

Just to follow up, that worked perfectly! My "Riverbottom Smoother" script:

Code: Select all

// This smooths the terrain of river bottoms (and anything else with a low elevation so BEWARE).
// Requires Estate Manager, or equivalent High Risk permissions

float HIGHEST = 14.0;  // The script will touch no points higher than this
float TOWARDS = 9.0;  // The script will move all points lower than HIGHEST toward this height
float FRACTION = 0.9;  // Set to 1.0 to make the bottoms completely flat, or lower numbers to keep some bumps
integer HALFSIDE = 63;  // The script will act on a square this many meters N, E, S, and W of the prim's location

default
{
    state_entry()
    {
        llSay(0, "Script running");
    }
    
    touch_start(integer n) {
        vector v = llGetPos();
        integer i = llFloor(0.5+v.x);
        integer j = llFloor(0.5+v.y);
        integer x;
        integer y;
        for (x=i-HALFSIDE; x<=i+HALFSIDE; x++) {
            for (y=j-HALFSIDE; y<=j+HALFSIDE; y++) {
                float h = osGetTerrainHeight(x,y);
                if (h<=HIGHEST) {
                    float newh = h + FRACTION * (TOWARDS - h);
                    // llOwnerSay("Would set "+(string)x+", "+(string)y+" to "+(string)newh);
                    osSetTerrainHeight(x,y,newh);
                }
            }
        }
        osTerrainFlush();
        llOwnerSay("Done");
    }
}