Overzelous HTTP request filter

Ask technical support questions about other topics
User avatar
Kayaker Magic
Posts: 354
Joined: Sun Dec 01, 2013 8:40 am
Has thanked: 52 times
Been thanked: 393 times

Overzelous HTTP request filter

Post by Kayaker Magic »

I use llRequestURL to get the address of a prim, so I can send messages to it from outside. In Kitely I can test that by putting a long URL in my browser, and see my prim react to it.
HOWEVER, if I send the message from the server at my hosting service, I get an error message:
[10:02] llHttpRequest: Request to http://ec2-54-177-102-174.us-west-1.com ... 0to%20race! disallowed by filter
I can cut the URL out of that error, paste it into my browser and it works, it gets past the filter. (Kid's don't try this at home, the address is temporary and changes all the time, also doesn't work when my world is not up).
I can run the same script in a prim in SL and IW and OSgrid, even on the OpenSim server running in my barn, and it works. The only place this does not work is Kitely. Why does Kitely allow me to make HTTP requests from my browser but not from a server in Ohio? Is there something I can change in the request from the server that the Kitely filter does not like (I've tried changing from GET to PUT requests, doesn't help). Can I get more information about the request and why it was disallowed?

This is NOT an HTTP request from one prim to another, it is a request from an outside server.
These users thanked the author Kayaker Magic for the post:
Graham Mills
Graham Mills
Posts: 1314
Joined: Sun Dec 23, 2012 2:26 pm
Has thanked: 1134 times
Been thanked: 1141 times

Re: Overzelous HTTP request filter

Post by Graham Mills »

History to this: viewtopic.php?f=11&t=2914
User avatar
Kayaker Magic
Posts: 354
Joined: Sun Dec 01, 2013 8:40 am
Has thanked: 52 times
Been thanked: 393 times

Re: Overzelous HTTP request filter

Post by Kayaker Magic »

Graham: It has been a while since that other thread, I started a new one to make sure it was clear I was NOT trying to make HTTP requests from one prim to another this time.

This is just an aside, but I have not figured out how this attack works: How can sending messages back and forth between prims allow you to hack into the asset database? Ok, assuming that there is a way to do that, we threw out the baby with the bathwater here. We threw out the only way prims have of communicating across a SIM border. So I must resort to sending messages to an outside server and having that server relay the message back to another prim. Wait a minute, if I can still send messages back and forth between prims (with an outside server in the middle). does that re-open the security issue that made the devs put in this restriction in the first place?
User avatar
Kayaker Magic
Posts: 354
Joined: Sun Dec 01, 2013 8:40 am
Has thanked: 52 times
Been thanked: 393 times

Re: Overzelous HTTP request filter

Post by Kayaker Magic »

OK, the error I am getting is not happening the way I thought. There is a place in my script where it turns out I did try to send a prim to prim message.

The LSL Wiki warns you that the URLs from llRequestURL are temporary. They must be requested again at several events, such as script reset and region crossings. They even suggested that the URL could be disconnected at random at any time and had sample code (I cannot find the link to it now) that showed the best practice of setting a timer and checking the validity of the URL every 300 seconds. My code is based on that example, and every 300 seconds it makes an HTTP request to itself to make sure the URL is still valid and requests a new one if it is not. So that request (a prim trying to send an HTTP request to itself) is the one that is displaying error messages!

The thing is, I am NOT getting this error message when I run the same code on OpenSim 0.9.0.0, even though I did not disable the new security feature. Either 0.9.0.0 has the security feature off by default, or it allows a prim to send a message to itself, if not to other prims. I'm going to have to go test that....
User avatar
Ilan Tochner
Posts: 6500
Joined: Sun Dec 23, 2012 8:44 am
Has thanked: 4935 times
Been thanked: 4450 times
Contact:

Re: Overzelous HTTP request filter

Post by Ilan Tochner »

Hi Kayaker,

You know our feelings about OpenSim 0.9.0 at its current pre-release state. It's not done yet. There are quite a few things that don't work as they should and will likely change before that version is finally made production-ready. This may be one of those cases.
Graham Mills
Posts: 1314
Joined: Sun Dec 23, 2012 2:26 pm
Has thanked: 1134 times
Been thanked: 1141 times

Re: Overzelous HTTP request filter

Post by Graham Mills »

Hi Kayaker -- that was just to give others a little context. I agree that it's important functionality and some clarification/resolution of issues would be useful.
Graham Mills
Posts: 1314
Joined: Sun Dec 23, 2012 2:26 pm
Has thanked: 1134 times
Been thanked: 1141 times

Re: Overzelous HTTP request filter

Post by Graham Mills »

I took the URL persistence/visitor counter script from http://wiki.secondlife.com/wiki/LSL_htt ... r/examples and put it in a prim, touched it to get a pseudo-persistent URL (it'll change on the next restart). I then used that URL in my web viewer and it updated the count. I put it in a simple python script running locally and that worked too. Finally I put the python script on a server and that worked as well. Have I missed something?
User avatar
Oren Hurvitz
Posts: 361
Joined: Sun Dec 23, 2012 8:42 am
Has thanked: 19 times
Been thanked: 499 times
Contact:

Re: Overzelous HTTP request filter

Post by Oren Hurvitz »

Hi Kayaker,

For security reasons, OpenSim doesn't allow llHttpRequest() to call URL's inside the OpenSim server itself. The reason is that there's a difference between how URL's are handled inside the server and out of it. Outside the server, the URL's hostname (ec2-54-177-102-174.us-west-1.com in your example) is converted to the server's external IP. But inside the server, the URL's hostname is converted to an internal IP. The internal IP is dangerous because there are services running on the server that, for security reasons, are blocked to the outside world. This is done by allowing them to be accessed only over internal IP's, with the assumption that anyone able to call an internal IP is trusted. But llHttpRequest() can't be trusted, since anyone can write a script that uses llHttpRequest(). Therefore, it must be blocked from calling internal IP's.

There's a config option to disable this security feature: OutboundDisallowForUserScripts. This might be why you are able to make llHttpRequest calls inside the server in other instances of OpenSim. However, we're not going to disable this feature as that would be dangerous.

AFAIK, only Channels are a safe method for prim-to-prim communications within a server. I guess it would be possible to write a script that uses a combination of channels and HTTP to allow it to be called both from prims inside the simulator, and from outside servers.
These users thanked the author Oren Hurvitz for the post:
Graham Mills
User avatar
Kayaker Magic
Posts: 354
Joined: Sun Dec 01, 2013 8:40 am
Has thanked: 52 times
Been thanked: 393 times

Re: Overzelous HTTP request filter

Post by Kayaker Magic »

Oren Hurvitz wrote:AFAIK, only Channels are a safe method for prim-to-prim communications within a server. I guess it would be possible to write a script that uses a combination of channels and HTTP to allow it to be called both from prims inside the simulator, and from outside servers.
I know this thread is stale, but I am still struggling with a way to do prim-to-prim communication between worlds or across SIM borders.

I can communicate from a prim to an external server with llHTTPRequest, and this can return information. A prim in a different world or across a SIM border can poll my server to see if a message has been left for it, but polling uses up a lot of SIM resources, I'd rather not do this. Prims can llRequestURL to get a URL and send that to servers, but those URLs always have a port number in them. The WEB hosting company that I use, and most others, have a security requirement that they block all HTTP requests with port numbers. So I am unable to use those to asynchronously pass messages back to prims. I am setting up a server in my barn that allows URLs with port numbers, but to be honest my WEB hosting company in Ohio (I am in California) is more reliable than the server in my barn. It is difficult to make reliable scripted products that are dependent on this.

So my question to Oren is: How can a script in a prim use channels and HTTP to communicate with prims in a different world (across a SIM border)? Channels only work inside a region (world) and everywhere I turn HTTP is not allowed for prim-to-prim or not allowed for server-to-prim. (Prim-to-server is easy and reliable with llHTTPRequest, but that is only half of the equation).

It seems to me that the OpenSim developers threw out the baby with the bath-water. It was apparently possible for a script in a prim to make requests to servers on the same local area network, which is bad. But couldn't they have allowed prims to continue to communicate with each other and just disallowed communication with non-prim servers?

P.S. In some grids (like SL or InWorldz) it is still possible for prims to communicate with each other using llHTTPRequest to URLs returned by llRequestURL. Does this mean those systems have gaping holes in their security? Or do they allow prim-to-prim communication while not allowing prims to make requests to other severs on the LAN?
User avatar
Oren Hurvitz
Posts: 361
Joined: Sun Dec 23, 2012 8:42 am
Has thanked: 19 times
Been thanked: 499 times
Contact:

Re: Overzelous HTTP request filter

Post by Oren Hurvitz »

Hi Kayaker,

I think this should work:
  • Script A calls llRequestURL()
  • It sends this URL to your server
  • Script B polls the server, and gets this URL
  • Script B can now use llHttpRequest() to call Script A. It can pass it its own URL directly.
This does require polling, but there's no way around that unless you switch to a hosting company that allows calling URL's with ports. (Which I would recommend: this limitation makes them unsuitable for use by programmers.)
These users thanked the author Oren Hurvitz for the post:
Graham Mills
Post Reply