Page 1 of 1

Attempt to connect to Watson Assistant: llhttpRequest custom header for Authorization gives error

Posted: Tue Apr 06, 2021 8:59 pm
by Ramesh Ramloll
Hello Ilan and Oren,
I have been trying to enable our NPCs to acquire state of the Art machine learning capabilities. In particular, I want to make our NPCs connect to conversation engines hosted by the main players, DialogFlow, Watson Assistant, Lex and RASA. I have evaluated each one of them, and I settled on Watson Assistant. However, I have hit two problems with HTTP_CUSTOM_HEADER.
1. The first is that HTTP_CUSTOM_HEADERs are limited to 253 characters. So while I was able to get a Bearer Token from my instance of Waston Assistant running on IBM servers from within Kitely, I now cannot use it. The number of characters in the Bearer token is 1699 chars.
2. The second problem is already documented here http://opensimulator.org/mantis/view.php?id=8881&nbn=9
I also get the error
llHTTPRequest: Name is invalid as a custom header at parameter, the reason is that am using 'Authorize' in the custom header and this is a standard REST API requirement these days for authorization before you can actually use the service.

because it appears if the variable name is found in this list, from Jeff Kelley, an error will be issued.
private static readonly HashSet<string> HttpStandardHeaders = new HashSet<string>(StringComparer.InvariantCultureIgnoreCase)
{
"Accept", "Accept-Charset", "Accept-Encoding", "Accept-Language",
"Accept-Ranges", "Age", "Allow", "Authorization", "Cache-Control",
"Connection", "Content-Encoding", "Content-Language",
"Content-Length", "Content-Location", "Content-MD5",
"Content-Range", "Content-Type", "Date", "ETag", "Expect",
"Expires", "From", "Host", "If-Match", "If-Modified-Since",
"If-None-Match", "If-Range", "If-Unmodified-Since", "Last-Modified",
"Location", "Max-Forwards", "Pragma", "Proxy-Authenticate",
"Proxy-Authorization", "Range", "Referer", "Retry-After", "Server",
"TE", "Trailer", "Transfer-Encoding", "Upgrade", "User-Agent",
"Vary", "Via", "Warning", "WWW-Authenticate"
};

3. I feel a bit sad, here I was trying to raise the bar, and all these issues pop up. Just imagine having really intelligent NPCs roaming about in Opensim with hooks connecting them to the web etc... it would be really awesome. You could even order Pizza or book a hotel from within Kitely just by chatting with an NPC.

Can I get some help on these issues?

The easy one I think would be to remove Authorization from this list ... this term is really core and needed if we want to get authorized to use any external service. The second one would be to expand llhttpRequest so that the custom Headers can accomodate 1699 characters + say 10 for the variable such as "Bearer".

All the state of the Art AI engines recommend using a "Bearer" token to get authorized because the chat sessions will be processed faster and will be more responsive. I could try using apikeys as passwords but that is supposed to be done only for test environments.

Worst case, I would have to settle for the last option but I would still need to have 'Authorization' in custom header not flagged.

Let me know your thoughts.

Re: Attempt to connect to Watson Assistant: llhttpRequest custom header for Authorization gives error

Posted: Wed Apr 07, 2021 6:46 am
by Oren Hurvitz
I did some testing, and OpenSim doesn't enforce the 253-characters limit on custom HTTP headers. And as you've already seen, it doesn't actually forbid the headers that it claims are invalid: it shows an error when they're used, but it sends them anyway. Therefore, you should be able to complete these requests right now, and the only problem is the error message that shows up.

Please try sending the "Authorization" request (with no limit on its length), and confirm that it works as expected. If so, then I'll remove "Authorization" from the list of forbidden headers, which will remove the error message.

Re: Attempt to connect to Watson Assistant: llhttpRequest custom header for Authorization gives error

Posted: Wed Apr 07, 2021 2:48 pm
by Ramesh Ramloll
Hello Oren, I made some progress. I did get the session_id I wanted. So yes looks like opensim ignores the character limit on Custom REST headers.

Re: Attempt to connect to Watson Assistant: llhttpRequest custom header for Authorization gives error

Posted: Wed Apr 07, 2021 3:49 pm
by Ramesh Ramloll
Hey Oren, I can get Watson to talk in Kitely! Yes I vote for suppressing this Authorization 'error' ... it is dangerous for health :)
Ramesh

Re: Attempt to connect to Watson Assistant: llhttpRequest custom header for Authorization gives error

Posted: Thu Apr 08, 2021 7:00 am
by Oren Hurvitz
That's great to hear. I'll remove the error on the "Authorization" header for the next release.