Revigniter V2 sanitizing issue

[istech]

Hi All,

Maybe Ralf can answer this when he is free. A final issue has popped up when testing Revigniter V2 with my project. Upon POSTING an array to a controller it now sanitizes my POST before it gets to the controller. So my POST is empty. The error I get is attached. It seems as though input lib does not like any array keys with base64 encoded data. With V1 I could POST arrays with base64 encoded data with no issues. Has this changed in V2? If so is there a way around it with not too many changes? Would love to hear your thought on this.

Many thanks

PS - Some of my errors in the attachment are from my handler picking up on the empty POST. In my logs I also get "GLOBAL POST & COOKIE WHERE SANITIZED "

[Ralf Bitter]

Hi istech,

valid characters for POST keys are ”(?i)^[a-z0-9:_\/-]+$" (regular expression).
This has actually never changed, except for the brackets added in version 1.7.2.
The problem is that base64 encoded strings may have one or more equal signs at
the end of the line. I think you can safely remove these before posting, then
base64 encoded keys should work.

[istech]

Thanks for that Ralf.

Previously, I have added "a-z 0-9~%.:_\-?=+/" into gConfig["permittedUriChars"] in the config.lc file. Would this not allow the POST with the base64 to pass as is?

Many thanks

[Ralf Bitter]

Hi istech,

this setting applies only to URLs and has no effect
on POST data.
If you absolutely need equal signs in POST keys you
may consider to add an equal sign to the regular
expression in function _rigCleanInputKeys() in the
Input library.

[istech]

Hi Ralf,

Of course you are right. I have made your recommendation to try and add "=" to the above function however I don't think this is possible as the whole site can not be reached with this error displayed. "Disallowed Key Characters. ri_session" and my POST response error is now "Disallowed Key Characters. eyJhcGkiOiB7ImtleSI6ICJhQjJRZ0pka0VrNHM5cmpzb1FDRyIsInRhc2siOiAidXBkYXRl"

Very strange, It seems as though it sees my base64 encoded POST as an array key?

If you find the time can you let me know how the Input.livecodescript gets the POST/$_POST_RAW before releasing it to the controller?

private function _rigCleanInputKeys pKey
if matchchunk(pKey, "(?i)^[a-z0-9:_\/-]+=$") is FALSE then
put "Disallowed Key Characters." && pKey
exit to top
end if

return pKey
end _rigCleanInputKeys

[Ralf Bitter]

Hi istech,

I am in a hurry right now, but will deal with it later.
There seems to be a misunderstanding on my side.
Thought your POST array keys are base64 encoded.
Now it turns out your POST array is encoded. I assume
you followed the instructions regarding dealing with arrays
in POST data in chapter Input Library of the user guide.

[Ralf Bitter]

Hi istech,

could you provide a sample array so I can investigate
where the problem is?

[istech]

Hi Ralf,

Sorry for my late reply I was out all day yesterday.

I have found out the cause but do not know how to fix it as yet. The problem comes from some base64 encoded image information stored in my array. When I remove the based64 encoded image data the POST is processed fine.

The strange thing is as they are base64 encoded revigniter should not have a problem processing them. right? (I even removed the padding just to see "=")

I'll attach a sample so you can have a look today when I get back.

As always thanks for your time on this one.

PS. I am currently using an base64 encoded LC array to POST. However, If I wanted to POST a JSON array encoded with base64. Would this fail to pass the input lib because it is base64 encoded?

[Ralf Bitter]

Hi istech,

you can POST JSON base64 encoded,

you can POST JSON unencoded if you

Code: Select all

set the httpHeaders to "Content-Type: application/json”

or you can POST JSON in a base64 encoded arrayencoded LC array if you

Code: Select all

set the httpHeaders to "Content-Type: application/lc.array”

or if you are dealing with a compressed array

Code: Select all

set the httpHeaders to "Content-Type: application/lc.array.compressed”

[istech]

Thank you for your patience Ralf,

I have attached a sample base64 encoded post for you to review. I have narrowed it down to the base64 encoded image data. But not quite sure how to get around this one as it used to POST with Revigniter V1. Would love to get some feedback. Many thanks

[Ralf Bitter]

Hi istech,

have tested your sample data in a POST action. There was no
problem neither by sending the base64 encoded data nor
by sending the base64 decoded (JSON) data, in this case using
a "Content-Type: application/json" header.

To verify that everything works as expected, on the server side
I converted the JSON data to an array and returned an arbitrary
value (tYourDataArray[“link"]["todo"]) of the array data. The result
was “update”.

However, it is important to note that if you post base64 encoded
data you should

Code: Select all

put false into gConfig["csrf_protection"]

If you need CSRF protection you should post the JSON data
unencoded and

Code: Select all

set the httpHeaders to "Content-Type: application/json”

Furthermore I noticed that if gConfig["globalXssFiltering"] is true
the execution of the Input library stops while sanitising data.
It seems that there is a problem with the size of your data because
if I test less data it works. So, could you please check what happens
when you

Code: Select all

put false into gConfig["globalXssFiltering"]

Please let me know if this helps.

[istech]

Thanks for this Ralf and I really appreciate your dedication and time.

I can confirm the config.lc file settings are false.

put false into gConfig["globalXssFiltering"]

put false into gConfig["csrf_protection"]

and also confirm the settings for the header is "set the httpHeaders to "Content-Type: application/json”

So I can only assume it has got to be a server limit for the post. I will check my server logs for any information.

Did you POST from a Livecode application/IDE? Just covering all basis.

[Ralf Bitter]

Hi istech,

not that there is any misunderstanding, I thought you post
the JSON data base64 encoded. So, no need to set a
HTTP header. But previously mentioned restrictions do
not apply if you post unencoded JSON data using the
appropriate HTTP header.

And yes, I did post from the LC IDE. How do you post?

[istech]

Hi Ralf,

I have tried to POST with encoded and non-encoded. I have also tried both JSON and LC arrays with the same results.

I POST from my LC IDE so we have tried the same method.

I'm still investigating the issue and will update you when I have some more information. The feedback is priceless many thanks.

[istech]

Just a quick update on this issue.

After a break troubleshooting this issue and further investigation, I can see no obvious problem with the server as nothing appears in the logs and it is working correctly using a Livecode script where I post directly to it.

With this in mind, I must conclude the problem points to an issue with my version/setup of Revigniter. Maybe some corrupted files?

Moving forward after troubleshooting this for a while I can only try to install Revigniter V2 from scratch and see if it works. If you can think of anything else to try Ralf please let me know.

Many thanks