Using Cryptography:aesDecrypte() function returns extra bytes at the end of the returned string

simwhi · May 2016

Here is my test code.

 
  local key = "AAAAAAAAAAAAAAAA"
  local iv = "0123456789abcdef"
 
  local str = Cryptography.aesEncrypt("Hello, world!", key, iv, 1)
  local str_2 = Cryptography.aesDecrypt(str, key, iv, 1)
 
  print(str, str_2)

The string is encrypted, but when it's decrypted there are 3 extra characters at the end. These characters are EXT (end of text) codes.

Is this a bug?

hgy29 · May 2016

Hi @simwhi,
Aes 128 works on 16 bytes buffers, so anything you encrypt is internally rounded up to be a multiple of 16 bytes/characters. Your string is 13 chars in length, so gideros adds 3 extra chars as padding.

simwhi · May 2016

@hgy29 Thanks for the explanation.

SinisterSoft · May 2016

AES 128 is to 16 byte buffer boundaries or a min of 16 bytes?

I thought it was just a min of 16 bytes? The key has to be 16 bytes (32 bytes for AES256, etc) and the init vector for CBC is alway just 16 bytes. I might be wrong though?

(A min of 16 and not 0 because the first 16 bytes ensure you can't figure out the init block?)

Edit:
You use ciphertext stealing to allow it to work with at least 16 bytes, but after the first 16 you don't need a boundary.
https://en.wikipedia.org/wiki/Ciphertext_stealing

"Ciphertext stealing for CBC mode doesn't necessarily require the plaintext to be longer than one block. In the case where the plaintext is one block long or less, the Initialization vector (IV) can act as the prior block of ciphertext. In this case a modified IV must be sent to the receiver. This may not be possible in situations where the IV can not be freely chosen by the sender when the ciphertext is sent (e.g., when the IV is a derived or pre-established value), and in this case ciphertext stealing for CBC mode can only occur in plaintexts longer than one block."

hgy29 · May 2016

@sinistersoft, AES 128 operates on exactly 16 bytes buffers no more, no less. Of course you can overcome this limitation by using any kind of padding you want, eventually chaining blocks or mixing data in any way you want, but those are beyond the scope of aes, and applicable to any cipher.

SinisterSoft · May 2016

Yes, all the versions I've used before must have used cyphertext stealing to make it so you had to make it at least 16 bytes long but not have to worry about boundaries after that. I thought that was the normal implementation.

EricCarr · May 2016

Ideally Gideros would both ADD and STRIP padding so developers don't have to. If I call a method and tell it to expect PKCS7 padding when decrypting, I don't expect to have to strip it off myself.

Also, it would be a good helper to include a CSPRNG to generate IV on the fly or most ideally a helper method that does this and returns a single base64 rendition of IV + ciphertext to promote best practices. HMAC would be icing on the cake to prevent tampering.

Ideally Gideros could provide a proven encryption implementation that developers follow and then you don't run into issues of people making poor encryption choices.

On the other hand, at that point a would-be cracker might just dump the memory to find the key in lua, so there should be a way to store/pass in the key other than an immutable string.

Maybe it can be part of the export process, a global key that is more securely embedded somewhere in the project so LUA isn't even aware of the key?

hgy29 · May 2016

Well it is easy to add padding on encryption, however when decrypting you can't tell for sure wether the trailing bytes of the string are padding or were plain text. To differentiate, string length would have to be included somewhere (at the beginning), but if gideros did this inconditionnaly, then this make it impossible to interface to non gideros aware peers. The idea was to stay simple and let the user use the AES primitive according to their need. Adding a header can still be done at lua side after all.

Basically this AES block was added to allow ciphered communication to servers, in conjunction with UrlLoader. One could argue that HTTPS already add encryption, but HTTPS requires a certificate that many of us can't afford.

I agree that HMAC would be a nice addition, since it can be CPU intensive. As for embedding a key within a non lua accessible storage, it would be bound to source code attack since Gideros is open source, nothing can be hidden after all.

EricCarr · May 2016

Well it is easy to add padding on encryption, however when decrypting you can't tell for sure wether the trailing bytes of the string are padding or were plain text.

If you are correctly following the PKCS7 spec, then well, yes, you can.

The only time a payload could possibly look like real padding is if it is an exact multiple of the block size (16 bytes). In that case the PKCS7 spec calls for adding another block of 16 bytes, each byte equaling 16 (0x10). In this way you are always adding padding and can always be sure.

Most likely, any other encryption library the dev uses on the back end will correctly follow the PKCS7 spec, and if it is not implemented correctly in Gideros the devs will run into encryption/decryption issues down the line for payloads that are multiples of 16.

One could argue that HTTPS already add encryption, but HTTPS requires a certificate that many of us can't afford.

This would probably be the best route to go. Regarding HTTPS, you could generate a (free) self signed cert and pin that cert in the code to prevent MITM. A real certificate authority isn't needed, and you would want to pin just your cert anyway rather than trust the list of root CAs on the device by default. Otherwise a cracker with a rooted device could place a fake CA on their device, route through a proxy with a fake cert and easily view all of the transmission from their device to your server in plaintext.

As for embedding a key within a non lua accessible storage, it would be bound to source code attack since Gideros is open source, nothing can be hidden after all.

Good point about the open source attack. Unfortunately, this is already a problem in Gideros today. The entire project (lua files/images, etc) is encrypted via a key that is stored in a file in the apk. I made a POC program that takes any gideros apk and decrypts (just to confirm the flaw - I would never release this program). Worked for most gideros apps I tested from the app store, since the key is always in the same relative spot.

I assume that will get an overhaul at some point. Open source is tricky, since there is no security through obscurity, but maybe some sort of dynamic code generator that calls an obfuscated chain of methods using math operations (with a spot to add your custom flavor) to build the key instead of hard coding it will at least make brute force searching the key in the apk impossible. Then, this same "hardened" code could be used to embed a key for AES if desired. Maybe that's overkill?

I was planning on doing it for my projects. If I ever get around to it I will share the code.

Lastly, I realize I'm being nit-picky and security isn't always the highest priority in feature lists, but I figure i would put in my two cents. I think it's great that someone even thought to add encryption to Gideros in the first place.

hgy29 · May 2016

If you are correctly following the PKCS7 spec, then well, yes, you can. The only time a payload could possibly look like real padding is if it is an exact multiple of the block size (16 bytes). In that case the PKCS7 spec calls for adding another block of 16 bytes, each byte equaling 16 (0x10). In this way you are always adding padding and can always be sure.

You are right, you seem to know encryption better then I do

So I'll change the API so that when PKCS7 padding is selected (default is zero padding), padding will be checked and removed on decryption.

This would probably be the best route to go. Regarding HTTPS, you could generate a (free) self signed cert and pin that cert in the code to prevent MITM.

The issue with self signed certs is that most OS/underlying system simply disallow self-signed certs and won't give you the opportunity to supply your own CA list. Gideros relies on OS for HTTP/HTTPS, and cannot bypass whatever the OS decided to check, even if you could recognize your own cert.

Lastly, I realize I'm being nit-picky and security isn't always the highest priority in feature lists, but I figure i would put in my two cents. I think it's great that someone even thought to add encryption to Gideros in the first place.

Thanks

All in all in you think of some suitable way of 'hidding' a key within gideros source code, I'll be happy to discuss/help.

EricCarr · May 2016

Nice. I'm happy to hear you will update PKCS7 in the API.

Regarding pinning, it appears to be very possible in Android and iOS, and generally recommended.

Here is an overview and Android/iOS/openSSL code samples from OWASP, my go-to resource for all things security:

Cheat Sheet:
https://www.owasp.org/index.php/Pinning_Cheat_Sheet

Technical Guide (code samples in the page)
https://www.owasp.org/index.php/Certificate_and_Public_Key_Pinning

This is to protect from external MITM, say if your app users are on a public wifi.

An attacker could always root the phone and send made-up messages to a server. The only protection against that to validate input on the server side and use non enumerable ids like a guid instead of an integer for playerID. If someone really wants to hack their "save file" to skip to level 100, they can be my guest.

Howdy, Stranger!

Categories

In this Discussion

Top Posters

Using Cryptography:aesDecrypte() function returns extra bytes at the end of the returned string

Comments

Howdy, Stranger!

Quick Links

Categories

In this Discussion

Top Posters

Using Cryptography:aesDecrypte() function returns extra bytes at the end of the returned string

Comments