UrlLoader after adding 64 bit support on Android

PaulH

Hey, folks. In preparation for Google's upcoming 64 bit requirement I've enabled that in a game that didn't previously have it, and I've encountered some strange behavior. It's evident that for some portion of users, UrlLoader() isn't working when the app tries to access my server scripts. The app is still getting data from Google Play, since it's showing in-app products that it will only show if it received price data from the store. But when attempting to contact my server, nothing gets through. The server access code looks like this:

local loader = UrlLoader.new(url, UrlLoader.POST, {["Content-Type"] = "application/json"}, JSON:encode(data))

Not only is the app not receiving a response, but the server isn't receiving any data. I have a screen where users can enter a tech support code, and one just sends some diagnostic text to a PHP script that emails that text to me. Even that isn't working.

The only change I made was checking "enable 64 bit libs (android)" in the Require plugin, and moving to target Android version 28. In my own tests on multiple devices everything works fine, but I started hearing from some users who couldn't download content, and there was no trace of interaction between those users and my servers. I unchecked the 64 bit libs and rebuilt, targeting Android version 26, and those users are getting content again.

Any ideas? Are there additional headers, or other things that need to change to post JSON data via UrlLoader, either with 64 bit libraries or when targeting newer versions of Android?

This is a really tough one to troubleshoot, since I can't reproduce the error on any device I have, and my system for sending me diagnostic data from users who are having problems doesn't work in this case, since that uses the same network access that's failing in the first place. Supporting the 64 bit libraries is a must, but currently that causes users to be unable to get the content they're paying for.

Paul

totebo

I'm seeing the same issue in my game, and luckily I can replicate it on one of my phones. This is the code I use:

local ip_loader = UrlLoader.new("https://hub.nobrakes.io:2015/servers/all")

It results in an Event.ERROR, but with no information of the error passed to the function. Logcat is also not telling me what's going on.

PaulH

Very interesting... that rules out a couple things I'd been thinking about. It shows the problem isn't limited to Post data or other headers I'm using, or JSON data, or non-secure HTTP.

What kind of phone are you seeing the issue on?

hgy29

@PaulH, seems like it was caused by gideros using an old Apache HTTP Client library. Using the ones bundled with android fixed @totebo's issue.

PaulH

Glad to hear there's a fix! What would I need to do to make Gideros link the APK with the correct library from Android Studio?

hgy29

You need an updated Android template. The one I made yesterday is here: http://hieroglyphe.net/gideros/AndroidStudio.zip
Replace it in your Gideros installation (unzip it inside the Templates folder on Windows). You should probably make a backup of the official one, just in case.

totebo

PaulH said:

What kind of phone are you seeing the issue on?

When I uploaded the game to Google Play console, the pre-launch report showed it failing on a mix of phones, buy mostly more recent ones. The one that failed for me was OnePlus 6T.

@hgy29's fix, above, solved the issue for me. With luck, it will now run a while before the next required upgrade comes!

PaulH

I've put in the new Android template, and it seems to still be failing. In my case, the pre-launch tests weren't catching the problem, because a lack of network response wouldn't crash the app, just make it impossible for users to download additional content. Since I can't reproduce the issue on any device I have, I've added temporary code to intentionally crash if an attempt to check in with my server results in an ERROR event. Then I publish an alpha version that goes only to me, to trigger Google's automated testing of the new APK. That way I can see crashes in the pre-launch tests for devices where network access isn't working. I tested that before and after switching to the new Android template, and it passes on most devices, but crashes after the UrlLoader's ERROR event on two devices: The Google Pixel 2 and 3, which are the two devices they test using Android 9.

So I'm still unable to publish an update, as some fraction of my users would lose access to content they've paid for due to the network issue.

Any other ideas? Do I need to try building the APK via Android Studio rather than directly from Gideros? Or go buy a Pixel 2 or 3 phone to troubleshoot on a real device?

Paul

MoKaLux

I would go the android studio way first. See if any errors occur.

Apollo14

Maybe you can emulate Google Pixel in Android Studio?

PaulH

I've been trying to build the APK in Android Studio. I get an error that some licenses have not been accepted. The error includes a link to a page saying to go to Tools, SDK Manager, but SDK Manager isn't included under the Tools menu. I've been looking at a lot of posts from people missing the SDK Manager under the tools menu, hoping there's some setting I need to enable or other step to make that show up, but so far, no joy. It seems Android Studio, including the command line called by Gideros, can install newer SDKs as needed, if the licenses have been accepted, but the steps I see to accept them don't work with the current Android Studio, evidently. I'm on a fairly clean system at the moment, since a recent Windows update corrupted the OS, forcing me to format the main drive and reinstall Windows and all the applications I use, including Android Studio. I published apps for years from Eclipse, until Gideros supported APK exports.
I'd hoped Android Studio would be more straightforward than it was the last time I used it, but when it tells me to use a menu option that doesn't exist, well... it's not what I was hoping for.

I thought this might fix it: I clicked Help, Check for Updates, then accepted the license for all items available for updating. That stalled for half an hour, and evidently did nothing. I restarted AS. Now check for updates says I'm up to date, but there's still no SDK Manager under the Tools menu. And Build, Build APKs does nothing at all. And Build, Build Bundles tells me I need to update Gradle, gives me a button to update, and the button does nothing. I thought Xcode was the worst IDE I'd seen in my long career, but... I'll stop myself from further ranting. I'm going to try really hard not to be so cynical, but what seem like the obvious steps just aren't getting anywhere.

I must be missing something really obvious here. I've exported an Android Studio project from Gideros, opened it in Android Studio, and the build options seem to do nothing, and the SDK manager doesn't seem to be present.

Is there some trick to building an APK from an Android Studio project exported from Gideros? Or to launching the SDK manager, so both Android Studio and Gideros can link APKs with the up-to-date libraries?

PaulH

So I finally figured out how to get the SDK Manager to show up in Android Studio. Launch AS, but don't open a project, but select Configure, then SDK Manager.

Portions of support for Android 9 were not installed. Installing now. We'll see if that helps.

hgy29

@PaulH, if that doesn't work, then best way forward would be to print the details of the issue that caused the ERROR event to be reported in logcat. Ideally it would be great if it could be reported to lua, but that's more work.

I am compiling a new template with that error logged, if you are on gideros slack group we can chat about it today. It is 8:00 AM here, and I am back at my office after holidays...

PaulH

I've been trying various things, but still in the same situation - Some of Google's pre-launch tests work, but those on Android 9 still get errors with UrlLoader. If you have that new template available that will add details to the logcat, I'd love to try it.

hgy29

@PaulH, I uploaded the newly built template. It should log HTTP errors to logcat.
You can get it using the same link a few posts above.

vitalitymobile

In Manifest file, I suggest adding this line

<uses-library android:name="org.apache.http.legacy" android:required="false"/>

It will reduce crash and error on Android 9
https://developer.android.com/about/versions/pie/android-9.0-changes-28

hgy29

Already done in new template: https://github.com/gideros/gideros/blob/master/ui/Templates/AndroidStudio/Android Template/app/src/main/AndroidManifest.xml

PaulH

Thanks for the new template. In the logcat from a Pixel phone on Android 9, on which the network fails, I see several HTTP exceptions like this:

08-07 06:53:33.014: E/Gideros(12754): HTTP exception
08-07 06:53:33.014: E/Gideros(12754): java.io.IOException: Cleartext traffic not permitted: http://www.flyfishingsimulator.com
08-07 06:53:33.014: E/Gideros(12754): at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:439)
08-07 06:53:33.014: E/Gideros(12754): at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:560)
08-07 06:53:33.014: E/Gideros(12754): at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:492)
08-07 06:53:33.014: E/Gideros(12754): at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:470)
08-07 06:53:33.014: E/Gideros(12754): at com.giderosmobile.android.player.HTTPThread.run(HTTPManager.java:348)

From the Googling I've done so far it seems there might be a manifest change that might solve this. Anyone familiar with that exception?

Or, the other option would seem to be to encrypt all traffic between the app and my servers.

hgy29

I wasn’t aware Android was heading the same way as iOS, disallowing cleartext http by default. Good to know. found this: https://medium.com/@imstudio/android-8-cleartext-http-traffic-not-permitted-73c1c9e3b803

MoKaLux

crazy androids!?

PaulH

I'm adding the setting suggested (android:usesCleartextTraffic="true" in the Application tag) in that article to the manifest in the Android template, and I'll see what happens.

One thing I read about someone dealing with a similar issue on another platform was that enabling clear text traffic could conflict with some other libraries. I don't know if it will be an issue here, but I'll know in the next couple of hours if this gets HTTP working on Android 9 in this app.

Google's automated tests are still running, but my servers have heard from a Google Pixel 3 running the new build on Android 9, so early signs suggest HTTP is working with this change.

PaulH

According to Goole's pre-launch report, that manifest change solved it, with HTTP working on all the devices and Android versions they tested on. So might there be any reason to worry about side effects of the change causing other problems?

SinisterSoft

This manifest flag could be added to the requires plugin maybe?