Proper Lua Crash Logging for Gideros

bowerandy · April 2013

Folks,

Some time ago, following @atilim 's request, I added Crittercism crash logging to my app under development. Imagine my surprise when I found nothing in the logs following several "crashes". This was, of course, because these crashes I'd experienced weren't actually caused by real exceptions at all but rather they were "just" Lua errors. Unfortunately, in such circumstances Gideros just prints a stack trace to the console and then exits normally. Nothing get's sent to Crittercism for later perusal.

Recently, I got down to sorting this out and I've now written a plugin that allows you to detect Lua crashes *and* get hold of the Lua stack trace when things went wrong. You can find full details and a download link for the plugin in this blog article:

http://bowerhaus.eu/blog/files/luacrashlogging.html

As you'll see if you read the article, I found that neither Crittercism nor BugSense are suitable for this purpose and I opted for Crashlytics. This is no bad thing, since it turns out that Crashlytics is now completely free to use.

Here's a short (two-line) stack dump that shows how it appears in the Crashlytics dashboard (newlines are replaced by *** markers):

Note that the plugin, as it stands, is written for iOS. However, there's nothing in there that should preclude an easy port to Android. If anyone manages to do this, please let me know.

best regards

Scouser · April 2013

The only problem I can see about porting this to Android is the fact that Andoid isn't supported yet.

bowerandy · April 2013

@Scouser, oh I thought it was. When I Googled "Crashlytics Android" I got this:

http://support.crashlytics.com/knowledgebase/topics/18948-crashlytics-sdk-for-android

They've recently been bought by Twitter. I guess the Android SDK will be there soon if it isn't already.

best regards

Scouser · April 2013

Andy, I followed your original link and clicked the Try Crashlytics link where It says iOS now, Android soon!

Hopefully it will be sooner rather than later.

krisis · April 2013

Hey @bowerandy,

Thanks a lot for this. I set it up according to your instructions, but the crash report doesn't contain the lua backtrace.

I created a button with an onClick function that tries to run unknownFunctionCall(). I installed the app to my device and clicked the button. It generated a crash report, but under 'more details' it just shows LUASTACK as blank, and Log as : 0 * null
The main exception trace is:

0 stackeralpha -[Crashlytics crash] + 1145559
1 stackeralpha
BhCrashlytics.mm line 75
exit
2 stackeralpha std::_Rb_tree, std::_Select1st >, StringId::ltstr, std::allocator > >::_M_erase(std::_Rb_tree_node >*) + 82
3 stackeralpha
giderosapi.mm line 772
ApplicationManager::drawFrame()
4 stackeralpha
ViewController.m line 155
-[ViewController drawFrame]
5 QuartzCore CA::Display::DisplayLink::dispatch(unsigned long long, unsigned long long) + 160
6 QuartzCore CA::Display::IOMFBDisplayLink::callback(__IOMobileFramebuffer*, unsigned long long, unsigned long long, unsigned long long, void*) + 64
7 IOMobileFramebuffer IOMobileFramebufferVsyncNotifyFunc + 154
8 IOKit IODispatchCalloutFromCFMessage + 192
...
16 UIKit UIApplicationMain + 1120
17 stackeralpha
main.m line 12
main

Any ideas?

krisis · April 2013

I added a breadcrumb and retested crash and I see the breadcrumb under Log:
0 00:00:07:70 $ clicking crash button

but still no Lua stack trace. Exact code I'm using is:

local button = Button2.new("CrashMe!")
button:addEventListener("click", function()
	breadcrumb("clicking crash button")
	unknownFunctionCall()
end)
menu:addChild(button)

Running through Gideros Player, the actual stack trace it reports is:

options.lua:122: attempt to call global 'unknownFunctionCall' (a nil value)
stack traceback:
	options.lua:122: in function <options.lua:118>

This is what I'd expect to see on Crashlytics.

bowerandy · April 2013

@krisis, this could well be as a result of my incompetence with regex.

BhCrashlytics uses two regexes in a function called distillLuaStackTrace() to strip out the crap before the real stack trace and to shorten the contents of each line (so that they are more readable in the Crashlytics dashboard). However, I'm truly hopeless at regex so it's very likely that my attempt doesn't work correctly in all cases and this function might be stripping out everything.

Things to try:

1) Try removing the call to that function so that the whole file gets sent up to Crashlytics. See what you can see in the dashboard. If something gets there that will indicate that the regex is at fault.

2) Try using a tool like iExplorer or i-FunBox to explore the file system on your device after a crash occurs and look at the contents of stdout.txt in your apps /Documents directory. Do this after the app has crashed but before restarting (otherwise the file will get wiped). This will tell you if the stack trace is being written to the file and might explain why the regex is failing if it is.

Let me know how you get on.

best regards

bowerandy · April 2013

Here's a question for @atilim.

EDIT: I have changed this question from the original in the light of more information.

I'm trying to use the above crash reporting solution to find a bug in my application. I have a reproducible Lua crash but it doesn't get a stack dump with it. In this case I'm seeing a slightly different sort of exit message in the log than I normally do. It says:

Lua panicked and quit: /Users/bower/Documents/Apps/Gideros/XCode/Universal/spinoff/spinoff/assets/Library/BhHelpers.lua:942: bad argument #2 to 'format' (string expected, got userdata)

I haven't seen this "panicked and quit" exit before. Is there any reason why Gideros doesn't create a stack trace in this case.

Best regards

bowerandy · April 2013

@atilim, please ignore the previous query. The error exit is called in the BhWax library and so doesn't use the normal Gideros exit.

best regards

krisis · April 2013

@bowerandy, with the regex removed the stracktrace string I'm seeing in stdout.txt (and sent to Crashlytics) doesn't include the .lua filenames, which is why your regex is failing:

*CppLuaBridge::luaEvent* stack NOT ok begin:2 end:3 delta:0
*EventDispatcherBinder::dispatchEvent* stack NOT ok begin:2 end:3 delta:0
*visit(MouseEvent* v)* stack NOT ok begin:0 end:3 delta:0
*CppLuaBridge::luaEvent* stack NOT ok begin:0 end:3 delta:0
*enterFrame* stack NOT ok begin:0 end:3 delta:0
?:0: attempt to call global 'unknownFunctionCall' (a nil value)
stack traceback:
	?: in function <?:118>

Any idea why that would be the case?

FWIW, I'm testing this by creating an archive release build and pushing that to my device.

bowerandy · April 2013

@krisis, I'm not sure. Might need a comment from @atilim?

One possibility; are you encrypting the Lua files when exporting the project? If so, try turning that option off. Also, just check that you are using the latest version of Gideros, I thought I remembered something about the encryption being changed in this respect in this version. Ah, here it is, from the 2012.9.10 release notes:

"Change: While exporting, Lua bytecodes are not debug stripped to get better error descriptions"

I'm using the Crashlytics dumps with encryption turned on and it seems okay with Gideros 2012.9.10.

Best regards

krisis · April 2013

I'm not encrypting them, but I'm also on an older 2012.9.06 version, so there may be fixes somewhere. I get proper filenames in xcode debug or through Gideros Studio, just not these release builds.

krisis · April 2013

I just generated a crash using Gideros Player on device and I got a proper stacktrace. Not sure why my exported Gideros app isn't doing the same.

ar2rsawseen · April 2013

It seems that undefined function crash is catched by C to Lua bridge and not Lua itself. Might be a specific how Lua is handled internally.
Thus such reporting error, however (at least on Android) these kind of errors are also provided by Gideros, thus may not be included in crash reports on themselves.
Probably need some more sophisticated regex to anticipate these kind of situations also. Maybe looking for keywords stack and lua?

phongtt · April 2013

I'd love to see an ultimate solution for crash reporting. I believe it's extremely important for all pf us.

With 400.000+ downloads of Game42, I've received quite many crashes from Critterism but honestly don't have enough clue to fix the bugs.

Okay, I'm a pusher, not an implementer

)

fxone · April 2013

With 400.000+ downloads of Game42...

@phongtt huh..still on the wave: 100k per day?

bowerandy · May 2013

@phongtt, well done on the Game42 success!

I assume that you can't understand the Crittercism crash reports because they don't include a Lua stack trace? I know Crashlytics isn't available for Android yet but couldn't you try to adapt the stuff I did above and upload the error stacks to a website (either yours or a third party BAAS). So, what I'm saying is, continue to use Crittercism for raw crashes and then something else for the (probably more common) Lua crashes.

Before I came across Crashlytics I started working on this. I was going to use the Collections support (like a database) of Moai Direct cloud services. But since Crashlytics came along and did what I needed I didn't carry on with that idea. Anyone else want to pick up the ball and create a generic Lua crash reporting service for both IOS and Android?

Best regards

phongtt · May 2013

Thanks guys.

@fxone: no no, I just meant the total downloads was more than 400K. It's already back to Paid so not that many downloads per day. But still, it's been in top 100 paid in Board game category in US and some other countries

@bowerandy: I meant Game42 on iOS (actually the Android versions for Samsung phones and tablets have just arrived on Samsung store too

). But yes some solution that support both iOS and Android would be very helpful for many people.

keszegh · June 2013

@bowerandy, could you possibly update your cool tutorial for using with android at some point?
see: http://www.crashlytics.com/blog/its-finally-here-announcing-crashlytics-for-android/

bowerandy · June 2013

@keszegh, I saw that announcement too. Unfortunately, I was hoping someone else might take up the baton and run with it on the Android side. I have very little experience with Android, especially where plugins are concerned.

best regards

keszegh · June 2013

hopefully somebody will take it up, maybe @ar2rsawseen can be 'forced to', although for me it is not that high on the preference list. for gideros it would sound good if there would be official plugins for all mayor aspects of mobile development, like iap, crashlog, younameit.

ar2rsawseen · June 2013

@keszegh noted and added to experiment list

keszegh · June 2013

good enough, thanks

mrcoffee · June 2013

@bowerandy i have some problems with the BhCrashlyticsDemo demo. I have done all steps for setting up my gideros player for Crashlytics. But now when i try your demo in the gideros player no crash info is send to Crashlytics. Its working fine when i am crashing the player manually with the crash method of crashlytics but no lua crash was sent until now. I had a short look at your BhCrashlytics plugin and it seems that the exit function is never triggered!

bowerandy · June 2013

@mrcoffee,

Not sure what might be happening but here are a few ideas:

1) You won't get a Crashlytics log entry if the player is running under Xcode. Stuff only gets sent up to Crashlytics if you have launched the player (or your app) directly on a real device. I also suspect it may not work in the iOS simulator. It certainly won't work using the Gideros desktop player.

2) Sometimes Lua quits without calling exit(). I'm not sure when this happens but it is rare. Try adding "undefinedSymbol()" to your main.lua and see if this causes a log. If it doesn't try running the player (on a real device) under Xcode and put a breakpoint at that exit(status) function in BhCrashlytics.mm. If it's not getting there I'm a bit stumped.

best regards

mrcoffee · June 2013

@bowerandy

Thanks for your reply. I will check this out.

Howdy, Stranger!

Categories

In this Discussion

Top Posters

Proper Lua Crash Logging for Gideros

Comments

Howdy, Stranger!

Quick Links

Categories

In this Discussion

Top Posters

Proper Lua Crash Logging for Gideros

Comments