Invalid key to 'next' - Help needed

Mells · March 2013

Hi community,

Collisions

After 14min of intensive tests. Doesn't seem to be triggered by a specific event...

I’m facing this error :

invalid key to ‘next’
stack tracebkack:

The obvious answer would be "you are iterating through a table and modifying it at the same time" but please read more.
I think I have a good understanding of what this means and how to avoid it usually.
However this time I can’t solve it and it happens very randomly ( = I couldn’t reproduce it).
If my code was wrong, wouldn't my app crash right from the start? (If things were that easy...).

I have spent two days making web searches before asking here.
I nearly spent all night on this.

What I have tried

Based on the Lua Manual and the following resource, this typically happens when :

The behavior of next is undefined if, during the traversal, you assign any value to a non-existent field in the table. You may however modify existing fields. In particular, you may clear existing fields.

I have tried to avoid those situations and the app seems to run fine for an undetermined time, under normal conditions (= reasonable amount of objects on screen, simple collisions that happen in a linear way).

Sum-up

Let's say I’m creating a new instance of a “Bug” class every second.
“myBug[1]” is an instance of the Bug class that contains a sprite (myBug[1].sprite) and a box2d body (myBug[1].body).

Bug

OnCollision with myPlayer, bugs are destroyed.

The issue

The issue is very hard to reproduce, but “sometimes” an error is raised :

invalid key to ‘next’
stack tracebkack:

Note

This seem to happen when I’m doing stress tests (in that case spawn 3 objects every 400ms), when a lot of collisions are happening at the same time.
I couldn’t reproduce the issue when only a few bugs are on stage.

* Note : Images from the PlanetCute game prototyping assets pack at LostGarden.com

If somebody wants to help me, I believe it’s better if I give as many details as possible

More details

Creating new Bug instances

Every xxx ms, a new Bug instance is created.
I keep a list of all bodies (BODIES_LIST) in order to update their position each frame later.

Bug.lua

-- ---------- Create a new Bug instance
--
function Bug:init()
    (...)
    -- Add to the list of bodies to update each frame
    BODIES_LIST[self.body] = self   --### Modifying BODIES_LIST
end

MyScene:onEnterFrame()

-> Update bodies’ position

MyScene.lua

-- ---------- Update bodies' position
--
function MyScene:onEnterFrame()
    world:step(1/60, 8, 3)
    for body, bug in pairs(BODIES_LIST) do  --### Iterating through BODIES_LIST
        bug:setPosition(body:getPosition()) 
    end
end

onCollision()

MyScene.lua

local function onBeginContact(event)
    -- Test collision
    -- If true, get myBug's value and dispatchEvent
        myBug:dispatchEvent("collision")    
end
 
-- MyScene is listening to collisions
world:addEventListener(Event.BEGIN_CONTACT, onBeginContact)

Bug.lua

-- Listen to "collision"  and "destroy" events
function Bug:init()
    self:addEventListener("collision", self.onCollision, self)
    self:addEventListener("destroy", self.onDestroy, self)
end
 
function Bug:onCollision()      
    -- if collides with background, do nothing
        (...)
    -- if collides with player, dispatch "destroy" Event
        self:dispatchEvent("destroy")
end

Destroy and free()

Bug.lua

function Bug:onDestroy()
 
    -- Stop 
    self.body:setLinearVelocity(0, 0)       
 
    -- Free 
    BODIES_LIST[self.body] = nil   --### Modifying BODIES_LIST
 
    -- Clean body
    world:destroyBody(self.body)
    self.body = nil
 
    -- Clean sprite
    self.sprite:removeFromParent()
    self.sprite = nil
 
    -- Clean self
    self:removeFromParent()
    self = nil
 
end

Issue

MyScene.lua

Even when I’m printing a lot of things in the console, I can’t seem to find where the issue is coming from.
As far as I can see, the only place where I’m iterating through my table is in MyScene:onEnterFrame() :

-- ---------- Update bodies' position
--
function MyScene:onEnterFrame()
    world:step(1/60, 8, 3)
    for body, bug in pairs(BODIES_LIST) do
        bug(body:getPosition()) 
    end
end

But what I don’t understand is why would the app run for several minutes without raising an error and then suddenly it happens (even without user input).

If my code was totally wrong, an error would be raised right from when the app is launched, am I wrong?

Can you help?

I have been stuck with this issue for 2 days and I can’t stand it anymore.

Would you have any suggestion so that I could make some progress?
Would you make some progress anyway and release your game, knowing that this “should” not happen with a lot fewer objects on screen, and update later when you find out where this issue is coming from?

Thanks to anyone that can contribute.
I’m ready to send a bunch of virtual cookies to thank you!
@ar2rsawseen interested?

Mells

BJG · March 2013

Hi @Mells - 10/10 for post formatting.

So...bugs have bodies as well as other attributes, and you have a table of bugs indexed by bodies. You iterate over this and call a function on the bugs. Does this function destroy or modify the bodies in some cases; ie change the table index in the course of the loop...?

I'm wondering if you could use a table of bugs with a simple numerical index instead. Bugs and bodies are already associated, so you don't need a table that associates them again.

ar2rsawseen · March 2013

@Mells it's really hard to tell, all seems legit, but +1 to @BJG even if it won't solve the problem, it's much faster to loop through indexed array as:

local bug
 for i = 1, #BODIES_LIST do
    bug = BODIES_LIST[i]
    bug(bug.body:getPosition()) -- actually I don't understan how this one works, but that was in your code <img class="emoji" src="http://forum.gideros.rocks/resources/emoji/smile.png" title=":)" alt=":)" height="20" />
 end

Mells · March 2013

hey friends

@BJG, @ar2rsawseen

Does this function destroy or modify the bodies in some cases; ie change the table index in the course of the loop...?

Sorry I made a mistake while I was trying to simplify my code.

bug:setPosition(bug.body:getPosition())  -- and not bug(bug.body:getPosition())

So basically the list is not modified here, only the position of the bodies (and their parent 's object) are updated.
The position change will make some bodies to collide, an event is dispatched to the parent object (instance of Bug). Depending on the type of collision, the Bug instance dispatches a detroy event.
When destroyed, the BODIES_LIST list is modified.

I'm wondering if you could use a table of bugs with a simple numerical index instead.

I see, that's the first time I use box2D, so I looked at the example that is shipped with Gideros (Collision detection) and thought it was the best way to do it (see below).

local function createBox(x, y, name)
	local body = world:createBody{type = b2.DYNAMIC_BODY, position = {x = x, y = y}}
	actors[body] = sprite
end
 
local function onEnterFrame()
	world:step(1/60, 8, 3)
 
	for body,sprite in pairs(actors) do
		sprite:setPosition(body:getPosition())
		sprite:setRotation(body:getAngle() * 180 / math.pi)
	end
end

10/10 for post formatting

thank you, I know it's time-consuming for forum members to read/scan and answer.
A good presentation makes it easier.

I will try iterating through an indexed array.
The problem is that sometimes I have to play for more than 20 minutes to trigger the error, which is very frustrating (it's always triggered the moment that I think my problem is solved).

thank you again I will post an update later.

Mells · March 2013

Update : .. the error is still here.
This night will be long, again.

thank you for your help, and if anybody comes up with a suggestion I can not tell how much I'm interested to hear about it.

atilim · March 2013

@Mells Are you using the latest version of Gideros? Because with the latest version you can see where (name of .lua file and line number) this error occurs. For example, here is a sample code that causes this error:

local t = {a = 1}
for k,v in pairs(t) do
	t.a = nil
	t.b = 2
end

When I execute this code with latest version of Gideros, I can get this error:

invalid key to 'next'
stack traceback:
	main.lua:2: in main chunk

The name of .lua file and line number can help a lot while tracking the problem.

phongtt · March 2013

Maybe this line:

-- Free 
    BODIES_LIST[self.body] = nil   --### Modifying BODIES_LIST

conflicts with the iteration:

for body, bug in pairs(BODIES_LIST) do  --### Iterating through BODIES_LIST
        bug:setPosition(body:getPosition()) 
    end

Just a guess, don't blame me (haven't coded for years) :-\"

BJG · March 2013

I looked at the example that is shipped with Gideros (Collision detection) and thought it was the best way to do it

(I don't know much about Box 2D, but it sounds to me like in the built-in example, sprites and bodies are separate things which are associated via a table to give a set of "actor" pairs; whereas in your game the body is a property of the bug so you don't need the table for that.)

If you've now changed the code to avoid "pairs", maybe you could post up the new version...

Mells · March 2013

@atilim

Are you using the latest version of Gideros?

I was on version .09, updating to .10 gave me more informations thank you for the work done

gtween.lua:80: in function <.../gtween.lua:73>

Actually I am destroying my objects after a gtween is completed (alpha fade out animation). The error might come from that delay... but I have no idea why this happens so randomly.

@phongtt

Just a guess, don't blame me (haven't coded for years)

thank you for suggesting something

In that case I think the error would be raised right when the first instances are created. But I am not sure.

@BJG
Those are simplified versions of my current files.
I have added the fact that I am destroying objects after the end of a tween animation, I didn't think it was relevant but it seems that I was wrong.

Bug.lua

-- ---------- Create a new Bug instance
--
function Bug:init()
    (...)
    -- Add to the list of bodies to update each frame
    BODIES_LIST[#ACTORS_BODIES_LIST+1] = self
end

onEnterFrame()

MyScene.lua

-- ---------- Update bodies' position
--
function MyScene:onEnterFrame()
    world:step(1/60, 8, 3)
    local bug
    for i = 1, #BODIES_LIST, 1 do  --### Iterating through BODIES_LIST
        bug = ACTORS_BODIES_LIST[i]
        body = bug.body
        bug:setPosition(body:getPosition()) 
    end
end

onCollision()

-- Listen to "collision"  and "destroy" events
function Bug:init()
    self:addEventListener("collision", self.onCollision, self)
    self:addEventListener("destroy", self.onDestroy, self)
end
 
function Bug:onCollision()      
    -- if collides with background, do nothing
        (...)
    -- if collides with player, dispatch "destroy" Event
 
        ################
        self.animTween = GTween.new(self, .6, {y = self:getY() - 30, alpha = 0})				
        if not(self.animTween == nil) then
		self.animTween.onComplete = function()	
                      self:dispatchEvent("destroy")
                end
        end
        ################        
end

Destroy & free()

Bug.lua

 
function get_key_for_value( t, value )
     local k, v
     for k,v in pairs(t) do
          if v==value then return k end
     end
     return nil
end
 
function Bug:onDestroy()
    (...)
    -- Free 
    local id = get_key_for_value(ACTORS_BODIES_LIST, self)
    table.remove(ACTORS_BODIES_LIST, id)	
 
    -- Clean body
    world:destroyBody(self.body)
    self.body = nil
 
    -- Clean sprite
    self.sprite:removeFromParent()
    self.sprite = nil
 
    -- Clean self
    self:removeFromParent()
    self = nil
 
end

I really appreciate the help, thank you all.

fxone · March 2013

Just a guess, don't blame me (haven't coded for years) :-\"

@phongtt do you say serious? are you sitting only on the cash at Guava?!

atilim · March 2013

gtween.lua:80: in function <.../gtween.lua:73>

damn.. I'm supposed to fix it. I've been searching for this GTween bug for a long time.

@fxone

)

Mells · March 2013

@fxone -> @phongtt is showing us the way

@atilim
Ah? Do you mean that there is a (small) possibility that the error is not coming from my own code?

atilim · March 2013

@Mells I think there is big possibility that the bug is because of GTween and not coming from your code

Wait for my fix

Mells · March 2013

[-O<
If that's the case, it will *really* be a weight lifted off my shoulder.
I am awaiting for the fix

I've been searching for this GTween bug for a long time.

Does it mean you have an idea how to fix it now?

atilim · March 2013

@Mells oh yes. the line number (gtween.lua:80) is what I was searching for.

thank you and I'm sorry about wasting your time for all night :-\"

Mells · March 2013

thank you and I'm sorry about wasting your time for all night

No worries, we can say that this time was helpful if we could find the bug, and it will benefit all Gideros users

bowerandy · March 2013

@Mells, @atilim, I too am experiencing this problem but only very intermittently. I also think it is to do with GTween as it only seems to occur if I have a background tween running (a looping button "pulse" animation).

Looking forward to a fix.

best regards

bowerandy · April 2013

@Mells, @atilim, this has happened in my debugging five times today. I now think it's rather a show stopper for me.

It always happens on scene changes (although scene manager doesn't use tweens). However, that is the time I force a GC. I wonder if that is something to do with it.

Any news about the fix?

best regards

atilim · April 2013

@bowerandy this issue is in my urgent TODO list but I couldn't get to it yet. please wait a little bit more.

Mells · April 2013

@bowerandy
I have been waiting patiently for a fix but yes, it has been a showstopper for me since I reported the issue.
Still wondering why games that have been released didn't experience the issue, as my scene is pretty simple and this is happening very often when doing stress tests.

bowerandy · April 2013

@Mells, I'm trying out the following as a temporary fix.

function table.copy(t)
	-- Table shallow copy
	local u = {}
	for k, v in pairs(t) do u[k] = v end
	return setmetatable(u, getmetatable(t))
end

and in gtween.lua (line 80):

	for tween in pairs(table.copy(GTween.tickList)) do
		tween:setPosition(tween._position + (tween.useFrames and    
                      GTween.timeScaleAll or dt) * tween.timeScale)
	end

It's obviously rather inefficient but I haven't had a crash for two hours now. Perhaps you could try a similar fix and report back?

best regards

Mells · April 2013

@bowerandy
I will try as soon as I get the chance (currently away from home for a few days so can not guarantee it happens soon). Thank you for the alternative.
Hopefully @atilim will have come up with a magic trick before I get the chance to test your code.

atilim · April 2013

@bowerandy yes, that can be a temporary solution.

@bowerandy, @Mells One question: Are you calling GTween.stopAll() function anywhere? If so, that may be the problem and fix is easy.

bowerandy · April 2013

@atilim, my project does not have a call to stopAll() in it.

best regards

atilim · April 2013

@bowerandy damn!

let me investigate more

atilim · April 2013

After some hours looking at code, now I'm clueless.

@bowerandy I'll use your way to fix it but it's possible to make without creating a new table in each frame.

First create a local table for that purpose (You can put it at gtween.lua:66):

local copyTickList = {}

And change the loop at gtween.lua:80 with:

	for tween in pairs(GTween.tickList) do
		copyTickList[#copyTickList + 1] = tween
	end
	for i=1,#copyTickList do
		local tween = copyTickList[i]
		tween:setPosition(tween._position + (tween.useFrames and GTween.timeScaleAll or dt) * tween.timeScale)
	end	
	for i=#copyTickList,1,-1 do
		copyTickList[i] = nil
	end

It's fast and doesn't create any tables at each frame.

Let me organize the code and push it to github.

atilim · April 2013

Also I've look at the this code https://github.com/joshtynjala/gtween.lua/blob/master/gtween.lua shamelessly :-\" It also creates a copy of that table at each frame.

Mells · April 2013

@atilim

Are you calling GTween.stopAll() function anywhere? If so, that may be the problem and fix is easy.

In my scene GTween.stopAll() is never called before the issue happens (I use GTween.stopAll() to activate pause mode, only on user interaction).

Let me organize the code and push it to github.

Do you mean that this GTween.lua file will be updated in the coming days with a fix?
Also you both describe @bowerandy as a temporary solution, what are its limitations?

@atilim : I don't get the technical, so my question is : is it safe to use it in production and does it solve the issue in all cases?

bowerandy · April 2013

@Mells, I haven't seen any (confirmed) reoccurrence of the problem since I installed that fix but it has only been a couple of days. I say confirmed because I have seen a couple of crashes in a deployed app but I don't know what caused these. They could be normal Lua errors so I'm currently trying to add a stack trace to the report I get from Crittercism (crash logger) so I can see what's going on.

I haven't noticed any slow down with that fix (the table.copy version).

best regards

atilim · April 2013

Hi all,

Sorry for late reply. I've pushed the fix on github: https://github.com/gideros/GTween

The only unimportant downside of @bowerandy's approach is creating a table with each frame. I've just improved his solution a little bit.

best

bowerandy · April 2013

@atilim, thanks. I hadn't seen any further issues with my fix in place. I've now switched to using yours and will report back if I see anything further. Let's hope not!

best regards

Howdy, Stranger!

Categories

In this Discussion

Top Posters

Invalid key to 'next' - Help needed

What I have tried

Sum-up

The issue

Note

More details

Creating new Bug instances

MyScene:onEnterFrame()

onCollision()

Destroy and free()

Issue

Can you help?

Comments

onEnterFrame()

onCollision()

Destroy & free()

Howdy, Stranger!

Quick Links

Categories

In this Discussion

Top Posters

Invalid key to 'next' - Help needed

What I have tried

Sum-up

The issue

Note

More details

Creating new Bug instances

MyScene:onEnterFrame()

onCollision()

Destroy and free()

Issue

Can you help?

Comments

onEnterFrame()

onCollision()

Destroy & free()