AirPlay support in Gideros

Mells · October 2012

Hi,

I want to take a bet on the future of AirPlay and develop applications that take advantage of AirPlay.
I want to achieve the following.

My question are :

Is it possible to achieve it with Gideros?
Is Gideros one of the best way to achieve it or are there better alternatives for those specific needs.
I don't want to spend time digging into the framework : I am looking for a solution that has already made it easy to use AirPlay
Have you (dev) ever implemented AirPlay in your apps and are there issues that I should be aware of right from the beginning?
How hard would it be to develop an app that achieves the result above and what would be the major components?
- Best solution to implement Airplay as an extended screen, not only mirror mode = different visuals on the tv and on the ios device (imagine playing on your tv screen and having your inventory on the ipad in Zelda)
- Best solution for 2 (up to 8) player mode
- Best solution to make it possible to play on ipad/ipad, iphone/ipad, itouch/iphone, etc...
- Best solution to make additional sets of questions available through in app purchases
- Best solution to have some datas shared across all players in the world (like a leader board)
And ultimately, make it possible for somebody on the internet to join the game

If you have a few informations to share, that would already be really helpful.

Thank you!

OZApps · October 2012

@Mells, you might have already seen this post but still drawing your attention towards it

http://howto.oz-apps.com/2012/10/using-tv-out-for-extra-information.html

Mells · October 2012

@OZApps
Thank you for the link, there aren't many informations of that type available so that is helpful.
Btw, just recommended your book to a friend who preordered it.

Mells · October 2012

@OZApps
Could you elaborate on this please :

In other words, the native objects, images, rectangles, etc are only displayed on the OpenGL surface where as the new window we create is an iOS UIWindow with an UIView and this cannot display the native objects, so we almost have to create two sets of displays, one using the normal framework objects and an external one with UIKit.

I don't quite understand the limitation of creating an iOS UIWindow with an UIVIew (no experience), what would "creating a set of display with UIKit" mean?
Basically, does it mean I can't display a Gideros Sprite and if not, what should I do instead (or look at)?
I'm trying to figure out the next steps and to understand if it's realistic for me to even try with my current experience and skills.

@atilim Are there any plans to provide the ability to create several OpenGL surfaces?
(I don't even know if my question is realistic).

OZApps · October 2012

I think in a related thread, @Atilim did mention that there could be multiple OpenGL surfaces that we could use rather than simply stage:addChild(), we could use stage1:addChild() and stage2:addChild()

http://giderosmobile.com/forum/discussion/comment/13131

techdojo · October 2012

Going off topic slightly - but wasn't there plans for an app store for Apple TV devices.
You might be able to achieve this by actually running the quiz app on the AppleTV device using it's remote to setup the game etc and then just have a free "player" app running on an iDevice that just handles two way comms either via WiFi or Bluetooth - that certainly is doable.

Although maybe Ouya would be a better platform for the quiz "server"...

OZApps · October 2012

@techdojo,
there were some apps that create a server and then allow for polling activity like the devices used on game show like "Who want's to be a millionaire" when they want the public poll, or like e-clickers (used in some academic institutions) for feedback during a lecture.

While these are cool and fun, the idea of setting up the iPad to a TV or monitor is more like what is on the Who wants to be a millionaire, where the host sees a lot more while the monitor on the players screen is the question with the options in full screen. This can be used in various situations where the AppleTV solution might not.

Completely off-topic, if Apple add the Apple TV functionality into iMacs, so one can boot into the AppleTV or the iMac would be such a wonderful thing. If they would integrate a PIP and allow for input via HDMI that would be amazing.

Mells · October 2012

@OZApps
thank you for the links.
I will wait for @atilim to give us more informations about it.

@techdojo
What you describe seems doable (and interesting), but I think that this is different to what I am trying to achieve.

The following games use AirPlay and I would like to achieve the same kind of results :

Real Racing 2

MetalStorm : Wingman

When you are not connected to the Apple Tv, controls + game view both appear on the ipad or the iphone.

Update :

The MHL consortium is on a mission to take its technology of pushing mobile content onto big screens into every home.

Mobile High-Definition Link (MHL) links a smart phone to a TV's HDMI port through its patented connector, and the consortium which includes big names like Samsung, Sony and Toshiba is pushing hard on the technology to turn mobile games into the next generation of consoles.

Infographic: Mobile - The Top Gaming Platform Even At Home!

Source : Iqu.com

An interesting discussion :
What would be creative and innovative uses of AirPlay for an iOS app?

@OZApps
What is the feature that this would allow?
To use a Mac/iMac as an airplay receiver, or are you looking for extra features (video rental, hulu/netflix, etc)?

Currently it's possible to send an Airplay stream from Macs to Apple Tv (since Mountain Lion but only in Mirror Mode, not as an second screen).
It's also possible to use a Mac as an Airplay receiver (like AppleTv) sent by an iOs device through AirServer, ReflectionApp.
I believe there will be soon a solution to use AirPlay between Macs.

OZApps · October 2012

@Mells, no it is more to do with using the iMac as a TV, right now I can use iTunes and do whatever. I can use AirPlay to use the iMac as a second screen or a mirroring screen from my MBP or Windows Laptop, but cannot for the XBox or the PS3, so if Apple add the TV component into the iMac, that is the ultimate device. That's all.

Mells · October 2012

@OZApps
now I get it, I tried to tie it to the topic (Airplay) but you were talking about a wired solution. Hence the "off-topic"

OZApps · October 2012

@Mells, not exactly, when I look back, mentioning the PS3 and the XBox make it a wired option

If Apple did make the TV per-say then it would be in the housing of an iMac (literally) the size might be larger that's all. So why not integrate it right now? and also allow it to work with a lot more things wired and wireless.

However this is more thinking aloud and nothing to do with development or Gideros and hence Off-topic.

Mells · October 2012

@OZApps We all agree that the AppleTv and Airplay have the potential to open access to so many things, and that it is still in its infancy.
I am still wondering what are Apple's plans for it...
Anyway...

@OZApps now that we are here, assuming that we won't have access to several OpenGL surfaces for more than 6 months, can you elaborate on what you said above?

I don't quite understand the limitation of creating an iOS UIWindow with an UIVIew (no experience), what would "creating a set of display with UIKit" mean?
Basically, does it mean I can't display a Gideros Sprite and if not, what should I do instead (or look at)?
I'm trying to figure out the next steps and to understand if it's realistic for me to even try with my current experience and skills.

OZApps · October 2012

@Mells, in a very simple manner,

Gideros Objects are rendered via OpenGL and are displayed on the

stage

So when we create the external window and assign it a view, we can only display UIKit items on this UIWindow and UIView.

To have the TV Out, you will have to use BHWax from @Andy so while you use that, you can as well use UIView and the UIKit items.

techdojo · October 2012

@Mells - I wasn't aware those games provided that functionality (must take a look), do they actually require a wireless connection via an Apple TV or will they work via a HDMI connection kit (I have the latter, not the former

)

OZApps · October 2012

@Techdojo, I have not tried them with the AppleTV wireless connectivity however I have tried the HDMI and the Composite wired solution and both work.

The first game that I saw with this functionality was Max Adventure (https://itunes.apple.com/us/app/max-adventure/id396057778?mt=8) when connected to the TV, it displays the controls on the iOS device and the game on the TV. It has some very intelligent and advanced uses as @Mells has provided with the screenshots.

If you have the HDMI connection and an iPad2/iPhone4S or greater you are set.

Mells · November 2012

That technology (Airplay/Smartglass/etc...) opens a lot of possibilities.
That's why I would love if there were things to make the process of developing for it easier and faster.
From now I would like to make all my future apps support and exploit those features.

(Microsoft is in the game with Smartglass)

@OZApps
sorry for being so insistent, but as I have never touched BHWax and I don't know anything about UIView and UIKit, are there some limitations and/or difficulties by taking that way?
From experience, is it required to learn anything new other than my current Lua skills in order to manipulate those UI***?

Please understand that I am in no way asking for the job to be done for me and if I get a few more informations, I am ready to dive into those solutions.
But
I am trying to understand if it's worth investing my resources in exploring those dark sides of the dev universe

Or maybe it would be less expensive to hire a developer.
I don't know, just trying to get a better idea of the requirements and investments.

@techdojo
What Jayant said

But as I have never tried myself (only saw on the internet) I am wondering :
isn't the hdmi connection handled by iOs itself? If so, is iOs able to provide something else than a mirror mode (which is different from what I am describing above).

I don't know, if the Airplay feature is activated and handled ingame it's possible that the features available are a bit different (extended mode).

I just bought an airplay (works so great but is also so frustrating on a few points) but I don't own any Hdmi adapter for the ipad.

OZApps · November 2012

@Mells,
UIKit development is learning/knowing about Objective-C development. bhWax only provides a Lua wrapper to the same so you do not have to bother with the alloc, retain, release and autorelease anymore. The other thing is that all of objective-C APIs change and what looked like

[startCol setColor:[NSColor grayColor]]

will now look like

startCol:setColor(NSColor:grayColor())

and

[someObj doThis:<a href="http://forum.gideros.rocks/profile/this" rel="nofollow">@this</a> withThisNumber:5]

will look like

 somObj:doThis_withThisNumber("this", 5)

The one thing you will have to remember is that UIKit does not work the way Gideros does, so you are in essence creating a hybrid app one that displays stuff using UIKit and one that does with Gideros. Second and most important, whatever you display on the device will have the touch functionality unless you have a touch screen TV or monitor, so if you display the Gideros (openGL) on the device, then your game which is displayed on the external screen has to be written in objective-C (literally) and if you move the Gideros stage to the external screen then you have to write the touch code in Objective-C which will display and handle the touches.

All of this will be easier when @Atilim adds the multiple stages, that way you will be able to create a new stage and place it on the external screen window. Every object you add to either of the stages will be native Gideros objects and will work with the native Gideros API.

Hope that was not so confusing...

Mells · November 2012

@OZApps
That was a great explanation, thank you very much for taking the time.
That's all I wanted to know.

So my options are :

Wait for @atilim to implement multi stages
-> Being realistic I believe from what I have read on the forums that this won't happen within the 6 months coming... I would love to get an official confirmation, @atilim.
Learn Obj-c which will not happen because it will be very resources consuming and I started with Gideros to avoid that (though I understand I was trading features for ease of use)
Hire a developer -> which is a tough decision because I'll become very dependent from his progress if I want to release/modify often and (while cost-effective in some cases) it requires financial resources.
Also, using Airplay being something farily new, I will need to be able to experiment/test/iterate a lot.
Find a framework that allows me to use multiple stages : I don't even know if it exists, but if somebody knows about one, I'm interested to hear about it

Well...
it seems I can't make it happen for now and can't even make progress in that way.
My best chance lies in Atilim's answer.

OZApps · November 2012

@Mells, reference to #4, from what I know, there is no framework that offers multiple stages as yet, it might be in development stage for all you know, not in the production stage for sure.

Have you had a look at the posts on howto.oz-apps.com on using the UIKit controls with the bhWax plug-in from @Andy? You do not have to learn lot of Objective-C, it is rather easy. Mostly because the device will be used mainly for passing events to the main stage objects.

Mells · November 2012

As a reference for those following this discussion, @OZApps is referring to the following article on howto.oz-apps.com :

Get Developing for Free - Enterprise Features
@bowerandy's blog for additional informations : Hot Wax

@OZApps
I had read the article but thought that it was above my skills (looking at the API, something new to learn).
(Btw I encountered issues displaying articles on the ipad - probably a Blogger issue with the new layouts).

Now I took time to read it again, if you say that this is doable then I believe it is.
So if I understand my next steps are :

Display the Gideros stage on the bigger screen (Tv) by following your article : Using the TV Out for extra information (Obj-C)
On the ipad (touch screen) use BHWax from @Andy to create an iOS UIWindow with an UIView
- So I need to learn how to display sprites in Obj-C
- I need to learn (Obj-C again) how to handle events on displayed objects, and send those events to the main stage objects

#-o

OZApps · November 2012

@mells, it would be easier if you could share what you want to do either publicly or privately.

In short, yes you will create the images using UIImageView and UIImage and display/animate/reposition them on the device and set up touches or gesture controllers. BTW, you get Gestures for free... and yes you will have to learn a little Objective-C to manage the touches and update the display.

Another thing is that your code will still be in Lua, you are not really writing any Objective-C code, but you need to understand the iOS SDK to be able to work with it. There are plenty of entries on stackoverflow for snippets on objective-c so you can save some time there.

Mells · November 2012

Hi @OZApps
there would be many things that I would like to achieve, I am better at thinking about creative uses than finding out how to achieve it.
For example :

[ 1 ] I am working with a few teachers that are currently trying to bring tablet devices at school (ipad).
I would like to experiment with a setup where a child can draw a character on the ipad (free hand or with preexisting shapes Mii style), once done the character is sent to the big screen (next to the teacher), shapes are changed to box2D objects.
Each child could add his own character to the screen (something like this) :

Even better would be that each child can create a character on his own ipad and they all send/improve their characters simultaneously.

That would be great for a collective art experience.

[ 2 ]
I could start with something simpler (another situation, child playing alone in the living room) : the child plays on the ipad, and when he could find a solution to a problem a character is displayed on the big screen, an animation is played to congratulate him.

Recognize animals on the ipad (ex : elephant), a video showing a little bit more about elephants is played on the bigger screen.

[ 3 ]
Set up fireworks on your ipad, push "play" button, the fireworks are launched and played on the big screen.
Modify your fireworks setup, "restart", fireworks are updated on the big screen.

etc...
those are only a few ideas I got.

Ideally I would like to be able to do on both screens what Gideros currently allows.
If I understand well, I will have to create/update/modify my objects using iOS Api through BHWax, and manipulate them in Gideros with Lua.

@atilim
Can you give a time frame (+3months, +6months, +12months) for the implementation of multi stages in Gideros Mobile?

OZApps · November 2012

@Mells, here is something else to get you more excited, it might not be the right fit for the examples you provided, but it is still interesting and amazing...

It has to have a server that is sending the main image data, the devices are running just the client app so based on the pinch they determine the position and the orientation... or maybe something even more clever and interesting.

Mells · November 2012

@OZApps
Looks amazing, thanks for sharing.
Thinking about the kids I'm working with, we could pinch all screens and 'pair' them then give it randomly to children.
They would have to find the right place in a limited amount of time (Rubik's Cube style), or reproduce a drawing that appears on the bigger screen.

The big screen could even use a camera to display their placement on a floating window (PIP).

We could even make online and live contests between schools (reproduce picture, send photo online. The quickest school gets some points).

And if all those screens were displaying AR code in each, part of a bigger composition?
With cameras (smartphone), children would try to recompose the scenery by moving in the room, garden, etc...

I'm going to contact the PINCH developers and see if I can meet them, that would be an great opportunity to know more about it.

Anyway, I'm off-topic in my own discussion

OZApps · November 2012

@Mells, that is the Life of George, that is an amazing app/game and most of all LEGO!! There are two of these, i believe the older one is out of stock.

techdojo · November 2012

@Mell's I love the idea of all the kids "signing in" by sending their Mii's to a big screen, however rather than the multi-stage idea I would have thought that would be more in the domain of a client / server based app.

As @OzApp's said re the Pinch demo (amazing BTW) what your talking about is more a "distributed app" where all the devices have their own local apps running and then communicate via the main class room server to facilitate the sharing etc.

In fact I suspect you could probably do something akin to that already just by using a mobile optimised webapp running in safari, then it would (should) work on any device (or even a PC) which might make it a more realistic or feasible approach for a school to take.

Mells · November 2012

@OZApps Yes Life of George is something that I have kept an eye on for quite some time. There are a lot of things to learn from that project.

@techdojo
What you describe was another way to do things, but what I had in mind would be something a bit different : keep things local, just a router and an Airplay, no internet connection required. So that I don't need to handle a server, and at the same time I can bring it to many places where the internet is not available (I'm thinking about some hospitals where they have rooms for children but no internet - if wifi is allowed -, or I can bring it to some schools in the countryside).

Also I want to build a bridge between the apps and physical accessories (Life of George) and the way to monetize would be by selling it to each school, or directly in toy stores. That's why a "distributed app" was not what I had in mind.

But this is just an idea that came to my mind after I saw @OZApps post, I was already off topic, didn't take time to think serioulsy about it. It popped in my head like that

@ Atilim (I don't want to trigger another notification), can you give a time frame (+3months, +6months, +12months) for the implementation of multi stages in Gideros Mobile?

OZApps · November 2012

@mells, you can set up the server on the device itself instead of running a separate server. Look at AutoLAN from MyDevelopers, the code is open sourced (now) and is written in pure Lua implementation (however for C*SDK) and could be ported.

You haven't seen the post on Twitter on using magnets, have you? So many things to do and so little time ...

Mells · November 2012

@OZApps I am not sure I want to see that post on twitter

I'm still struggling to release my current app so I need to focus (buuuuut I'll check later).
(Will definitely check AutoLAN).

I will start my experiments with BHWax and TV out next week, thanks again for your help in that discussion.

OZApps · November 2012

@Mells, the link on twitter was about using magnets as a form of input on the mobile devices from a HCI point of view. Anyways, I know what you mean, I am unable to really focus on getting something done, maybe that is procrastination

techdojo · November 2012

@Mells - The thing I don't get is, (and correct me if I'm wrong), if you want to use AirPlay to control multiple displays then your talking about ONE app running on ONE device with ONE person orchestrating the show.

If you want collaboration between kids -> teacher, teacher -> kids and kids -> kids then your talking about multiple devices, each running their own app, controlling one display but sharing content (via Wifi or Bluetooth), as @OZApps said you don't need internet access, just a local adhoc wifi connection with one app running as the server (possibly the teachers) with the rest as clients (although a peer-to-peer layout would also possibly work) - this functionality already exists inside Gideros (given the right plugins)

Howdy, Stranger!

Categories

In this Discussion

Top Posters

AirPlay support in Gideros

Comments

Howdy, Stranger!

Quick Links

Categories

In this Discussion

Top Posters

AirPlay support in Gideros

Comments