Log in or register to post comments

get video texture

August 24, 2011 - 10:44pm #1

I want to be able to get the camera image stream in Unity without any augmentations. I want to take each camera frame, convert it to a texture, modify that texture, and use it in the augmented scene. I want to do something similar you can see in this video:
at 2:50
http://www.youtube.com/watch?v=Y9HMn6bd-v8

How can I do this in Unity3d?

Re: get video texture

September 5, 2011 - 6:07pm #7
Quote:

How do I get the camera frame/raw data in iOS without using unity?

Start with the State object returned by QCAR::Renderer::getInstance().begin() in the renderFrame method. From the State object you can get the Frame, and from the Frame object you can loop through the available Images and choose the one with the format that interests you (e.g. QCAR::RGB565). You can register for different image formats using the QCAR::setFrameFormat() method.

See the API reference for more details: https://ar.qualcomm.at/qdevnet/api

- Kim

Re: get video texture

September 5, 2011 - 7:21am #6

Two things:

1)How do I get the camera frame/raw data in iOS without using unity?

2)I did a partial port of PTAM to the iPhone a while ago and there are now two new students working on fixing it so that it works correctly. It is true that the narrow FOV of the iPhone makes it less stable but it should still work well enough for augmentation. Also, the iPhone 5 will probably have a better camera and a faster processor making it entirely feasible to use the version that we will hopefully release in the next couple of months.
It is important to note though that because we are using the original PTAM code with modifications that it all still falls under the same license and cannot be used for profit without obtaining a license from Isis innovations.

Re: get video texture

August 31, 2011 - 8:45am #5

http://www.robots.ox.ac.uk/~gk/

http://ewokrampage.wordpress.com/projects-using-ptam/

There's source code available, though the iPhone source must be licensed. One thing about PTAM on mobiles is that the narrow FOV of most mobile phone cameras is less than ideal for PTAM. It compromises the integrity of the map that is generated and so reduces the area of the visual region that you can work with. But it's very cool all the same.

Re: get video texture

August 30, 2011 - 7:49pm #4

Thanks Kim, I will have a look. The performance problem isn't too much of an issue. By the time I am done programming, the devices will be fast enough. As long as it is possible in the first place, I am happy.

Re: get video texture

August 30, 2011 - 11:15am #3

WOW!

Do you guys have any plans to implement markerless tracking? like the PTAM example in the video that elecman posted above?

That would take AR gaming to whole another level!

-kedar

Re: get video texture

August 28, 2011 - 9:55am #2

That effect is quite cool :)

In theory this is possible with our current extension, but I think it might be difficult to get acceptable performance. See this thread for a discussion of working with the camera image:

http://ar.qualcomm.at/node/2000492

- Kim

Log in or register to post comments