Great :-)

fixed

If you are running it on device, you need to check if the frame format is supported or not; for example, certain devices only support RGB565, but not RGB888 (and vice versa for certain other devices).

Any if you are running in Play Mode using the webcam of your PC, not that Play Mode only supports Grayscale.

The CameraDevice.Instance.SetFrameFromat() function returns true if the format is supported, false otherwise.

see also

https://developer.vuforia.com/forum/faq/unity-how-can-i-reset-frame-format-scene-reload

this is obv not working https://developer.vuforia.com/resources/dev-guide/unity-camera-image-access

if i try to get camera access with Webtexture logcat -> Failed to connect to camera service

script from middle of the page logcat->  RGB888 image is not available yet (once)

modified script with "until image not null" coroutine http://pastebin.com/kF0ydze3 logcat-> 

Hi,

I was trying to implement the drag object feature. I was refering to this link.
https://developer.vuforia.com/resources/dev-guide/unity-drag-ar-object-screen-your-finger

I followed the instructions (loaded the script to ARCamera and The teapot is a child of image tracker) but the object is not moving when I tried to move it.  I loaded the teapot and applied a texture and mesh collider. The tutorial mention about Mesh Collider. Is there anything I have to config special for that ?
Can you please help me on this. 
Looking forward for a sooner reply. 

Thanks a lot, 
Amal.

Hi,

When I further test it I saw its moving but its very slow. Its almost undetectable... is it normal ? How can I improve the drag ? make it more faster ?

Hi, 

Thank you very much DavidBeard and AlessandroB !!  sorry I didnt have an idea that my question is not related to Vuforia. I thought it has something to do with Vuforia. I will research more in Unity.

Thanks a lot for the support. Appriciate a lot. :)
 

Hi, happy to see that it works. 

Concerning your further question I would recommend then that you keep learning a bit about Android and Unity.

Please avoid posting questions that are not relevant to Vuforia.

Thanks.

try..

    if (Application.platform == RuntimePlatform.Android)
            {
                if (Input.GetKey(KeyCode.Escape))
                {
                   Application.LoadLevel('welcome_scene');
                    return;
                }

        }

Android's back button is mapped to the Escape KeyCode, which you can catch as an Input. You can add the above to an OnGUI script or Update callback so that it's checked for each frame. Application.LoadLevel can be called using either a scene name or index number based on its load order in Build Settings - http://docs.unity3d.com/Documentation/ScriptReference/Application.LoadLevel.html

Cool thanks. I have another small question. I need to check for colour ranges in pixels. I have done something like this to check the read colour range..

float r = pixels[3*((newWid-j-1)*origWid + (newHgt-i-1)) + 0] / 255.0f;

I need to get this value to a range from 0 to 255. So if I do 

int r =  pixels[3*((newWid-j-1)*origWid + (newHgt-i-1)) + 0];

will this do the trick ? 
Thank you very much for early reply and support. 
Appriciate a lot. :)

Hi,

Its working :) I tested it now :)

A another small question.

How can I switch between applications ? I have 2 different scenes in Unity. Now I need to prompt a welcome screen and promt the user to select which section to go. Then when he/she touch the back button on android device, it should go batck to welcome screen. Currently the back button also doesnt work when I install the apk to my android device. I have to terminate the task from task manager everytime. How can I configure this and give the user to navigate between scenes ?

Highly appriciate for the support. :)
Thank you very much.

Hi, the transparency is controlled by the alpha component, so for example the pixel at position (i, j) can be set to transparent by setting alpha = 0:

colors[i*newWid + j].a  =  0.0f;

Partial transparency can be obtained using values between 0 and 1 (for example 0.5f for a half-translucent pixel)

Hi,
In the algorithm that makes the image rotation by flipping the pixels, How can I make some selected area of pixels transparent ? 
e.g:    

colors[i*newWid + j].r  =  pixels[3*((newWid-j-1)*origWid + (newHgt-i-1)) + 0] / 255.0f;

colors[i*newWid + j].g  =  pixels[3*((newWid-j-1)*origWid + (newHgt-i-1)) + 1] / 255.0f;

colors[i*newWid + j].b =  pixels[3*((newWid-j-1)*origWid + (newHgt-i-1)) + 2] / 255.0f;

colors[i*newWid + j].a  =  1.0f

What are the values I should load to make the current pixel transparent ? 
Looking forward for a early reply. 
Thanks a lot.

you're welcome.

Cool thanks :) I will try it out and buzz if I get any issues on projecting the tracker coordinates to extract the image. Thanks again :)
Appriciate your support.

To do what you describe here, you have probably many possibilities (and this is a really wide feature);

one thing I would maybe consider is to try something like:

- compute the 3D coordinates of the 4 vertices (corners) of the trackable

- project each of those points from 3D space to 2D camera image coordinates (pixels)

- once you have the 4 corners in your camera image coordinates, you could write a simple algorithm to extract the part of the image which is delimited by the 4 segments that connect the 4 corners in the image space.

Note: this is just a general guideline of how you could proceed, but here I don't have any sample code snippet to share about this, so you will need to implement it on your own.

Also, other approaches might be possible as well, of course.

Thanks I will dig into more in Unity website. The base question is I am planning to do some image recognition algorithm to integrate with Vuforia. Actually the images I capture through camera, I need to remove their backgrounds and use them as icons. Is it possible ? if you have any clues please direct me. 
Thanks a lot :)

Glad to help !

concerning your .NET question: Unity C# scripts rely on "Mono" which is a kind of .NET equivalent (it should contain a subset of the whole .NET framework, but is is already very large and comprehensive).

So, you can use everything which is included with Mono.

However, I encourage you to search for more information in the Unity website, as you can get more up-to-date information there.

Awesome !! it works !! :)
Once I used the code it was flipped upside down. So I worked on the algorithm you gave and modified the code. Now its capturing the images perfectly. 
Thanks a million !!
By the way quick question. Can I use .Net frameworks in Unity scripts ? Will it work ? and is there anything special I have to do to use .Net framework ?

Here is the modified code so that other developers could refer. :)

for (int i = 0; i < newHgt; i++)
{
  for (int j = 0; j < newWid; j++) 
  {
    colors[i*newWid + j].r = pixels[3*((newWid-j-1)*origWid + (newHgt-i-1)) + 0] / 255.0f;
    colors[i*newWid + j].g = pixels[3*((newWid-j-1)*origWid + (newHgt-i-1)) + 1] / 255.0f;
    colors[i*newWid + j].b = pixels[3*((newWid-j-1)*origWid + (newHgt-i-1)) + 2] / 255.0f;
    colors[i*newWid + j].a = 1.0f;
  }
}

Thanks again ! :)

Hi, glad to see it works!

Now, about rotating the image, this is just a matter of swapping rows with columns in your pixel array;

I think you can just modify the code as follows:

int orig_wid = image.Width; //original width
int orig_hgt = image.Height; //original height

int new_wid = orig_hgt;//swap width with height
int new_hgt = orig_wid;//swap height with width
Texture2D caption = new Texture2D(new_wid, new_hgt, TextureFormat.ARGB32, false);
 
byte[] pixels = image.Pixels;
 
int numPixels = orig_wid * orig_hgt;
 
Color[] colors = new Color[numPixels];
for (int i = 0; i < new_hgt; i++)
{
  for (int j = 0; j < new_wid; j++) 
  {
    colors[i*new_wid + j].r = pixels[3*(j*orig_wid + i) + 0] / 255.0f;
    colors[i*new_wid + j].g = pixels[3*(j*orig_wid + i) + 1] / 255.0f;
    colors[i*new_wid + j].b = pixels[3*(j*orig_wid + i) + 2] / 255.0f;
    colors[i*new_wid + j].a = 1.0f;
  }
}
             
caption.SetPixels(0,0, hgt, wid, colors);
caption.Apply();
byte[] bytes = caption.EncodeToPNG();
File.WriteAllBytes(Application.persistentDataPath+"/caption.png", bytes);

Awesome !!! It works !!! No words to thank you enough :) It works like a charm.
Now the image is getting saved in the SD card. It saves the image 90 degrees rotated anticlockwise. How can I rotate the image back to normal when it saves?
Appriciate your help a lot.
Thank you very much. :)

Hi, you need to use a color array which represents your camera image;

what I would try is something like this:

Texture2D caption = new Texture2D(image.Width, image.Height, TextureFormat.ARGB32, false);

byte[] pixels = image.Pixels;

int numPixels = image.Width * image.Height;

Color[] colors = new Color[numPixels];
for (int i = 0; i < numPixels; i++)
{
  colors[i].r = pixels[3*i] / 255.0f;
  colors[i].g = pixels[3*i + 1] / 255.0f;
  colors[i].b = pixels[3*i + 2] / 255.0f;
  colors[i].a = 1.0f;
}
            
caption.SetPixels(0,0, image.Width, image.Height, colors);
caption.Apply();
byte[] bytes = caption.EncodeToPNG();
File.WriteAllBytes(Application.persistentDataPath+"/caption.png", bytes);

Hi,

I modified the code according to the link you sent me. Now it is saving an png image. But it's totally black. can you please point me the wrong that I have done.
here is the code I used to save the image I get from CameraDevice.GetCameraImage(m_PixelFormat) method. 

Texture2D caption = new Texture2D(image.Width, image.Height, TextureFormat.ARGB32, false);
Color[] colors = new Color[3];
colors[0] = Color.red;
colors[1] = Color.green;
colors[2] = Color.blue;
			
caption.SetPixels(0,0, image.Width, image.Height, colors);
caption.Apply();
byte[] bytes = caption.EncodeToPNG();
File.WriteAllBytes(Application.persistentDataPath+"/caption.png", bytes);

Thank you very much for helping me out. Appriciate a lot.
 

Hi,

glad that you solved your problem ! (and glad to help).

Concerning the saving of the image,

in your code, you are calling SetPixels(0, 0, image.Width, image.Height), which is probably incorrect;

actually you need to set the pixels that come from the Image camera buffer.

I would recommend to read this:

http://docs.unity3d.com/Documentation/ScriptReference/Texture2D.SetPixels.html

That's the only issue I see in the code, but also make sure to check in the Unity website as well, as this is very Unity-specific (not very much about Vuforia).

Hi, 

Thanks a million !!! Finally it works !! I dont know how but when I created a new project and applied the script to ARCamera it started working :)
Thanks a lot for the help. 

Next step is to save that image to my SD card in the phone. I used this code to save it. But seems not working. Could you please check and let me know what's wrong here or can you please guide me with a correct code.

Texture2D caption = new Texture2D(image.Width, image.Height, TextureFormat.ARGB32, false);
caption.SetPixels(0,0, image.Width, image.Height);
caption.Apply();
byte[] bytes = caption.EncodeToPNG();
File.WriteAllBytes(Application.dataPath + "/../testscreen.png", bytes);

Highly appriciate your help.
Thanks again. :)

Hi,

I tried the code and yes I attached it to the ARCamera. But it gives "RGB888  image is not available yet" :(
I tried changing the RGB888 to RGB565 but still the same. It then gives "RGB565  image is not available yet"  :(

What can I do to solve this problem ? Is it possible for you to mail me the sample project if I post my mail address ? 
It will be a great help to me.
Thank you,
Amal.

Ok. Just to be sure.

Have you attached the CameraImageAccess script to the ARCamera ? or to an ImageTarget ?

The right place is to attach it to the ARCamera (just in case).

Secondly, I have slightly modified the script, so that it does not set the mRegisteredFormat to true until the SetFrameFormat is successfull, see here below (you can try this and see if it makes any difference):

public class CameraImageAccess : MonoBehaviour , ITrackerEventHandler
{
    private Image.PIXEL_FORMAT m_PixelFormat = Image.PIXEL_FORMAT.RGB888;
    private bool m_RegisteredFormat = false;
    private bool m_LogInfo = true;

	void Start()
    {
        QCARBehaviour qcarBehaviour = (QCARBehaviour) FindObjectOfType(typeof(QCARBehaviour));
        if (qcarBehaviour)
        {
            qcarBehaviour.RegisterTrackerEventHandler(this);
        }
    }

	public void OnInitialized() 
	{
		//
	}
	
	public void OnTrackablesUpdated()
    {
        GetFrame();
    }
	
	private void GetFrame() 
	{
		if (!m_RegisteredFormat)
        {
            bool formatOK = CameraDevice.Instance.SetFrameFormat(m_PixelFormat, true);
			if (formatOK) {
            	m_RegisteredFormat = true;
			}
			else {
				Debug.Log("Format NOT OK, failed to set " + m_PixelFormat);
				return;
			}
        }
        if (m_LogInfo)
        {
            CameraDevice cam = CameraDevice.Instance;
            Image image = cam.GetCameraImage(m_PixelFormat);

			if (image == null)
            {
                Debug.Log(m_PixelFormat + " image is not available yet");
            }
            else
            {
                string s = m_PixelFormat + " image: \n";
                s += "  size: " + image.Width + "x" + image.Height + "\n";
                s += "  bufferSize: " + image.BufferWidth + "x" + image.BufferHeight + "\n";
                s += "  stride: " + image.Stride;
                Debug.Log(s);
                m_LogInfo = false;
            }
        }
	}
}

Hi,

I tried adding both RGB888 and RGB565 but none of them work. My device is a latest Samsung Galaxy S2 with Anrdoid version Ice cream sandwich update. So I'm sure it should be capable of capturing the image. But still it gives "Image.PIXEL_FORMAT.RGB565 image is not available yet"

here is my complete script. Additionally I added the OnGUI() method, so that I can see the debug messages on the device screen.

using UnityEngine;
using System.Collections;
public class CameraImageAccess : MonoBehaviour, ITrackerEventHandler
{
    private Image.PIXEL_FORMAT m_PixelFormat = Image.PIXEL_FORMAT.RGB888;
    private bool m_RegisteredFormat = false;
    private bool m_LogInfo = true;
    private string s = "";
    void Start()
    {
        QCARBehaviour qcarBehaviour = (QCARBehaviour) FindObjectOfType(typeof(QCARBehaviour));
        if (qcarBehaviour)
        {
            qcarBehaviour.RegisterTrackerEventHandler(this);
        }
    }
	
	void OnGUI() {
        GUI.skin.label.fontSize = 25;
		GUI.Label(new Rect(10, 700, 800, 100), s);
    }
	
    public void OnTrackablesUpdated()
    {
        if (!m_RegisteredFormat)
        {
            CameraDevice.Instance.SetFrameFormat(m_PixelFormat, true);
            m_RegisteredFormat = true;
        }
        if (m_LogInfo)
        {
            CameraDevice cam = CameraDevice.Instance;
            Image image = cam.GetCameraImage(m_PixelFormat);
            if (image == null)
            {
                Debug.Log(m_PixelFormat + " image is not available yet");
				s="image is not available yet";
            }
            else
            {
                string s = m_PixelFormat + " image: ";
                s += "  size: " + image.Width + "x" + image.Height + "";
                s += "  bufferSize: " + image.BufferWidth + "x" + image.BufferHeight + "";
                s += "  stride: " + image.Stride;
                Debug.Log(s);
                m_LogInfo = false;
            }
        }
    }
	public void OnInitialized()
	{
		
	}
}

I added this script as a component in the camera. Is that correct ? Is there anything wrong in the script ? Please help me out. Highly appriciate your help.
Thank you very much,
Amal. 

Hi, the script code that is in camera image access tutorial is correct (except that you need to manually add OnInitialized(), as you already did,  but everything else is fine)

I just tested it on my device and it runs Ok;

the only issue is that, if you run in Play Mode (using the PC webcam) the SetFrameFormat will fail, but on the device it should run fine;

if you see the error on the device, then it could be that your device is not capable of using RGB888; if that's the case, try using RGB565 instead.

Hi,

Yes that is the first thing I tried. But it didnt work. When I create that file the Unity says that I haven't Implemented the OnInitialized() function. So I added the additional code line to the file and then I added the component to camera.

public void OnInitialized()
	{
		
	}

It was suggested by you early in this post as well. Once I add this, it always gives

"Image.PIXEL_FORMAT.RGB888  image is not available yet" message.

I am using a Android Galaxy S2 to deploy the application. Do you have the properly implemented code ? Can you please help me out ? Appriciate your help.

Thanks a lot,
Amal.

Hi,

have you already read this article about how to get the camera image?

https://developer.vuforia.com/resources/dev-guide/unity-camera-image-access

Hi again,

For the past few days I'm trying to capture the camera image and save it in my device SD card. But still I cannot save the image. 
The only way I was able to save is a screenshot. But this will save the rendered AR content as well. I need only to save the plain camera image. I tried many ways like 

1)
Image img = CameraDevice.Instance.GetCameraImage(Image.PIXEL_FORMAT.RGB888); // this give a null pointer :(

2)
Application.CaptureScreenshot("Screenshot.png"); // this will save the virtually rendered AR content too :(

3)
//below code save a blank image :(
Texture2D captured = new Texture2D (640, 800);
Camera.current.Render();
RenderTexture.active = Camera.current.targetTexture;
captured.ReadPixels(new Rect(0,0,640,800),0,0);
captured.Apply();
RenderTexture.active = null;
byte[] bytes = captured.EncodeToPNG();
File.WriteAllBytes(Application.dataPath + "/testscreen.png", bytes);

Please can someone help me to write a complete C# code to save the plain camera image ? 
Highly appriciate your help. Looking forward for a early reply.

Thanks,
Amal.

You're welcome. Happy Unity learning !

Hi, I managed to open the project. After Importing the package I had to open the file from the project tab browser. I am new to Unity3D. Forgive me for my poor experience on Unity3D. Now I'm practicing Unity3D and getting used to it functionalities. Highly appreciate your help.

Thanks.

Hi, the easiest way to setup UDT sample is to load the existing sample (.unitypackage), via.

Assets -> Import Custom Package...

and select the userdefinedtargets-unity.unitypackage from the sample directory

There is no specific tutorial for that.

Hi,

Thanks a lot for the guidance. By following the information you gave I managed to save the camera image and put the button to GUI. Thanks again for that valuable information.

But still I couldnt figure our a way to configure the User-Defined targets. I downloaded the sample Unity3D project provided by Vuforia. But when I open it, there is nothing in the scene. I build it and install in my phone. Nothing happens. 

I think after importing them we have to put the camera and tracker and config the settings. I added the camera and tracker, but settings to config I dont know. Are there any tutorials regarding User-Defined Target Unity3D project setup ? Please help me. highly appreciate your help.

Thanks,
Amal

Hi,

1) How to add a button on the screen display (not virtual buttons define in image targets)  using C# code ? is it possible to do in C# script or do I have to jump to android code to add the button ?

You can do it in a C# script, no need to go to Android; you can use the GUI.Button method, placing your code in the OnGUI method of your Behaviour class, as explained here (look at the code snippet):

http://docs.unity3d.com/Documentation/ScriptReference/GUI.Button.html 

2) How can we capture an image from the camera and save it in the galery (using c# scripting in Unity3D) ?

We have a tutorial about capturing the camera image in Unity with the Vuforia API; have a look here: https://developer.vuforia.com/resources/dev-guide/unity-camera-image-access

How to save the image into the gallery is not a Vuforia-related question, you should probably search the Unity website for such information.

3) I downloaded the User-Defined Targets sample unity project. But when I open it, there is no setup on the view. So I add the AR camera to the screen and added the UserDefinedTargetBuilder to the screen. Rest of the steps I'm confused. Can you please guide me on the rest of the steps please?

Your best bet is to take a look at the existing UDT sample, and with a bit of patience, uderstand how the scripts, prefabs and components are working together.

HTH.