Log in or register to post comments

Dynamic image not rendered correctly on iPad

October 30, 2015 - 12:57am #1

Hi,

I have really frustrating problem. I have very simple app with just one frame marker (0). I load a jpeg dynically from a URL. I then render the jpeg as a texture on a plane game object. This works as expected on MacBook Pro in Unity Player (see attachment MacBook Pro), but is displaying as magenta square on iPad (see attachment iPad). It's running exactly the same code (see below).

Can someone tell me what I'm doing wrong? The frustrating part is that it works in Unity Player, but not on the device, which means I'm now hopelessly stuck. Can someone suggest an alternate way of doing this?

 

I'm using;

Vuforia 5.0.6

Unity 5.2.1

OS X El Capitan

Xcode 7.1

iPad 4th Gen iOS 8.3

 

Source code:

 

using UnityEngine;
using System.IO;
using System.Collections;

namespace Vuforia
{
    /// <summary>
    /// A custom handler that implements the ITrackableEventHandler interface.
    /// </summary>
    public class DynamicTextureTrackableEventHandler : MonoBehaviour,
                                                ITrackableEventHandler
    {
        #region PRIVATE_MEMBER_VARIABLES
 
        private TrackableBehaviour mTrackableBehaviour;

        private Texture2D texture;
        private string filePath;
    
        #endregion // PRIVATE_MEMBER_VARIABLES

        #region UNTIY_MONOBEHAVIOUR_METHODS
    
        IEnumerator Start()
        {
            mTrackableBehaviour = GetComponent<TrackableBehaviour>();
            if (mTrackableBehaviour)
            {
                mTrackableBehaviour.RegisterTrackableEventHandler(this);
            }
            texture = new Texture2D(400, 400, TextureFormat.DXT1, false);

            filePath = "http://10.20.11.38/Server/Services/Media1.jpg";

            WWW www = new WWW (filePath);

            yield return www;

            this.texture = www.texture;
        }

        IEnumerator LoadImageUrl(WWW www) {
            yield return www;
        }

        #endregion // UNTIY_MONOBEHAVIOUR_METHODS

        void OnGUI()
        {
            GUI.Box (new Rect (10, 20, 780, 35), filePath);
            GUI.Box (new Rect (10, 70, 600, 600), this.texture);
        }

        #region PUBLIC_METHODS

        /// <summary>
        /// Implementation of the ITrackableEventHandler function called when the
        /// tracking state changes.
        /// </summary>
        public void OnTrackableStateChanged(
                                        TrackableBehaviour.Status previousStatus,
                                        TrackableBehaviour.Status newStatus)
        {
            if (newStatus == TrackableBehaviour.Status.DETECTED ||
                newStatus == TrackableBehaviour.Status.TRACKED ||
                newStatus == TrackableBehaviour.Status.EXTENDED_TRACKED)
            {
                OnTrackingFound();
            }
            else
            {
                OnTrackingLost();
            }
        }

        #endregion // PUBLIC_METHODS

        #region PRIVATE_METHODS

        private void OnTrackingFound()
        {
            Renderer[] rendererComponents = GetComponentsInChildren<Renderer>(true);
            Collider[] colliderComponents = GetComponentsInChildren<Collider>(true);

            // Enable rendering:
            foreach (Renderer component in rendererComponents)
            {
                component.enabled = true;
            }

            // Enable colliders:
            foreach (Collider component in colliderComponents)
            {
                component.enabled = true;
            }

            Debug.Log("Trackable " + mTrackableBehaviour.TrackableName + " found");

            GameObject go = GameObject.CreatePrimitive (PrimitiveType.Plane);
            go.transform.parent = mTrackableBehaviour.transform;
            go.transform.localPosition = new Vector3 (0f, 0f, 0f);
            go.transform.localRotation = new Quaternion (0f, 90f, 0f, 0f);
            go.transform.localScale = new Vector3 (0.15f, 0.15f, 0.15f);
            go.GetComponent<Renderer>().material.mainTexture = this.texture;
            go.SetActive (true);
        }

        private void OnTrackingLost()
        {
            Renderer[] rendererComponents = GetComponentsInChildren<Renderer>(true);
            Collider[] colliderComponents = GetComponentsInChildren<Collider>(true);

            // Disable rendering:
            foreach (Renderer component in rendererComponents)
            {
                component.enabled = false;
            }

            // Disable colliders:
            foreach (Collider component in colliderComponents)
            {
                component.enabled = false;
            }

            Debug.Log("Trackable " + mTrackableBehaviour.TrackableName + " lost");
        }

        #endregion // PRIVATE_METHODS
    }
}

 

AttachmentSize
Image icon iPad.JPG1.32 MB
Image icon MacBook Pro.jpg1.33 MB

Dynamic image not rendered correctly on iPad

April 14, 2016 - 6:13pm #6

Hi,

I've come back to this problem and been trying to solve it, and it appears to be iOS specific. Basically as described below I have a very simple scene with AR Camera (using Vuforia extension for Unity) and one frame marker. When frame marker is recognised (by Vuforia), the OnTrackingFound() method is called. In this method I load a JPG image, convert it to texture (doesn't matter whether I use PVRTC compression or not), create a 3D plane programatically and render the texture on the plane (see code below). 

Now this works fine in Unity Editor. However, when I build for iOS and then deploy to iPad (via Xcode) the texture is not rendered (magenta coloured shape with the correct dimensions is rendered). I have definitively proven that this is NOT a texture loading/compression problem. Because in Unity I can add the 3D plane at design time (rather than programatically as in my code below) while adding the texture at run-time and such setup works on iPad (i.e. correct image is rendered).

So my question is why is adding 3D plane in Unity Editor produces the correct texture being rendered on iPad while adding 3D plane in code (like below) doesn't. Can some one tell what is missing in my code?

 

Now I'm using:

Unity 5.3.4
Vuforia 5.5.9
iPad Air 2 (16GB)
iOS 9.2.1

 

private void OnTrackingFound()

{

bool planeFound = false;


string filename = "/Users/E-Learning/Documents/media/Media2.jpg";


byte[] bin = File.ReadAllBytes (filename);

//Create texture. Size doesn't matter because LoadImage will replace with the incoming image size

var tex = new Texture2D (2, 2);

tex.LoadImage (bin);
 
float ws = Convert.ToSingle (tex.width) / 1000;
float hs = Convert.ToSingle (tex.height) / 1000;
Debug.Log (string.Format ("Height scale: {0}. Width scale: {1}", hs, ws));
 
foreach (Transform child in mTrackableBehaviour.transform.GetComponentInChildren<Transform>())
{
if (child.gameObject.name == (mTrackableBehaviour.TrackableName + "Plane"))
{
Debug.Log (mTrackableBehaviour.TrackableName + "Plane found.");
planeFound = true;
child.gameObject.transform.localScale = new Vector3 (ws, 0.1f, hs);
break;
}
}
 
if (!planeFound)
{
GameObject plane = GameObject.CreatePrimitive (PrimitiveType.Plane);
plane.name = mTrackableBehaviour.TrackableName + "Plane";
plane.transform.parent = mTrackableBehaviour.transform;
//plane.transform.localScale = new Vector3 (0.1f, 0.1f, 0.1f);
plane.transform.localScale = new Vector3 (ws, 0.1f, hs);
plane.transform.localPosition = new Vector3 (0f, 0.01f, 0f);
plane.transform.localRotation = Quaternion.Euler (new Vector3 (0f, 180f, 0f));
 
MeshRenderer renderer = plane.gameObject.GetComponent<MeshRenderer> ();
renderer.material.mainTexture = tex;
renderer.enabled = true;
 
plane.SetActive (true);
}
 
Renderer[] rendererList = GetComponentsInChildren<Renderer>(true);
foreach (Renderer renderer in rendererList) {
//renderer.material.mainTexture = tex;
renderer.enabled = true;
}
Collider[] colliderComponents = GetComponentsInChildren<Collider>(true);
foreach (Collider collider in colliderComponents)
{
collider.enabled = true;
}
}

Dynamic image not rendered correctly on iPad

April 13, 2016 - 2:05am #5

Ok, the work-around that I found (i.e. creating 3D Plane object in Unity Editor) is no longer appropriate. The reason is that I now have a requirement to load a different prefab at run-time. That is, based on metadata that another app passes to Unity/Vuforia app I need to either render 3D Plane with JPG image as texture, or to render Video prefab with MP4 residing in some shared folder (not streaming assets).

So I have confirmed what I have said in this forum. The magenta box (i.e. the issue) is NOT due to the way I load/initialise JPG/texture. I have proven this by creating the 3D Plane in Unity Editor, as opposed to programmatically (as I have shown below in this post). If the plane is created in Unity Editor, iPad app displayes image correctly. If the plane is created at run-time, the image is a magenta box.

Running on:

iPad Air 2 (16GB)

iOS 9.2.1

Unity 5.3.4

Vuforia 5.5.9

 

The commented out code below is how I create the plane at run-time. So as I said, if I statically create the plane in the Editor, it works. If I dynamically create the plane (commented out code), then it doesn't work.

 

        private void OnTrackingFound()
        {
            TextureWrapper wrapper;
            bool planeFound = false;

            if (TrackableNameToTextureMap != null) {
                if (TrackableNameToTextureMap.TryGetValue (mTrackableBehaviour.TrackableName, out wrapper)) {

                    float ws = Convert.ToSingle (wrapper.Tex.width) / 1000 * Convert.ToSingle (wrapper.ScalingFactor);
                    float hs = Convert.ToSingle (wrapper.Tex.height) / 1000 * Convert.ToSingle (wrapper.ScalingFactor);
                    Debug.Log (string.Format ("Height scale: {0}. Width scale: {1}", hs, ws));

                    foreach (Transform child in mTrackableBehaviour.transform.GetComponentInChildren<Transform>())
                    {
                        if (child.gameObject.name == (mTrackableBehaviour.TrackableName + "Plane"))
                        {
                            Debug.Log (mTrackableBehaviour.TrackableName + "Plane found.");
                            planeFound = true;
                            child.gameObject.transform.localScale = new Vector3 (ws, 0.1f, hs);
                            break;
                        }
                    }

//                    if (!planeFound)
//                    {
//                        GameObject plane = GameObject.CreatePrimitive (PrimitiveType.Plane);
//                        plane.name = mTrackableBehaviour.TrackableName + "Plane";
//                        plane.transform.parent = mTrackableBehaviour.transform;
//                        //plane.transform.localScale = new Vector3 (0.1f, 0.1f, 0.1f);
//                        plane.transform.localScale = new Vector3 (ws, 0.1f, hs);
//                        plane.transform.localPosition = new Vector3 (0f, 0f, 0f);
//                        plane.transform.localRotation = Quaternion.Euler (new Vector3 (0f, 180f, 0f));
//                        plane.SetActive (true);
//                    }

                    //MeshRenderer renderer = plane.gameObject.GetComponent<MeshRenderer> ();
                    //renderer.material.mainTexture = wrapper.Tex;
                    //renderer.enabled = true;

                    Renderer[] rendererList = GetComponentsInChildren<Renderer>(true);
                    foreach (Renderer renderer in rendererList) {
                        renderer.material.mainTexture = wrapper.Tex;
                        renderer.enabled = true;
                    }
                    Collider[] colliderComponents = GetComponentsInChildren<Collider>(true);
                    foreach (Collider component in colliderComponents)
                    {
                        component.enabled = true;
                    }
                }
            }

            Debug.Log("Trackable " + mTrackableBehaviour.TrackableName + " found");
        }

Dynamic image not rendered correctly on iPad

November 4, 2015 - 10:17pm #4

I figured out how to make it work. So basically I created a 3D plane in Unity (GameObject menu->3D Object->Plane) and placed it as a child of the frame marker. I then modified the tracking method like so:

private void OnTrackingFound()
        {
            Renderer[] rendererComponents = GetComponentsInChildren<Renderer>(true);
            Collider[] colliderComponents = GetComponentsInChildren<Collider>(true);

            // Enable rendering:
            foreach (Renderer component in rendererComponents)
            {
                if (mTrackableBehaviour.TrackableName == "FrameMarker0")
                {
                    component.material.mainTexture = this.texture;
                    Debug.Log ("TEXTURE HAS BEEN SET.");
                }
                component.enabled = true;
            }

            // Enable colliders:
            foreach (Collider component in colliderComponents)
            {
                component.enabled = true;
            }

            Debug.Log("Trackable " + mTrackableBehaviour.TrackableName + " found");

}

So there was nothing wrong with the way I was loading the image and converting it to texture. The problem is that when I create the plane dynamically at run-time it doesn't work. That is my question, how do you create this Plane object dynamically? In other words, why the following code doesn't work (I guess this question belongs on the Unity forum rather than here):

            if (go == null) {
                go = GameObject.CreatePrimitive (PrimitiveType.Plane);
                go.transform.parent = mTrackableBehaviour.transform;
                go.transform.localPosition = new Vector3 (0f0.01f0f);
                go.transform.localRotation = new Quaternion (0f90f0f0f);
                go.transform.localScale = new Vector3 (0.2f0.2f0.2f);
                go.GetComponent<Renderer> ().material.mainTexture = this.texture;
            }
            go.GetComponent<Renderer> ().enabled = true;
            go.GetComponent<Collider> ().enabled = true;
            go.SetActive (true);

Dynamic image not rendered correctly on iPad

November 4, 2015 - 4:12am #3

... When I built Unity project and deployed it the screen was always black. I then unticked "Auto Graphics API" in Player Settings and instead added "OpenGLES2" option. The deployed app then had a functioning camera (i.e. not black screen), ...

Good to hear that Vuforia works fine. By the way, the OpenGL settings for Vuforia and Unity can also be found here:

https://developer.vuforia.com/forum/faq/unity-player-settings

 

...but the jpeg was still NOT rendered (i.e. it is still magenta-coloured box)....

That part is about Unity programming (how to create/render a Texture2D object from a JPEG), not about Vuforia. Have you seacrhed the Unity docs and Forums ? 

for example, here is one potentially related:

http://answers.unity3d.com/questions/432655/loading-texture-file-from-pngjpg-file-on-disk.html

 

 

 

Dynamic image not rendered correctly on iPad

November 1, 2015 - 11:02pm #2

Today I've tested with the same environment, but using iPad Air with iOS 9.1. When I built Unity project and deployed it the screen was always black. I then unticked "Auto Graphics API" in Player Settings and instead added "OpenGLES2" option. The deployed app then had a functioning camera (i.e. not black screen), but the jpeg was still NOT rendered (i.e. it is still magenta-coloured box).

Can anyone please help by explaining how to dynamically render image (jpeg)? Or if there's nothing wrong with my code, then what should I be checking/configuring/etc???

Log in or register to post comments