Log in or register to post comments

Can't load dataset from obb Vuforia 3

June 18, 2014 - 3:36am #1
The new version of Vuforia assure us that splitting application binary in Android should work when  loading datasets from the obb file as stated in the release notes:
  • "Added support to load device databases from APK Expansion files on Android. The Unity Extension will now associate targets authored in the scene using the dataset name, not the full path, so that databases can be loaded from a .obb file. The DatasetLoadBehaviour will also try to load databases in the StreamingAssets folder from common APK extension directories automatically."

But I am getting the following error:

I/Unity   ( 7319): Data set HOC_TARGET not present in application package, checking extended root locations

I/Unity   ( 7319):  

I/Unity   ( 7319): (Filename: ./artifacts/AndroidManagedGenerated/UnityEngineDebug.cpp Line: 49)

I/Unity   ( 7319): 

D/STATUSBAR-NetworkController( 2484): refreshSignalCluster: data=0 bt=false

D/IconMerger( 2484): overflowShown = false

D/IconMerger( 2484): moreRequired = false

V/WindowOrientationListener( 2298): nearestRotation : 0   Angle: 356   tilt: 57

I/Unity   ( 7319): NullReferenceException: Object reference not set to an instance of an object

I/Unity   ( 7319):   at DataSet.Exists (System.String path, StorageType storageType) [0x00000] in <filename unknown>:0 

Is there a special precedure to follow to make it work?

Thank you

 

Can't load dataset from obb Vuforia 3

September 29, 2015 - 9:39am #28

Hey,

You're right, application.version returns the unity version, not the bundle version. Sorry, my bad.

Here is a version that uses the created date of the original instead.

Best,

Mattijs

 

AttachmentSize
Package icon dataSetCopier.cs_.zip1.53 KB

Can't load dataset from obb Vuforia 3

September 29, 2015 - 8:01am #27

Hi MattijKneppers and thanks for this! I was aware of the problem but due to my workload I didn't have a time to do it.

I was going to add this to the package but looks like Application.version doesn't work...it should as on the Unity Ref file it exists but the only thing that I can find is application.unityVersion (but this is for something else)...

Can you tell me if there is any library or trick to make it work so I can update the package?

 

Thanks again!

 

Ale.

PS: Looks like Google will accept 100Mb APKs so maybe we no longer need this (except for apps bigger than 100mb of course!)

Can't load dataset from obb Vuforia 3

September 29, 2015 - 4:49am #26

Hi,

Thanks for this.

One issue I found: it looks like when selecting Run Only Once (which is highly preferable when working with large datasets), if a new version of the app is out containing a new dataset, the new version is never copied.

In an attempt to fix this, I changed your file dataSetCopier.cs to add checking for changes in data set names and the application version number.

I attached the new file.

Best,

Mattijs

AttachmentSize
Package icon dataSetCopier.cs_.zip1.29 KB

Can't load dataset from obb Vuforia 3

August 14, 2015 - 1:19pm #25

this worked incredibly well and easily. I have huge video files in my obb that I thought wouldnt work and they run perfectly. thank you so much.

Jordan

Can't load dataset from obb Vuforia 3

August 9, 2015 - 2:00pm #24

iguana02 wrote:

Hi, weeks ago I wrote a small plugIn to avoid hardcoding datasets names and amounts. It is really simple to use and you don't need to write a single line of code to move the data on the persistentDatapath. Most of my apps are split binary and with many dataSets so I had to avoid hardcoding stuff.

Here:

https://www.dropbox.com/s/jg71h8hontby1j7/dataSetLoader.unitypackage?dl=0

Hope it helps,

Ale.

 

Thank You very much. It save me lot of work. Good job!

Can't load dataset from obb Vuforia 3

March 28, 2015 - 8:55am #23

Never mind. I found your instructions.

:)

 

Thanks.

Can't load dataset from obb Vuforia 3

March 28, 2015 - 8:54am #22

Thanks. But, could you please explain how to use your plugin?

Can't load dataset from obb Vuforia 3

February 25, 2015 - 1:28pm #21

Hi, weeks ago I wrote a small plugIn to avoid hardcoding datasets names and amounts. It is really simple to use and you don't need to write a single line of code to move the data on the persistentDatapath. Most of my apps are split binary and with many dataSets so I had to avoid hardcoding stuff.

Here:

https://www.dropbox.com/s/jg71h8hontby1j7/dataSetLoader.unitypackage?dl=0

Hope it helps,

Ale.

 

Can't load dataset from obb Vuforia 3

February 24, 2015 - 3:17am #20

I have Solved this problem.

Just  follow exactly all those steps including the DAT/XML given in link below.

https://developer.vuforia.com/forum/faq/unity-how-can-i-handle-large-android-apps

than, just Open your DataSetLoadBehaviour.cs File

 

Add the Following lines of code in that file :

void Awake() {		
  AddOSSpecificExternalDatasetSearchDirs ();
}

What i found is that Vuforia SDK was not searching for ExternalDatabaseDirs, because this function ( AddOSSpecificExternalDatasetSearchDirs ) was not getting call from anywhere. 

Hope it resolves your problem.

Can't load dataset from obb Vuforia 3

September 2, 2014 - 1:04am #19

Ok,

so, for anyone interested in trying this out,

let me describe here my test project setup and also a small code snippet for the datasets extraction (working for me) that anyone in this thread may want to try out

 

Scene 0 (called "scene-0"):

  • only contains a standard Main Camera (Unity camera), with the following script attached as component:

public class ObbExtractor : MonoBehaviour {

  void Start () {
    StartCoroutine(ExtractObbDatasets());
  }
	
  private IEnumerator ExtractObbDatasets () {
    string[] filesInOBB = {"StonesAndChips.dat", "StonesAndChips.xml"};
    foreach (var filename in filesInOBB) {
      string uri = Application.streamingAssetsPath + "/QCAR/" + filename;

      string outputFilePath = Application.persistentDataPath + "/QCAR/" + filename;
      if(!Directory.Exists(Path.GetDirectoryName(outputFilePath)))
        Directory.CreateDirectory(Path.GetDirectoryName(outputFilePath));

        var www = new WWW(uri);
        yield return www;

        Save(www, outputFilePath);
        yield return new WaitForEndOfFrame();
    }

    // When done extracting the datasets, Start Vuforia AR scene
    Application.LoadLevel("scene-vuforia");
  }

  private void Save(WWW www, string outputPath) {
    File.WriteAllBytes(outputPath, www.bytes);
		
    // Verify that the File has been actually stored
    if(File.Exists(outputPath))
      Debug.Log("File successfully saved at: " + outputPath);
    else
      Debug.Log("Failure!! - File does not exist at: " + outputPath);
  }
}

Scene 1 (called "scene-vuforia"):

  • contains an ARCamera with StonesAndChips dataset, XML and DAT files are located under StreamingAssets/QCAR
  • the ARCamera has Activate and Load Dataset checkboxes checked in the Inspector for the Stones and Chips dataset
  • contains an Image Target with the Stones target selected
    • a simple Cube is added as child of the Image Target, to show some kind of target augmentation
  • contains a Directional light

 

Player Settings:

  • The Split Binary Option is enabled in the Inspector
  • The Write Access (in Player Settings for Android) is set to "External SD Card"

 

Additional notes:

  • the project also contains some additional large asset in the Asset folder, just for the sake of esxceeding the 50 MB size limit for the APK;
  • the one and only Image Target is the Stones one as described in Scene-1, and there are NO other custom scripts used to load or to activate the dataset (as these would conflict with the Data Set Load Behaviour of the ARCamera and may result in failure to properly load or activate the dataset).

 

 

Can't load dataset from obb Vuforia 3

September 1, 2014 - 8:19am #18

Allesandro, 

Thanks for your reply but as I stated in my postings, I did all that.  I followed exactly all those steps including the DAT/XML being present first,  and no, it didn't work.  And it appears I am not alone.

I even only used the samples, a completely clean environment, and so on.  So I don't know what to tell you beyond there is a problem of some kind, perhaps a testing problem against all possible devices or configs (mine is a Windows 8.1 system running your stuff, unity 4.5.1, and Eclipse Juno downloading to a 2012 version of Nexus 7 with 4.4.4 os).  I used to work in dev support, so I know how difficult it is to find complex interactions and cover all configs.

It is puzzling,but again, as I said, I followed the steps you outlined to the letter, and like others, it did not work.  But your other workaround did.

Best

 

Can't load dataset from obb Vuforia 3

September 1, 2014 - 5:45am #17

@CGA: happy that you were able to achieve the loading using my previous workaround.

 

However, it should be perfectly possible to Load datasets using the approach described here:

https://developer.vuforia.com/forum/faq/unity-how-can-i-handle-large-android-apps

If that approach does not work for you, you may want to make sure to check these points:

- make sure that the Load Data Set and Activate Data Set checkboxes are enabled in the ARCamera inspector

- make sure that the Image Targets in your scene are correctly associated to the targets belonging to one of the Datasets tht you have loaded and activated in the ARCamera inspector

- make sure that the Split Binary option is enabled

- and, VERY IMPORTANT, make sure that the dataset files (.DAT and .XML) are successfully written to the External path BEFORE Vuforia initializes (otherwise Vuforia may try to load the Datasets before these are actually available); this can be achieved by creating 2 scenes, as explained in the article above (I copy-paste the text here below):

  • If the loading is done asynchronously, you need to make sure that the Datasets have been saved to the destination locations (external storage) before the Vuforia initialization process is complete; to this aim, one option is to perform the Dataset extraction + saving using a separate Unity Scene, and then load the Vuforia scene when the extraction is complete. For example:
    • create an empty scene containing just one empty gameObject and attach it a script that performs the Dataset extraction + storing of the Datset files to the external storage location (as explained above)
    • when the extraction is complete, load a second scene that contains the Vuforia content (with your ARCamera, Image targets, etc.)

 

I have just verified once again (with SDK 3.0.9) the approach above with 2 scenes, an ARCamera, an Image Targets and a total Asset size of about 80 Megs, running on a Samsung S3 and onan Asus MemoPad HD7, and it worked fine on both devices.

 

I hope this helps.

 

Can't load dataset from obb Vuforia 3

August 28, 2014 - 1:06pm #16

I also couldn't get the 3.0.9 fix to work so had to fall back on Alessandro's earlier quick fix method (Thanks Alessandro!), which works fine for me (using 3.0.9 stuff, by the way).  

I don't know if it will help you or not, but I summarized my steps in another thread:  

https://developer.vuforia.com/forum/android/android-split-issue-unity3d

Hope this helps some.  

Now I can get some sleep finally...(long two days of this...)

Best,

 

 

Can't load dataset from obb Vuforia 3

August 20, 2014 - 5:33am #15

Hi,

Unfortunately it is not fixed. I've tried multiple things but I just cant seem to get it to work.

Any other fixes avaiable?

 

Cheers

Can't load dataset from obb Vuforia 3

July 22, 2014 - 12:08pm #14

UPDATE:

Vuforia 3.0.9 is now available for download. This version should fix the issue described below.

The complete procedure for OBB handling is described here:

https://developer.vuforia.com/forum/faq/unity-how-can-i-handle-large-android-apps

 

Can't load dataset from obb Vuforia 3

June 26, 2014 - 4:07am #13

You're welcome.

I will update this thread once the next release with the fix will be released (at the moment I do not have any ETA for this). Until then, you can use the workaround below.

Also, I have updated this FAQ with equivalent information:

https://developer.vuforia.com/forum/faq/unity-how-can-i-handle-large-android-apps

 

 

Can't load dataset from obb Vuforia 3

June 26, 2014 - 3:12am #12

Thank you so much Alessandro, your workaround works perfectly fine.

Hope the proper fix will be included in the next version ;)

Can't load dataset from obb Vuforia 3

June 25, 2014 - 4:32am #11

Update:

we have found the cause of the issue reported below in the latest Unity Extension 3.0.5 / 3.0.6. 

 

 

At present we don't have an estimate for a release with this fix, but you can correct this by:

 

1. Editing the DataSetLoadBehaviour.cs script (located under Assets/Qualcomm Augmented Reality/Scripts) by commenting out (or removing) the code inside the public override void AddOSSpecificExternalDatasetSearchDirs() {  ... } method (i.e. the implementation body of that method should be empty)

public override void AddOSSpecificExternalDatasetSearchDirs()
{  
  // empty implementation
}

 

2. Creating a C# script and attach it to your ARCamera, containing this code snippet (note: you will need to replace the mDataSetNames elements with your own dataset names):


private bool mDataSetsLoaded = false;
private string[] mDataSetNames = new string[] {"StonesAndChips.xml"};
	
// Update is called once per frame
void Update () 
{
  ImageTracker itracker = TrackerManager.Instance.GetTracker<ImageTracker>(); 
  if (itracker == null) return;

  if (!mDataSetsLoaded) {
    string dataSetRootDir = Application.persistentDataPath + "/QCAR/";
    foreach(var datasetName in mDataSetNames) {
      DataSet dataSet = itracker.CreateDataSet();
      if (dataSet.Load (dataSetRootDir + datasetName, QCARUnity.StorageType.STORAGE_ABSOLUTE))
        itracker.ActivateDataSet(dataSet);
    }

    mDataSetsLoaded = true;
  }

 

 

 

 

Can't load dataset from obb Vuforia 3

June 25, 2014 - 2:07am #10

Hi,

We are using the system you described AlessandroB, however when the AR scene is loaded after the files have been copied over, the application just quits, and we receive no logs identifying why this is happening.

It seems yo crash just after the application starts to check for a data set in an external location, and all we have to go on is this:

06-25 10:07:06.690: W/WindowManager(447): Force-removing child win Window{41e23bd0 u0 SurfaceView} from container Window{41d24f70 u0 uk.co.omobile.eurekar/com.qualcomm.QCARUnityPlayer.QCARPlayerNativeActivity}
06-25 10:07:06.690: W/ActivityManager(447): Force removing ActivityRecord{4186d818 u0 uk.co.omobile.eurekar/com.qualcomm.QCARUnityPlayer.QCARPlayerNativeActivity t21}: app died, no saved state
06-25 10:07:06.700: W/WindowManager(447): Failed looking up window
06-25 10:07:06.700: W/WindowManager(447): java.lang.IllegalArgumentException: Requested window android.os.BinderProxy@41dd9970 does not exist
06-25 10:07:06.700: W/WindowManager(447): 	at com.android.server.wm.WindowManagerService.windowForClientLocked(WindowManagerService.java:7923)
06-25 10:07:06.700: W/WindowManager(447): 	at com.android.server.wm.WindowManagerService.windowForClientLocked(WindowManagerService.java:7914)
06-25 10:07:06.700: W/WindowManager(447): 	at com.android.server.wm.WindowState$DeathRecipient.binderDied(WindowState.java:1047)
06-25 10:07:06.700: W/WindowManager(447): 	at android.os.BinderProxy.sendDeathNotice(Binder.java:493)
06-25 10:07:06.700: W/WindowManager(447): 	at dalvik.system.NativeStart.run(Native Method)
06-25 10:07:06.700: I/WindowState(447): WIN DEATH: null
06-25 10:07:06.735: D/Zygote(120): Process 4519 terminated by signal (11)
06-25 10:07:06.735: D/mali_winsys(634): new_window_surface returns 0x3000
06-25 10:07:06.760: W/InputMethodManagerService(447): Got RemoteException sending setActive(false) notification to pid 4519 uid 10127
06-25 10:07:06.765: W/Binder(601): Caught a RuntimeException from the binder stub implementation.
06-25 10:07:06.765: W/Binder(601): java.lang.NullPointerException
06-25 10:07:06.765: W/Binder(601): 	at android.inputmethodservice.IInputMethodWrapper.setSessionEnabled(IInputMethodWrapper.java:280)
06-25 10:07:06.765: W/Binder(601): 	at com.android.internal.view.IInputMethod$Stub.onTransact(IInputMethod.java:129)
06-25 10:07:06.765: W/Binder(601): 	at android.os.Binder.execTransact(Binder.java:404)
06-25 10:07:06.765: W/Binder(601): 	at dalvik.system.NativeStart.run(Native Method)

 

Can't load dataset from obb Vuforia 3

June 24, 2014 - 8:11am #9

Thanks fo confirming,

I am looking into this issue (and also using your code snippet for extraction, which is helpful );  

I will report back as soon as I have some updates...

 

Can't load dataset from obb Vuforia 3

June 24, 2014 - 8:04am #8

Yes, the write access option is set to external(sd card) in the player settings, and I can confirm that the files are correctly extracted to the persistant data path, I can even browse to the location and confirm that the extraction has been successful. I even tested it adding a wait of 10 seconds at the end of the coroutine that saves the files into the persistentDataPath in case it tries to load them before the are extracted from the obb, and it doesn't work either.

Can't load dataset from obb Vuforia 3

June 24, 2014 - 6:57am #7

kmo82win 

what do the Logs say ? do they confirm that the files are correctly extracted and stored under persistentDataPath ?

Also, have you set the "Write Access" setting  to "External (SD card)" in the Player Settings ?

 

Can't load dataset from obb Vuforia 3

June 24, 2014 - 4:01am #6

I have the same problem. My code worked with the older version of vuforia (2.x) but now it crashes with the 3.0.6 and obb split.

Can't load dataset from obb Vuforia 3

June 24, 2014 - 3:34am #5

Alessandro, thank you for clarifying the direction the team is taking to support Split Application Binary for Android builds.

But having followed your steps, I am still getting a crash in the application when trying to load the Dataset.

So this are the steps that I've followed:

1. created a first scene to extract and copy the Datasets to the suggested path /<external storage>/Android/data/your.package.name/files/QCAR/”:

void Start () {
    StartCoroutine(GetDatasets ());
}
	
protected IEnumerator GetDatasets () {
    string[] filesInOBB = {"QCAR/BM.dat", "QCAR/BM.xml", "QCAR/british_museum.dat", "QCAR/british_museum.xml", "QCAR/Hebe.dat", "QCAR/Hebe.xml", "QCAR/Hebe5Ang.dat", "QCAR/Hebe5Ang.xml", "QCAR/HOC_TARGET.dat", "QCAR/HOC_TARGET.xml", "QCAR/hoc2_2Wheel.dat", "QCAR/hoc2_2Wheel.xml", "QCAR/HocNew.dat", "QCAR/HocNew.xml", "QCAR/LAYER_GAME.dat", "QCAR/LAYER_GAME.xml", "QCAR/PedimentAng.dat", "QCAR/PedimentAng.xml"} ;
foreach (var filename in filesInOBB) {
  Debug.Log("Will load: " + filename);
  string uri = Application.streamingAssetsPath + "/" + filename;
  string targetFilePath = Application.persistentDataPath + "/" +filename;
  if(!Directory.Exists(Path.GetDirectoryName(targetFilePath)))
	Directory.CreateDirectory(Path.GetDirectoryName(targetFilePath));
	Debug.Log("WWW: " + uri);
	var www = new WWW(uri);
	yield return www;
	if (www.error != null) {
		Debug.Log("WWW Error!!!!!: " + www.error);
		yield break;
	}
	Debug.Log("Success!!!!! "+ Application.persistentDataPath + "/" + filename);
	File.WriteAllBytes(Application.persistentDataPath + "/" + filename, www.bytes);
	yield return new WaitForEndOfFrame();
  }
  StartGame();
}

private void StartGame() {
  Application.LoadLevel("BMStartScene");
}

2. Load the scene with all my augmentation as normal

Of course the project works perfectly without the split of the application binary, but when spplication is splitted, I am getting this error:

I/Unity   (22274): SetHint
I/Unity   (22274):  
I/Unity   (22274): (Filename: ./artifacts/AndroidManagedGenerated/UnityEngineDebug.cpp Line: 49)
I/Unity   (22274): 
D/dalvikvm(22274): Trying to load lib /data/data/com.gamar.bmgame/lib/libQCARWrapper.so 0x41818aa0
D/dalvikvm(22274): Shared lib '/data/data/com.gamar.bmgame/lib/libQCARWrapper.so' already loaded in same CL 0x41818aa0
D/dalvikvm(22274): Trying to load lib /data/data/com.gamar.bmgame/lib/libQCARWrapper.so 0x41818aa0
D/dalvikvm(22274): Shared lib '/data/data/com.gamar.bmgame/lib/libQCARWrapper.so' already loaded in same CL 0x41818aa0
D/QCAR    (22274): Setting Unity version 4.5.0
D/dalvikvm(22274): Trying to load lib /data/data/com.gamar.bmgame/lib/libQCARWrapper.so 0x41818aa0
D/dalvikvm(22274): Shared lib '/data/data/com.gamar.bmgame/lib/libQCARWrapper.so' already loaded in same CL 0x41818aa0
D/dalvikvm(22274): Trying to load lib /data/data/com.gamar.bmgame/lib/libQCARWrapper.so 0x41818aa0
D/dalvikvm(22274): Shared lib '/data/data/com.gamar.bmgame/lib/libQCARWrapper.so' already loaded in same CL 0x41818aa0
D/QCAR    (22274): dataSetExists
I/Unity   (22274): Data set HOC_TARGET not present in application package, checking extended root locations
I/Unity   (22274):  
I/Unity   (22274): (Filename: ./artifacts/AndroidManagedGenerated/UnityEngineDebug.cpp Line: 49)
I/Unity   (22274): 
I/ActivityManager( 2139): Process com.gamar.bmgame (pid 22274) (adj 0) has died.
D/SensorService( 2139): SensorDevice::activating sensor handle=0 ns=66667000 
V/LvOutput( 1832): initCheck: 0
D/iNemoSensor( 2139): setDelay(): handle(0), en(66667000)

Any further help will be aprreciated...

Can't load dataset from obb Vuforia 3

June 23, 2014 - 4:10am #4

Can't load dataset from obb Vuforia 3

June 23, 2014 - 3:51am #3

Hi,

let me bring some little clarifications on the statements that you took from the release notes:

 The DatasetLoadBehaviour will also try to load databases in the StreamingAssets folder from common APK extension directories automatically.

This is referring to the fact that the DataSetLoadBehaviour.cs component of the ARCamera in Vuforia will look for datasets in the following directories (in addition to the normal StreamingAssets folder)

  1.  "/<external storage>/Android/data/your.package.name/files/
  2. /<external storage>/Android/data/your.package.name/files/QCAR/

where your.package.name here refers to your app package name (or 'bundle identifier', as you specify it in the Unity  Player Settings, for example "com.my.org.myapp")

NOTE: the <external storage>/Android/data/your.package.name/files/ above is the location where the official Google Android Developer Guide suggests to extract the OBB package content.

see also http://developer.android.com/google/play/expansion-files.html

The 2 paths above do not represent however the storage location of your OBB files;

actually, when you enable the APK Split Option, the OBB expansion files will be stored at this path:

/<external storage>/Android/obb/your.package.name/main.1.your.package.name.obb
 
Note: <external_storage> here refers to one of these (equivalent) paths on your device storage:
  • /sdcard
  • /mnt/sdcard
  • /storage/sdcard0
  • /storage/extSdCard

So if you are reading the Obb file from the Android/obb location where the OBB are stored (see above "/<external storage>/Android/obb/....") and you are extracting (unzipping) it programmatically and saving the extracted content into one of the above directories under "Android/data", then Vuforia will be able to locate your Datasets in one of those directories and load them automatically (while in previous versions, this was not possible).

However, Vuforia does not provide a full end-to-end management system for your OBB packages, i.e. the OBB extraction part is still your responsibility and needs to be implemented in the application code.

 

See also this:

https://developer.vuforia.com/forum/faq/unity-loading-dataset-sd-card

 

Can't load dataset from obb Vuforia 3

June 20, 2014 - 1:09am #2

EDIT: Found it.

Any admins out there that can help?

Where did you see this?  I didn't see it in the release notes.  I hope it's fixed.  I have been waiting for a fix like this for months.

Log in or register to post comments