BLOBs in Rho

Hi, All.

I am trying to bring some order and clarity into my understanding of Rho BLOBs. For someone who's never been dealing with something like Rho, I find it incredibly hard (and I have to tech newcomers).

There is a lot of docs/references and RhoConnect BLOB sync parths, but not a single tutorial on BLOBs in general.

So, below is "Rho BLOB beginner guide, as I understand them" draft .

Can you please provide feedback (is my understanding correct and full?), answer the questions in red and share your thoughts and experiences with BLOBs?

Let's build a community guide!

>>>>>>>>>>>>

Introduction

Normally, RHOM stores data in the local SQLite DB. However, large binary objects (pictures, files, etc) are not stored there.

[ Any particular reason for that? Performance? Memory? OS restrictions? Is this the reason why: Re: What is the maximum size of the Web SQL database? ]

Instead, RHOM stores them is a special folder on the device (Rho::Application.databaseBlobFolder, default: “<app_path>/db/db-files/”) and stores a relative PATH (URI, to be precise) to them in the SQLite database.

Working with BLOBs

Thus, every time you want to address the BLOB:

  • Get the BLOB URI from the Model records: @product.myblob
  • Convert that URI to an absolute file path: localpath = Rho::RhoApplication::get_blob_path(@product.myblob)
    • [I found it to work with relative URIs pretty well, why do I need to convert to absolute path every time?]
    • [Why wouldn't I store the absolute path in the DB instead of URI then? RhoConnect requirements?]
  • Do something with the file: <img src="localpath">, File.delete(locapath), etc..

    Sounds simple, right? We'll it's not that hard, really. However, there are some important things to remember:

    RhoConnect and sync in general

    RhoConnect sync client (client! In the app!) will recognize BLOBS as BLOBS if model has property set up as BLOB: property :myblob, :blob

    Then, RhoConnect will sync the file DATA (and update URIs in SQL database automatically). You still need to implement a source adapter method to deal with those BLOBS.

    More here: Rhomobile | Binary Objects and Blob Sync

    If you're not using RhoConnect for sync - you're on your own. Also see the next section.

    BLOB File Management

    Other than RhoConnect, no file management happens with BLOBs [true?]. For example, when you have a record with a BLOB in it (and associated file), the file will not be deleted after you delete the record.

    Same, if you, say, change the picture, associated with your records - old file will remain. With time, this junk accumulates to great proportions.

    Thus, you have to implement file management yourself, in controller or model (where applicable). Here's a quick example of hacking update_parameters in model

    alias_method :update_attributes_orig, :update_attributes

    def update_attributes(update_hash)

      if update_hash['Picture']

        # This is equivalent to: old_url = @vars[:Picture] ? @vars[:Picture] : ''

        old_url = @vars[:Picture] || ''

        File.delete(old_url) if File.exists?(old_url) && update_hash['Picture'] != old_url

      end

      update_attributes_orig(update_hash) #call the original method

    end

    Summary

    • BLOB is a file stored in a special folder
    • Only reference stored in the URI, which has to be converted to absolute path
    • RhoConnect handles sync automatically at the client side
    • But for everything else you have to implement your own file management and clean up.
    • As a workaround - can change the DB size and store BLOBS as Base64 encoded strings. Then decode and use them where needed.

    >>>>>>>

    Thanks for your time. Please let me know your thoughts!

    Jeffrey Cookle
    Hi Arsen, great information,

    Hi Arsen, great information, thanks for putting this together.    Were you able to display the images/blobs on the screen if you used <img src="localpath">?    I get a broken image if I attempt to do it that way by using the local path.    When using RhoSimulator, I am able to do this =>  <img src="/rhosimulator<%= @fsimages.image_uri %>"> and have it display the image properly.  But, when I build the application, I cannot get to the /rho/db... using that same method where the blobs are stored. 

    Are we able to change the Rho::Application.databaseBlobFolder default path to something inside of the /app directory to have the images synch into?   I would like to synch the files down to the client(which I currently can do), but them display them directly in the application, instead of having to copy or read them back in.

    Thanks!

    Vote: 
    Vote up!
    Vote down!

    Points: 0

    You voted ‘up’


    Jon Tara
    Given that the most common

    Given that the most common use of Blobs in Rhodes apps is likely images, putting the blobs in files makes the most sense, IMO.

    With the blobs in files, you can just reference the images directly from the filesystem path in your HTML.

    If the blobs were in the database, it would be necessary to copy them from the database into a file in order to use them from your HTML. Either that, or the Rhodes default controller would need to do that for you. e.g. something like /app/ModelName/{id}/blob/{field_name}. It would not need to copy them to the filesystem, but would still have to read them from the database and serve them to the browser. Depending on the size of your blobs, this could be very inefficient. Keep in mind, too, that the entire blob will have to be held in memory during this.

    As well, I came across some timing tests for SQLite. Blobs < 100K are indeed faster when accessed from the database. Blobs > 100K are faster from the filesystem. (Of course, that's a pretty general statement, would depend on specifies of filesystem efficiency.)

    I assume relative pats are stored because they are shorter.

    Vote: 
    Vote up!
    Vote down!

    Points: 0

    You voted ‘up’


    Jeffrey Cookle
    I must be doing something

    I must be doing something incorrect then.  The files have synched down to the local file system as files and when I try to reference the file via the full path from the c:/ drive, I get a broken image using img src=.

    If I read the file contents of the image and write it to a location inside the /app folder before it is viewed, then it works using the new uri location to the file.  I am using RhoStudio 5, maybe something changed in there(?).  I will keep looking.

    Thanks

    Vote: 
    Vote up!
    Vote down!

    Points: 0

    You voted ‘up’


    Jon Tara
    You should still construct

    You should still construct the path using code. Don't hard-code an absolute path. You shouldn't be coding "C:", etc. Android and iOS don't have a C: drive!

    You get a single blob folder. You can create subdirectories within that folder. You might organize the files by model or any other criteria of your choosing.

    Here's an example <img> tag from one of our apps, while running on the iOS simulator:

    <img class="base" src="/Users/jon/Library/Application Support/iPhone Simulator/7.1/Applications/066E4421-9BD8-4377-BB5B-621E7DAF7BB4/Documents/db/db-files/NPEKWE/000000_NPEKWE_02_Image_140903192126_sketch.png">

    Of course, this will be completely different on iOS device, Android simulator, Android device, Windows, etc. etc.

    db-files is the blob directory. We organize our image files by something we call a "short code" rather than by model. (It associates our photos/sketches with a particular home/commercial energy audit.) So, NPEKWE above is a "short code".

    In the database, you store the relative path from the blob directory. You need to create the full path using some Ruby code, making sure to use Rho::RhoFile.join() rather than pasting '/' between parts (because different on different OS.)

    Use Rho::Application.databaseBlobFolder to get the path to the blob folder. Do not assume it is called db-files or anything else about the path.

    BTW, if you want to paste the path into your browser URL bar, make sure to prefix it with file://

    Vote: 
    Vote up!
    Vote down!

    Points: 0

    You voted ‘up’


    Kutir Mobility
    Hi Have you tried the

    Hi

    Have you tried the suggestion from Jon Tara.

    Please let us know if you need further help.

    Thank you.

    Visnupriya R

    Kutir Mobility

    Vote: 
    Vote up!
    Vote down!

    Points: 0

    You voted ‘up’


    Jeffrey Cookle
    Hi, I did look at Jon's

    Hi,

    I did look at Jon's suggestion and I may have not explained what I did very well.  I am using this to get the local path of the file by passing in the URI of the file that was downloaded to the local file system, like Arsen mentioned above:

    <%localpath=Rho::RhoApplication.get_blob_path(@fsimages.image_uri)%>

    When I use localpath in an <img src="<%=localpath%>">   I get a broken image.  The path shows it to the proper location(on win32 it is c:/...) and when using RhoSimulator it shows the path as C:/Rhoprojects/fs_files/FSFiles/rhosimulator/db/db-files/id_090320141312133445.jpg

    I would like to directly use the file that was synched down to the file system, instead of my work around, which is to read and write the file to something inside the /app folder and then reference it as a URI =>   <img src="/tmp_image.<%= @fsimages.image_ext %>"/> when I wish to view it.

    Thank you for your assistance.

    Vote: 
    Vote up!
    Vote down!

    Points: 0

    You voted ‘up’


    Jon Tara
    Files do not have to be

    Files do not have to be within the /app folder, though, in order to reference them from HTML in the webview. See my example <img> tag above.

    As well, you should not be writing files in the /app directory.

    Vote: 
    Vote up!
    Vote down!

    Points: 0

    You voted ‘up’


    James Howard
    I have this same issue as

    I have this same issue as well. Rho::RhoApplication::get_blob_path returns a system path on win32 (and rhosimulator) which includes the drive letter. What I've had to do as a workaround is something like this:

    if System::get_property('device_name') == 'RhoSimulator'
      return '../rhosimulator/' + p.image_uri
    else
      return  Rho::RhoApplication::get_blob_path(p.image_uri)
    end
    

    This only accounts for rhosimulator, but the win32 fix should be similar ... but I wish get_blob_path would just return a webserver relative path.

    Vote: 
    Vote up!
    Vote down!

    Points: 0

    You voted ‘up’


    Jeffrey Cookle
    Hi James,Thanks for the

    Hi James,

    Thanks for the update.  I did submit this to support so a work around wouldn't be needed for rhosimlator.  I have not been able to get this to work in Win32 application, except having to read, copy, and write the files into the /app directory and use that path to there.

    Thank you!

    Vote: 
    Vote up!
    Vote down!

    Points: 0

    You voted ‘up’


    Kutir Mobility
    Hi 1.To address your blob

    Hi

    1.To address your blob problem, we would like to build a sample small application using blob and share the source and .cab with you. Could you please illustrate what specific small feature we can cover in it ?

    2. May we know on what devices you test your application?

    Thank you.

    Visnupriya

    Kutir Mobility

    Vote: 
    Vote up!
    Vote down!

    Points: 0

    You voted ‘up’


    Jon Tara
    It can be rather confusing,

    It can be rather confusing, as the paths are different on different OSs and whether you are running a simulator and then what kind of simulator!

    As well, the utility methods moved from RhoApplication to Application starting with 4.0 and the method names changed. (However, the 2.x RhoApplication with the old method names remains supported at least in 5.x).

    When get back to San Diego, I will try to make sense of it and post a 2.x -> 5.x "translation". It's a good exercise, as we have code that was written with 2.x that we've moved to 5.x but that we have not fully converted to the new APIs. (Though we have mostly - for example, we've re-written all of our network code - which is extensive - and application.rb.

    But currently, some of our blob code uses Application methods to create the directory paths, and some of it uses the old RhoApplication methods, depending on when the code was written. I'd like to get it all converted to the new API.

    Vote: 
    Vote up!
    Vote down!

    Points: 0

    You voted ‘up’


    Jeffrey Cookle
    I have everything working on

    I have everything working on both iOS and Android Devices with RhoConnect and Images now.

    The issue started with using RhoSimulator for win32(or others on Windows Platform) because the drive of C: was always being prefixed to the path.   I had to add code to remove the C: that was mentioned by James Howard's reply earlier .


    There is still an issue when building and using blobs with a win32 build and I have a support ticket open for that still.

    Thanks everyone for your assistance.

    Vote: 
    Vote up!
    Vote down!

    Points: 0

    You voted ‘up’


    Log in to post comments