12 Replies Latest reply on Sep 12, 2014 6:25 AM by Jeffrey Cookle

    BLOBs in Rho

    Arsen Bandurian

      Hi, All.

       

      I am trying to bring some order and clarity into my understanding of Rho BLOBs. For someone who's never been dealing with something like Rho, I find it incredibly hard (and I have to tech newcomers).

      There is a lot of docs/references and RhoConnect BLOB sync parths, but not a single tutorial on BLOBs in general.

       

      So, below is "Rho BLOB beginner guide, as I understand them" draft .

      Can you please provide feedback (is my understanding correct and full?), answer the questions in red and share your thoughts and experiences with BLOBs?

      Let's build a community guide!

       

      >>>>>>>>>>>>

      Introduction

      Normally, RHOM stores data in the local SQLite DB. However, large binary objects (pictures, files, etc) are not stored there.

      [ Any particular reason for that? Performance? Memory? OS restrictions? Is this the reason why: Re: What is the maximum size of the Web SQL database? ]

      Instead, RHOM stores them is a special folder on the device (Rho::Application.databaseBlobFolder, default: “<app_path>/db/db-files/”) and stores a relative PATH (URI, to be precise) to them in the SQLite database.

       

      Working with BLOBs

      Thus, every time you want to address the BLOB:

      • Get the BLOB URI from the Model records: @product.myblob
      • Convert that URI to an absolute file path: localpath = Rho::RhoApplication::get_blob_path(@product.myblob)
        • [I found it to work with relative URIs pretty well, why do I need to convert to absolute path every time?]
        • [Why wouldn't I store the absolute path in the DB instead of URI then? RhoConnect requirements?]
      • Do something with the file: <img src="localpath">, File.delete(locapath), etc..

         

        Sounds simple, right? We'll it's not that hard, really. However, there are some important things to remember:

         

        RhoConnect and sync in general

        RhoConnect sync client (client! In the app!) will recognize BLOBS as BLOBS if model has property set up as BLOB: property :myblob, :blob

        Then, RhoConnect will sync the file DATA (and update URIs in SQL database automatically). You still need to implement a source adapter method to deal with those BLOBS.

        More here: Rhomobile | Binary Objects and Blob Sync

        If you're not using RhoConnect for sync - you're on your own. Also see the next section.

         

        BLOB File Management

        Other than RhoConnect, no file management happens with BLOBs [true?]. For example, when you have a record with a BLOB in it (and associated file), the file will not be deleted after you delete the record.

        Same, if you, say, change the picture, associated with your records - old file will remain. With time, this junk accumulates to great proportions.

        Thus, you have to implement file management yourself, in controller or model (where applicable). Here's a quick example of hacking update_parameters in model

         

        alias_method :update_attributes_orig, :update_attributes
        def update_attributes(update_hash)
        
          if update_hash['Picture']
            # This is equivalent to: old_url = @vars[:Picture] ? @vars[:Picture] : ''
            old_url = @vars[:Picture] || ''
            File.delete(old_url) if File.exists?(old_url) && update_hash['Picture'] != old_url
          end
        
          update_attributes_orig(update_hash) #call the original method
        end
        
        
        

         

        Summary

        • BLOB is a file stored in a special folder
        • Only reference stored in the URI, which has to be converted to absolute path
        • RhoConnect handles sync automatically at the client side
        • But for everything else you have to implement your own file management and clean up.
        • As a workaround - can change the DB size and store BLOBS as Base64 encoded strings. Then decode and use them where needed.

        >>>>>>>

         

        Thanks for your time. Please let me know your thoughts!

          • Re: BLOBs in Rho
            Jeffrey Cookle

            Hi Arsen, great information, thanks for putting this together.    Were you able to display the images/blobs on the screen if you used <img src="localpath">?    I get a broken image if I attempt to do it that way by using the local path.    When using RhoSimulator, I am able to do this =>  <img src="/rhosimulator<%= @fsimages.image_uri %>"> and have it display the image properly.  But, when I build the application, I cannot get to the /rho/db... using that same method where the blobs are stored. 

             

            Are we able to change the Rho::Application.databaseBlobFolder default path to something inside of the /app directory to have the images synch into?   I would like to synch the files down to the client(which I currently can do), but them display them directly in the application, instead of having to copy or read them back in.

             

            Thanks!

              • Re: BLOBs in Rho
                Jon Tara

                Given that the most common use of Blobs in Rhodes apps is likely images, putting the blobs in files makes the most sense, IMO.

                 

                With the blobs in files, you can just reference the images directly from the filesystem path in your HTML.

                 

                If the blobs were in the database, it would be necessary to copy them from the database into a file in order to use them from your HTML. Either that, or the Rhodes default controller would need to do that for you. e.g. something like /app/ModelName/{id}/blob/{field_name}. It would not need to copy them to the filesystem, but would still have to read them from the database and serve them to the browser. Depending on the size of your blobs, this could be very inefficient. Keep in mind, too, that the entire blob will have to be held in memory during this.

                 

                As well, I came across some timing tests for SQLite. Blobs < 100K are indeed faster when accessed from the database. Blobs > 100K are faster from the filesystem. (Of course, that's a pretty general statement, would depend on specifies of filesystem efficiency.)

                 

                I assume relative pats are stored because they are shorter.

                  • Re: BLOBs in Rho
                    Jeffrey Cookle

                    I must be doing something incorrect then.  The files have synched down to the local file system as files and when I try to reference the file via the full path from the c:/ drive, I get a broken image using img src=.

                     

                    If I read the file contents of the image and write it to a location inside the /app folder before it is viewed, then it works using the new uri location to the file.  I am using RhoStudio 5, maybe something changed in there(?).  I will keep looking.

                     

                    Thanks

                      • Re: BLOBs in Rho
                        Jon Tara

                        You should still construct the path using code. Don't hard-code an absolute path. You shouldn't be coding "C:", etc. Android and iOS don't have a C: drive!

                         

                        You get a single blob folder. You can create subdirectories within that folder. You might organize the files by model or any other criteria of your choosing.

                         

                        Here's an example <img> tag from one of our apps, while running on the iOS simulator:

                         

                        <img class="base" src="/Users/jon/Library/Application Support/iPhone Simulator/7.1/Applications/066E4421-9BD8-4377-BB5B-621E7DAF7BB4/Documents/db/db-files/NPEKWE/000000_NPEKWE_02_Image_140903192126_sketch.png">

                         

                        Of course, this will be completely different on iOS device, Android simulator, Android device, Windows, etc. etc.

                         

                        db-files is the blob directory. We organize our image files by something we call a "short code" rather than by model. (It associates our photos/sketches with a particular home/commercial energy audit.) So, NPEKWE above is a "short code".

                         

                        In the database, you store the relative path from the blob directory. You need to create the full path using some Ruby code, making sure to use Rho::RhoFile.join() rather than pasting '/' between parts (because different on different OS.)

                         

                        Use Rho::Application.databaseBlobFolder to get the path to the blob folder. Do not assume it is called db-files or anything else about the path.

                         

                        BTW, if you want to paste the path into your browser URL bar, make sure to prefix it with file://

                          • Re: BLOBs in Rho
                            Kutir Mobility

                            Hi dd7d9349-5416-4aa2-9266-180824a24df4

                            Have you tried the suggestion from Jon Tara.

                            Please let us know if you need further help.

                             

                            Thank you.

                            Visnupriya R

                            Kutir Mobility

                              • Re: BLOBs in Rho
                                Jeffrey Cookle

                                Hi,

                                 

                                I did look at Jon's suggestion and I may have not explained what I did very well.  I am using this to get the local path of the file by passing in the URI of the file that was downloaded to the local file system, like Arsen mentioned above:

                                 

                                <%localpath=Rho::RhoApplication.get_blob_path(@fsimages.image_uri)%>

                                 

                                When I use localpath in an <img src="<%=localpath%>">   I get a broken image.  The path shows it to the proper location(on win32 it is c:/...) and when using RhoSimulator it shows the path as C:/Rhoprojects/fs_files/FSFiles/rhosimulator/db/db-files/id_090320141312133445.jpg

                                 

                                I would like to directly use the file that was synched down to the file system, instead of my work around, which is to read and write the file to something inside the /app folder and then reference it as a URI =>   <img src="/tmp_image.<%= @fsimages.image_ext %>"/> when I wish to view it.

                                 

                                 

                                Thank you for your assistance.

                                  • Re: BLOBs in Rho
                                    Jon Tara

                                    Files do not have to be within the /app folder, though, in order to reference them from HTML in the webview. See my example <img> tag above.

                                     

                                    As well, you should not be writing files in the /app directory.

                                    • Re: Re: BLOBs in Rho
                                      James Howard

                                      I have this same issue as well. Rho::RhoApplication::get_blob_path returns a system path on win32 (and rhosimulator) which includes the drive letter. What I've had to do as a workaround is something like this:

                                      if System::get_property('device_name') == 'RhoSimulator'
                                        return '../rhosimulator/' + p.image_uri
                                      else
                                        return  Rho::RhoApplication::get_blob_path(p.image_uri)
                                      end
                                      

                                       

                                      This only accounts for rhosimulator, but the win32 fix should be similar ... but I wish get_blob_path would just return a webserver relative path.

                          • Re: BLOBs in Rho
                            Jon Tara

                            It can be rather confusing, as the paths are different on different OSs and whether you are running a simulator and then what kind of simulator!

                             

                            As well, the utility methods moved from RhoApplication to Application starting with 4.0 and the method names changed. (However, the 2.x RhoApplication with the old method names remains supported at least in 5.x).

                             

                            When get back to San Diego, I will try to make sense of it and post a 2.x -> 5.x "translation". It's a good exercise, as we have code that was written with 2.x that we've moved to 5.x but that we have not fully converted to the new APIs. (Though we have mostly - for example, we've re-written all of our network code - which is extensive - and application.rb.

                             

                            But currently, some of our blob code uses Application methods to create the directory paths, and some of it uses the old RhoApplication methods, depending on when the code was written. I'd like to get it all converted to the new API.

                              • Re: BLOBs in Rho
                                Jeffrey Cookle

                                I have everything working on both iOS and Android Devices with RhoConnect and Images now.

                                 

                                The issue started with using RhoSimulator for win32(or others on Windows Platform) because the drive of C: was always being prefixed to the path.   I had to add code to remove the C: that was mentioned by James Howard's reply earlier .


                                There is still an issue when building and using blobs with a win32 build and I have a support ticket open for that still.

                                 

                                Thanks everyone for your assistance.