9 Replies Latest reply on May 22, 2013 12:58 PM by Lars Burgess

    Datastorage in RhoConnect

      We have some queries regarding Storage mechanisms in RhoConnect. We have created a RhoConnect instance, where the data is fetched from the back end database systems. The source adapter method to fetch the data is invoked when sync is triggered from the device.

      Questions we have are :

       

      1.      1. Is it possible to automatically fetch data from Backend system from RhoConnect (without sync initiation from the device)

      2.      2. Is it possible to store the data that is fetched above in RhoConnect instance?

      3.      3. Can the data stored be queried later ?

        • Re: Datastorage in RhoConnect

          1.      1. Is it possible to automatically fetch data from Backend system from RhoConnect (without sync initiation from the device)

          Yes, you can use something like resque-scheduler to queue jobs automatically at given intervals.  Note you will need to pass the username/partitionname to the job as the adapter needs some context when it is executed: https://github.com/bvandenbos/resque-scheduler

           

          2.      2. Is it possible to store the data that is fetched above in RhoConnect instance?

          The data while automatically be stored in redis when the adapter runs.  That is effectively what storing the data in the rhoconnect instance means.

           

          3.      3. Can the data stored be queried later ?

          Starting with 4.0 there is a simple public interface for querying rhoconnect adapter data:  http://edgedocs.rhomobile.com/rhoconnectapi/source-adapter-model-api-ruby#getdata

           

          For pre-4.0 versions it is a little bit more code but still possible:

           

          data = Store.get_data(@source.docname(:md))
          
            • Re: Datastorage in RhoConnect

              Thanks a lot Lars for the inputs.

               

              We have a system in which there is a large volume of data stored in back end database system that needs to be synchronized with the devices. We use an intermediate database between the actual backend and the devices. This helps in keeping the system up when the actual backend is down. There are also a few data manipulations that happen in the intermediate database.

               

              With the solutions you have mentioned,

              1. Is it possible for RhoConnect to play the role of the intermediate database by holding the data when the backend is down or the devices are offline?

              2. How much of data can RhoConnect cache, (we have checked the plans available for RhoHub hosting and saw that the maximum amount of storage is 5GB) given the system may be down for a longer time frame?. We have a data volume which could be much larger than this in a worst case scenario. What do you suggest if this is the case?

              3. Can the data stored on the redis instance be queried and edited from within RhoConnect ?

                • Re: Datastorage in RhoConnect

                  1. Is it possible for RhoConnect to play the role of the intermediate database by holding the data when the backend is down or the devices are offline?

                  Yes, though remember that redis, the data store RhoConnect uses, is an in-memory data store.  Therefore, if you're using it as a backup database or intermediate database make sure to have aggressive persistence settings: http://redis.io/topics/persistence

                   

                  2. How much of data can RhoConnect cache, (we have checked the plans available for RhoHub hosting and saw that the maximum amount of storage is 5GB) given the system may be down for a longer time frame?. We have a data volume which could be much larger than this in a worst case scenario. What do you suggest if this is the case?

                  This is really a "how much data can redis handle", since RhoConnect uses redis to store data.  Redis can handle enormous amounts of data, and starting with 4.0, you can shard the data across multiple redis servers, effectively giving you unlimited data storage capacity (as much ram as you can buy across all your servers).

                   

                  3. Can the data stored on the redis instance be queried and edited from within RhoConnect ?

                  Yes, using the Store api and the model data access methods that I mentioned earlier in this thread.