6 Replies Latest reply on Apr 19, 2012 1:50 PM by Lars Burgess

    How to sync dependent objects

      I’ve been looking at RhoMobile over the last day or two and have achieved basic object transfer from our SQL Server to a Rhodes client app. I now need to understand a deeper level on RhoXX to understand how I proceed.

          

      I am trying to understand how dependant objects are handled. However looking into this has raised a lot of unknowns.

          

      So to solve my ‘problem’ I need to know how to sync a dependent object (I have a Job which has an FK to a customer).  I have set the belongs_to in the job model to indicate the relationship, but cannot see how the correct subset of customer data will be downloaded to Rhodes.

       

      This then opens up the wider unknowns of just how much data is moved around the system.  My basic app is syncing ‘Jobs’ and my source adaptor filters by the logged on user so I just get (say) two jobs going down to the device.  Now am I supposed to do this or should I be loading the entire Jobs table into the source adaptor and getting it all to the client and filtering there (probably not) – but then how do I get only the current customer entries round the system.

       

      The params argument on Query is null (I guess this gets passed through from the client SyncEngine.do_sync argument of query_params) which is never set in the default application code.  I’ve looked at search on the source adaptor but this has to be invoked. It would have expected the framework to deal with syncing just the data needed (using the belongs_to markup to decide what is needed) but can’t see how it works.  Of course the alternative is that there is a disconnect (via redis) of the data from the source adaptor and the data sent to Rhodes on the device, but in that case how is that filtering performed and controlled.

       

      So – does that mean I need all customers on my device to satisfy the two Jobs or ….

       

      Docs on docs.rhomobile.com seem a little sparse - are there any more detailed anywhere ?

       

      Thanks in advance

        • Re: How to sync dependent objects

          So to solve my ‘problem’ I need to know how to sync a dependent object (I have a Job which has an FK to a customer).  I have set the belongs_to in the job model to indicate the relationship, but cannot see how the correct subset of customer data will be downloaded to Rhodes.


          This then opens up the wider unknowns of just how much data is moved around the system.  My basic app is syncing ‘Jobs’ and my source adaptor filters by the logged on user so I just get (say) two jobs going down to the device.  Now am I supposed to do this or should I be loading the entire Jobs table into the source adaptor and getting it all to the client and filtering there (probably not) – but then how do I get only the current customer entries round the system.

          belongs_to in rhom only defines a trigger for a newly created record.  It is not used when data is downloaded.  This is useful when a parent and child object are created, the child object's reference to its parent will be updated automatically when the parent is synchronized.  This only applies to objects created on the device since temporary IDs are used locally before the record is pushed up to the server.

           

          You should synchronize all of the adapter data that you need on the client (filtered appropriately as you mentioned).  You can think of source adapters as namespaces for your data (or the @result hash you build in each one).  They don't have any inherent relationship to each other as it relates to how they are processed.

           

          The params argument on Query is null (I guess this gets passed through from the client SyncEngine.do_sync argument of query_params) which is never set in the default application code.  I’ve looked at search on the source adaptor but this has to be invoked. It would have expected the framework to deal with syncing just the data needed (using the belongs_to markup to decide what is needed) but can’t see how it works.  Of course the alternative is that there is a disconnect (via redis) of the data from the source adaptor and the data sent to Rhodes on the device, but in that case how is that filtering performed and controlled.

          Params are an additional argument you need to pass in your do_sync method.  The boilerplate client code just assumes you want to synchronize everything and you can add filtering as needed.  Search is different in that your search @result is always added to the adapter document (as opposed to replaced completely in query).  These are two different use cases:  query for complete dataset on each execution, search for incremental buildup of dataset as needed.

           

          So – does that mean I need all customers on my device to satisfy the two Jobs or ….

          You only need to synchronize enough customers to satisfy any related Jobs logic you need to perform.  To filter you can use the params approach from above or, if you know enough just based on who the user is, take advantage of the current_user variable in your adapter to filter the data.

           

          Docs on docs.rhomobile.com seem a little sparse - are there any more detailed anywhere ?

          We are continuously adding content to the site and would certainly welcome any suggestions you have on which content is missing or what you'd like to see covered in more detail.  It is also open source so we definitely welcome contributions too!  https://github.com/rhomobile/rhomobile-docs/

           

          -Lars

          1 of 1 people found this helpful
            • Re: How to sync dependent objects

              Thanks Lars, that's helpful

               

              So, I have to perform any 'joins' needed in my source adaptor to ensure the correct data set gets to the client.

               

              Following on from that is there any transactional concept to this - I can set sync priorities on sources (so customers are synced before contacts) but I presume that I could end up with all the customers on the device but not all the appropriate contacts.  Possibly a better example is order and orderlines - lets assume an ERP extension to mobile where we grab the order history for a customer:

               

              You'll have order model and orderline model (with orderline joining to order) sync priorities would grab the orders first and lines second.  Assuming a large number of records you could get a 'batch' that has orders, and then the next bach has half the lines and rthe final batch has the remaining lines.  If you lose connection between 2 and 3 you'll get an incomplete set on the PDA - can this be prevented?

               

              Next: what about transactions from client to server.  Lets assume I have a job object that has a status (assigned, in progress, complete). On completion I get a signature which is stored in a signature object which keys back to the job.  Jobs would have to have a higher priroity to be sync'd first.  Lets assume a job hits the mobile device status assigned, gets update to in progress, then a signature created, then job to complete.  When the next sync occurs the job will go first with status complete and the signature second. On my back end I will have a completed job with no signature (until the signature has sync'd). How can I stop this integrity violation ?

               

              Thanks

               

              Simon

                • Re: How to sync dependent objects

                  You'll have order model and orderline model (with orderline joining to order) sync priorities would grab the orders first and lines second.  Assuming a large number of records you could get a 'batch' that has orders, and then the next bach has half the lines and rthe final batch has the remaining lines.  If you lose connection between 2 and 3 you'll get an incomplete set on the PDA - can this be prevented?

                  Yes, rhoconnect does not assume to know about the structure of your app's data, so ordering the source adapters with priority as you mention would be the way to go.  You could prevent the above scenario by having a "Check" adapter run at the end and set a flag that everything succeeded.  In any case, if there is an error during the sync process, your callback is fired anyway so you can logout the user and delete the data (using the Rhom method) so you don't wind up with partial data on the client.

                   

                  Next: what about transactions from client to server.  Lets assume I have a job object that has a status (assigned, in progress, complete). On completion I get a signature which is stored in a signature object which keys back to the job.  Jobs would have to have a higher priroity to be sync'd first.  Lets assume a job hits the mobile device status assigned, gets update to in progress, then a signature created, then job to complete.  When the next sync occurs the job will go first with status complete and the signature second. On my back end I will have a completed job with no signature (until the signature has sync'd). How can I stop this integrity violation ?

                  One approach is to "aggregate" the data from your child adapters in redis using the Store.put_data method and have a "container" adapter pull out the data so you can make a single backend call with all the data in a transactional way.  Adapters are just containers for business logic, you don't need to hit the backend in every adapter, instead redis could be used as a staging area.

                  1 of 1 people found this helpful
                    • Re: How to sync dependent objects

                      I should also mention there is a conflict helper for client->rhoconnect changes that you should check out: http://edgedocs.rhomobile.com/rhoconnect/cud-conflicts

                      • Re: How to sync dependent objects

                        Ok - so is there anywhere that describes what happens to the data as it moves from client to sqlite to OTA to redis to adaptor so I can figure out how that might work.

                         

                        Where is the API for Store described ?  Presuambly the client updates the redis 'db' with changes and the adaptors then fire to undertake the operation required - can we query the store from an adaptor - this would enable a sequenced model where you have a 'transaction' model/adaptor that had lowest sync priority with attributes keying back to the master object involved - of course this assumes you can determine the changes that occured on the other 'objects' invovled from (presuamed) metadata in redis (?)

                          • Re: How to sync dependent objects

                            Ok - so is there anywhere that describes what happens to the data as it moves from client to sqlite to OTA to redis to adaptor so I can figure out how that might work.

                            The synchronization workflow (authorization & query) is described here: http://edgedocs.rhomobile.com/rhodes/synchronization#sync-workflow

                            Data moving from the client to rhoconnect and the adapter follows the same pattern: http post -> adapter method called with parsed post body data from redis -> backend system receives new data.

                             

                            Where is the API for Store described ?  Presuambly the client updates the redis 'db' with changes and the adaptors then fire to undertake the operation required - can we query the store from an adaptor - this would enable a sequenced model where you have a 'transaction' model/adaptor that had lowest sync priority with attributes keying back to the master object involved - of course this assumes you can determine the changes that occured on the other 'objects' invovled from (presuamed) metadata in redis (?)

                             

                            The store/redis interface is described here: http://edgedocs.rhomobile.com/rhoconnect/source-adapters#redis-interface

                            You are correct, data is first stored in redis and later processed by your adapter (this allows support for adapters to run in background jobs or separate processes).  RhoConnect doesn't currently flag dependent object adapter for you, but your dependent adapters could preserve this type of state in redis (via the Store interface) that is later consumed by the parent adapter.  We plan on providing a high level API around this use case in a future version, but it requires client as well as server changes so it isn't ready yet.