7 Replies Latest reply on Oct 14, 2013 5:45 AM by erika contiero

    Sync model ends with Nginx HTTP error 499

    erika contiero

      Hi all,

       

      we developed an enterprise app for a customer using Rhomobile on Android Motorola ET1, with RhoConnect as sync server on a CentOS server.

      In synchronizing a model with many records (retrieved by a backend DB select which takes some minutes), after 90 seconds we ran into a HTTP error 499, the Nginx error "Client Closed Request - Used in Nginx logs to indicate when the connection has been closed by client while the server is still processing its request, making server unable to send a status code back". Then our app tries again to sync this failed model, and the sync has success.

      My question is: in these situation, which is the "client" who closed the connection? The device, nginx, thin, backend? How can I set a higher timeout to avoid this error? I already set keepalive_timeout in ngnix.conf and proxy_connect_timeout, proxy_send_timeout, proxy_read_timeout in rhoconnect.conf to 3000...

       

      This is the interesting part of nginx log:  /opt/nginx/logs/access.log

      193.205.50.71 - - [12/Jun/2013:10:59:11 +0200] "GET /application?client_id=eaed62cb2c8e424a8ffc88dc56478262&p_size=500&version=3&source_name=Model1 HTTP/1.1" 200 83 "-" "-" "109.53.195.209" "0.220"

      193.205.50.71 - - [12/Jun/2013:10:59:12 +0200] "GET /application?client_id=eaed62cb2c8e424a8ffc88dc56478262&p_size=500&version=3&source_name=Model2 HTTP/1.1" 200 84 "-" "-" "109.53.195.209" "19.170"

      193.205.50.71 - - [12/Jun/2013:10:59:21 +0200] "GET /application?client_id=0558a1e85b86405f80b7c21466b3c68c&p_size=500&version=3&source_name=Model3 HTTP/1.1" 499 0 "-" "-" "83.103.24.202" "90.090"

       

      I thank you so much for your attention.

       

      Erika

        • Re: Sync model ends with Nginx HTTP error 499
          Pietro Francesco Maggi

          Hi Erika,

          do you've any log about this event on the device? (if this is a deployed app, you can setup a remote log to test using the RhoLog on github as a starting point).

           

          When this 90s expires, RhoConnect has already started to send data back to the device? e.g. what are the benchmarks on the relevant RhoConnect Source Adapter?

          You can enable statistics in RhoConnect for this.

           

          Ciao

          Pietro

          1 of 1 people found this helpful
            • Re: Sync model ends with Nginx HTTP error 499
              erika contiero

              Hi Pietro,

               

              I thank you for your answer and for your suggestion about logging tool and statistics.

               

              Now I can write here the device log at the end of a failed sync:

               

              Net| Operation finished with error 52: Server returned nothing (no headers, no data)

              Net| CURLNetRequest: METHOD = [GET] UEL = [http://myrhoconnecturl/application?client_id=fc1543b96da346f69f1ef3cfab863f84&p_size=500&version=3&source_name=Model3] BODY = []

              RhodesService | PING network FAILED.

              Sync | Fire notification. Source : Model3; Url :http://hocalhost:53876/app/Settings/sync_notify; Params:

              HttpServer | Process URI: '/app/Settings/sync_notify'

              APP | RHO serve: /app/Settings/sync_notify

              APP | Params: {"total_count"=>"0", "processed_count"=>"0", "cumulative_count"=>"0", "source_id"=>"6", "source_name"=>"ODL", "sync_type"=>"incremental", "status"=>"error", "error_code"=>"1", "error_message"=>"", "rho_callback"=>"1"}

               

              In the most cases the error comes before the server has sent any data, but I see also some cases of error after 3 or more responses with 500 records have been sent.

              I'm going to enable stats to have more information to provide about this error, and update this post.

               

              Thank you so much for your precious help.

              Erika

              • Re: Re: Sync model ends with Nginx HTTP error 499
                erika contiero

                Hi Pietro and all,

                 

                now we have the same error also in update of the same model: when the sync of the model starts, first of all the modified objects are sent to rhoconnect to update them, but in our case the set of modified objects can be huge (40 or 50 objects), and the operations of updating them in rhoconnect take more than 90 seconds. After this time, rhoconnect writes this line in /opt/nginx/logs/access.log

                 

                193.205.50.71 - - [14/Oct/2013:11:20:36 +0200] "POST /application HTTP/1.1" 499 0 "-" "rhodes-wm" "83.103.24.202" "90.090" upstream_addr: unix:/tmp/thin.0.sock

                 

                and the device receives this response:

                 

                I 10/14/2013 11:20:36:284 00001a0c              APP | Params: {"total_count"=>"0", "processed_count"=>"0", "cumulative_count"=>"0", "source_id"=>"6", "source_name"=>"ODL", "sync_type"=>"incremental", "status"=>"error", "error_code"=>"2", "error_message"=>" [...] The request timed out before the page could be retrieved. [...] }

                 

                in the error_message there is the html code of a page, I write here only the relevant message. I attach to this post the entire rholog.txt of the app at the moment of sync failure (with customer data replaced with samples).

                When this error occurs RhoConnect has not already started to send data back to the client, I suppose he sends something back only at the end of source adapter's "query".

                 

                Thank you for your attention and help.

                 

                Erika