"502 Proxy Error" and SizeLimit server-side errrors during upload.

[nataliaratnikova]

Site reported the error [1] returned by spacecount when trying to upload ~123K entries.
associated with the errors [2] in the web-server logs.
A work around was to use --level option for a higher aggregation and shorter upload record.

Several solutions to explore:

• increase size limit on the server side (estimate what is the reasonable size )
• calculate record size on the client side and issue a warning if it is too big (possible with a suggested work around).

[1]
Error from server 502(Proxy Error), output below:

<title>502 Proxy Error</title>

Proxy Error

The proxy server received an invalid response from an upstream server.
The proxy server could not handle the request POST /auth/complete/dmwmmon/datasvc/perl/storageinsert.

Reason: Error reading from remote server

[2] vocms0132/dmwmmon/error_log_20150506.txt:[Wed May 6 20:37:11 2015](3774) Apache2::SizeLimit httpd process too big, exiting at SIZE=589660 KB SHARE=8204 KB UNSHARED=581456 REQUESTS=3101 LIFETIME=556353 seconds

[nataliaratnikova]

Note, "proxy error" in this case is related to the proxy server, and has nothing to do with the user's proxy used for authentication.