Client-side processing

Answered

Alden Gannon

• March 04, 2018 17:33

It would be great if the http get came from a browser instance.  I need that instance to run javascript ajax postbacks because my cache is filled by ajax and the simple get only loads the page container, not the content.  I use the cron job to keep the cache fresh.

0

• Official comment

  

  Nguyễn An Thuận

  • March 23, 2018 11:45
  • Edited

  Hi Alden,

  It's possible, but it'll require more server resources. It's not scalable either. We're sending thousands of HTTP requests every minute, but I don't think our servers are enough to launch a few hundred browser instances, even headless browser for really short web visits.

  However, you may try some website screenshot takers service with an API URL, so you can set cronjob with their endpoint and pass parameters via POST data.

  I've justed phantomjscloud.com service, and it seems working. WARNING: I do not own nor operate the service, and can't guarantee its availability/reliability in the long run.

  For example, I set cronjob with this URL:

  http://PhantomJScloud.com/api/browser/v2/a-demo-key-with-low-quota-per-ip-address/

  then at section Send HTTP Request, select HTTP Method: POST, enter

  { "url":"http://example.com", "renderType":"plainText" }

  into POST Data field, and Save cronjob.

  It will load and render your URL, execute JavaScript, and render as plain text.

  It gives away 500 pages a day (each page = 2 seconds AND 50KB) with its free account, so I guess it's fine if you set cronjob to run once or twice a day.

  I hope this helps.
• 

  Alden Gannon

  • March 04, 2018 18:34

  I've done a workaround by doing each http post manually.  Works for simple pages, but could quickly get very complicated.  Better to simply have a browser instance.

  0
• 

  Alden Gannon

  • March 23, 2018 16:45

  Actually, this looks like a great solution.  I'll look into it.  Thanks!

  0

Please sign in to leave a comment.