Proxy Configuration Examples – JavaScript

Request Library

JavaScript has a Request library designed for simplicity and ease in making HTTP calls. It supports HTTPS and follows redirects by default. Here is the basic request structure:

const request = require('request');
request('http://www.google.com', function(error, response, body) {
  console.error('error:', error); 
// Print the error if one occurred
  console.log('statusCode:', response && response.statusCode); 
// Print the response status code if a response was received
  console.log('body:', body); 
// Print the HTML for the Google homepage.
});

Proxy Environment Variables

The Requests library recognizes the following environment variables:

  • HTTP_PROXY / http_proxy. The value should look like:http://PROXYHOST:PORT
  • HTTPS_PROXY / https_proxy. The value should look like:https://PROXYHOST:PORT

These should contain the hostname and port of the proxy server, and look like the above example. You can also include a username & password, so that the proxy value looks like http://username:[email protected]:port. For more details on environment variables with the Requests library, please see Controlling proxy behaviour using environment variables.

Proxy Option

Instead of using environment variables, you can also pass a proxy option to your request. The proxy option should contain the same kind of values as the environment variables above.

Here is a JavaScript code example using a single external proxy parameter. This sample is from Stackoverflow.

var request = require('request');
request({
  'url':'https://example.com',
  'method':"GET",
  'proxy':'http://PROXYHOST:PORT'
}, function(error, response, body) {
  if (!error && response.statusCode == 200) {
    console.log(body);
  }
})

More about nodejs

Follow the link for a brief overview of nodejs.

And, if you’re interested in the overall process of nodejs HTTP handling, you can read about it at Anatomy of an HTTP Transaction.

Axios Library

Axios is a library used with nodejs. Popular features include request and response interception and support for progress reporting and for older browsers. Axios is widely used for HTTP requests as an alternative to the Fetch API.

Axios can also make requests over HTTPS. However, there may be support issues affecting HTTPS proxies; for example, use of the GET method may trigger error responses. This Stackoverflow thread offers three possible solutions:

  • Pass the proxy through httpsProxyAgent using HTTP.
  • Via npm, install axios-https-proxy-fix. Then import axios from axios-https-proxy-fix.
  • Explicitly identify the port. (For ProxyEgg, this is 8080.)

Further Information

Selenium WebDriver and JavaScript

Selenium is a great tool for automated functional testing on websites in many programming/scripting languages. You can use Selenium and JavaScript together, along with nodejs, to run tests across most popular browsers.

The blog Selenium + JavaScript Best Practices offers tips on combining them for best results, as well as test writing in Selenium.

PhantomJS and CasperJS

PhantomJS is a headless web browser scriptable with JavaScript. It runs on Windows, Mac OS, Linux, and FreeBSD.

CasperJS enables scripting of full navigation scenarios in a simple interface and testing for PhantomJS.

PhantomJS is currently suspended and archived. For those who want to continue using it, version 2.1.1 is the most recent stable release. Continued use may not be advisable, however, because archiving and suspension have been in effect since March 2018.

Downloads and Information

  • PhantomJS use cases and features. This page also links to full documentation.
  • Download PhantomJS. Instructions for Windows, MAC OS, Linux, and FreeBSD.
  • CasperJS has a download link. The page also offers code examples for navigating to PhantomJS, testing, and scraping.

Proxy and auth Settings

PhantomJS has a --proxy setting for specifying the proxy host:port. It also has a --proxy-auth setting that unfortunately doesn’t work. 

So, to use username:password authentication, you must do the following in your code:

page.customHeaders = {'Proxy-Authorization': 'Basic '+btoa('USERNAME:PASSWORD')};

With CasperJS, the syntax is slightly different:

casper.page.customHeaders = 'Proxy-Authorization': "Basic #{btoa('USERNAME:PASSWORD')}"

 

IP Rotating Proxy Onsale

SPECIAL LIMITED TIME OFFER

00
Months
00
Days
00
Hours
00
Minutes
00
Seconds
First month free with coupon code FREE30