sys-proxy v1.0.3

SystemYA Proxy

Quickstart:

git clone https://github.com/sysce/proxy ./sys-proxy

node ./sys-proxy/demo

Installation:

npm i sys-proxy

Demo:

See the demo folder for more usage examples

var nodehttp = require('sys-nodehttp'),
	rewriter = require('sys-proxy'),
	server = new nodehttp.server({
		port: 7080,
		static: path.join(__dirname, 'public'),
	}),
	rw = new rewriter({
		prefix: '/service',
		codec: rewriter.codec.xor,
		server: server,
		title: 'Service',
		// http_agent: ..,
		// https_agent: ..,
		// ruffle: ..,
		// adblock: ..,
	});

// [0000] server listening on http://localhost:7080/

API:

Table of Contents

• index
  • Parameters
  • Properties
  • url
    • Parameters
  • unurl
    • Parameters
  • js
    • Parameters
  • css
    • Parameters
  • manifest
    • Parameters
  • html
    • Parameters
  • html_attr
    • Parameters
  • plain
    • Parameters
  • decode_blob
    • Parameters
  • attr_type
    • Parameters
  • headers_decode
    • Parameters
  • headers_encode
    • Parameters
  • cookie_encode
    • Parameters
  • cookie_decode
    • Parameters
  • decode_params
    • Parameters
  • decompress
    • Parameters
  • valid_url
    • Parameters
  • str_conf
  • globals
    • Parameters
  • html_serial
    • Parameters
  • wrap
    • Parameters
  • checksum
    • Parameters

index

Rewriter

Parameters

• config Object
  • config.adblock Boolean? Determines if the adblock.txt file should be used for checking URLs
  • config.ruffle Boolean? Determines if ruffle.rs should be used for flash content
  • config.ws Boolean? Determines if websocket support should be added
  • config.codec Object? The codec to be used (rewriter.codec.plain, base64, xor)
  • config.prefix Boolean? The prefix to run the proxy on
  • config.interface Boolean? The network interface to request from
  • config.timeout Boolean? The maximum request timeout time
  • config.title Boolean? The title of the pages visited
  • config.http_agent Object? Agent to be used for http: / ws: requests
  • config.https_agent Object? Agent to be used for https: / wss: requests
• server Object nodehttp/express server to run the proxy on, only on the serverside this is required

Properties

• mime Object Contains mime data for categorizing mimes
• attr Object Contains attribute data for categorizing attributes and tags
• attr_ent Object Object.entries called on attr property
• regex Object Contains regexes used throughout the rewriter
• config Object Where the config argument is stored
• URL Object class extending URL with the fullpath property

url

Prefixes a URL and encodes it

Parameters

• value
• data Object Standard object for all rewriter handlers (optional, default {})
  • data.origin Object The page location or URL (eg localhost)
  • data.base Object? Base URL, default is decoded version of the origin
  • data.route Object? Adds to the query params if the result should be handled by the rewriter
  • data.type Object? The type of URL this is (eg js, css, html), helps the rewriter determine how to handle the response
  • data.ws Object? If the URL is a WebSocket
• null-null (String | URL | Request) URL value

Returns String Proxied URL

unurl

Attempts to decode a URL previously ran throw the URL handler

Parameters

• value
• data (optional, default {})
• null-null String URL value

Returns String Normal URL

js

Scopes JS and adds in filler objects

Parameters

• value String JS code
• data Object Standard object for all rewriter handlers (optional, default {})
  • data.origin Object The page location or URL (eg localhost)
  • data.base Object? Base URL, default is decoded version of the origin
  • data.route Object? Adds to the query params if the result should be handled by the rewriter

Returns String

css

Rewrites CSS urls and selectors

Parameters

• value String CSS code
• data Object Standard object for all rewriter handlers (optional, default {})
  • data.origin Object The page location or URL (eg localhost)
  • data.base Object? Base URL, default is decoded version of the origin
  • data.route Object? Adds to the query params if the result should be handled by the rewriter

Returns String

manifest

Rewrites manifest JSON data, needs the data object since the URL handler is called

Parameters

• value String Manifest code
• data Object Standard object for all rewriter handlers (optional, default {})
  • data.origin Object The page location or URL (eg localhost)
  • data.base Object? Base URL, default is decoded version of the origin
  • data.route Object? Adds to the query params if the result should be handled by the rewriter

Returns String

html

Parses and modifies HTML, needs the data object since the URL handler is called

Parameters

• value String Manifest code
• data Object Standard object for all rewriter handlers (optional, default {})
  • data.snippet Boolean? If the HTML code is a snippet and if it shouldn't have the rewriter scripts added
  • data.origin Object The page location or URL (eg localhost)
  • data.base Object? Base URL, default is decoded version of the origin
  • data.route Object? Adds to the query params if the result should be handled by the rewriter

Returns String

html_attr

Validates and parses attributes, needs data since multiple handlers are called

Parameters

• node (Node | Object) Object containing at least getAttribute and setAttribute
• name String Name of the attribute
• data Object Standard object for all rewriter handlers
  • data.origin Object The page location or URL (eg localhost)
  • data.base Object? Base URL, default is decoded version of the origin
  • data.route Object? Adds to the query params if the result should be handled by the rewriter

plain

Soon to add removing the servers IP, mainly for converting values to strings when handling

Parameters

• value (String | Buffer) Data to convert to a string
• data Object Standard object for all rewriter handlers

decode_blob

Decoding blobs

Parameters

• data
• Blob

Returns String

attr_type

Determines the attribute type using the attr_ent property

Parameters

• name String Property name
• tag String? Element tag

Returns String

headers_decode

Prepares headers to be sent to the client from a server

Parameters

• value
• data (optional, default {})
• null-null Object Headers

Returns Object

headers_encode

Prepares headers to be sent to the server from a client, calls URL handler so data object is needed

Parameters

• value
• data Object Standard object for all rewriter handlers (optional, default {})
  • data.origin Object The page location or URL (eg localhost)
  • data.base Object? Base URL, default is decoded version of the origin
  • data.route Object? Adds to the query params if the result should be handled by the rewriter
  • data.type Object? The type of URL this is (eg js, css, html), helps the rewriter determine how to handle the response
  • data.ws Object? If the URL is a WebSocket
• null-null Object Headers

Returns Object

cookie_encode

Prepares cookies to be sent to the client from a server, calls URL handler so

Parameters

• value String Cookie header
• data Object Standard object for all rewriter handlers (optional, default {})
  • data.origin Object The page location or URL (eg localhost)
  • data.url Object Base URL (needed for hostname when adding suffix)

Returns Object

cookie_decode

Prepares cookies to be sent to the server from a client, calls URL handler so

Parameters

• value String Cookie header
• data Object Standard object for all rewriter handlers (optional, default {})
  • data.origin Object The page location or URL (eg localhost)
  • data.url Object Base URL (needed for hostname when adding suffix)

Returns Object

decode_params

Decode params of URL, takes the prefix and then decodes a querystring

Parameters

• url
• URL (URL | String) to parse

Returns URLSearchParams

decompress

Decompresses response data

Parameters

• req
• res
• callback
• Client Object request
• Request Object response
• Callback Function

valid_url

Validates a URL

Parameters

• args ...any
• URL (URL | String) to parse
• Base (URL | String)?

Returns (Undefined | URL) Result, is undefined if an error occured

str_conf

Returns a string version of the config`

Returns Object

globals

Globals, called in the client to set any global data or get the proper fills object

Parameters

• url
• rw
• Local URL URL incase the global object is not set
• Rewriter Object instance

Returns Object Fills

html_serial

Serializes a JSDOM or DOMParser object

Parameters

• dom
• DOM Document

Returns String

wrap

Wraps a string

Parameters

• str
• String

Returns String

checksum

Runs a checksum on a string

Parameters

• r
• e (optional, default 5381)
• t (optional, default r.length)
• String

Returns Number

How it works:

Recieve request => parse URL => send request to server => rewrite content => send to client

JS:

To achieve accuracy when rewriting, this proxy uses "scoping". All js is wrapped in a closure to override variables that otherwise are not possible normally (window, document)

Proxies are used to change or extend any value to be in line with rewriting URLs

Any occurance of this is changed to call the global rw_this function with the this value, if the this value has a property indicating that the value has a proxied version, return the proxied version.

An example:

Call the rewriter and parse:

if(window.location == this.location)alert('Everything checks out!');

Expected result:

{let fills=<bundled code>,window=fills.this,document=fills.document;if(window.location == rw_this(this).location)alert('Everything checks out!');
//# sourceURL=anonymous:1
}

this in the input code is defined as window, the window has a proxied version that will also determine if any properties are proxied and give a result.

this => fills.this

this.location => fills.url

HTML rewriting:

A part of getting down full HTML rewriting is also making sure any dynamically made elements are rewritten.

Getters and setters are used for properties on the Node.prototype object for such as but not limited to:

• outerHTML
• innerHTML
• getAttribute
• setAttribute
• setAttributeNS
• insertAdjacentHTML
• nonce
• integrity

• every attribute that is rewritten in the HTML side of things

Any property/function that inserts raw html code that is not rewritten is ran through the rewriters HTML handler. Properties are handled by the rewriters HTML property handler (for consistency)

CSS:

A basic regex to locate all url() blocks is used and the rewriters URL handler is called with the value and meta for the page

HTML:

A bundled version of JSDOM is used to achieve accuracy and consistency when rewriting and DOMParser in the browser.

Each property is iterated and in the rewriter a huge array containing information for determining the type of attribute being worked with is used ( this includes tag and name).

• If the type is a URL then the resulting value is determined by the rewriters URL handler
• If the type is JS then the resulting value is determined by the rewriters JS handler along with being wrapped for encoding
• If the type is CSS then the resulting value is determined by the rewriters CSS handler along

Manifest:

A basic JSON.stringify checking if the key is src or key or start_url and if it is then the rewriters URL handler is used to determine the result.