mitm-play v0.10.3

Man in the middle

Using Playwright to intercept traffic in between and do lots of stuff for Developer to exercise

• Installation
• Features
• Concept
• Object & function
• Route Sections
• _global_ Route
• ~/.mitm-play
• HTTP_PROXY
• CLI Options
• Macros
• Macro Keys
• Persistent
• ws__send
• User Route
• Use Cases
• Early Stage

Installation

npm install -g mitm-play

Execute mitm-play command with demo route, or add -h to see help screen:

mitm-play -Gdr  #-OR-
NODE_OPTIONS='--inspect' mitm-play -Gdr 

// create file: ~/user-route/keybr.com/index.js & add this content:
const css = `
body>div,.Body-header,.Body-aside {
  display: none !important;
}`;

const route = {
  url: 'https://keybr.com',
  'mock:no-ads': {
    'cloudflareinsights.com': '',
    '#201:google.+.com': '',
    'doubleclick.net': '',
    'cookiebot.com': '',
    'btloader.com': '',
    'pub.network': '',
  },
  css: {
    'GET:no-ads:/assets/[a-z0-9]+': `=>${css}`,
  },
}
module.exports = route;

// create file: ~/user-route/_global_/index.js & add this content:
const route = {
  'args': {
    debug: true
  },
  'flag': {
    'ws-connect': true,
    'ws-message': true,
  }
}
module.exports = route;

# 1st run will be to save all cli option to 'default'
mitm-play keyb --delete --save  # --OR--
mitm-play keyb -ds

# next run should be simple as:
mitm-play #-OR-
NODE_OPTIONS='--inspect' mitm-play

Routing definition having remove-ads tag, it will be shown on chrome dev-tools "mitm-play" "tags" as an option to enabled / disabled rules. You can see the togling process on this video.

Features

Concept

Mitm intercept is hierarchical checking routes.

First check is to match domain on the url with route-folder as a domain namespace.

Next check is to match full-url with regex-routing of each section/rule. the regex-routing having two type:

• An Array nosocket, nonproxy, proxy, noskip, skip
• Object Key:
  1. General request, mock, cache, log, response
  2. Specific to content-type html, json, css, js

if match, then based on the route section / rules meaning, the next process can be carry over, detail explanations will be on the title of: "Route Section".

/**
 * Folder structure:
 * = user-route       // folder
 * \=== abc.com       // folder
 *    |--- index.js   // file
 *    \--- index.json // autogenerated file
*/
{
  'abc.com': { // route-folder mapped to object as namespace
    request: { // sections can be: skip, proxy, <etc...> 
      '/assets/main.js': {     // regex-routing
        request(reqs, match) { // handler 
          const {body} = reqs;
          ...
          return {body}
        }
      }
    }
  }
}

If the process of checking is not match, then it will fallback to _global_ namespace for checking, and the operation is the same as mention in above paragraph: 'Next check...'.

Usually html page load with several assets (image, js & css) that not belong to the same domain, and to match those type of assets, it use browser headers attributes: origin or referer, in which will scoping to the same namespace.

Object & function

Detail structure of Object and Function shared accros Section

Objects

/**
 * match: {
 *   tags       : {},
 *   route      : {}, 
 *   contentType: {}, 
 *   workspace  : '',/undefined,
 *   namespace  : '', 
 *   pathname   : '', 
 *   hidden     : true,/false 
 *   search     : '',
 *   host       : '',
 *   arr        : [],
 *   url        : '',
 *   key        : '',
 *   log        : '',
 *   typ        : 'cache:tag'
 * }
*/

/**
 * reqs/request: {
 *   url        : '',
 *   method     : 'GET',/PUT/POST/DELETE 
 *   headers    : {}, 
 *   oriRef     : '',
 *   body       : '',/null,
 *   browserName: 'chromium',/webkit/firefox
 * }
*/

/**
 * resp/response: {
 *   url    : '',
 *   status : 200,/302/400/500/etc.. 
 *   headers: {},
 *   body   : '',
 * }
*/

Functions

/**
 * arguments:
 * - <reqs: object>
 * - <match: object>
 * 
 * return: <filename: string>/false
 * 
 * False value indicate skiping rule
*/
file(reqs, match) {
  match.path = 'some/path' // superseded match.route.path
  ...
  return 'common.js'; //return {path: 'some/path', file: 'common.js'}
},

/**
 * arguments:
 * - <reqs: object>
 * - <match: object>
 * 
 * return: <reqs: object>
*/
request(reqs, match) {
  const {headers} = reqs;
  headers['new-header'] = 'with some value';
  ...
  return {headers};
},

/**
 * arguments:
 * - <resp: object>
 * - <reqs: object>
 * - <match: object>
 * 
 * return: <resp: object>
*/
response(reqs, reqs, match) {
  const {headers} = reqs;
  headers['new-header'] = 'with some value';
  ...
  return {headers};
},

Route Section

on each route you can add section supported:

route = {
  url:     '',
  urls:    {},
  title:   '',
  jsLib:   [],
  workspace: '',
  screenshot: {}, //user interaction rules & DOM-element observer
  nosocket:[],
  noproxy: [], 
  proxy:   [], //request with proxy
  noskip:  [], //start routing rules
  skip:    [],
  request: {},
  mock:    {}, 
  cache:   {},
  response:{},
  html:    {},
  json:    {},
  css:     {},
  js:      {},
  log:     {}, //end routing rules
}

Title: provide basic information about this route.

Url: when user enter cli with 1st args, it will try to find in url, then open the browser with that location.

Urls: additional search urls key, the 1st args can be split by ,, if find more than one, it will open multi tabs.

workspace: will be use as the base folder for file option in Mock and Cache.

lib: inject js library into html which having websocket, it can be jquery.js, faker.js, chance.js, log-patch.js, axe.js

route = {
  title: 'Amazon - amazon',
  url:  'https://www.amazon.com/b?node=229189',
  urls: {
    ama1: 'https://www.amazon.com/b?node=229100',
    ama2: 'https://www.amazon.com/b?node=229111',
  },
  workspace: '~/Projects',
  jsLib: ['chance.js'],
};
// cli: mitm-play ama -dpsr='.' 
// search: 'ama' and it will open two browser tabs

Capture/Screeshot when user click specific DOM-Element match with selector or state-change, like DOM-Element getting insert or remove and match selector inside observer key.

Below example show three selector in observer:

• '.field.error' -> filename: field-error -> state: insert or remove
• '.input.focus' -> filename: input -> state: insert or remove
• '.panel.error' -> filename: panel-error -> state: insert

Caveat: observer is an experimental feature, take it as a grain salt, expectation of selector should be like toggling and it need a unique match to the DOM-Element, please do test on chrome-devtools before reporting a bug.

Caveat 2: this Screenshot rule(s), required successful injection of websocket client to html document, if it not success (error can be seen on chrome dev-tools),might be content-security-policy restriction.

Caveat 3: process screenshot sometime take times and for SPA, transition between page usually instantly and it lead to capturing next page, even if the trigger come from button in previouse page, there is a CLI option: -z/--lazy to delay click action for about ~400ms

screenshot: {
  selector: '[type=button],[type=submit],button,a', //click event
  observer: {
    /***
     * selector must be uniq, represent not in the dom 
     * state change couse element tobe insert or remove,
     * or can be just class change 
    */
    '.field.error': 'field-error:insert,remove',
    '.input.focus': 'input:insert,remove',
    '.panel.error': 'panel-error:insert',
  },
  at: 'sshot', //'^sshot' part of log filename
},

at is a partion of filename and having a simple rule attach on it. Guess what is it?.

No WebSocket Injection to html, mitm-play will process further.

nosocket: ['sso'],

if proxy config was set to all request/response, noproxy will exclude it from proxy. Example below will set domain nytimes.com with direct access and the rest will go thru proxy.

// HTTP_PROXY env need to be set, cli: --proxy .. --noproxy ..
noproxy: ['nytimes.com'],
proxy:   ['.+'],

Certain domain will go thru proxy, proxy & noproxy will make sanse if command line contains -x/--proxy

// HTTP_PROXY env need to be set, cli: --proxy ..
proxy: [
  'google-analytics.com',
],

Forces to some domains not to be skip it

noskip: ['wp-admin'],
skip  : ['.+'], // put it in on global routes

Skipping back url to the browser if partion of url match text in array of skip section, mitm-play will not process further.

skip: ['.+'],

Manipulate request with request function

request: {
  'GET:/disqus.com/embed/comments/': {
    request(reqs, match) {
      const {headers} = reqs;
      headers['new-header'] = 'with some value';
      ...
      return {headers};
    },
    session: true, // optional - set session id
  }
},

Mock the response.

Basic rule:

Replace response body with the matcher value

mock: {'/mock': 'Hi!'},

Replace response body with content from file

mock: {'/mock1': {file: 'mocks/mock1.json'}},

mock: {'/mock2': {path: 'mocks', file: 'mock2.html'}}, // match.route.path

Manipulate response with response function

mock: {
  '/mock3': {
    file(reqs, match) {
      match.path = 'mocks' // superseded match.route.path
      return 'mock2.html' //return {path: 'some/path', file: 'filename'}
    },
    response(resp, reqs, match) {
      let {body} = resp
      body += '<h2>there!</h2>'
      return {body} // {status, headers, body} or false to skip
    },
    log: true, // optional - enable logging
    ws: true,  // inject web socket (html)
  },
},

Replace response body with content from remote

'/mock4': {path: 'https://www.lipsum.com/feed', file: 'html'}, 

Below is the logic of file getting translate combine with path or workspace, if workspace exists, and file value not start with dot(.), it will use workspace (ie: ${workspace}/${file}) and the path will be ignore.

mock: {
  'mitm-play/twitter.js': {
    file: 'relative/to/workspace/file.html', // --OR--
    // file: '../relative/to/route/folder/file.html',
    // file: './relative/to/route/folder/file.html',
    // file: '~/relative/to/home/folder/file.html',
    // file: (reqs, match) => 'filename'
  },
},

Concatenation of JS code js at the end of the mock body

const unregisterJS = () => {
  ...
  console.log('unregister service worker')
};

mock: {
  'mitm-play/twitter.js': {
    js: [unregisterJS],
  },
},

If both options are defined: response, js, js will be ignored.

Save the first request to your local disk so next request will serve from there.

cache: {
  'amazon.com': {
    contentType: ['javascript', 'image'], //required!
    jsonHeader: ['nel'],// convert hearder(s) to json
    querystring: true,  // hash of unique file-cache
    hidden: true,       // optional - no consolo.log
    log: true,          // optional - enable logging
    path: './api',      // optional cache file-path
    file: ':1.png',     // optional cache file-name
    tags: 'js-img',     // optional route by tags
    at: 'mycache',      // part of filename
  }
},

logic for file is the same as in mock, if workspace exists and file value not start with dot(.), it will use workspace (ie: ${workspace}/${file}) and the path will be ignore.

cache: {
  'amazon.com': {
    file: 'relative/to/workspace/file.html', // --OR--
    // file: '../relative/to/route/folder/file.html',
    // file: './relative/to/route/folder/file.html',
    // file: '~/relative/to/home/folder/file.html',
    // file: (reqs, match) => 'filename'
  },
},

cache support response function, it means the result can be manipulate first before send to the browser.

cache: {
  'amazon.com': {
    contentType: ['json'], //required! 
    response(resp, reqs, match) {
      const {body} = resp;
      ...
      return {body} // {status, headers, body} or false to skip
    },    
  }
},

Manipulate response with response function

response: {
  '.+': {
    response(resp) {
      const {headers} = resp;
      headers['new-header'] = 'with some value';
      ...
      return {headers};
    },
    tags: 'all-response',
  }
},

Manipulate the response.

Basic rule:

Replace response body with some value

html: {'twitter.net': ''},

Insert js script element into specific area in html document:

• el: 'head'   // default, no need to add el key
• el: 'body'

html: {
  'https://keybr.com/': {
    // el: 'head', // JS at <head> 
    js: [()=>console.log('Injected on Head')],
  },
},

Insert <script src="..."></script> into <head> section

html: {
  'https://keybr.com/': {
    src: ['http://localhost:/myscript.js'],
    ws: true, // inject web socket
  },
},

Manipulate response with response function

html: {
  'https://keybr.com/': {
    response(resp, reqs, match) {
      const {body} = resp;
      ....
      return {body} // {status, headers, body} or false to skip
    },
    tags: 'response' // enable/disable route by tags
    hidden: true, // optional - no consolo.log
  },
},

Manipulate the response.

Basic rule:

Replace response body with some value

json: {'twitter.net': '{}'},

Manipulate response with response function

json: {
  'twitter.com/home': {
    response(resp, reqs, match) {
      const {body} = resp;
      ....
      return {body} // {status, headers, body} or false to skip
    },
    tags: 'json-manipulate',
  },
},

Manipulate the response.

Basic rule:

Replace response body with some value -or- add to the end of response body by adding FAT arrow syntax =>${style}

const style = 'body: {color: red}';
...
css: {'twitter.net': style}, //or `=>${style}`

Manipulate response with response function

css: {
  'twitter.com/home': {
    response(resp, reqs, match) {
      const {body} = resp;
      ....
      return {body} // {status, headers, body} or false to skip
    },
    tags: 'css-manipulate',
  },
},

Manipulate the response.

Basic rule:

Replace response body with some value -or- add to the end of response body by adding FAT arrow syntax =>${style}

const code = 'alert(0);'
...
js: {'twitter.net': code}, //or `=>${code}`

Manipulate response with response function

js: {
  'twitter.com/home': {
    response(resp, reqs, match) {
      const {body} = resp;
      ....
      return {body} // {status, headers, body} or false to skip
    },
    tags: 'js-manipulate',
  },
},

Save the response to your local disk. by default contentType json will log complete request / response, for different type default log should be response payload.

Special usacase like google-analytic will send contentType of gif with GET request, and response payload is not needed, there is an option log to force log with json complete request / response.

log: {
  'amazon.com': {
    contentType: ['json'],
    jsonHeader: ['nel'], // convert hearder(s) to json
    tags: 'json-bo'      // optional route by tags
    at: 'myjson',        // part of log filename
  },
  'google-analytics.com/collect': {
    contentType: ['gif'],
    log: true,      // '<remove>'
  }
},

log support response function, it means the result can be manipulate first before send to the browser or save to logs file.

log: {
  'amazon.com': {
    contentType: ['json'], //required! 
    response(resp, reqs, match) {
      const {body} = resp;
      ...
      return {body} // {status, headers, body} or false to skip
    },
  }
},

_global_ Route

A special route to handle global scope (without namespace)

_global_ = {
  jsLib:   [],
  skip:    [], //start routing rules
  proxy:   [], //request with proxy
  noproxy: [], 
  nosocket:[],
  request: {},
  mock:    {}, 
  cache:   {},
  log:     {},
  html:    {},
  json:    {},
  css:     {},
  js:      {},
  response:{}, //end routing rules
}

Two additional Section only appear in __global__

args, flag and it can be served as a section-tags

_global_ = {
  args: { // part of cli options
    activity,  // rec/replay cache activity*
    cookie,    // reset cookies expire date*
    fullog,    // show detail logs on each rule*
    lazyclick, // delay ~700ms click action*
    nosocket,  // no websocket injection to html page*
    nohost,    // set logs without host name*
    nourl,     // set logs without URL*
    csp,       // relax CSP unblock websocket*
  }
}

_global_ = {
  flag: { // toggle to show/hide from console.log()
    'referer-reqs': true,
    'no-namespace': true,
    'ws-broadcast': false, // true if --verbose
    'ws-connect': false,   // true if --verbose
    'ws-message': false,   // true if --verbose
    'frame-load': false,   // true if --verbose
    'page-load': false,    // true if --verbose
    'mitm-mock': false,    // true if --verbose
    'file-log': false,     // true if --verbose
    'file-md': false,      // true if --verbose  
    silent:   false,       // true: hide all
    skip:     false,
    nosocket: true,
    request:  true,
    mock:     true,
    cache:    true,
    log:      true,
    html:     true,
    json:     true,
    css:      true,
    js:       true,
    response: true,
  }
}

~/.mitm-play

By default all save file are on the ~/.mitm-play profile folder.

HTTP_PROXY

mitm-play support env variable HTTP_PROXY and NO_PROXY if your system required proxy to access internet. Please check on CLI Options > -x --proxy section for detail explanation.

CLI Options

when entering CLI commands, mitm-play support two kind of arguments:

• args:
  • 1st for searching url/urls
  • 2nd for loading profile
• options.

# syntax
$ mitm-play [args] [-options]

# create 'secure' profile with -s/--save option # OR
$ mitm-play yahoo --lazyclick --incognito -s='secure'
$ mitm-play yahoo -zts='secure'

# search yahoo route and use 'secure' profile & add -k/--cookie option 
$ mitm-play yahoo secure -k

# if no profile, fallback to 'default'
$ mitm-play yahoo --cookie

To show all the options Command Line Interface (CLI). this option can be arbitrary position on cli, the result should be always display this messages:

$ mitm-play -h  <OR>
$ mitm-play --help

  Usage: mitm-play [args] [options]

  args:
    1st for searching url/urls
    2nd for loading profile

  options:
    -h --help          show this help
    -u --url           go to specific url
    -s --save          save as default <profl>
    -r --route         userscript folder routes
    -a --activity      rec/replay cache activity*
    -b --basic         login to http authentication
    -c --clear         clear/delete cache & log(s)
    -d --devtools      show chrome devtools on start
    -e --device        resize to mobile screen device
    -f --fullog        show detail logs on each rule*
    -i --insecure      accept insecure cert in nodejs env
    -j --jformat       JSON save as human readable format
    -n --nosocket      no websocket injection to html page*
    -o --offline       console log withount new-line
    -k --cookie        reset cookies expire date*
    -l --light         unset devtools dark mode
    -g --group         create cache group/rec
    -p --csp           relax CSP unblock websocket*
    -t --incognito     set chromium incognito
    -w --worker        enable service worker
    -x --proxy         a proxy request
    -z --lazyclick     delay ~700ms click action*

    -A --a11y          axe-core a11y checker
    -D --debug         show Playwright debugger
    -E --websecure     enable web security 
    -G --nogpu         set chromium without GPU
    -H --nohost        set logs without host name*
    -L --showsql       show sqlite generated commands
    -R --redirect      set redirection: true/false/manual
    -Q --nosql         disabling persist data using sqlite
    -S --session       sqlite session from requst header
    -U --nourl         set logs without URL*
    -V --verbose       show more detail of console log
    -X --proxypac      set chromium proxypac

    -C --chromium      run chromium browser
    -F --firefox       run firefox browser
    -W --webkit        run webkit browser

  * _global_.config.args
    
  v0.10.xxx

Open Browser to specific URL

$ mitm-play -u='https://google.com'  <OR>
$ mitm-play --url='https://google.com'

Save CLI options with default or named so later time you don't need to type long CLI options

$ mitm-play -s  <OR>
$ mitm-play --save
  <OR>
$ mitm-play -s='google'  <OR>
$ mitm-play --save='google'

Specify which folder contains routes config

$ mitm-play -r='../user-route'  <OR>
$ mitm-play --route='../user-route'

Flag the caching with sequences, they are three mode of activity:

• rec:activity to record cache w/ seq, all cache always recorded
• mix:activity to record cache w/ seq, non seq behave as std cache
• play:activity to replay cache w/ seq, non seq behave as std cache

Tag activity need to be add to html - rule to indicate the point when sequences cached will be start.

$ mitm-play -a='rec:activity'  <OR>
$ mitm-play --activity='rec:activity'

The first step is to record the flow and do the navigation

$ mitm-play -a='rec:activity'

Next step is to replay the flow

$ mitm-play -a='play:activity'

When page required HTTP Authentication, this parameters will be passs to the newly created Page Context with login and password supplied to this params

$ mitm-play -b='MYCREAD:MYPASSWORD'  <OR>
$ mitm-play --basic='MYCREAD:MYPASSWORD'

Delete logs or cache, can be all or specific one

$ mitm-play -c  <OR>
$ mitm-play --clear
  <OR>
$ mitm-play -c='log'  <OR>
$ mitm-play --clear='log'
  <OR>
$ mitm-play -c='cache'  <OR>
$ mitm-play --clear='cache'

Show chrome devtools on start up on ech tabs

$ mitm-play -d  <OR>
$ mitm-play --devtools

Resize screen to specific mobile device (still buggy)

$ mitm-play -e  <OR>
$ mitm-play --device
  <OR>
$ mitm-play -e='iPhone 11 Pro'  <OR>
$ mitm-play --device='iPhone 11 Pro'

Set NodeJS to operate within insecure / no https checking

$ mitm-play -i  <OR>
$ mitm-play --insecure

Set Saving Json with human readable format

$ mitm-play -j  <OR>
$ mitm-play --jformat

unset devtools dark mode, this option effected only when theme set to System preference.

$ mitm-play -l  <OR>
$ mitm-play --light

If only the params with no value, it will act as No Injection on HTML Page, meaning no open websocket on the page

$ mitm-play -n  <OR>
$ mitm-play --nosocket

if params contain value off ie: -n=off, there will be Injection into HTML Page with no open websocket connection, this options is to get alternative for macros automation tobe send via [POST] request.

$ mitm-play -n=off  <OR>
$ mitm-play --nosocket=off

change console.log to print the logs only when the log-message is unique from the previous log

$ mitm-play -o  <OR>
$ mitm-play --offline

Set proper cache retriver with an update expiry of the cookies

$ mitm-play -k  <OR>
$ mitm-play --cookie

Add group name to file cache/logs, if necessary when large capturing is done and difficult to check the files.

There is an option at on the rules of cache/log for additional filename grouping path.

$ mitm-play -g='mygroup'  <OR>
$ mitm-play --group='mygroup'

By Default program will run in normal browser, adding this option will result in Incognito mode.

$ mitm-play -t  <OR>
$ mitm-play --incognito

enable service worker, current release playwirght cannot intercept request that came from service worker.

$ mitm-play -w  <OR>
$ mitm-play --worker

Some traffict with domain match to proxy section will use proxy.

this option serving two kind of needs: 1. if --proxy without value, mitm-play traffict will get thru proxy. Proxy configuration will get from ENV variable. 2. if --proxy with string domain, all (mitm-play or browser) traffict will get thru proxy. (ie: --proxy='http://username:pass@my.proxy.com')

$ mitm-play -x  <OR>
$ mitm-play --proxy
  <OR>
$ mitm-play -x='http://username:pass@my.proxy.com'  <OR>
$ mitm-play --proxy='http://username:pass@my.proxy.com'

Delay click action ~700ms or you can provide value in milisecond, to provide enough time for screenshot to be taken

$ mitm-play -z  <OR>
$ mitm-play --lazyclick
  <OR>
$ mitm-play -z=400  <OR>
$ mitm-play --lazyclick=400

Update CSP header on Html Page injected with wws-client.js to unblock Websocket communication

$ mitm-play --csp

Enable Axe-core a11y checker, when actiaved, buttons to check a11y visible on top-left side of screen, click the button or use short-cut Ctl+Alt+(yyy or yy or y or c) to execute :

• strict-[yyy] - most stricted rules
• wcag:AA[yy-] - WCAG AA rules
• a11y---[y--] - Base Axe-core rules
• clear--[c--] - Clear/reset the page

When Axe-core a11y checker is finished, it will show result in:

• 2-border: Violation, Wcag:AAA, & Best-practice
• 1-border: Incomplete

$ mitm-play -A  <OR>
$ mitm-play --a11y

More information will be shown in console.log from DEBUG=pw:api, including info from Mitm-play debug logs.

$ mitm-play -D <OR> #pw:api
$ mitm-play -D=b <OR>
$ mitm-play --debug=bc

Option can having combine chars, lowercase represent sepecific Playwright type of logs, but if all Playwiright, use "V"
|char|value | |:--:|-----------| | V |pw: | | a |pw:api | | b |pw:browser | | c |pw:channel| | p |pw:protocol| | B |*browser* | | F |fetch req-H| | P |page load | | S |sqlite logs| | W |websocket |

Enable web security

$ mitm-play -E  <OR>
$ mitm-play --websecure

Necessary option for Macbook owner.

Options can be added with value -G=all to disabled all gpu (might hang notebook)

$ mitm-play -G  <OR>
$ mitm-play --nogpu

set logs without host name

$ mitm-play -H  <OR>
$ mitm-play --nohost

To switch on / show sqlite generated syntax.

$ mitm-play -L  <OR>
$ mitm-play --showsql

Change mechanism of redirection

$ mitm-play -R  <OR>
$ mitm-play --redirect

set logs without URL

$ mitm-play -U  <OR>
$ mitm-play --nourl

Add additional info in console.log

$ mitm-play -V  <OR>
$ mitm-play --verbose

When network on your having a proxypac settings, might be usefull to use the same. This option only in Chromium

$ mitm-play -X='w3proxy.netscape.com:8080'  <OR>
$ mitm-play --proxypac='w3proxy.netscape.com:8080'

Launch Chromium browser

$ mitm-play -C  <OR>
$ mitm-play --chromium

Preset either chrome or msedge

If in the system having stock browser of chrome or msedge

• chrome
• msedge
• chrome-dev
• msedge-dev
• chrome-beta
• msedge-beta

$ mitm-play -C="chrome"  <OR> 
$ mitm-play --chromium="chrome"

Can be a path to Chrome installation ie on MAC

$ mitm-play -C="/Applications/Google\ Chrome.app"  <OR> 
$ mitm-play --chromium="/Applications/Google\ Chrome.app"

Launch Firefox browser

$ mitm-play -F  <OR>
$ mitm-play --firefox

Launch Webkit browser

$ mitm-play -W  <OR>
$ mitm-play --webkit

Macros

When creating rule for specific website site (ie: autologin to gmail), inside folder you can add macros.js to contains what automation need to be run. macros is a Javascript getting injected into the browser, by default if there is a html request then this macro will be included on the injection. To run different macros in the same SPA, just create another a named-macros ie: login -> login@macros.js and to load that macro, URL need to have a query params of '?mitm=login'.

# folder
./accounts.google.com/index.js
./accounts.google.com/_macros_/macros.js
./accounts.google.com/_macros_/login@macros.js

// .../_macros_/macros.js
module.exports = () => {
  const observeOnce = async function() {
    console.log('Getting execute one time')
  }
  return {
    '^/signin/v2/identifier?'() {
      console.log('login to google account...!');
      window.mitm.autofill = [
        '#identifierId => myemailname',
        '#identifierId -> press ~> Enter',
      ];
    },
    '^/signin/v2/challenge/pwd?'() {
      window.mitm.autofill = [
        'input[type="password"] => password',
        'input[type="password"] -> press ~> Enter',
      ];
      // executed when DOM changes, use MutationObserver event
      // postfix "Once" indicate one-time execution
      return observeOnce
    }
  }
}

// will be send to playwright to execute when user click button "Autofill"
window.mitm.autofill = [...]

// it will run on interval 500ms
window.mitm.autointerval = () => {...};

// additinal buttons to be visible on the page top-right
// buttons can be toggle show / hide by clicking [Ctrl] + [SHIFT]
window.mitm.autobuttons = {
  'one|blue'() {console.log('one')},
  'two|green'() {console.log('two')}
}

// A macro keys can be set as a hotkey!
window.mitm.macrokeys = {...}

Macro Keys

A hot keys that can be press on specific page and it will do similar thing with a macro from mechanical keyboard, except its generated from injected mitm-play macros.js,

Example below show a defined macro keys: code:KeyA or code:KeyP & To activate, it need to press combination buttons of Ctrl + Alt + KeyA/KeyP.

list of event.code : https://developer.mozilla.org/en-US/docs/Web/API/KeyboardEvent/code/code_values

// .../_macros_/macros.js
module.exports = () => {
  return {
    '^/signin/v2/identifier?'() {
      window.mitm.macrokeys = {
        'code:KeyA'() {
          alert('Alert KeyA')
        }
      }
      // -- OR --
      window.mitm.fn.hotKeys({
        'code:KeyP'() {
          // chance is a javascript faker defined in jsLib
          const name = chance.email().split('@')[0];
          return [
            `=> ${name}@mailinator.com`,
            '-> press ~> Enter',
          ]  
        }
      })
    }
  }
}    

Automation commands return from KeyP function don't include selector, means it will run from current input focused.

Recomended macro keys

Combination Ctrl + Alt + ... will work on Mac/Windows.

Suport all event.code & lowercase event.key

window.mitm.macrokeys = {
  'key:a'()          { console.log('key in: Ctrl + Alt + a') }, // take presedance over code:KeyA
  'key:ab'()         { console.log('key in: Ctrl + Alt + ab') },// take presedance over code:KeyA:KeyB
  'code:KeyA'()      { console.log('key in: Ctrl + Alt + KeyA') },
  'code:KeyA:KeyB'() { console.log('key in: Ctrl + Alt + KeyA:KeyB') },
}

Feature to provide shortcut with option of _keys as condition logic.

if Shift key pressed, it will serve as saving the key into windows.mitm.lastKey._keys.

Ie: how to type shortcut: KeyL with same keys: 'one' save to windows.mitm.lastKey._keys:

* press:   `Ctrl + Alt + Shift + one`, then
* release: `Shift` and press: `KeyL`
// complete press/release on oneliner
* press: `Ctrl + Alt + Shift + one` release: `Shift` press: `KeyL`

Not Recomended macro keys - may conflict with reserved keys on OS/Chrome

Conflict with Chrome shortcut keys or in Windows conflict with Ctrl + J

Suport all event.code & event.key

window.mitm.macrokeys = {
  'key:<a>'()          { console.log('key in: .... + Ctrl + a') }, // take presedance over code:KeyA
  'key:<A>'()          { console.log('key in: .... + Ctrl + A') }, // take presedance over code:KeyA
  'key:<aA>'()         { console.log('key in: .... + Ctrl + aA') },// take presedance over code:KeyA:KeyA
  'code:<KeyA>'()      { console.log('key in: .... + Ctrl + KeyA') },
  'code:<KeyA:KeyA>'() { console.log('key in: .... + Ctrl + KeyA:KeyA') },
}

Not Recomended macro keys - may conflict with reserved keys on OS/Chrome

In windows conflict with Alt + D, unless need to combine with Shift ie: Shift + Alt + D

Suport all event.code & event.key

window.mitm.macrokeys = {
  'key:{a}'()          { console.log('key in: .... + Alt + a') }, // take presedance over code:KeyA
  'key:{A}'()          { console.log('key in: .... + Alt + A') }, // take presedance over code:KeyA
  'key:{aA}'()         { console.log('key in: .... + Alt + aA') },// take presedance over code:KeyA:KeyA
  'code:{KeyA}'()      { console.log('key in: .... + Alt + KeyA') },
  'code:{KeyA:KeyA}'() { console.log('key in: .... + Alt + KeyA:KeyA') },
}

Persistent

isomorphic - persistent is currently implement as a global function under namespace mitm.fn.sql....:

when params is a string, should be sql like statement where condition (no need to put quote) with an option of orderby, the order orientation need to be added after fieldname with colon either :a for asc and :d for desc, other type is an object params with combination of keys:

• _where_ - string sql like statement as state above
• _limit_ + _offset_ - number for pagination result set
• _pages_ - boolean to calculate how many pagination pages

await mitm.fn.sqlList()
// (*sqlite sqlList*)
// select * from `kv` []

await mitm.fn.sqlList('(hst like %o%) orderby hst id:d')
// (*sqlite sqlList where:(hst LIKE ?) orderby:hst asc, id desc, ["%o%"]*)
// select * from `kv` where (hst LIKE ?) order by `hst` asc, `id` desc [ '%o%' ]

await mitm.fn.sqlList('(hst like %o%) && id=20 orderby hst id:d')
// (*sqlite sqlList where:(hst LIKE ?) AND id = ? orderby:hst asc, id desc, ["%o%","20"]*)
// select * from `kv` where (hst LIKE ?) AND id = ? order by `hst` asc, `id` desc [ '%o%', '20' ]

await mitm.fn.sqlList('(hst like %o%) && (id=20 || id=21) orderby hst id:d')
// (*sqlite sqlList where:(hst LIKE ?) AND (id = ? OR id = ?) orderby:hst asc, id desc, ["%o%","20","21"]*)
// select * from `kv` where (hst LIKE ?) AND (id = ? OR id = ?) order by `hst` asc, `id` desc [ '%o%', '20', '21' ]

await mitm.fn.sqlList({
  _where_:'(hst like %o%) orderby dtu:d',
  _limit_: 15,
  _offset_: 0,
  _pages_: true
})
// (*sqlite sqlList where:{"_where_":"(hst like %o%) orderby dtu:d","_limit_":15,"_offset_":0,"_pages_":true}*)
// select count(`id`) as `ttl` from `kv` where (hst LIKE ?) order by `dtu` desc [ '%o%' ]
// select * from `kv` where `id` in (select `id` from `kv` where (hst LIKE ?) order by `dtu` desc limit ?) [ '%o%', 15 ]

parameters is required, the string parameters having same rules as sqlList excluding orderby

await mitm.fn.sqlDel('(hst like %o%) && app=WOW')
// (*sqlite sqlDel where:(hst LIKE ?) AND app = ?, ["%o%","WOW"]*)
// delete from `kv` where (hst LIKE ?) AND app = ? [ '%o%', 'WOW' ]

await mitm.fn.sqlDel({_hold_:'id>1 orderby hst:d', _limit_: 15})
// (*sqlite sqlDel where:{"_hold_":"id>1 orderby hst:d","_limit_":15}*)
// delete from `kv` where `id` in (select `id` from `kv` where id > ? order by `hst` desc limit ? offset ?) [ '1', -1, 15 ]

await mitm.fn.sqlDel({id:1, _hold_:'id>1 orderby hst:d', _limit_: 15})
// (*sqlite sqlDel where:{"id":1,"_hold_":"id>1 orderby hst:d","_limit_":15}*)
// delete from `kv` where `id` in (select `id` from `kv` where id > ? order by `hst` desc limit ? offset ?) or (`id` = ?) [ '1', -1, 15, 1 ]

parameters is required, an object literal at minimum should be 2 field and the first field either id or _where_ to indentify record that need to be updated.

await mitm.fn.sqlUpd({id:14, app: 'LOL2'})
// (*sqlite sqlUpd set:{"id":14,"app":"LOL2"}*)
// update `kv` set `app` = ?, `dtu` = CURRENT_TIMESTAMP where `id` = ? [ 'LOL2', 14 ]

await mitm.fn.sqlUpd({_upd_:'id<10', app: 'below10'})
// (*sqlite sqlUpd set:{"_upd_":"id<10","app":"below10"}*)
// update `kv` set `app` = ?, `dtu` = CURRENT_TIMESTAMP where id < ? [ 'below10', '10' ]

parameters is required, an object literal. it will serve two purpose: first just insert a record or second to delete record(s) before insert with _hold_, _limit_, _del_ keys.

await mitm.fn.sqlIns({hst: 'demo2', grp: 'group2', typ: 'type2', name: 'name2', meta: 'meta2', data: 'data2'})
// (*sqlite sqlIns set:{"hst":"demo2","grp":"group2","typ":"type2","name":"name2","meta":"meta2","data":"data2"}*)
// insert into `kv` (`data`, `dtc`, `dtu`, `grp`, `hst`, `meta`, `name`, `typ`) values (?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, ?, ?, ?, ?, ?) [ 'data2', 'group2', 'demo2', 'meta2', 'name2', 'type2' ]

await mitm.fn.sqlIns({
  _hold_:'id>1 orderby hst:d',  
  hst: 'demo3', grp: 'group3', typ: 'type3', name: 'name3', meta: 'meta3', data: 'data3'
})
// (*sqlite sqlIns set:{"_hold_":"id>1 orderby hst:d","hst":"demo3","grp":"group3","typ":"type3","name":"name3","meta":"meta3","data":"data3"}*)
// delete from `kv` where `id` in (select `id` from `kv` where id > ? order by `hst` desc limit ? offset ?) [ '1', -1, 1 ]
// insert into `kv` (`data`, `dtc`, `dtu`, `grp`, `hst`, `meta`, `name`, `typ`) values (?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, ?, ?, ?, ?, ?) [ 'data3', 'group3', 'demo3', 'meta3', 'name3', 'type3' ]

await mitm.fn.sqlIns({
  _hold_:'id>1 orderby hst:d', _limit_: 15,  
  hst: 'demo4', grp: 'group4', typ: 'type4', name: 'name4', meta: 'meta4', data: 'data4'
})
// (*sqlite sqlIns set:{"_hold_":"id>1 orderby hst:d","_limit_":15,"hst":"demo4","grp":"group4","typ":"type4","name":"name4","meta":"meta4","data":"data4"}*)
// delete from `kv` where `id` in (select `id` from `kv` where id > ? order by `hst` desc limit ? offset ?) [ '1', -1, 15 ]
// insert into `kv` (`data`, `dtc`, `dtu`, `grp`, `hst`, `meta`, `name`, `typ`) values (?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, ?, ?, ?, ?, ?) [ 'data4', 'group4', 'demo4', 'meta4', 'name4', 'type4' ]

await mitm.fn.sqlIns({
  _hold_:'id>1 orderby hst:d', _limit_: 15, _del_:'id<10', 
  hst: 'demo5', grp: 'group5', typ: 'type5', name: 'name5', meta: 'meta5', data: 'data5'
})
// (*sqlite sqlIns set:{"_hold_":"id>1 orderby hst:d","_limit_":15,"_del_":"id<10","hst":"demo5","grp":"group5","typ":"type5","name":"name5","meta":"meta5","data":"data5"}*)
// delete from `kv` where id < ? [ '10' ]
// delete from `kv` where `id` in (select `id` from `kv` where id > ? order by `hst` desc limit ? offset ?) [ '1', -1, 15 ]
// insert into `kv` (`data`, `dtc`, `dtu`, `grp`, `hst`, `meta`, `name`, `typ`) values (?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, ?, ?, ?, ?, ?) [ 'data5', 'group5', 'demo5', 'meta5', 'name5', 'type5' ]

There are three tables available: kv(default), log & cache. log & cache are preserved, not yet used.

ws__send

Create socket custom command and later it can be use to update/manipulate object, it utilize ws_send function with built-in random keys to make command send to BE is unique

// from browser CLI terminal 
ws__send('ping', 'hi', d=>console.log(`result ${d}`)) // >>> ws-message: `ping:G2kGPCYj{"data":"pong hi!"}`

// example of socket custom command built in for the purpose of testing and validate the custom command
window.mitm.wsrun.$ping = ({ data }) => { // it become: window.mitm.wsrun.$ping
  return `pong ${data}!`
},

User Route

User-route are available on this repo: https://github.com/mitmplay/user-route and it should be taken as an experiment to test mitm-play functionality.

If you think you have a nice routing want to share, you can create a PR to the user-route or add a link to your repo.

Use Cases

There are several strategy to reduce internet usage, user commonly use different tools to achieve, either install new browser (ie: Brave) or install Add Blocker (ie: uBlock). Using mitm-play, developer can controll which need to be pass, blocked or cached.

Cache any reguest with content type: font, image, javascript, css, if url contains cached busting, it may miss the cached, you can experiment by turning off querystring to false.

cache: {
  '.+': {
    contentType: ['font','image','javascript','css'],
    querystring: true,
  }
},

Block/Mock unnecessary javascript with an empty result, be careful to not block UX or content navigation.

mock: {
  'block/w/empty.js': '',
  'some/url/with/adv.js': {
    response(resp, reqs, match) {
      const {body} = resp;
      ...
      return {body: '/* content is blocked! */'}
    },
  },
},

as developer sometime we need to get access to lots website in which some of the page need to be automated fill in and submit to the next page. With Macros it can be done!

Early Stage

Expect to have some rule changed as feature/fix code are incrementally committed.

.

Goodluck!,

-wh.

Known Limitation

Issue or Limitation on Playwright:

• Route handler to support redirects #3993 / Disallow intercepting redirects #2617
  • or alternative intercept response is implemented #1774