simple-caching-proxy v1.0.0

Simple-caching-proxy

Simple and transparent caching forward proxy server written in Nodejs.

Features

• It has only one dependency ( node-forge )
  • it is used for creating self-signed certificates dynamically for MITM https proxy server.
• By default configuration, it will indefinitely cache all http requests irrespective of their cache headers.
• All cached data is transparently saved to a cache directory with simple file structure.
  • Contents are saved as it & headers are saved as a json file. It can be viewed/edited later
• If Root CA certificates are provided, then MITM HTTPS proxy server will get enabled and HTTPS Traffic also will get cached.
• We can specify URL rewrite rules to avoid caching of same data from different mirror sites.
  • This feature will help to work with default configuration of YUM/DNF utilities which will be using different mirrors in each time.

Installation

npm i simple-caching-proxy

Configuration

• simple-caching-proxy will listen on 127.0.0.1:8080 by default ( without any configuration )
• All the configuration variables can be permanently stored in configuration file
• We can even specify different configuration file using SCPROXY_CONFIG environment variable.
• All the values specified in configuration file can be overridden by setting corresponding SCPROXY_<config key> environment variable.
• Multiple url_rewrites rules can be specified by providing space separated list of rewrite rules.
• url_rewrite rule, follows format used by s subcommand of sed command. The first and last charecter of the rewrite rule should be same and the same charecter is used as separator.
• To install root CA in a CentOS-7 system, do the following
  • root@host# cp <rootCA.*> /etc/pki/ca-trust/source/anchors/
  • root@host# update-ca-trust

Example Usage

env SCPROXY_host=0.0.0.0 npx simple-caching-proxy

shell Output

Proxy server running...

Using env variables
  SCPROXY_CONFIG=/home/user/.config/simple-caching-proxy/config.json

Current Configuration ( edit /home/user/.config/simple-caching-proxy/config.json  or set env variable SCPROXY_<config-key>=<value> to change )
  "port": 8080,
  "host": "0.0.0.0",
  "cache_dir": "/home/user/.cache/simple-caching-proxy",
  "root_ca_key": "/home/user/.config/simple-caching-proxy/rootCA.key",
  "root_ca_cert": "/home/user/.config/simple-caching-proxy/rootCA.pem",
  "url_rewrites": "#http://(.*)/7.7.1908/#http://mirrors.centos/7.7.1908/#"

https requests are not cached since it is not configured.
  Make sure that the files
    /home/user/.config/simple-caching-proxy/rootCA.pem
    /home/user/.config/simple-caching-proxy/rootCA.key
  exists and accessible to the process.
  Refer documentation more details.

Run the following command shell to start using this proxy
  export http_proxy=http://0.0.0.0:8080


2020-03-21T19:53:10.864Z Miss GET https://github.com/harish2704/node-simple-caching-proxy/archive/master.zip => /home/user/.cache/simple-caching-proxy/github.com/GET/harish2704/node-simple-caching-proxy/archive/master.zip
2020-03-21T19:53:25.549Z Hit! GET https://github.com/harish2704/node-simple-caching-proxy/archive/master.zip => /home/user/.cache/simple-caching-proxy/github.com/GET/harish2704/node-simple-caching-proxy/archive/master.zip

Configuring MITM proxy for HTTPS traffic.

To cache HTTPS traffic , a root CA certificate has to provided to proxy server. Then the same root CA has to be installed as a trusted CA on all the client systems. Otherwise "invalid issuer" error will raise during any https request.

openssl command line can be used to generated Root CA certificate. For details please use / refer the gist generate-certificate-openssl.sh

Root CA certificates has to be placed in the location pointed by root_ca_cert & root_ca_key configuration values.

Example structure of cache directory

.
├── mirrorlist.centos.org
│   └── GET
│       ├── ?release=7&arch=x86_64&repo=extras&infra=container
│       ├── ?release=7&arch=x86_64&repo=extras&infra=container.scpData
│       ├── ?release=7&arch=x86_64&repo=os&infra=container
│       ├── ?release=7&arch=x86_64&repo=os&infra=container.scpData
│       ├── ?release=7&arch=x86_64&repo=updates&infra=container
│       └── ?release=7&arch=x86_64&repo=updates&infra=container.scpData
└── mirrors.centos
    └── GET
        └── 7.7.1908
            ├── os
            │   └── x86_64
            │       ├── Packages
            │       │   ├── wget-1.14-18.el7_6.1.x86_64.rpm
            │       │   ├── wget-1.14-18.el7_6.1.x86_64.rpm.scpData
            │       └── repodata
            │           ├── 04efe80d41ea3d94d36294f7107709d1c8f70db11e152d6ef562da344748581a-primary.sqlite.bz2
            │           ├── 04efe80d41ea3d94d36294f7107709d1c8f70db11e152d6ef562da344748581a-primary.sqlite.bz2.scpData
            │           ├── 4af1fba0c1d6175b7e3c862b4bddfef93fffb84c37f7d5f18cfbff08abc47f8a-c7-x86_64-comps.xml.gz
            │           ├── 4af1fba0c1d6175b7e3c862b4bddfef93fffb84c37f7d5f18cfbff08abc47f8a-c7-x86_64-comps.xml.gz.scpData
            │           ├── repomd.xml
            │           └── repomd.xml.scpData
            └── updates
                └── x86_64
                    ├── Packages
                    │   ├── iproute-4.11.0-25.el7_7.2.x86_64.rpm
                    │   ├── iproute-4.11.0-25.el7_7.2.x86_64.rpm.scpData
                    └── repodata
                        ├── 06a1f304fa4be7195913e76e0f1914c7090b87d8cb46a227b7389288920bbf17-primary.sqlite.bz2
                        ├── 06a1f304fa4be7195913e76e0f1914c7090b87d8cb46a227b7389288920bbf17-primary.sqlite.bz2.scpData
                        ├── repomd.xml
                        └── repomd.xml.scpData