@ddn/server v0.0.2

DDN Server

ddn-server is a Data Distribution Node (DDN) server based on DDN with the Dat protocol.

ddn-server is for you if:

• You're comfortable with some server administration (or want to learn!)
• You want to keep your dat:// website/s online
• You want to publish your dat:// website/s to a domain name

Table of contents

• Features
• Install
• Running ddn-server
• Examples
• Configuration
• Advanced examples
• Troubleshooting
• Support
• Changelog

Features

• Keeps dat:// websites online
• Publishes dat:// websites to DNS shortnames
• Mirrors dat:// websites to https://
• Provisions and manages HTTPS with Let's Encrypt
• Provides optional Pinning Service API endpoints

Install

If you already have Node.js (8.0+) and npm installed on your server, get started by installing DDN server with npm or npx.

npm install -g @ddn/server

Otherwise, install Node.js and npm first:

• Install Node.js
• nvm for managing Node versions
• Fixing npm permissions problems

Having trouble installing? See Troubleshooting.

Running server

To run ddn-server manually, simply invoke the ddn-server command:

ddn-server

To keep ddn-server running, you'll need to daemonize it. We like using pm2.

# install pm2
npm i -g pm2

# start server with pm2
pm2 start ddn-server

To stop the daemon, run:

# stop server
pm2 stop ddn-server

Command line flags

• --config <path>
  • Use the config file at the given path instead of the default ~/.ddn-server.yml. Overrides the value of the DDN_SERVER_CONFIG env var.

Environment variables

• DDN_SERVER_CONFIG=cfg_file_path
  • Specify an alternative path to the config than ~/.ddn-server.yml
• NODE_ENV=debug|staging|production
  • Set to debug or staging to use the Let's Encrypt testing servers.

Examples

ddn-server uses a configuration file (~/.ddn-server.yml by default) for managing its behavior. These examples show various configurations.

See all configuration options

Example: set up a website with a domain name and HTTP mirroring

This configuration file will "seed" or "pin" the files at dat://123...456, publish those files to dat://alice.com, and mirror them to https://alice.com.

This example uses a domain name, so in order for the domain name to resolve correctly, you'll need to update your DNS configuration first. In this case, you could set an A record that points to the ddn-server server's IP address.

dats:
  - url: dat://123...456
    domains:
      - alice.com
httpMirror: true
letsencrypt:
  email: alice@mail.com
  agreeTos: true

Example: seed multiple websites, with no domain names

This configuration simply "seeds" or "pins" the files at dat://123...456 and dat:///456...789. No domain name is required for this configuration.

dats:
  - url: dat://123...456
  - url: dat://456...789

Example: seed multiple websites, enable Web API

In addition to being used to manage a website, ddn-server can be used with more specific configurations, like providing a mini-pinning service for a group of friends.

This example enables the Pinning Service API, which makes it possible for you (and maybe a few friends!) to publish to your ddn-server instance with ddn, the Dat CLI, or any client that supports the Pinning Service API.

If you choose to use a domain name for your API endpoint, be sure to configure your DNS to point to your ddn-server server.

See full webapi reference

# enable publishing to `ddn-server` from Eok and the Dat CLI
webapi:                # set to false to disable
  domain:              # the domain name for the web API (optional)
  username:            # the username for publishing from ddn or the Dat CLI
  password:            # the password for publishing from ddn or the Dat CLI

Configuration

• Configuration file
• dashboard
  • dashboard.port
• dats
  • dats.*.url
  • dats.*.domains
  • dats.*.name
  • dats.*.otherDomains
• directory
• domain
• httpMirror
• letsencrypt
  • letsencrypt.email
  • letsencrypt.agreeTos
• ports
  • ports.http
  • ports.https
• proxies
  • proxies.*.from
  • proxies.*.to
• redirects
  • redirects.*.from
  • redirects.*.to
• webapi
  • webapi.username
  • webapi.password
  • webapi.domain

Configuration file

ddn-server uses ~/.ddn-server.yml as its default configuration file. You can specify an alternative config file using a command line flag or an environment variable.

directory: ~/.ddn-server # where your data will be stored
httpMirror: true       # enables HTTP mirroring
ports:
  http: 80             # HTTP port for redirects or non-TLS serving
  https: 443           # HTTPS port for serving mirrored content and DNS data
letsencrypt:           # set to false to disable Let's Encrypt
  email:               # you must provide your email to LE for admin
  agreeTos: true       # you must agree to the LE terms (set to true)
dashboard:             # set to false to disable
  port: 8089           # port for accessing the metrics dashboard

# enable publishing to server from ddn and the Dat CLI
webapi:                # set to false to disable
  domain:              # enter your web API's domain here (optional, unless Let's Encrypt TLS is wanted on the web API)
  username:            # the username for publishing from ddn or the Dat CLI
  password:            # the password for publishing from ddn or the Dat CLI

# enter your pinned dats here
dats:
  - url:               # URL of the dat to be pinned
    domains:           # (optional) the domains of the dat

# enter any proxied routes here
proxies:
  - from:              # the domain to accept requests from
    to:                # the domain (& port) to target

# enter any redirect routes here
redirects:
  - from:              # the domain to accept requests from
    to:                # the domain to redirect to

dashboard

Default false

Set to true to enable the Prometheus metrics dashboard.

dashboard:             # set to false to disable
  port: 8089           # port for accessing the metrics dashboard

dashboard.port

Default: 8089

The port to serve the Prometheus metrics dashboard.

dats

A listing of the Dat archives to seed.

dats:
  - url: dat://1f968afe867f06b0d344c11efc23591c7f8c5fb3b4ac938d6000f330f6ee2a03/
    domains:
      - mysite.com
      - my-site.com

dats.*.url

The Dat URL of the site to host. Should be a 'raw' dat url (no DNS hostname).

Example values:

# raw key
1f968afe867f06b0d344c11efc23591c7f8c5fb3b4ac938d6000f330f6ee2a03

# URL with trailing slash
dat://1f968afe867f06b0d344c11efc23591c7f8c5fb3b4ac938d6000f330f6ee2a03/

# URL with no trailing slash
dat://1f968afe867f06b0d344c11efc23591c7f8c5fb3b4ac938d6000f330f6ee2a03

dats.*.domains

Additional domains of the Dat archive. Can be a string or a list of strings. Each string should be a domain name.

To use dats.*.domains, you'll first need to configure the DNS entry for your domain name to point to your server. For instance, to point alice.com with ddn-server, you'll need to update your DNS configuration to point alice.com to your server server's IP address.

Example values:

mysite.com
foo.bar.edu
best-site-ever.link

dats.*.name

DEPRECATED. See the v2.0.0 migration guide.

The name of the Dat archive. Sets a subdomain relative to the Web API domain, similar to the way that Hashbase does. Must be unique on the ddn-server instance.

dats.*.otherDomains

DEPRECATED. Use the domains field instead.

Additional domains of the Dat archive. Can be a string or a list of strings. Each string should be a domain name. Example values:

mysite.com
foo.bar.edu
best-site-ever.link

directory

Default: ~/.ddn-server

The directory where ddn-server will store your Dat archive's files.

domain

DEPRECATED. See the v2.0.0 migration guide.

The DNS domain of your server instance.

httpMirror

Default: false

Set to true to provide https mirroring of your Dat archives.

letsencrypt

Default: false

Set to true to enable Lets Encrypt's automatic TLS certificate provisioning.

letsencrypt:           # set to false to disable Let's Encrypt
  email:               # you must provide your email to LE for admin
  agreeTos: true       # you must agree to the LE terms (set to true)

letsencrypt.email

The email to send Let's Encrypt notices to.

letsencrypt.agreeTos

Do you agree to the terms of service of Lets Encrypt? Required, must be true.

ports

The ports for HTTP and HTTPS.

ports:
  http: 80             # HTTP port for redirects or non-TLS serving
  https: 443           # HTTPS port for serving mirrored content and DNS data

ports.http

Default: 80

The port for serving HTTP sites.

HTTP automatically redirects to HTTPS.

ports.https

Default: 443

The port for serving HTTPS sites.

proxies

A listing of domains to proxy. Useful when your server has other services running that you need available.

proxies:
  - from: my-proxy.com
    to: http://localhost:8080

proxies.*.from

The domain to proxy from. Should be a domain name.

Example values:

mysite.com
foo.bar.edu
best-site-ever.link

proxies.*.to

The protocol, domain, and port to proxy to. Should be an origin.

Example values:

https://mysite.com/
http://localhost:8080/
http://127.0.0.1:123/

redirects

A listing of domains to redirect.

redirects:
  - from: my-old-site.com
    to: https://my-site.com

redirects.*.from

The domain to redirect from. Should be a domain name.

Example values:

mysite.com
foo.bar.edu
best-site-ever.link

redirects.*.to

The base URL to redirect to. Should be an origin.

Example values:

https://mysite.com/
http://localhost:8080/
http://127.0.0.1:123/

webapi

Default: false

Set to true to enable the Pinning Service API, which enables publishing to DDN server with ddn, the Dat CLI, or any other client that supports the Pinning Service API.

# enable publishing to `ddn-server` from ddn and the Dat CLI
webapi:                # set to false to disable
  domain:              # the domain of the web api (optional)
  username:            # the username for publishing from ddn or the Dat CLI
  password:            # the password for publishing from ddn or the Dat CLI

webapi.username

The username for your pinning service API.

webapi.password

The password for your pinning service API.

webapi.domain

The DNS domain of your server Web API. Optional, but required if you want Let's Encrypt to provide your Web API with a TLS certificate.

Advanced examples

Example: proxies

If your ddn-server instance is running on ports 80/443, and you have other Web servers running on your server, you might need ddn-server to proxy to those other servers. You can do that with the proxies config. Here's an example proxy rule:

See full proxies reference

proxies:
  - from: my-proxy.com
    to: http://localhost:8080

Example: redirecting requests

Sometimes you need to redirect old domains to new ones. You can do that with the redirects rule. Here's an example redirect rule:

See full redirects reference

redirects:
  - from: my-old-site.com
    to: https://my-site.com

Example: using a metrics dashboard

ddn-server has built-in support for Prometheus, which can be visualized with Grafana.

DDN server exposes its metrics at port 8089. Prometheus periodically scrapes the metrics and stores them in a database. Grafana uses those metrics and provides a provides a nice dashboard visualization. It's a little daunting at first, but setup should be relatively painless.

Steps:

1. Install Prometheus on your server
2. Install Grafana on your server
3. Update the prometheus.yml config
4. Start Prometheus and Grafana
5. Login to Grafana
6. Add Prometheus as a data source to Grafana (it should be running at localhost:9090
7. Import this Grafana dashboard

Your prometheus.yml config should include have the scrape_configs option set like this:

scrape_configs:
  - job_name: 'prometheus'
    static_configs:
      - targets: ['localhost:9090']
  - job_name: 'server'
    static_configs:
      - targets: ['localhost:8089']

Example: running server behind Apache or Nginx

If you're running ddn-server on a server that uses Apache or Nginx, you may need to change your config to disable HTTPS. For instance, if you're using nginx and proxying to port 8080, update your config to disable Let's Encrypt and to set the HTTP port:

letsencrypt: false
ports:
  http: 8080

You will need to add all domains to your Nginx/Apache config.

Example: running server in a docker container

1. Install Docker. If you're on Linux, remember to configure Docker to start on boot. Don't know of the equivalent for other systems.

2. Clone the project. Edit .ddn-server.yml according to your needs. Most importantly: Change username and password.
   If you don't want to think of a username and a password, just use this but increase the length.

3. In the project root, run this command:

docker build -t ddn-server:latest . && docker run -d --name=ddn-server --restart=always -p 80:80 -p 443:443 -p 3282:3282 ddn-server:latest

Notes:
1. Not an expert in Docker security or configuration.
2. if you have ddn on the same machine, you may want to change the dat port -p 3282:3282 to something like -p 9999:3282.
3. To debug the running container:

• Run docker ps -a to see the container running status.
• Run docker logs ddn-server to see the logs.
• Run docker exec -it ddn-server sh to get into a terminal.

4. Didn't think about how you'd install a newer version of ddn-server while keeping the old configuration and data.

Troubleshooting

Installing build dependencies

When installing ddn-server, you may need to install additional build dependencies:

sudo apt-get install libtool m4 automake libcap2-bin build-essential

Port setup (EACCES error)

For ddn-server to work correctly, you need to be able to access port 80 (http), 443 (https), and 3282 (dat). Your firewall should be configured to allow traffic on those ports.

If you get an EACCES error on startup, you either have a process using the port already, or you lack permission to use the port. Try lsof -i tcp:80 or lsof -i tcp:443 to see if there are any processes bound to the ports you need.

If the ports are not in use, then it's probably a permissions problem. We recommend using the following command to solve that:

# give node perms to use ports 80 and 443
sudo setcap cap_net_bind_service=+ep `readlink -f \`which node\``

This will give nodejs the rights to use ports 80 and 443. This is preferable to running ddn-server as root, because that carries some risk of a bug in ddn-server allowing somebody to control your server.

Support

ddn-server is built by the DDN team. You can help us continue our work on ddn, ddn-server, and more. Thank you!

Licence

Copyright (c) 2019 DDN.link Copyright (c) 2018 Blue Link Labs