@tty-pt/logs NPM

@tty-pt/logs

We don't need no - data structures. Pam pam pam pam. No custom implementations - in the classroom. Engeneers don't - reinvent the weel

Intro

There are many libraries that deal with writing logs to the backend. And many others to present them. Unfortunately or fortunately, I did't know of any that matched MOV.AI's requirements. Something that could keep the logs updated in a central place, without having to burden the maintainer or developer with a hard job.

This is what came out of me wanting to find a solution.

It combines an "interval tree" and a normal binary search tree into one data structure that fits our needs. While meanwhile making the everyday's developer job as easy as "I'm interested in logs like these".

You can fetch logs any way you want, stream, do it over intervals.. Whatever.

But you won't have to do it for every card. Just do it once in a central place. And every place needing them can request this API.

It's smart enough to know how to handle it.

Usage

// firstly, we inherit the base class:

export default class MyLogs extends Logs {
  static CONSTANTS = {
    DEFAULT_LEVELS,
    DEFAULT_SERVICE,
  };

  constructor() {
    super({
      url: 'v1/logs/',
      transform,

      stream: true,
      streamUrl: "/ws/logs",
      streamTransform,

      fields: {
        level: { type: "enumeration", default: DEFAULT_LEVELS },
        service: { type: "enumeration", default: DEFAULT_SERVICE },
        tags: { type: "tags" },
        message: { type: "string" },
        robot: { type: "enumeration" },
      },
    });
  }
}

// we also implement a simple hook to use the class more easily in react:
export function useLogs(myQuery, dependencies) {
  const logs = new MyLogs();

  const getFiltered = useCallback(
    ({ limit = 0, ...query }) => {
      const filtered = logs.filter(query);
      return limit ? filtered.slice(0, limit) : filtered;
    },
    [],
  );

  const [logsData, setLogsData] = useState(getFiltered(myQuery));

  useEffect(
    () => {
      setLogsData(getFiltered(myQuery));
      return logs.subscribe(() => setLogsData(getFiltered(myQuery)), myQuery);
    },
    dependencies,
  );

  return logsData;
}

// then, in a component, we do:
const filteredLogs = useLogs(
  {
    level: levels,
    service,
    tags,
    message,
    robot: robots,
    fromDate: selectedFromDate,
    toDate: selectedToDate,
  },
  [levels, service, tags, message, robots, selectedFromDate, selectedToDate],
);

// or even:
const { robot } = props;
const logsData = useLogs(
  {
    robot: {
      [robot.name]: true,
    },
    tags: {
      ui: true,
    },
    level: {
      "INFO": true,
    }
    limit: 2,
  },
  [robot.name],
);

And that's all you need.

Options

url

The url to fetch the initial list of past logs.

fetch

A function (assumed async) to fetch a selection of logs based on an url parameter.

default: async (url) => (await fetch(url)).json()

transform

A function that transforms items from a fetch response into what we store.

default: log => log

signature:

log: the item to transform
index: the index of the item in the received data
array: the full array of data to be added
isStream: a boolean that identifies if it came from the stream

stream

A boolean that indicates if we will do real-time streaming or not (if not, we can still use fetch requests over an interval).

default: false

streamUrl

A url to use to open a stream (for example a websocket stream). This is used for keeping logs up to date real-time.

streamOpen

A function that opens a stream connection. It should return an object with an "onmessage" property that is a function that will get called for every new log that is received over this connection.

default: streamUrl => new WebSocket(streamUrl)

streamTransform

This is a function that runs for each log message received in the "onmessage" I just mentioned. Then it passes the result to the "transform" function.

default: log => log

signature: