critique-ingest v1.0.0

critique-ingest

• allow authorized uploads to local disk
  • √ Accept same payload that storage does currently (POST files[] and X-Auth-Token)
  • authorize incoming uploads by via REST API call to storage:
    • √ Request: POST /assets/upload Headers: PROXIED, Body: PROXIED but with any file data removed
    • Positive Response: {"_id": "asset_id"}
      • √ Storage creates asset in MongoDB with status 0 (pending) if it does not exist
        • √ This handles reloading and allows user to re-attempt uploads or clear failed uploads and gives room for future improvement to transport, etc
      • √ Storage provides ability to subscribe (WS) to updates on this transient asset on the UI pending future REST calls from ingest
      • √ Ingest accepts file (UI progress bar will move now)
        • on transfer completion, enqueues main worker
          • updates storage with PUT /assets { status: 1 } (processing)
    • √ Negative Response: 401 -- Drop request, delete cached request body, end of flow
• Main Worker:
  • metadata extraction via mediainfo
    • Request: PATCH /assets/upload Header: {X-App-Auth-Token} Body: { "status": "extracting metadata" }
    • if h264 and not mp4
      • container rewrap via ffmbc
        • Request: PATCH /assets/upload Header: {X-App-Auth-Token} Body: { "status": "wrapping as mp4" }
    • and so on ...
  • moves derived data to CDN
  • notifies storage of completion with a final request
    • updates storage with PUT /assets { status: 2, urls: {...}, metadata: {...} } (completed)

Proposed changes to storage

• modify POST /assets/upload to provide authentication, authorization, and to create a transient asset
  • Security -> Must include API token to be honored.
• migrate asset model to support transient assets / (read: ingestions)
• addition of PATCH /assets/upload for updating and finalizing ingest jobs

Proposed changes to web

• add ingest server to config
• send upload requests to ingest server instead of storage server

Dependencies

• ffmbc
• mediainfo

Running

npm start

Testing

npm test