Multi-db-seeder NPM

multi-db-seeder 🌱

multi-db-seeder is a powerful and efficient npm package designed to make database seeding a breeze for developers. Whether you're working with SQL or NoSQL databases, multi-db-seeder has you covered. This package allows you to seed thousands of data records into your databases within seconds, enhancing your development workflow with speed and ease.

🌱 Key Features

Universal Database Support: Seamlessly seed both SQL and NoSQL databases.
Blazing Fast Performance: Seed thousands of records in seconds.
Flexible Data Sources: Use Local JSON Data or Generate Fake Data on the fly.
Multiple Seeding Options: Customize your seeding process with various configuration options.
Ease of Use: Intuitive designed for quick setup and usage.

🌱 Installation

npm install multi-db-seeder

🌱 Quickstart (SQL Database Seed)

// Import the required dependencies
const {
  DATABASE_TYPE,
  SEED_INSERTION_TYPE,
  runDatabaseSeeder,
  generateFakeData,
} = require("multi-db-seeder");
const { v4: uuidv4 } = require("uuid");
const { faker } = require("@faker-js/faker");

// Database seeder configuration
const DATABASE_SEEDER = {
  DATABASE_TYPE: DATABASE_TYPE.MY_SQL,
  SEED_INSERTION_TYPE: SEED_INSERTION_TYPE.INSERT_IF_EMPTY,
  SQL_DB_CONNECTION_PROPS: {
    HOST: "localhost",
    USER: "root",
    PASSWORD: "12345678",
    DATABASE: "db-seed-test",
    PORT: 3306,
  },
  IS_DISABLE_LOCAL_DATA_POOL: false,
  LOCAL_DATA_POOL: {
    seeds: [
      {
        ENTITY_NAME: "TechStack", // Database table name
        DATA: [
          { id: "e715fe66-c01c-4c39-aa54-f19a3a604ae0", name: "React" }, // Data to be seed. "id" filed is required.
          { id: "8c1803fd-7e3c-4c06-8edb-b12d09e4830a", name: "Angular" },
        ],
      },
      {
        ENTITY_NAME: "UserRole", // Database table name
        DATA: [
          { id: "acb29e1a-2344-4db8-b988-3e85a3a9ac52", name: "ADMIN" }, // Data to be seed. "id" filed is required.
          { id: "7ccd4a67-f88d-403c-8b66-3735c91ab12d", name: "MANAGER" },
          { id: "38959582-221a-45af-821c-a0fb7b392720", name: "CLIENT" },
          { id: "62ec4aa1-17fa-46bd-94c9-fda7a928886a", name: "USER" },
        ],
      },
    ],
  },
  IS_DISABLE_FAKE_DATA_POOL: false,
  FAKE_DATA_POOL: {
    seeds: [
      {
        ENTITY_NAME: "Product", // Database table name
        DATA: generateFakeData(
          {
            id: () => uuidv4(), // Generating the unique ID for each record using uuid
            name: () => faker?.commerce?.product(), // Column name and Generating the fake data using the faker-js
            price: () => faker?.commerce?.price(), // Column name and Generating the fake data using the faker-js
          },
          10
        ),
      },
    ],
  },
};

// Function to run the database seeder
const runSeed = async () => {
  await runDatabaseSeeder(DATABASE_SEEDER);
};

// Execute the seed function
runSeed();

🌱 Quickstart (NoSQL Database Seed)

You can use this mongodb-objectid-generator web tool to generate _id for your local data pool when using NoSQL databases.

// Import the required dependencies
const {
  DATABASE_TYPE,
  SEED_INSERTION_TYPE,
  runDatabaseSeeder,
  generateFakeData,
} = require("multi-db-seeder");
const { v4: uuidv4 } = require("uuid");
const { faker } = require("@faker-js/faker");

// Database seeder configuration
const DATABASE_SEEDER = {
  DATABASE_TYPE: DATABASE_TYPE.MONGO_DB,
  SEED_INSERTION_TYPE: SEED_INSERTION_TYPE.INSERT_IF_EMPTY,
  NO_SQL_DB_CONNECTION_PROPS: {
    URI: "mongodb://localhost:27017/",
    DATABASE: "db-seed-test",
  },
  IS_DISABLE_LOCAL_DATA_POOL: false,
  LOCAL_DATA_POOL: {
    seeds: [
      {
        ENTITY_NAME: "TechStack", // Database collection name
        DATA: [
          { _id: "66778c34c6eeec8eb178b9f6", name: "React" }, // Data to be seed. "_id" filed is required with valid MongoDB Object ID.
          { _id: "66778c41185a3e276ccfca74", name: "Angular" },
        ],
      },
      {
        ENTITY_NAME: "UserRole", // Database collection name
        DATA: [
          { _id: "66778c48bd6f32d0ee7ac27a", name: "ADMIN" }, // Data to be seed. "_id" filed is required with valid MongoDB Object ID.
          { _id: "66778c4f5dfafba120a66b4d", name: "MANAGER" },
          { _id: "66778c564bac2f0959ca4296", name: "CLIENT" },
          { _id: "66778c5d70f9c2a54726a3fb", name: "USER" },
        ],
      },
    ],
  },
  IS_DISABLE_FAKE_DATA_POOL: false,
  FAKE_DATA_POOL: {
    seeds: [
      {
        ENTITY_NAME: "Product", // Database collection name
        DATA: generateFakeData(
          {
            id: () => uuidv4(), // Generating the unique ID for each record using uuid
            name: () => faker?.commerce?.product(), // Column name and Generating the fake data using the faker-js
            price: () => faker?.commerce?.price(), // Column name and Generating the fake data using the faker-js
          },
          10
        ),
      },
    ],
  },
};

// Function to run the database seeder
const runSeed = async () => {
  await runDatabaseSeeder(DATABASE_SEEDER);
};

// Execute the seed function
runSeed();

🌱 Sample code demos

🌱 Environment Variables (Optional)

You can use the environment variable values from your local .env file to define the database connection.

const { generateFakeData } = require("multi-db-seeder");
const dotenv = require("dotenv");

// Load environment variables from .env file into process.env
dotenv.config();

// Example for the SQL Databases
const DATABASE_SEEDER = {
  // Remaining config options go here...
  SQL_DB_CONNECTION_PROPS: {
    HOST: process.env.MYSQL_HOST,
    USER: process.env.MYSQL_USER,
    PASSWORD: process.env.MYSQL_PASSWORD,
    DATABASE: process.env.MYSQL_DATABASE,
    PORT: process.env.MYSQL_PORT,
  },
};

// Example for the NoSQL Databases
const DATABASE_SEEDER = {
  // Remaining config options go here...
  NO_SQL_DB_CONNECTION_PROPS: {
    URI: process.env.MONGO_URI,
    DATABASE: process.env.MONGO_DATABASE,
  },
};

🌱 Configuration Options

DATABASE_TYPE: Specifies the type of database to seed (e.g., MY_SQL, MONGO_DB, POSTGRESQL). Determines the type of database connection and operations.
- MY_SQL: Relational database
- MONGO_DB: NoSQL database
- POSTGRESQL: Relational database
SEED_INSERTION_TYPE: Defines how seeding should handle existing data. Options include:
- INSERT_IF_EMPTY: Inserts data only if the database is empty.
- UPDATE_AND_INSERT: Updates existing records and inserts new ones.
- DELETE_AND_INSERT: Deletes existing records and then inserts new ones.
SQL_DB_CONNECTION_PROPS: Configuration object for SQL database connections:
- HOST: Hostname or IP address of the SQL server. (e.g., "localhost")
- USER: Username used to authenticate against the SQL server. (e.g., 3306)
- PASSWORD: Password used to authenticate against the SQL server. (e.g., "root")
- DATABASE: Name of the SQL database to connect to. (e.g., "12345678")
- PORT: Port number where the SQL server is listening. (e.g., "db-seed-test")
NO_SQL_DB_CONNECTION_PROPS: Configuration object for NoSQL database connections:
- URI: Connection URI for MongoDB or similar NoSQL databases. (e.g., "mongodb://localhost:27017/")
- DATABASE: Name of the MongoDB database to connect to. (e.g., "db-seed-test")
LOCAL_DATA_POOL: Class or module handling local JSON data for seeding. (e.g., LocalDataPool in your implementation)
IS_DISABLE_LOCAL_DATA_POOL: Flag to disable usage of local JSON data pool if set to true.
FAKE_DATA_POOL: Class or module responsible for generating fake data dynamically. (e.g., FakeDataPool in your implementation)
IS_DISABLE_FAKE_DATA_POOL: Flag to disable usage of fake data pool if set to true.

Explanation

DATABASE_TYPE and SEED_INSERTION_TYPE set the foundational parameters for database operations, determining the type of database being seeded and the method used for insertion, updating and deleting of data.
SQL_DB_CONNECTION_PROPS is used to configure essential connection parameters required for SQL databases such as MySQL and PostgreSQL, including hostname, user credentials, database name, and port.
NO_SQL_DB_CONNECTION_PROPS is intended for configuring connection details specific to NoSQL databases like MongoDB. It includes the connection URI and database name necessary for establishing a connection to MongoDB or similar NoSQL databases.

Note: You should include one of these configurations (SQL_DB_CONNECTION_PROPS for SQL databases or NO_SQL_DB_CONNECTION_PROPS for NoSQL databases like MongoDB) in your configuration setup, depending on the type of database your application uses for seeding.

LOCAL_DATA_POOL and FAKE_DATA_POOL enable flexibility in data seeding by allowing developers to choose between seeding from local JSON files or generating synthetic data.
IS_DISABLE_LOCAL_DATA_POOL and IS_DISABLE_FAKE_DATA_POOL provide control over whether these seeding options are active, enhancing customization and adaptability.

How the Data Pool Seeds work?

App Screenshot

🌱 Contributions Welcome

Contributions are welcome! If you have ideas for new features, improvements, or bug fixes, feel free to create an issue or submit a pull request on our GitHub repository. Your input is invaluable in helping to make multi-db-seeder even better. Thank you for your support! ❤️

database seeding sql nosql mysql mongodb workflow multi-db-seeder seeder postgresql

@faker-js/faker mongoose mysql2 mysql2-promise pg uuid

9 months ago

9 months ago

12 months ago

12 months ago

12 months ago

12 months ago

12 months ago

12 months ago