🚀 MiniDB - Build Your Own Database Engine

A lightweight database engine built with Node.js that stores data in JSON files and implements core database concepts such as CRUD operations, indexing, query processing, and storage management.

This project is designed to help developers understand how databases work internally instead of simply using PostgreSQL, MySQL, or MongoDB.

---

📖 Overview

Modern databases provide powerful APIs for storing and retrieving data. However, most developers never explore what happens behind the scenes when executing a query.

MiniDB is a simplified database engine that demonstrates:

• Data Storage
• Record Management
• CRUD Operations
• Query Processing
• Indexing
• File System Operations
• Database Architecture

By building MiniDB, you will learn the fundamental concepts used by real-world database systems.

---

🎯 Objectives

• Understand database internals
• Learn how data is stored on disk
• Implement CRUD operations from scratch
• Build indexing mechanisms
• Apply Data Structures and Algorithms in real-world systems
• Simulate database query execution

---

🏗️ Architecture

Client Request
      │
      ▼
 Query Engine
      │
      ▼
 Storage Manager
      │
      ▼
 JSON Storage

Future architecture:

Client
  │
  ▼
Query Parser
  │
  ▼
Query Engine
  │
  ▼
Index Manager
  │
  ▼
Storage Engine
  │
  ▼
Disk Storage

---

📂 Project Structure

mini-db/
│
├── data/
│   └── users.json
│
├── indexes/
│   └── userIndex.json
│
├── db/
│   ├── database.js
│   ├── insert.js
│   ├── find.js
│   ├── update.js
│   ├── delete.js
│   └── query.js
│
├── app.js
│
└── README.md

---

⚡ Features

Insert Records

insert({
  id: 1,
  name: "Harsh",
  age: 20
});

Find Records

findById(1);

Update Records

update(1, {
  age: 21
});

Delete Records

deleteUser(1);

---

💾 Storage Engine

MiniDB stores records in JSON files.

Example:

[
  {
    "id": 1,
    "name": "Harsh",
    "age": 20
  }
]

This simulates how databases persist information on disk.

---

🔍 Query Engine

Example:

find({
  age: 21
});

Internal execution:

users.filter(
  user => user.age === 21
);

This demonstrates how databases process search conditions.

---

📌 Indexing

Without indexing:

findById(1000);

Time Complexity:

O(n)

The database scans every record.

With indexing:

{
  "1": 0,
  "2": 1,
  "3": 2
}

Time Complexity:

O(1)

The database directly accesses the required record.

---

🧠 Data Structures Used

Feature	Data Structure
Storage	Array
Records	Objects
Indexing	Hash Map
Search	Linear Search
Future Index	B+ Tree

---

📊 Time Complexity

Operation	Complexity
Insert	O(1)
Find	O(n)
Update	O(n)
Delete	O(n)
Indexed Search	O(1)

---

🛠️ Technologies

• Node.js
• JavaScript (ES6+)
• File System (fs)
• JSON Storage

---

📚 Concepts Implemented

Database Concepts

• Storage Engine
• Query Engine
• Indexing
• Data Persistence
• Record Management

DSA Concepts

• Arrays
• Objects
• Hash Maps
• Linear Search
• Time Complexity Analysis

Backend Concepts

• File Handling
• JSON Serialization
• CRUD Operations
• System Design Fundamentals

---

🚀 Future Improvements

Phase 2

• Hash Indexing
• Secondary Indexes
• Fast Record Lookup

Phase 3

• SQL-like Query Language

Example:

db.query(
  "SELECT * FROM users WHERE age = 20"
);

Phase 4

• B+ Tree Indexes

Benefits:

• O(log n) Search
• O(log n) Insert
• O(log n) Delete

Phase 5

• REST API Layer

POST   /users
GET    /users/:id
PATCH  /users/:id
DELETE /users/:id

Phase 6

• Transactions
• Write-Ahead Logging (WAL)
• Caching
• Concurrency Control

---

🎓 Learning Outcomes

After completing this project, you will understand:

• How databases store data
• How query engines work
• Why indexing improves performance
• Why Hash Maps are used in databases
• Why B+ Trees are the industry standard
• How backend systems manage persistent data

This project bridges the gap between:

• Data Structures & Algorithms
• DBMS
• Backend Development
• System Design

making it an excellent educational and portfolio project for aspiring software engineers.

---

👨‍💻 Author

Harsh Chauhan

BCA Student | Backend Developer | Future Software Engineer