redis migration

Signed-off-by: ale <ale@manalejandro.com>

QUICK_REFERENCE.md

@@ -17,6 +17,12 @@ npm run index-file <file> -- --batch-size N  # Custom batch size
 npm run index-file -- --help                 # Show help
 ```
 
+### Duplicate Removal
+```bash
+npm run remove-duplicates -- --field md5 --dry-run    # Preview duplicates
+npm run remove-duplicates -- --field md5 --execute    # Remove duplicates
+```
+
 ## 🔍 Hash Detection Patterns
 
 | Type   | Length | Example |
@@ -45,32 +51,38 @@ GET /api/health
 - **Web Interface**: http://localhost:3000
 - **Search API**: http://localhost:3000/api/search
 - **Health API**: http://localhost:3000/api/health
-- **Elasticsearch**: http://localhost:9200
+- **Redis**: localhost:6379
 
-## 📊 Elasticsearch Commands
+## 📊 Redis Commands
 
 ```bash
-# Health
-curl http://localhost:9200/_cluster/health?pretty
+# Test connection
+redis-cli ping
 
-# Index stats
-curl http://localhost:9200/hasher/_stats?pretty
+# Get database stats
+redis-cli INFO stats
 
-# Document count
-curl http://localhost:9200/hasher/_count?pretty
+# Count all keys
+redis-cli DBSIZE
 
-# Search
-curl http://localhost:9200/hasher/_search?pretty
+# List all hash documents
+redis-cli KEYS "hash:plaintext:*"
 
-# Delete index (CAUTION!)
-curl -X DELETE http://localhost:9200/hasher
+# Get document
+redis-cli GET "hash:plaintext:password"
+
+# Get statistics
+redis-cli HGETALL hash:stats
+
+# Clear all data (CAUTION!)
+redis-cli FLUSHDB
 ```
 
 ## 🐛 Troubleshooting
 
 | Problem | Solution |
 |---------|----------|
-| Can't connect to ES | Check `ELASTICSEARCH_NODE` env var |
+| Can't connect to Redis | Check `REDIS_HOST` and `REDIS_PORT` env vars |
 | Port 3000 in use | Use `PORT=3001 npm run dev` |
 | Module not found | Run `npm install` |
 | Build errors | Run `npm run build` to see details |
@@ -81,17 +93,18 @@ curl -X DELETE http://localhost:9200/hasher
 |------|---------|
 | `app/page.tsx` | Main UI component |
 | `app/api/search/route.ts` | Search endpoint |
-| `lib/elasticsearch.ts` | ES configuration |
+| `lib/redis.ts` | Redis configuration |
 | `lib/hash.ts` | Hash utilities |
 | `scripts/index-file.ts` | Bulk indexer |
+| `scripts/remove-duplicates.ts` | Duplicate remover |
 
 ## ⚙️ Environment Variables
 
 ```bash
-# Required
-ELASTICSEARCH_NODE=http://localhost:9200
-
 # Optional
+REDIS_HOST=localhost
+REDIS_PORT=6379
+REDIS_PASSWORD=your-password
 NODE_ENV=production
 ```
 
@@ -135,6 +148,7 @@ curl http://localhost:3000/api/health
 
 ```bash
 npm run index-file -- --help  # Indexer help
+npm run remove-duplicates -- --help  # Duplicate remover help
 ```
 
 ---