ig-harvester

What is ig-harvester?

ig-harvester is a free, open-source (MIT) Instagram OSINT tool for legitimate research, journalism, security audits and education. It attaches to a Chrome window you are already logged into (via CDP), so it sees exactly what your session sees — then harvests a target profile into structured, analyzable data: posts, comment threads with replies, follower and following lists, media URLs, hashtags and mentions.

Everything is exported as JSON, CSV and SQLite, with a resume cache that lets interrupted runs continue where they stopped. A companion downloader saves every post image — each carousel slide included — as a full-resolution, timestamped PNG, and the ig-analyzer dashboard turns the harvest into engagement, posting-pattern, network and bio-intel metrics on a local web page.

Features

Everything below is in the box — no plugins, no paid tiers.

Quick Start

Three steps. Prerequisites: Node.js ≥ 20, system Chrome, and a burner Instagram account you are authorized to use.

Option 1 PowerShell — single entry point Windows, recommended

One script drives the whole suite. Run it bare for an interactive menu, or give it an action to run once (automation-friendly, exit codes propagate):

# Interactive menu — asks for the target, walks you through the run settings, confirms before running
.\ig-harvester.ps1

# One-click setup & run (installs deps, Playwright Chromium, merges MCP config)
.\dependencies\start.ps1 --profile someuser --comments --followers --following --sqlite

# Run one action directly (aliases: Both, Combo, scrape+images)
.\ig-harvester.ps1 scrape someuser --posts 50 --comments
.\ig-harvester.ps1 ScrapeImages someuser --posts 50 --comments

# Non-interactive full pipeline: scrape -> images -> archive, with a step summary
.\ig-harvester.ps1 -Action Full -User someuser -Posts 50 -Comments -Yes

# Preview exactly what would run, without running it
.\ig-harvester.ps1 -Action Images -User someuser -Force -DryRun

# Environment health check (Node version, deps, CDP endpoint)
.\ig-harvester.ps1 -Action Doctor

Preparing the browser (both options)

# 1. Launch Chrome with remote debugging on port 9222 (dedicated profile)
.\dependencies\launch-chrome-debug.ps1

# 2. Log into a BURNER Instagram account in that window, then keep it open

Option 2 Node.js — direct CLI cross-platform

git clone https://github.com/anurag-panda-dev/ig-harvester.git
cd ig-harvester
npm install

# Full OSINT harvest: posts + comments + follower/following lists + SQLite
node dependencies/scrape-ig.mjs --profile someuser --posts 50 --comments --followers --following --sqlite

# Every post the profile grid serves (no 30-post cap)
node dependencies/scrape-ig.mjs --profile someuser --posts all

# Download every post image (carousels included) as timestamped PNGs
node dependencies/ig-images.mjs --profile someuser --posts 50

# Analyze the results in a live local dashboard -> http://localhost:8080
node dependencies/ig-analyzer.mjs --user someuser

# With a proxy, or from a config file (defaults < config.json < CLI)
node dependencies/scrape-ig.mjs --profile someuser --proxy http://127.0.0.1:8080
node dependencies/scrape-ig.mjs --config config.json

npm scripts

Full CLI flags reference

Every scraper flag works in both entry points — --flag value, --flag=value and PowerShell-style -Flag value are all accepted. Unknown flags are forwarded verbatim to the Node tools.

Flag groups by task

ig-harvester.ps1 — actions

Bare invocation opens a numbered, looping menu (target first, then run settings, then a Run … with current settings? [Y/n] confirmation). Naming an action runs exactly one job and returns its exit code.

• GNU passthrough — every scraper flag works in both shapes: --delay-min 900 --log-level debug and -Posts 50 -Comments.
• Post limits — -Posts <n> sets the cap (tools default to 30); -Posts all, -AllPosts or --posts all harvest everything the grid serves.
• -Yes = non-interactive (answers every prompt with the default); -DryRun = print the exact commands without executing them.
• Preflight — Node ≥ 20, npm dependencies and the CDP endpoint are checked before anything runs; if CDP is down it offers to launch Chrome for you.
• Exit codes — 0 success · 1 step/preflight failed · 2 usage error or cancelled input.

Output layout

All files are saved inside a folder named after the username.

out/someuser/
  someuser.json             # full payload + analytics
  someuser-posts.csv        # one row per post
  someuser-comments.csv     # one row per comment
  someuser-followers.csv    # one row per follower
  someuser-following.csv    # one row per following
  someuser.db               # SQLite database (--sqlite)
  screenshots/*.png         # screenshots (--shots)
  images/*.png              # post images (ig-images)
  .cache.db                 # resume cache (internal)

Archiving to IG-DATA

out/ is scratch space — the next scrape can overwrite it. dependencies/import-data.ps1 copies each run into a structured archive, keeps every import as a dated snapshot, and regenerates the analysis reports and master index:

.\dependencies\import-data.ps1 --source out/someuser   # archive + report
.\dependencies\import-data.ps1 --reindex               # rebuild INDEX.md only
.\dependencies\import-data.ps1 --source out/someuser --no-report

Media is stored once: runs/<timestamp>/ keeps JSON, CSV, .db and screenshots, while multi-GB image folders live only in latest/images/.

Built-in analytics

The scraper computes OSINT metrics automatically on every run:

{
  "engagement": {
    "totalLikes": 1234, "totalComments": 56,
    "avgLikes": 45, "avgComments": 2,
    "engagementRate": "1.23%", "postsAnalyzed": 30
  },
  "posting": {
    "postsPerWeek": "0.21",
    "bestPostingHour": "18:00",
    "bestPostingDay": "Saturday",
    "daysActive": 1000
  },
  "account": { "estimatedCreated": "2024-01-01T00:00:00.000Z", "estimatedAgeDays": 1000 },
  "content": {
    "topHashtags": [{ "tag": "DevProfile", "count": 5 }],
    "topMentions": [{ "user": "friend1", "count": 3 }],
    "mostLiked": [{ "shortcode": "ABC123", "likes": 500 }],
    "carouselPosts": 10, "reelPosts": 5, "photoPosts": 15
  },
  "network": {
    "followerCount": 299, "followingCount": 925,
    "followerFollowingRatio": "0.32",
    "verifiedFollowers": 2, "verifiedFollowing": 5
  },
  "bio": { "hasEmail": true, "email": "user@example.com", "hasUrl": true, "length": 150 }
}

ig-analyzer — live dashboard

After scraping, analyze the data with a live web dashboard on http://localhost:8080:

node dependencies/ig-analyzer.mjs                      # auto-detect latest scraped user
node dependencies/ig-analyzer.mjs --user someuser --port 8080
.\dependencies\ig-analyzer.ps1 --user someuser         # one-click on Windows

# Also reads archived profiles directly (profile folder, latest/ or a single run)
.\dependencies\ig-analyzer.ps1 --dir ..\IG-DATA\someuser

Troubleshooting & FAQ

Why do I get exactly 30 posts when the profile claims more?

Post links are read from the profile grid, so a run returns only what Instagram serves to your session. The default cap is --posts 30 — raise it (--posts 500) or use --posts all for every post the grid serves.

I only got 12 / 24 / 36 posts — is it broken?

The grid stopped paginating (slow or gated layout). Re-run: the resume cache continues where it stopped, and the log line grid: N permalink(s) - <reason> names the exact stop reason. grid stopped early: N of M posts found is printed whenever the profile header is higher than what the grid returned.

0 posts plus "This account is private"

The logged-in session doesn't follow the target, or the follow is still pending (Requested). Attach Chrome logged in as the burner that already follows the account — a pending request serves nothing.

0 posts and no warning?

The profile doesn't exist or the account is blocked. Check the spelling and account status.

Where does the tool write, and can I re-run safely?

Everything lands in out/<username>/. The SQLite resume cache (.cache.db) makes re-runs incremental by shortcode; pass --no-resume to start fresh. Archive finished runs into IG-DATA before the next scrape of the same profile to keep history intact.

Which flags does the image downloader accept?

ig-images shares the scraper's targeting and connection flags (--profile/--url, --posts, --out, --cdp, --headless, --proxy, --delay-*, --log-level, --config) plus --force — and collects nothing else: no JSON/CSV, no comments, no follower lists.

How do I run it from CI or a script?

Use action mode with -Yes (non-interactive) and read the exit code: 0 success, 1 step/preflight failed, 2 usage error or cancelled input. -DryRun prints the exact commands first.

Project

npm run lint
npm test

Links

GitHub repository · Releases · Documentation (README) · Guide · Contributing · MIT License · Author — Anurag Panda