fab-free-watcher

Good question — and it's very much in the "weekend project, not a moonshot" category. Let me verify the current state of the main asset source (Fab) before I give you specifics, since that's the linchpin.

• ToolSearch select:WebSearch,WebFetch 6ms 100 B

• Read error /home/megaproxy/claude/research 4ms 80 B

• WebSearch Fab.com free assets API scraping unofficial endpoint 2025 7.8s 2.3 KB

• WebSearch Fab limited time free monthly assets list free for the month 2026 6.8s 2.1 KB

Very plausible — this is squarely a weekend-to-a-few-evenings self-hosted project, not a hard one. The honest answer is that the *scraping* is easy and the *value* depends on which problem you're actually solving. Let me break it down.

## The short version

**Plausibility: high (8/10).** The single most important source — Fab, Epic's marketplace that absorbed the old Unreal Marketplace and Quixel Megascans — exposes a filterable JSON-backed search you can hit directly. The hard part isn't getting the data; it's deciding whether you want a *notifier* (claim the free-for-now stuff before it expires) or a *catalog* (browse everything that's permanently free), because those are different builds.

## The sources, ranked by value

The whole project lives or dies on Fab, since it's where ~90% of free UE content now lives:

- **Fab "Limited-Time Free"** (`fab.com/limited-time-free`) — this is the killer feature. Epic rotates ~3 UE assets free every **two weeks**; you must "purchase" them at $0 before the window closes or you lose them forever. *This* is the thing worth a dashboard — missing the window is the actual pain. ([80.lv coverage](https://80.lv/articles/claim-new-fab-s-free-content-before-mid-july), [Fab page](https://www.fab.com/limited-time-free))
- **Fab permanently-free** — `fab.com/search?ui_filter_price=1&is_free=1` lists every always-free asset (Megascans, sponsored content, etc.). Good for a catalog, low urgency. ([fabfreecontent](https://www.unrealengine.com/en-US/fabfreecontent))
- **Secondary sources** (nice-to-have): itch.io game-assets RSS/API, Humble Bundle, GameFromScratch's roundup feed (already does human-curated aggregation — [example](https://gamefromscratch.com/june-2026-free-asset-round-up/)).

## What's easy vs. what bites

**Easy:**
- Fab's search is a React/Next front-end over a JSON listings endpoint. The community has already reverse-engineered it — there are [Tampermonkey "claim all free"](https://github.com/noslipper/FAB-Free-Asset-Getter-Latest) scripts and [gists](https://gist.github.com/Subtixx/47f6db9ceabf51c3f689db1594a50b37) hitting it. You can lift their endpoint/params instead of reverse-engineering from scratch.
- Hourly/daily polling of a handful of endpoints is trivial load — a cron + SQLite + a small web UI. No scraping farm needed.

**Bites (the real friction):**
1. **No official public API** for the marketplace listings — you're on undocumented endpoints that Epic can change without notice. Build the fetcher as one swappable module so a breakage is a 20-minute fix, not a rewrite.
2. **Auto-*claiming* needs your logged-in session.** Scraping the *list* is anonymous and clean. Actually adding free assets to your library requires your Epic auth cookies/token — that's where it gets fiddly and ToS-grey. I'd strongly suggest **notify-only** (dashboard shows "3 new free assets, claim by June 16" + deep links) and you click through to claim. Same pattern I flagged for the flex-block-grabber thing — notify-only sidesteps the auth/ToS/arms-race mess.
3. **Cloudflare / bot detection.** Fab sits behind Cloudflare. A plain `requests` call may get challenged; you might need a real-browser fetch (Playwright headless) for reliability. Not hard, just heavier than curl.

## Suggested shape

A minimal, robust build:

```
cron (hourly) → fetcher (Playwright or requests, per-source module)
              → normalize → SQLite (dedupe on asset id)
              → diff against last run → "new since you last looked"
              → small FastAPI/Flask + HTMX dashboard
              → optional: Discord/ntfy push when limited-time-free rotates
```

The Discord/ntfy ping on rotation is honestly the 80% of the value for 20% of the work — you might not even need the dashboard.

---

Two questions shape what this actually becomes — worth pinning down before any code:

• AskUserQuestion questions 83578.6s 199 B

Good — that's the shape I'd have pushed you toward anyway. Notify-only on Unraid is the lean, high-value version. Here's what that specifically looks like.

## The build, concretely

**One container, two moving parts: a poller and a notifier.** No web UI needed — the notification *is* the product.

```
Unraid Docker container
  └─ cron (every 6h is plenty — rotation is bi-weekly, not hourly)
       └─ fetch  fab.com/limited-time-free  (Playwright headless, Cloudflare-safe)
       └─ parse → [{id, title, url, free_until}]
       └─ diff against state.json (seen ids)
       └─ on new asset → push to ntfy/Discord with title + deadline + claim link
       └─ optional reminder ping 24h before any free_until expires
```

Realistically ~150–250 lines of Python. SQLite is overkill for this — a flat `state.json` of seen asset IDs is enough. The only genuinely fiddly bit is the Cloudflare-fronted fetch, which is why I'd reach for Playwright headless over `requests` from the start rather than fighting 403s later.

## What "done" gives you

A Discord/[ntfy](https://ntfy.sh) ping like:

> 🆓 **3 new free on Fab** — claim by **Jun 16**
> • Stylized Nature Pack → [claim]
> • Sci-Fi Modular Corridor → [claim]
> • …

You tap through and click Epic's own $0 "purchase" button. No stored Epic credentials, no auto-claim brittleness, no ToS grey zone. The whole risky surface — your logged-in session — never enters the system.

## Two honest caveats for *this* host

1. **Your Unraid box hard-freezes** (the ongoing investigation). For a tool whose entire job is "don't miss a deadline," running it on an unreliable host is a mild irony. Mitigation: make the poller stateless and idempotent (re-reads `state.json` on boot, so a freeze just delays a ping, never drops one), and add a dead-man's-switch — if you get *no* ping for 14 days, something's wrong. Or honestly, this is light enough it could live as a tiny cron on any always-on box.
2. **Undocumented endpoint risk.** Fab's structure can change with no notice. Keep the `fetch+parse` as one isolated module so a break is a quick patch. The existing [Tampermonkey claim-scripts](https://github.com/noslipper/FAB-Free-Asset-Getter-Latest) are a useful reference for the current request shape, and a canary in the coal mine — if they break, yours probably did too.

## Scope ladder

- **v0 (an evening):** poll limited-time-free → Discord webhook on new. Done. This is 90% of the value.
- **v1:** add the expiry-reminder ping + dead-man's-switch.
- **v2 (only if you want it):** fold in secondary sources (itch.io, Humble) and add a little catalog page — but that's the *other* build you de-scoped, and I wouldn't bother unless v0 proves you actually use it.

---

Want me to scaffold v0 as a real research project — `~/claude/research/fab-free-watcher/` with the poller, a Dockerfile, and an Unraid-friendly compose file — or keep this as a feasibility writeup for now and let it sit?

I can also save a short project memory so this thread is recoverable later either way. Your call on both.

Let's build it. First let me look at the template and confirm tooling, then scaffold.

• Bash ls -la ~/claude/_templates/research/ && echo "---TEMPLATE FILES---" && find ~/claude/_templates/research/ -type f | head… 2.3s 1.3 KB

• Bash echo "=== memory.md ===" && cat ~/claude/_templates/research/memory.md && echo && echo "=== .gitignore ===" && cat ~/cla… 1.5s 3.7 KB

Tooling's all there. Let me scaffold the project and write the code. I'll track this with a task list since it's a multi-part build.

• ToolSearch select:TaskCreate 3ms 52 B

• Bash cp -rT ~/claude/_templates/research/ ~/claude/research/fab-free-watcher/ && cd ~/claude/research/fab-free-watcher && mkd… 1.9s 471 B

• TaskCreate Dockerfile on Playwright base image, docker-compose with data volume + cron loop, .env.example, requirements.txt. 9ms 72 B

• TaskCreate Playwright-based capture fetcher, defensive parser, state.json diff, ntfy+discord notifier, main orchestrator with expir… 5ms 90 B

• TaskCreate Project framing docs, seed memory.md decisions/TODOs, README usage incl. first-run dry-run to tune parser. 13ms 63 B

• TaskCreate git init -b main, commit scaffold, tea repo create --private, wire remote, push -u. 16ms 64 B

Now let me try to glimpse the live page to ground the parser, while I start writing the core files.

• WebFetch https://www.fab.com/limited-time-free 2.7s 215 B

403 — Cloudflare, exactly as predicted. That confirms the plain-HTTP approach is a dead end and Playwright (real browser engine) is mandatory. It also means I can't see the live JSON shape from here, so I'm building the fetcher to be **self-tuning**: it captures whatever JSON the page actually loads and anchors results to the listing cards visible in the rendered DOM, plus a `DRY_RUN` mode that dumps raw captures so first-run-on-the-box reveals the real shape. Writing the code now.

• Write /home/megaproxy/claude/research/fab-free-watcher/watcher/__init__.py 77ms 166 B

• Write /home/megaproxy/claude/research/fab-free-watcher/watcher/config.py 19ms 164 B

• Write /home/megaproxy/claude/research/fab-free-watcher/watcher/fetch.py 20ms 163 B

• Write /home/megaproxy/claude/research/fab-free-watcher/watcher/parse.py 22ms 163 B

• Write /home/megaproxy/claude/research/fab-free-watcher/watcher/state.py 29ms 163 B

• Write /home/megaproxy/claude/research/fab-free-watcher/watcher/notify.py 22ms 164 B

• Write /home/megaproxy/claude/research/fab-free-watcher/watcher/main.py 29ms 162 B

Core package done. Let me sanity-check the Python compiles, then write the Docker/config layer.

• ToolSearch select:TaskUpdate 2ms 52 B

• Bash cd ~/claude/research/fab-free-watcher && python3 -m py_compile watcher/*.py && echo "OK: all modules compile" 1.7s 23 B

• TaskUpdate taskId, status 17ms 22 B

• TaskUpdate taskId, status 18ms 22 B

Now the Docker/config layer. Using `python:3.12-slim` + `playwright install chromium` so the build is self-contained and version-pinned.

• Write /home/megaproxy/claude/research/fab-free-watcher/requirements.txt 33ms 163 B

• Write /home/megaproxy/claude/research/fab-free-watcher/Dockerfile 14ms 157 B

• Write /home/megaproxy/claude/research/fab-free-watcher/entrypoint.sh 32ms 160 B

• Write /home/megaproxy/claude/research/fab-free-watcher/docker-compose.yml 25ms 165 B

• Write /home/megaproxy/claude/research/fab-free-watcher/.env.example 22ms 159 B

Now lock down runtime artifacts in `.gitignore` and keep the `data/` dir tracked but empty.

• Edit error /home/megaproxy/claude/research/fab-free-watcher/.gitignore 6ms 96 B

• Read /home/megaproxy/claude/research/fab-free-watcher/.gitignore 21ms 459 B

• Edit /home/megaproxy/claude/research/fab-free-watcher/.gitignore 25ms 167 B

• Bash cd ~/claude/research/fab-free-watcher && touch data/.gitkeep && bash -n entrypoint.sh && echo "entrypoint.sh: syntax OK" 1.2s 24 B

• TaskUpdate taskId, status 746ms 22 B

• TaskUpdate taskId, status 251ms 22 B

Now the docs — README, plus filling in CLAUDE.md and memory.md.

• Write /home/megaproxy/claude/research/fab-free-watcher/README.md 22ms 156 B

• Edit error /home/megaproxy/claude/research/fab-free-watcher/CLAUDE.md 12ms 96 B

• Read /home/megaproxy/claude/research/fab-free-watcher/CLAUDE.md 13ms 2.7 KB

• Edit /home/megaproxy/claude/research/fab-free-watcher/CLAUDE.md 29ms 166 B

• Edit /home/megaproxy/claude/research/fab-free-watcher/CLAUDE.md 11ms 166 B

Now seed `memory.md`:

• Write error /home/megaproxy/claude/research/fab-free-watcher/memory.md 1ms 96 B

• Read /home/megaproxy/claude/research/fab-free-watcher/memory.md 14ms 712 B

• Write /home/megaproxy/claude/research/fab-free-watcher/memory.md 106ms 166 B

Docs done. Now git init, Forgejo repo, and first push.

• TaskUpdate taskId, status 15ms 22 B

• TaskUpdate taskId, status 26ms 22 B

• Bash cd ~/claude/research/fab-free-watcher && git init -b main -q && git add -A && git status --short && echo "--- check no s… 1.5s 359 B

• Bash cd ~/claude/research/fab-free-watcher && git commit -q -m "Initial scaffold: notify-only Fab limited-time-free watcher" … 13.5s 304 B

• Bash cd ~/claude/research/fab-free-watcher && git remote add origin https://git.rdx4.com/megaproxy/fab-free-watcher.git && gi… 2.1s 133 B

• TaskUpdate taskId, status 8ms 22 B

Done — built, committed, and pushed. Let me record a project memory so this is recoverable in future sessions.

• Write /home/megaproxy/.claude/projects/-home-megaproxy-claude-research/memory/project_fab_free_watcher.md 30ms 197 B

• Read /home/megaproxy/.claude/projects/-home-megaproxy-claude-research/memory/MEMORY.md 7ms 556 B

• Edit /home/megaproxy/.claude/projects/-home-megaproxy-claude-research/memory/MEMORY.md 39ms 189 B

Built and shipped. Here's where things stand.

## What exists now

`~/claude/research/fab-free-watcher/` — committed and pushed to `git.rdx4.com/megaproxy/fab-free-watcher` (private).

```
watcher/
  config.py    env-driven config
  fetch.py     Playwright Chromium → captures page JSON + DOM listing cards
  parse.py     defensive parser, anchors results to what's actually on the page
  state.py     atomic flat-file state.json (crash-safe, idempotent)
  notify.py    ntfy + Discord push (stdlib only)
  main.py      fetch → parse → diff → notify + pre-deadline reminders
Dockerfile · entrypoint.sh (6h bash-loop) · docker-compose.yml · .env.example · README.md
```

All modules byte-compile; entrypoint passes `bash -n`. Verified Fab returns **403 to plain GETs** (Cloudflare), which is exactly why it's built on Playwright rather than `requests`.

## Two design calls worth knowing

- **No hardcoded Fab API.** Their internal listings endpoint is undocumented and changes, so `fetch.py` grabs *every* JSON response the page makes and keeps only the listings that are also rendered as DOM cards. If Fab renames an endpoint, this keeps working — and even if the JSON shape drifts entirely, a DOM fallback still produces correct titles + URLs.
- **Notify-only, by design.** No Epic login, no auto-claim. The risky surface (your session auth) never enters the system.

## The one real open item

I **couldn't validate the parser against live Fab** from here — I can't see its actual JSON shape (403 + Cloudflare). So the first thing to do on the Unraid box is a dry run:

```bash
cd <repo> && cp .env.example .env
docker compose build
DRY_RUN=1 RUN_ONCE=1 docker compose run --rm fab-free-watcher
```

Then check `data/last_capture.json` — if `parsed N assets` matches what you see on the page in a browser, go live; if not, the key heuristics at the top of `parse.py` need a small tweak (README walks through it). The thing most likely to need attention is `free_until` (the deadline reminders) — Fab may only render the countdown client-side, in which case we'd pull it from the DOM instead.

Everything's logged in `memory.md` and a project memory, so a future session picks up cleanly.

Want me to keep going and **deploy it** — build the image, push to the Forgejo registry, and write the Unraid container template — or leave that for when you're at the box and can do the dry-run validation first? I'd lean toward the latter, since tuning the parser against live data should happen before it goes live.