Goal
Make all PWA data sources truly cloud-independent so the dashboard stays live with fresh data even when the Mac is off or sleeping.Why: Joseph travels and the Mac goes to sleep. Currently Track and KeyData metrics go stale because extraction runs on the Mac.
Current state (2026-05-10)
| Source | Mac needed? | Notes |
|---|---|---|
| SmartLead | ❌ No | CF Worker (skyrun-sync) fetches from SL API directly |
| BeenVerified | ❌ No | Health = expiry date math; enrichment still needs Chrome |
| KeyData | ✅ Yes | extract_keydata_metrics.py uses Playwright on Mac → pushes to KV |
| Track | ✅ Yes | extract_track_metrics.py scrapes dashboard on Mac → pushes to KV |
Work to do
Track — CF Worker form login + scrape
The original plan (reference_session_keep_alive.md) was a CF Worker doing form login + HTML scrape.Implementation path:
1. Add a new route in skyrun-sync worker (or a dedicated track-sync worker)
2. Worker does: GET / → extract security CSRF token → POST login with Keychain creds → follow redirect → scrape dashboard HTML
3. Problem: Track credentials need to be in CF Secrets (not Keychain). Store via wrangler secret put TRACK_USERNAME and TRACK_PASSWORD.
4. Worker writes scraped metrics to SKYRUN_DATA KV key track_metrics
5. Trigger: CF Cron Trigger (0 /1 = hourly, or /15 * = every 15 min)
6. _worker.js already reads track_metrics from KV — no PWA changes needed
Key constraint: Track's form POST requires a security CSRF token from the login page GET. CF Workers can do this (fetch the GET, parse the token, POST). Tested programmatically in Python — works.
KeyData — CF Worker with stored session token OR API key
Two options:Option A (simpler): Store session token in CF KV / Secrets, fetch via authenticated API calls
- KeyData uses Next-Auth (
__Secure-next-auth.session-token) - Session expires 2026-06-29 — re-extract via
extract_keydata_session.pyafter Chrome login - Worker reads token from CF Secret → makes authenticated requests to KeyData API
- Problem:
session-tokenis a JWT that expires; need periodic refresh mechanism
Option B (proper): KeyData API key
- KeyData (DexAI) likely has a REST API with API keys that don't expire
- Joseph would need to provision an API key from KeyData dashboard
- Worker uses API key → no session management needed
- This is the cleanest long-term solution
How to apply
When Joseph is ready post-usage-reset, start with Track CF Worker (most impactful — Track is the primary ops data source). KeyData Option B is the ideal but requires operator action to get API key.Preconditions:
- CF account ID:
e38dfdc47924abcebc2227f3a2b8c9e2 - KV namespace:
e7ce6b5359a84d5eb55f3d7679805239(SKYRUN_DATA) - Track credentials already in Keychain (
track_username,track_password) — move to CF Secrets _worker.jsalready handles KV reads for both sources — no PWA-side changes needed