Shot Caller.

From scrape to score.

1. 01

   Daily ingestion

   Cron · daily

   A cron-driven pipeline scrapes Twitter, Reddit, and YouTube transcripts through a rotating proxy pool. Node and Python services run side by side — Node handles API-bound work, Python handles transcript extraction and NLP.

   • Twitter API for influencer feeds and replies
   • Reddit API for finance-adjacent subreddit threads
   • YouTube transcript extraction routed through proxies to evade rate-limits and IP bans
   • One full sweep per day, written through a queue
   ↓
2. 02

   Cheap matcher

   Filter

   Before any LLM touches the data, a fast string/regex matcher culls the corpus to posts that mention a known asset. We resolve against a curated table of asset IDs and aliases — Bitcoin and BTC collapse to the same canonical row.

   • Curated table of asset IDs with name / ticker / alias variants
   • Cuts 90%+ of irrelevant chatter before it can hit an LLM bill
   • Collapses surface forms into a single canonical asset id for downstream joins
   ↓
3. 03

   LLM enrichment

   LLM

   Surviving posts go through more expensive LLM calls that classify each recommendation into a small set of structured fields: action, sentiment, conviction, and time horizon. The calls run on a BullMQ worker pool backed by Redis — parallelizing the slow LLM step for throughput, with the queue architecture already in place to absorb backpressure if ingestion ever outpaces enrichment.

   • Action — buy / sell / hold
   • Sentiment — bullish / bearish / neutral
   • Conviction — scale
   • Time horizon — intraday, swing, weeks, months, long-term
   ↓
4. 04

   Hourly price tracking

   Parallel · hourly

   Running in parallel with the ingestion pipeline, a separate process records hourly price data for every tracked asset. This is the ground truth that every recommendation is back-tested against.

   • Per-asset, per-hour OHLC data
   • Continuous — independent of the daily scrape cadence
   • The substrate for every ROI calculation downstream
   ↓
5. 05

   ROI scoring

   Serial · after 03

   A second process runs serially after enrichment. It joins each structured recommendation against the price series and computes realized ROI at multiple horizons.

   • Trailing ROI (since the call was made)
   • 7-day ROI
   • 30-day ROI
   • Action and time-horizon from stage 03 drive how each ROI is interpreted
   ↓
6. 06

   Automated content generation: newsletter, social posts

   Distribution

   Once influencers are scored, the data flows into the editorial side — auto-generated leaderboards, a daily newsletter, and shareable social infographics, all published from the same underlying data without manual editorial intervention.

   

   • Influencer ROI Ratings: Continuously-updated leaderboard surfacing who is consistently positive and at which horizon
   • Newsletter Generator: Daily email digest auto-generated from the day's mined data — top movers, who called it correctly, who missed badly — drafted into a publish-ready brief
   • Social Infographics: Instagram and social-channel infographics auto-generated from the same data — leaderboards, contrarian wins, weekly callouts — formatted for each channel's aspect ratio and voice
   ↓
7. 07

   Evals & self-tuning

   Meta · continuous

   Every step that touches an LLM is wrapped in evals so the pipeline holds up over time. Outputs are graded by judge models, disagreements get flagged for review, and the prompts themselves adapt based on what's actually working in production.

   • LLM-as-Judge Ensemble: Multiple judge models grade each enrichment independently. Disagreements between judges flag the underlying call for review — consensus is the green light to advance
   • Self-Optimizing Prompts: Prompts update themselves based on which variants outperform on a held-out set. New candidates compete against the incumbent and only get promoted when they win on the eval suite