@spinhire
Raf's answer about Perplexity Pro for the competitor column is the part I'd poke at, not the scraping. Store data is a fact you can re-fetch; "who competes now" is a model's answer on the day it was asked, and that's the exact field someone bets a month of rebuilding on. We run an aggregated job board, around 6,700 open listings re-pulled from dozens of sources every six hours, and the columns we generate rather than fetch are the ones that quietly go wrong, because nothing errors when they age, they just stop being true. A dead-by-Manifest-V3 row also reads as a very different bet from a dead-by-no-demand row, since only the first leaves the userbase genuinely orphaned. Does the competitor field get re-run on a row before someone buys, or is it the single snapshot from when the 9,656 were collected?