Claude Skills and Bs4 for intelligent scraping

while building https://polyscores.xyz (soccer betting for polymarket) as a side project, I was scraping a number of data sources that feed the front end and my decision making during live games. but the problem that I have is that there is so many different data models and parts of the source websites that I am scraping - I don't want to have a dedicated solution for all them. I want something that can do all at once with a simple prompt

While doing all this scraping without any agent help, seeing claude skills' (I am an anthropic power user) ability to run scripts - I want to build a generic scraper that can do a number of things:

1) create data models based on the data it sees

2) create target data stores, like a postgres table based on a model from 1

3) scrape the data using claude agent sdk + playwright mcp

4) dump the data into the storages it creates

this can be utilised for so many different problems. but looking at the scope of the problem, this might be its own company :D

my questions are more of a market discovery:

- if people know of anyone building something similar?

- is someone from the audience would find it useful?

I think I will build a small version for myself without 1 and 2 but would love to know what people think