Hi guys I am running a cluster for our team, my collections are all well designed on the schema, but I have a new team who want to full text index a website that has had many developers working on it over time. So as you can imagine the structure and naming styles are a bit of a mixed bag and we have no set crawler tags. What is the best way to crawl this?

May be a SaaS service like this could help: https://apify.com/

thanks for this @Jason Bosco, i have dm'd to run a couple of concepts past u on this