Approximate Time to Scrape Website Documentation
TLDR Jean-Baptiste asked about the average time for website documentation scraping. Jason shared that scraping the Typesense docs site takes around 15 minutes.
2
Jun 29, 2023 (3 months ago)
Jean-Baptiste
12:38 PM
Jason
04:39 PM1
Jean-Baptiste
05:07 PM1
Typesense
Indexed 2779 threads (79% resolved)
Similar Threads
Building Website Search Engine with Typesense
andy inquired about examples of live sites using Typesense for website link search. Jason suggested the search bar on Typesense's docs site as a reference.
Fetching All Docs from a Collection in Typesense
Julian asked if all docs could be fetched from a Typesense collection, and Kishore Nallan explained there's a 250 result limit due to performance considerations. Andrew suggested using the export function, explaining their operations and performance.
Solving Typesense Docsearch Scraper Issues
Sandeep was having issues with Typesense's docsearch scraper and getting fewer results than with Algolia's scraper. Jason helped by sharing the query they use and advised checking the running version of the scraper. The issue was resolved when Sandeep ran the non-base regular docker image.