Finally, I manage to fix the issue. In documentation, its not mentioned that sitemap.xml need to be there as pre requisite before running the crawler. I followed following steps to resolve the issue: 1- Configure the sitemap plugins: https://docusaurus.io/docs/api/plugins/@docusaurus/plugin-sitemap 2- run

npm run build

3- Now server my site in local on port 80.

npm run serve

4- Run the typesense server 5 - Run the scrapper Thanls @Jason Bosco for your proactive support.