Hi. I would like to scan the site with some crawler. To see what errors there are and in general what sub-pages the site contains. Unfortunately, after crawling it, I don’t see any subpage. In wordpress there is such a function - blocking the site from indexing. Is it the same here? Is there some kind of slider or button you have to press to make the site get scanned? How to unblock it? I have such a problem for two sites.
Divhunt produces SPA websites, meaning there is no HTML on render time, rather afterwards JavaScript generates it, so I imagine that tool is not waiting for JavaScript, resulting in not finding any other pages.
However we have a Prerender tool which returns full HTML on request, but we do not support the tool you’ve sent.
So if you can send me a tool link, or even better their User-Agent of crawl bot so we can add it to the list.