Crawl log
What the crawler saw and when
SharePoint's crawl log for a site, a library or a single address: when the crawler visited, whether it succeeded, and the error code and message when it did not. Filter by level, content source and dates, look at every value SharePoint recorded for an entry, and export to Excel.
What does it tell you?
- Has the crawler visited this site since the migration?
- Which items failed to crawl, and with what error?
- When was a particular file last crawled?
How it runs
- Sign in. One sign-in when you click the tile. A SharePoint admin centre browser session is recommended; it needs no app consent.
- Choose what to read. An address (the tenant root to start), Starts with for everything below it, a level, Sites or User profiles, the dates and a row limit.
- Read crawl log. Entries come back newest first. Click one to see every value SharePoint recorded for it. If SharePoint refuses, click Add (account) to crawl log permissions.
- Export. Excel, with the filters you used on the Summary sheet.
How does it work?
Each read is one request to SharePoint's crawl log for the address you give, either that exact address or every address that starts with it, filtered by level, by content source (your sites, or user profiles) and by dates, up to the row limit. When the read stops at the row limit, the page says there may be more entries than shown.
An entry SharePoint records as a success but that carries an error code or a status message is shown as a warning, so it is not lost among the successes. Every value SharePoint returned for an entry is kept and shown in its details.
If the account is neither a SharePoint administrator nor granted crawl log access, SharePoint refuses the read, and the page says so in one sentence with a button that opens the admin centre page where an administrator can grant it. ShareMaster never grants that access itself, and nothing on this page changes SharePoint.
What do you need?
- A grant on the tenant's Crawl Log Permissions page, in the SharePoint admin centre's search settings. SharePoint and global administrators need it too: being an administrator is not enough on its own.
- Without the grant, the page names the account SharePoint refused, links that tenant's Crawl Log Permissions page, and offers "Add (account) to crawl log permissions". That button adds the account with the same sign-in, then reads again, waiting a little between tries because SharePoint can take a few minutes to apply it. If the change cannot be made for you, a message says why and what Microsoft's Crawl Log Permissions page will ask; Open the page opens it signed in, with the account already in its box and on your clipboard. Click OK there and ShareMaster reads the crawl log again on its own.
What will it not do?
- The crawl log itself is only read. The one change this page can make is the grant above, and only when you click its button: it adds the account to whoever is on the list already, removes no one, and stays in place afterwards. The export's Summary sheet says when this run added it.
- Large row limits can take minutes, and SharePoint ends one request after 90 seconds. Narrow the dates or the address if a read stops.
- The count says when the row limit was reached: there may be more entries than shown.
- Certificate sign-in is not offered here. SharePoint only lets a person on the Crawl Log Permissions page read the crawl log, and that page cannot list an app. Choose User or App authentication.
Read next
- Item search properties: The crawl log's last entries for one item, with the rest of the story.