.Google's Gary Illyes uncovers Googlebot collects links as opposed to "complying with" all of them, negating main documents.Googlebot accumulates hyperlinks rather than observing all of them in real-time.Gary Illyes discussed this relevant information in Google.com's 'Look Off The Report' podcast.Google.com's documents opposes the actual spider behavior.