Info about web crawlers, spiders and bots
Info about web crawlers, spiders and bots
I’ve had a few people ask about these so here’s some info on them.
Have you seen a user on here with a gray username that can’t be clicked?
That “user” is a web crawler, also known as a spider or bot. It’s not a real person, but rather an automated script that goes about the web and indexes information it finds.
In other words, they help improve search engines such as Google.
PhpBB automatically reserves the first 45 IDs or so (excluding the anonymous user and me) for these bots. Their accounts get activated when their scripts comes across Shibaface.
Bots have access to only some forums. They can’t go onto the staff board or the parts of the site that only mods and admins can view.
Some bots aren’t recognized by phpBB, and appear as guests. As of April 30th, the date this post was written, a total of 5 unique bots have visited the site.
You can read more about web crawlers here:
https://en.m.wikipedia.org/wiki/Web_crawler
TL;DR: Web crawlers/bots aren’t bad. They just help improve search engines.
Have you seen a user on here with a gray username that can’t be clicked?
That “user” is a web crawler, also known as a spider or bot. It’s not a real person, but rather an automated script that goes about the web and indexes information it finds.
In other words, they help improve search engines such as Google.
PhpBB automatically reserves the first 45 IDs or so (excluding the anonymous user and me) for these bots. Their accounts get activated when their scripts comes across Shibaface.
Bots have access to only some forums. They can’t go onto the staff board or the parts of the site that only mods and admins can view.
Some bots aren’t recognized by phpBB, and appear as guests. As of April 30th, the date this post was written, a total of 5 unique bots have visited the site.
You can read more about web crawlers here:
https://en.m.wikipedia.org/wiki/Web_crawler
TL;DR: Web crawlers/bots aren’t bad. They just help improve search engines.
Need help with Shibaface? Feel free to PM me or a staff member!
I go by they/them pronouns.
I am slightly forgetful, please resend PMs if they're important!
___
Outside of SF, I enjoy fursuit making, drawing, and video games!
I go by they/them pronouns.
I am slightly forgetful, please resend PMs if they're important!
___
Outside of SF, I enjoy fursuit making, drawing, and video games!
- Lexi
- First flight
- Posts: 248
- Joined: Thu Jan 10, 2019 3:00 am
- Tagline: Maybe one day, I will be looked up to for my art.
- Mood: Focused
- Contact:
Re: Info about web crawlers, spiders and bots
Thanks Blu. I've always wondered about those.

𝒮𝒽𝑒/𝐻𝑒𝓇 ♛ 𝒜𝓇𝓉𝒾𝓈𝓉
╚══════════════╝
◤XxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxXxxxxxxxxxxxxxxxx◥
Hi! I'm Lexi. I'm an artist here on SF. ^w^
Feel free to PM me any time!
I love; Birds, Shibas, Pokémon, Studio Ghibli, and Anime in general :')
Hope you like my designs!
◣XxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxXxxxxxxxxxxxxxxxx◢

- Oblivion
- Well-known
- Posts: 2548
- Joined: Sat Jan 05, 2019 10:44 am
- Tagline: When life gives you lemons
- Mood: I like bunnies
- Contact:
Re: Info about web crawlers, spiders and bots
ahh ok
winstalgia#1031
Call me: Steven
"Soon you shall become Steven (2000 posts), then Stephan, the fancy Steven (3000 posts), then Stefani, much like Gwen (4000 posts). No one has ever achieved higher than Stefani." -Purple. 5/6/19 (link)
Artist - INFP T
Call me: Steven
"Soon you shall become Steven (2000 posts), then Stephan, the fancy Steven (3000 posts), then Stefani, much like Gwen (4000 posts). No one has ever achieved higher than Stefani." -Purple. 5/6/19 (link)
Artist - INFP T
Re: Info about web crawlers, spiders and bots
That makes a ton of sense, thanks for explaining!
Hey, I'm Mat, nice to meet you!
- Milk
- Well-known
- Posts: 2072
- Joined: Wed Feb 06, 2019 4:28 pm
- Tagline: wataei canon
- Mood: tired & gay
Re: Info about web crawlers, spiders and bots
Abd then there are real bots
we uh
we don't talk about those
Jk, i actually never knew what those did, i just knew they were fake for some purpose. Thanks for the info!
we uh
we don't talk about those
Jk, i actually never knew what those did, i just knew they were fake for some purpose. Thanks for the info!
btw ive had this ranpo icon for like 2 years and ive only seen like 2 episodes of bungo stray dogs
- Coffee
- Spreading the wings
- Posts: 129
- Joined: Mon Apr 29, 2019 5:55 pm
- Tagline: Everyday can be a better day, despite the challenge!
- Mood: man.
Re: Info about web crawlers, spiders and bots
Well, the more you know.


𝙷𝚎𝚢𝚊. 𝙸'𝚖 𝙲𝚘𝚏𝚏𝚎𝚎, 𝚢𝚘𝚞𝚛 𝚕𝚘𝚟𝚊𝚋𝚕𝚎 (𝚎𝚡)𝚌𝚊𝚏𝚏𝚎𝚒𝚗𝚎 𝚊𝚍𝚍𝚒𝚌𝚝. 𝙸'𝚖 𝚊𝚕𝚠𝚊𝚢𝚜 𝚘𝚙𝚎𝚗 𝚝𝚘 𝚌𝚑𝚊𝚝 𝚊𝚗𝚍 𝚜𝚞𝚙𝚙𝚘𝚛𝚝 𝚢𝚘𝚞 𝚗𝚘 𝚖𝚊𝚝𝚝𝚎𝚛 𝚠𝚑𝚊𝚝! 𝙸'𝚖 𝚊 𝚝𝚛𝚊𝚗𝚜 𝚏𝚎𝚖𝚊𝚕𝚎 𝚏𝚛𝚘𝚖 𝙶𝚎𝚘𝚛𝚐𝚒𝚊, 𝚄𝚂𝙰. 𝙸 𝚑𝚘𝚙𝚎 𝚝𝚘 𝚋𝚎𝚌𝚘𝚖𝚎 𝚢𝚘𝚞𝚛 𝚋𝚎𝚜𝚝 𝚏𝚛𝚒𝚎𝚗𝚍!!


- Lilith
- Well-known
- Posts: 1308
- Joined: Mon Dec 31, 2018 11:42 am
- Tagline: I’m in the writing zone today.
- Mood: Chillin’
- Contact:
Re: Info about web crawlers, spiders and bots
I asked about these forever ago lol
Gotta redo this.
Message me if you need anything!
Message me if you need anything!
- Derpy Memester
- Ready to fly
- Posts: 76
- Joined: Sun Jan 26, 2020 11:09 am
- Tagline: You can never have too much TFP fanart...
- Mood: Meh
- Contact:
Re: Info about web crawlers, spiders and bots
Hmmmmmm good info
----------------------------------------------------------------------------------------------
Last edited by Knock Out on Tue Jan 07, 2020 5:00 pm, edited 1 time in total.
Reason: Post lacked proper flair.
ӇЄƳ ƬӇЄƦЄ, Ɩ'M ƊЄƦƤƳ.
í'm αn uncσnvєntíσnαl chríѕtíαn whσ dσєѕ nσt cєlєвrαtє cσnvєntíσnαl hσlídαчѕ. í líkє wαtchíng tfp mєmєs αnd drαwíng cσmícs αвσut tfp/mч custσm trαnsfσrmєrs; í'm σpєn tσ pms íf чσu nєєd hєlp wíth αnчthíng σr just wαnt tσ tαlk αвσut fαndσms. αu rєvσír, rєαdєr

"𝙳𝚎𝚊𝚝𝚑 𝚒𝚜 𝙶𝚘𝚍'𝚜 𝚠𝚊𝚢 𝚘𝚏 𝚜𝚊𝚢𝚒𝚗𝚐 '𝚈𝚘𝚞'𝚛𝚎 𝚏𝚒𝚛𝚎𝚍'.
𝚂𝚞𝚒𝚌𝚒𝚍𝚎 𝚒𝚜 𝚖𝚊𝚗'𝚜 𝚠𝚊𝚢 𝚘𝚏 𝚜𝚊𝚢𝚒𝚗𝚐, '𝚈𝚘𝚞 𝚌𝚊𝚗'𝚝 𝚏𝚒𝚛𝚎 𝚖𝚎, 𝙸 𝚚𝚞𝚒𝚝!"
Last edited by Knock Out on Tue Jan 07, 2020 5:00 pm, edited 1 time in total.
Reason: Post lacked proper flair.
ӇЄƳ ƬӇЄƦЄ, Ɩ'M ƊЄƦƤƳ.
í'm αn uncσnvєntíσnαl chríѕtíαn whσ dσєѕ nσt cєlєвrαtє cσnvєntíσnαl hσlídαчѕ. í líkє wαtchíng tfp mєmєs αnd drαwíng cσmícs αвσut tfp/mч custσm trαnsfσrmєrs; í'm σpєn tσ pms íf чσu nєєd hєlp wíth αnчthíng σr just wαnt tσ tαlk αвσut fαndσms. αu rєvσír, rєαdєr

"𝙳𝚎𝚊𝚝𝚑 𝚒𝚜 𝙶𝚘𝚍'𝚜 𝚠𝚊𝚢 𝚘𝚏 𝚜𝚊𝚢𝚒𝚗𝚐 '𝚈𝚘𝚞'𝚛𝚎 𝚏𝚒𝚛𝚎𝚍'.
𝚂𝚞𝚒𝚌𝚒𝚍𝚎 𝚒𝚜 𝚖𝚊𝚗'𝚜 𝚠𝚊𝚢 𝚘𝚏 𝚜𝚊𝚢𝚒𝚗𝚐, '𝚈𝚘𝚞 𝚌𝚊𝚗'𝚝 𝚏𝚒𝚛𝚎 𝚖𝚎, 𝙸 𝚚𝚞𝚒𝚝!"