Thread
:
Want a browser that tells sites it's a bot
View Single Post
Reply
Posted 2008-10-14, 01:07 PM in reply to
WetWired
's post
"Want a browser that tells sites it's a..."
WetWired said:
[Goto]
After getting lots of search hits into documents that the sites want me to pay to view, I had an idea: Why not tell the site that I'm a Google bot? That's how Google got the full text of the document in the first place, no? Because the site saw it was a Google bot and gave it full access?
Does anyone know of a browser that lets you set the user-agent header to make sites think it's a bot/different browser?
Not that I know much about it, but is it possible that your IP address has something to do with it? I think google bots all have similar IP addresses. Maybe websites allow certain ranges to view everything because they're used by bots.
Dunno, just a thought. I really don't know anything about it.
Profile
PM
WWW
Search
!King_Amazon!