Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I'm sure that Baidu respects robots.txt.

This site coolshell.cn is boycotting baidu, so it tells Baiduspider that it doesn't want to be indexed by baidu.

Baidu respects this and doesn't index anything from this website.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: