黑帽联盟

标题: Apache与Nginx环境屏蔽YisouSpider爬虫的方法 [打印本页]
作者: yun    时间: 2017-2-8 12:49
标题: Apache与Nginx环境屏蔽YisouSpider爬虫的方法
早上一个朋友告知其没有流量的博客突然出现访问很慢，然后打不开提示信息主机资源超量等信息（具体的提示我也忘记复制下来），大概意思就是占用资源过大短期被屏蔽。因为是虚拟主机，采用的是APACHE架构服务，所以比较简单，直接帮他登录CPANEL面板取得日志文件，然后查看后吓一跳。
16.png

原来是来自YisouSpider蜘蛛的爬行导致占用内存和CPU过大导致的资源消耗，一个小小的博客没有流量的怎么可能会这样呢，我看他的博客只有几篇文章而已。
解决方法很简单，我们直接在.htaccess文件中屏蔽这个IP就可以。
  1. Order Allow,Deny
  2. Allow from all
  3. Deny from 42.120.161.43
复制代码
在.htaccess文件中增加上面三行，如果有多个IP地址的话，用空格隔开。搜索了一下，发现出现这样问题的还真不少，而且还看到有人专门整理了所有的YisouSpider蜘蛛IP地址，我们可以全部屏蔽未雨绸缪，也可以等有问题的时候一个个添加。
网友整理的所有YiSouSpider IP地址：
  1. Deny from 42.156.136.95
  2. Deny from 42.156.137.67
  3. Deny from 42.156.137.62
  4. Deny from 42.156.137.63
  5. Deny from 42.156.137.60
  6. Deny from 42.156.138.113
  7. Deny from 42.120.161.37
  8. Deny from 42.120.161.32
  9. Deny from 42.120.161.65
  10. Deny from 42.156.139.95
  11. Deny from 42.156.136.113
  12. Deny from 42.156.136.44
  13. Deny from 42.156.136.42
  14. Deny from 42.156.136.40
  15. Deny from 42.156.139.51
  16. Deny from 42.156.139.53
  17. Deny from 42.156.139.55
  18. Deny from 42.156.139.54
  19. Deny from 42.156.139.57
  20. Deny from 42.156.138.63
  21. Deny from 42.156.138.62
  22. Deny from 42.156.138.61
  23. Deny from 42.156.138.60
  24. Deny from 42.156.138.67
  25. Deny from 42.120.161.62
  26. Deny from 42.156.138.65
  27. Deny from 42.120.160.67
  28. Deny from 42.120.161.95
  29. Deny from 42.120.160.113
  30. Deny from 42.120.160.63
  31. Deny from 42.120.160.62
  32. Deny from 42.120.160.61
  33. Deny from 42.120.160.60
  34. Deny from 42.156.138.81
  35. Deny from 42.156.138.80
  36. Deny from 42.156.138.83
  37. Deny from 42.156.138.82
  38. Deny from 42.156.136.23
  39. Deny from 42.156.137.53
  40. Deny from 42.156.137.51
  41. Deny from 42.156.137.57
  42. Deny from 42.156.138.104
  43. Deny from 42.156.139.5
  44. Deny from 42.156.139.4
  45. Deny from 42.120.161.23
  46. Deny from 42.120.161.24
  47. Deny from 42.156.136.107
  48. Deny from 42.156.137.61
  49. Deny from 42.156.136.50
  50. Deny from 42.156.136.51
  51. Deny from 42.156.136.53
  52. Deny from 42.156.136.54
  53. Deny from 42.156.136.55
  54. Deny from 42.156.136.57
  55. Deny from 42.156.139.42
  56. Deny from 42.156.138.75
  57. Deny from 42.156.139.40
  58. Deny from 42.156.138.77
  59. Deny from 42.156.138.74
  60. Deny from 42.120.160.74
  61. Deny from 42.120.160.75
  62. Deny from 42.120.160.109
  63. Deny from 42.120.160.77
  64. Deny from 42.156.137.40
  65. Deny from 42.156.138.95
  66. Deny from 42.120.160.16
  67. Deny from 42.120.160.17
  68. Deny from 42.120.160.15
  69. Deny from 42.120.160.13
  70. Deny from 42.120.160.10
  71. Deny from 42.120.160.28
  72. Deny from 42.156.139.77
  73. Deny from 42.120.161.57
  74. Deny from 42.156.139.74
  75. Deny from 42.120.161.51
  76. Deny from 42.120.161.53
  77. Deny from 42.120.161.18
  78. Deny from 42.156.136.83
  79. Deny from 42.156.136.27
  80. Deny from 42.120.160.81
  81. Deny from 42.120.160.80
  82. Deny from 42.120.160.83
  83. Deny from 42.120.160.82
  84. Deny from 42.156.136.28
  85. Deny from 42.120.161.63
  86. Deny from 42.156.137.32
  87. Deny from 42.156.137.37
  88. Deny from 42.120.160.23
  89. Deny from 42.156.136.82
  90. Deny from 42.156.136.81
  91. Deny from 42.156.136.80
  92. Deny from 42.120.160.24
  93. Deny from 42.156.136.4
  94. Deny from 42.156.136.5
  95. Deny from 42.156.136.2
  96. Deny from 42.120.160.9
  97. Deny from 42.120.160.4
  98. Deny from 42.120.160.5
  99. Deny from 42.120.160.2
  100. Deny from 42.156.139.60
  101. Deny from 42.156.139.61
  102. Deny from 42.156.139.62
  103. Deny from 42.156.139.63
  104. Deny from 42.120.161.42
  105. Deny from 42.120.161.40
  106. Deny from 42.156.139.67
  107. Deny from 42.156.138.20
  108. Deny from 42.156.138.16
  109. Deny from 42.156.138.17
  110. Deny from 42.156.138.15
  111. Deny from 42.156.136.37
  112. Deny from 42.120.160.95
  113. Deny from 42.156.136.32
  114. Deny from 42.156.136.31
  115. Deny from 42.156.136.24
  116. Deny from 42.156.137.23
  117. Deny from 42.156.137.20
  118. Deny from 42.120.160.37
  119. Deny from 42.120.160.32
  120. Deny from 42.156.137.28
  121. Deny from 42.156.139.82
  122. Deny from 42.156.139.83
  123. Deny from 42.156.139.80
  124. Deny from 42.156.139.81
  125. Deny from 42.120.161.4
  126. Deny from 42.156.138.107
  127. Deny from 42.156.138.24
  128. Deny from 42.156.138.23
  129. Deny from 42.120.161.74
  130. Deny from 42.120.160.19
  131. Deny from 42.156.138.28
  132. Deny from 42.156.139.15
  133. Deny from 42.156.139.17
  134. Deny from 42.156.139.19
  135. Deny from 42.156.139.18
  136. Deny from 42.120.160.40
  137. Deny from 42.120.160.42
  138. Deny from 42.120.161.20
  139. Deny from 42.156.138.19
  140. Deny from 42.120.161.68
  141. Deny from 42.156.137.2
  142. Deny from 42.120.161.66
  143. Deny from 42.120.161.67
  144. Deny from 42.120.161.60
  145. Deny from 42.120.161.61
  146. Deny from 42.156.137.5
  147. Deny from 42.156.137.4
  148. Deny from 42.156.138.32
  149. Deny from 42.156.138.68
  150. Deny from 42.156.136.18
  151. Deny from 42.156.136.19
  152. Deny from 42.156.138.37
  153. Deny from 42.156.136.15
  154. Deny from 42.156.136.16
  155. Deny from 42.156.136.17
  156. Deny from 42.156.136.13
  157. Deny from 42.156.139.28
  158. Deny from 42.120.160.53
  159. Deny from 42.120.160.50
  160. Deny from 42.120.160.51
  161. Deny from 42.120.161.5
  162. Deny from 42.120.160.57
  163. Deny from 42.120.160.55
  164. Deny from 42.120.161.9
  165. Deny from 42.120.161.19
  166. Deny from 42.156.139.107
  167. Deny from 42.156.139.22
  168. Deny from 42.120.161.15
  169. Deny from 42.120.161.17
  170. Deny from 42.156.137.24
  171. Deny from 42.120.161.83
  172. Deny from 42.156.136.61
  173. Deny from 42.156.136.60
  174. Deny from 42.156.136.63
  175. Deny from 42.156.136.62
  176. Deny from 42.156.136.67
  177. Deny from 42.156.137.107
  178. Deny from 42.156.139.32
  179. Deny from 42.156.139.31
  180. Deny from 42.120.161.81
  181. Deny from 42.156.139.37
  182. Deny from 42.156.138.40
  183. Deny from 42.156.137.19
  184. Deny from 42.156.138.42
  185. Deny from 42.156.137.17
  186. Deny from 42.156.137.15
  187. Deny from 42.156.137.13
  188. Deny from 42.156.137.10
  189. Deny from 42.120.161.55
  190. Deny from 42.120.161.113
  191. Deny from 42.120.161.77
  192. Deny from 42.156.137.74
  193. Deny from 42.156.137.77
  194. Deny from 42.156.137.95
  195. Deny from 42.156.138.2
  196. Deny from 42.156.138.4
  197. Deny from 42.156.138.5
  198. Deny from 42.156.139.113
  199. Deny from 42.120.160.20
  200. Deny from 42.156.137.80
  201. Deny from 42.156.137.81
  202. Deny from 42.156.137.82
  203. Deny from 42.156.137.83
  204. Deny from 42.156.137.113
  205. Deny from 42.156.139.71
  206. Deny from 42.156.139.24
  207. Deny from 42.156.139.20
  208. Deny from 42.156.136.77
  209. Deny from 42.156.136.74
  210. Deny from 42.156.139.23
  211. Deny from 42.120.161.82
  212. Deny from 42.156.138.57
  213. Deny from 42.120.161.80
  214. Deny from 42.156.138.55
  215. Deny from 42.156.138.53
  216. Deny from 42.156.138.50
  217. Deny from 42.156.138.51
复制代码
PS：如果我们是NGINX环境系统，我们也可以到自己当前网站的.CONF文件中添加
  1. deny 42.120.161.43;
复制代码
这样我们一样可以屏蔽当前这个IP地址的访问。
PS：这里在补充Nginx环境屏蔽YisouSpider爬虫的方法，我们可以直接将整个爬虫User Agent端口屏蔽，反正也没有多大的用途。
  1. if ($http_user_agent ~* "YisouSpider") {
  2. return 403;
  3. }
复制代码
在我们当前站点的.conf文件中添加上面的脚本，保存替换之后重启Nginx生效，这样我们就可以让所有的YisouSpider返回403。






欢迎光临 黑帽联盟 (https://bbs.cnblackhat.com/) Powered by Discuz! X2.5