最近接了一个老外的项目,需要模拟浏览器行为 - V2EX
zixiaomeng

最近接了一个老外的项目,需要模拟浏览器行为

  •  
  •   zixiaomeng Nov 28, 2018 6463 views
    This topic created in 2744 days ago, the information mentioned may be changed or developed.

    例如用多个地区的 ip 地址,模拟自然人访问某个网站,注册账号。需要绕过反爬虫和机器人识别。

    有没有库推荐?

    15 replies    2018-11-29 16:08:18 +08:00
    Kilerd
        1
    Kilerd  
       Nov 28, 2018
    https://github.com/kennethreitz/requests-html
    试试这个? 不行就用 headless 来实现咯
    EvilCult
        2
    EvilCult  
       Nov 28, 2018
    菜鸟表示能想到的最简单的就是
    python + Selenium
    使用 chrome driver
    设置--proxy-server http 代理
    kljsandjb
        3
    kljsandjb  
       Nov 28, 2018 via iPhone
    chrome headless ?
    jydeng
        4
    jydeng  
       Nov 28, 2018
    puppeteer
    zr8657
        5
    zr8657  
       Nov 28, 2018
    selenium,最近发现代理 ip 真是贵
    onedayoneapple
        6
    onedayoneapple  
       Nov 28, 2018
    @zr8657 有好的 ip 推荐吗
    q4336431
        7
    q4336431  
       Nov 28, 2018
    selenium+phantomjs,只不过 ip 这个不好处理,代理是真滴贵
    zr8657
        8
    zr8657  
       Nov 28, 2018
    @onedayoneapple 我自己买的最便宜的 60 一个月 1000 条,但是速度很慢,1M 带宽都不到,我也想找更便宜的。。有大肘子科普下为什么这么贵吗
    andylsr
        9
    andylsr  
       Nov 28, 2018 via Android
    adsl 服务器
    KingFong
        10
    KingFong  
    PRO
       Nov 28, 2018
    我觉得主要资源是 IP,以前买过美国的 private proxy,一个月 5 美元,速度非常快。
    akira
        11
    akira  
       Nov 28, 2018
    这个不是某一个库可以做到的事情吧
    hadoop
        12
    hadoop  
       Nov 28, 2018
    @yanjieee 关键词?
    Daveedo
        13
    Daveedo  
       Nov 28, 2018
    ip 代理去买就行了。几块钱买 500 个
    takato
        14
    takato  
       Nov 28, 2018
    如果对方对抗措施充分,完成这个就至少需要良好的数学和统计学功底。。。

    可能一般的库或方法是不行哦。。

    举个子问题的例子:如何用 code 控制鼠标光标的移动轨迹,使其符合人类使用物理鼠标移动光标的特征。
    clap3times
        15
    clap3times  
       Nov 29, 2018
    有个爬虫私活,有人感兴趣吗
    About     Help     Advertise     Blog     API     FAQ     Solana     5387 Online   Highest 6679       Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 116ms UTC 07:07 PVG 15:07 LAX 00:07 JFK 03:07
    Do have faith in what you're doing.
    ubao msn snddm index pchome yahoo rakuten mypaper meadowduck bidyahoo youbao zxmzxm asda bnvcg cvbfg dfscv mmhjk xxddc yybgb zznbn ccubao uaitu acv GXCV ET GDG YH FG BCVB FJFH CBRE CBC GDG ET54 WRWR RWER WREW WRWER RWER SDG EW SF DSFSF fbbs ubao fhd dfg ewr dg df ewwr ewwr et ruyut utut dfg fgd gdfgt etg dfgt dfgd ert4 gd fgg wr 235 wer3 we vsdf sdf gdf ert xcv sdf rwer hfd dfg cvb rwf afb dfh jgh bmn lgh rty gfds cxv xcv xcs vdas fdf fgd cv sdf tert sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf shasha9178 shasha9178 shasha9178 shasha9178 shasha9178 liflif2 liflif2 liflif2 liflif2 liflif2 liblib3 liblib3 liblib3 liblib3 liblib3 zhazha444 zhazha444 zhazha444 zhazha444 zhazha444 dende5 dende denden denden2 denden21 fenfen9 fenf619 fen619 fenfe9 fe619 sdf sdf sdf sdf sdf zhazh90 zhazh0 zhaa50 zha90 zh590 zho zhoz zhozh zhozho zhozho2 lislis lls95 lili95 lils5 liss9 sdf0ty987 sdft876 sdft9876 sdf09876 sd0t9876 sdf0ty98 sdf0976 sdf0ty986 sdf0ty96 sdf0t76 sdf0876 df0ty98 sf0t876 sd0ty76 sdy76 sdf76 sdf0t76 sdf0ty9 sdf0ty98 sdf0ty987 sdf0ty98 sdf6676 sdf876 sd876 sd876 sdf6 sdf6 sdf9876 sdf0t sdf06 sdf0ty9776 sdf0ty9776 sdf0ty76 sdf8876 sdf0t sd6 sdf06 s688876 sd688 sdf86