是否有像Perl的LWP for Ruby这样的模块？

发布时间：2020-12-15 21:54:04 所属栏目：大数据来源：网络整理

导读：在Perl中有一个 LWP module： The libwww-perl collection is a set of Perl modules which provides a simple and consistent application programming interface (API) to the World-Wide Web. The main focus of the library is to provide classes and f

在Perl中有一个 LWP module：

The libwww-perl collection is a set of Perl modules which provides a simple and consistent application programming interface (API) to the World-Wide Web. The main focus of the library is to provide classes and functions that allow you to write WWW clients. The library also contain modules that are of more general use and even classes that help you implement simple HTTP servers.

Ruby有类似的模块(gem)吗？

更新

以下是我从特定网站中提取URL的功能示例.

use LWP::UserAgent;
use HTML::TreeBuilder 3;
use HTML::TokeParser;

sub get_gallery_urls {
    my $url = shift;

    my $ua = LWP::UserAgent->new;
    $ua->agent("$0/0.1 " . $ua->agent);
    $ua->agent("Mozilla/8.0");

    my $req = new HTTP::Request 'GET' => "$url";
    $req->header('Accept' => 'text/html');

    # send request
    $response_u = $ua->request($req);

    die "Error: ",$response_u->status_line unless $response_u->is_success;

    my $root = HTML::TreeBuilder->new;
    $root->parse($response_u->content);

    my @gu = $root->find_by_attribute("id","thumbnails");

    my %urls = ();

    foreach my $g (@gu) {
        my @as = $g->find_by_tag_name('a');

        foreach $a (@as) {
            my $u = $a->attr("href");

            if ($u =~ /^//) {
                $urls{"http://example.com"."$u"} = 1;
            }
        }
    }

    return %urls;
}

解决方法

最接近的匹配可能是 httpclient,其目的是相当于LWP.但是,根据您的计划,可能会有更好的选择.如果您打算关注链接,填写表单等以便抓取Web内容,您可以使用 Mechanize,它与perl模块的名称相同.还有更多特定于Ruby的宝石,例如优秀的 Rest-client和 HTTParty(我个人最喜欢的).有关更大的列表,请参阅 HTTP Clients category of Ruby Toolbox.

更新：这是一个如何在Mechanize中找到页面上所有链接的示例(Ruby,但它在Perl中类似)：

require 'rubygems'
require 'mechanize'

agent = Mechanize.new

page = agent.get('http://example.com/')

page.links.each do |link|
  puts link.text
end

附：作为一个前Perler,我曾经担心放弃优秀的CPAN – 我会把自己画成Ruby的角落吗？难道我无法找到与我依赖的模块相当的东西吗？事实证明这根本不是问题,事实上最近恰恰相反：Ruby(以及Python)往往是第一个获得客户支持新平台/ Web服务等的人.

（编辑：李大同）

【声明】本站内容均来自网络，其相关言论仅代表作者个人观点，不代表本站立场。若无意侵犯到您的权利，请及时与联系站长删除相关内容!