用perl获取google搜索的结果
发布时间:2020-12-16 00:21:16 所属栏目:大数据 来源:网络整理
导读:以前google的API还开放的时候,可以直接使用CPAN上的包来进行获取,现在只能自己码代码了。。。 代码如下 sub google_search{my $keyword = $_[0];print "keyword is $keywordn";#we must first convert our search term into utf8,then can we use uri_esc
以前google的API还开放的时候,可以直接使用CPAN上的包来进行获取,现在只能自己码代码了。。。 代码如下 sub google_search{ my $keyword = $_[0]; print "keyword is $keywordn"; #we must first convert our search term into utf8,then can we use uri_escape to encode the term to search-engine like encoding my $search = encode("utf-8",decode("gb2312",$keyword)); $search = uri_escape($search); print "keyword is $searchn"; my $ua = new LWP::UserAgent; #Construct the request object $ua->agent( "Mozilla/5.0 (X11; Linux i686; rv:2.0.0) Gecko/20100101" ); #Google checks UAs,so set this to something common $ua->timeout( 10 ); #Give up after 10 seconds of waiting $ua->max_redirect( 0 ); #We only want the URL that google sends,so stop after we submit $ua->env_proxy; #Load proxy info from the environment,if any is set my $response = $ua->get( "http://www.google.com/search?q=".$search #The URL to request ) or die( $! ); #my $firstUrl = $response->contents; #This contains the link of the result page return encode("utf8",$response->decoded_content()); #convert to utf8 }这里用到了3个package: LWP::UserAgent 用来下载网页 URI::Escape 用来对中文进行转义,生成google链接中的奇奇怪怪的编码 use Encode 对我们输入的检索关键词编码至utf8 (编辑:李大同) 【声明】本站内容均来自网络,其相关言论仅代表作者个人观点,不代表本站立场。若无意侵犯到您的权利,请及时与联系站长删除相关内容! |