perl – 带有Search :: ElasticSearch :: Scroll的ElasticSearch
发布时间:2020-12-15 23:33:09 所属栏目:大数据 来源:网络整理
导读:我正在使用Search :: Elasticsearch和Search :: Elasticsearch :: Scroll进行搜索并滚动到我的elasticsearch服务器. 在滚动过程中,对于某些查询,我在滚动搜索结果时看到下一个错误: 2016/03/22 11:03:38 - 265885 FATAL: [Daemon.pm][8221]: Something gone
我正在使用Search :: Elasticsearch和Search :: Elasticsearch :: Scroll进行搜索并滚动到我的elasticsearch服务器.
在滚动过程中,对于某些查询,我在滚动搜索结果时看到下一个错误: 2016/03/22 11:03:38 - 265885 FATAL: [Daemon.pm][8221]: Something gone wrong,error $VAR1 = bless( { 'msg' => '[Missing] ** [http://localhost:9200]-[404] Not Found,called from sub Search::Elasticsearch::Scroll::next at searcher.pl line 92. With vars: {'body' => {'hits' => {'hits' => [],'max_score' => '0','total' => 5215},'timed_out' => bless( do{(my $o = 0)},'JSON::XS::Boolean' ),'_shards' => {'failures' => [{'index' => undef,'reason' => {'reason' => 'No search context found for id [4920053]','type' => 'search_context_missing_exception'},'shard' => -1},{'index' => undef,'reason' => {'reason' => 'No search context found for id [5051485]','reason' => {'reason' => 'No search context found for id [4920059]','reason' => {'reason' => 'No search context found for id [5051496]','reason' => {'reason' => 'No search context found for id [5051500]','shard' => -1}],'failed' => 5,'successful' => 0,'total' => 5},'_scroll_id' => 'c2NhbjswOzE7dG90YWxfaGl0czo1MjE1Ow==','took' => 2},'request' => {'serialize' => 'std','path' => '/_search/scroll','ignore' => [],'mime_type' => 'application/json','body' => 'c2Nhbjs1OzQ5MjAwNTM6bHExbENzRDVReEc0OV9UMUgzd3Vkdzs1MDUxNDg1OnJrQ3lsUkRKVHRxRWRWeURoOTB4WVE7NDkyMDA1OTpscTFsQ3NENVF4RzQ5X1QxSDN3dWR3OzUwNTE0OTY6cmtDeWxSREpUdHFFZFZ5RGg5MHhZUTs1MDUxNTAwOnJrQ3lsUkRKVHRxRWRWeURoOTB4WVE7MTt0b3RhbF9oaXRzOjUyMTU7','qs' => {'scroll' => '1m'},'method' => 'GET'},'status_code' => 404} ','stack' => [ [ 'searcher.pl',92,'Search::Elasticsearch::Scroll::next' ] ],'text' => '[http://localhost:9200]-[404] Not Found','vars' => { 'body' => { 'hits' => { 'hits' => [],'max_score' => '0','total' => 5215 },'timed_out' => bless( do{(my $o = 0)},'JSON::XS::Boolean' ),'_shards' => { 'failures' => [ { 'index' => undef,'reason' => { 'reason' => 'No search context found for id [4920053]','type' => 'search_context_missing_exception' },'shard' => -1 },{ 'index' => undef,'reason' => { 'reason' => 'No search context found for id [5051485]','reason' => { 'reason' => 'No search context found for id [4920059]','reason' => { 'reason' => 'No search context found for id [5051496]','shard' => -1 },'reason' => { 'reason' => 'No search context found for id [5051500]','shard' => -1 } ],'failed' => 5,'successful' => 0,'total' => 5 },'_scroll_id' => 'c2NhbjswOzE7dG90YWxfaGl0czo1MjE1Ow==','took' => 2 },'request' => { 'serialize' => 'std','path' => '/_search/scroll','ignore' => [],'mime_type' => 'application/json','body' => 'c2Nhbjs1OzQ5MjAwNTM6bHExbENzRDVReEc0OV9UMUgzd3Vkdzs1MDUxNDg1OnJrQ3lsUkRKVHRxRWRWeURoOTB4WVE7NDkyMDA1OTpscTFsQ3NENVF4RzQ5X1QxSDN3dWR3OzUwNTE0OTY6cmtDeWxSREpUdHFFZFZ5RGg5MHhZUTs1MDUxNTAwOnJrQ3lsUkRKVHRxRWRWeURoOTB4WVE7MTt0b3RhbF9oaXRzOjUyMTU7','qs' => { 'scroll' => '1m' },'method' => 'GET' },'status_code' => 404 },'type' => 'Missing' },'Search::Elasticsearch::Error::Missing' ); 我正在使用的代码是下一个(简化): # Retrieve scroll my $scroll = $self->getScrollBySignature($item); # Retrieve all affected documents ids while (my @docs = $scroll->next(500)) { # Do stuff with @docs } 函数getScrollBySignature具有下一个代码,以便调用elasticSearch my $scroll = $self->{ELASTIC}->scroll_helper( index => $self->{INDEXES},search_type => 'scan',ignore_unavailable => 1,body => { size => $self->{PAGINATION},query => { filtered => { filter => { bool => { must => [{term => {signature_id => $item->{profileId}}},{terms => {channel_type_id => $type}}] } } } } } ); 正如你所看到的,我正在做滚动而不传递滚动参数然后正如文档所说,滚动活着的时间是1分钟. elasticSearch是一个由3个服务器组成的集群,以该错误结束的查询检索的文档超过5000个. 我的第一个解决方案是将滚动的生命周期更新为5分钟,并且没有出现错误. 问题是,正如我所知,每当我调用$scroll-> next()时,受影响的滚动生命周期会升级1m,那么如何才能接收这些与上下文相关的错误? 我做得不好吗? 谢谢你们. 解决方法
首先想到的是计时器没有更新.你检查过这个吗?例如,你可以每隔10秒进行一次查询,看看第6次查询是否会给你错误…
(编辑:李大同) 【声明】本站内容均来自网络,其相关言论仅代表作者个人观点,不代表本站立场。若无意侵犯到您的权利,请及时与联系站长删除相关内容! |