在执行pool.map时,是否可以在python中每x秒执行一次函数?
发布时间:2020-12-20 11:37:40 所属栏目:Python 来源:网络整理
导读:我在大数据阵列上运行pool.map,我想每分钟在控制台中打印报告. 可能吗?据我所知,python是同步语言,它不能像nodejs那样做. 也许它可以通过线程来完成..或者如何? finished = 0def make_job(): sleep(1) global finished finished += 1# I want to call this
我在大数据阵列上运行pool.map,我想每分钟在控制台中打印报告.
可能吗?据我所知,python是同步语言,它不能像nodejs那样做. 也许它可以通过线程来完成..或者如何? finished = 0 def make_job(): sleep(1) global finished finished += 1 # I want to call this function every minute def display_status(): print 'finished: ' + finished def main(): data = [...] pool = ThreadPool(45) results = pool.map(make_job,data) pool.close() pool.join() 解决方法
您可以使用永久线程计时器,例如来自此问题的计时器:
Python threading.timer – repeat function every ‘n’ seconds
from threading import Timer,Event class perpetualTimer(object): # give it a cycle time (t) and a callback (hFunction) def __init__(self,t,hFunction): self.t=t self.stop = Event() self.hFunction = hFunction self.thread = Timer(self.t,self.handle_function) def handle_function(self): self.hFunction() self.thread = Timer(self.t,self.handle_function) if not self.stop.is_set(): self.thread.start() def start(self): self.stop.clear() self.thread.start() def cancel(self): self.stop.set() self.thread.cancel() 基本上这只是Timer对象的包装器,每次调用所需的函数时都会创建一个新的Timer对象.不要指望毫秒精度(甚至接近),但为了您的目的,它应该是理想的. 使用此示例将成为: finished = 0 def make_job(): sleep(1) global finished finished += 1 def display_status(): print 'finished: ' + finished def main(): data = [...] pool = ThreadPool(45) # set up the monitor to make run the function every minute monitor = PerpetualTimer(60,display_status) monitor.start() results = pool.map(make_job,data) pool.close() pool.join() monitor.cancel() 编辑: 更清洁的解决方案可能是(感谢下面的评论): from threading import Event,Thread class RepeatTimer(Thread): def __init__(self,callback,event): Thread.__init__(self) self.stop = event self.wait_time = t self.callback = callback self.daemon = True def run(self): while not self.stop.wait(self.wait_time): self.callback() 然后在你的代码中: def main(): data = [...] pool = ThreadPool(45) stop_flag = Event() RepeatTimer(60,display_status,stop_flag).start() results = pool.map(make_job,data) pool.close() pool.join() stop_flag.set() (编辑:李大同) 【声明】本站内容均来自网络,其相关言论仅代表作者个人观点,不代表本站立场。若无意侵犯到您的权利,请及时与联系站长删除相关内容! |