问题描述
如何在多进程 python 程序中捕获 Ctrl+C 并优雅地退出所有进程,我需要该解决方案在 unix 和 windows 上都可以工作.我尝试了以下方法:
How do I catch a Ctrl+C in multiprocess python program and exit all processes gracefully, I need the solution to work both on unix and windows. I've tried the following:
import multiprocessing import time import signal import sys jobs = [] def worker(): signal.signal(signal.SIGINT, signal_handler) while(True): time.sleep(1.1234) print "Working..." def signal_handler(signal, frame): print 'You pressed Ctrl+C!' # for p in jobs: # p.terminate() sys.exit(0) if __name__ == "__main__": for i in range(50): p = multiprocessing.Process(target=worker) jobs.append(p) p.start()
它有点工作,但我认为这不是正确的解决方案.
And it's kind of working, but I don't think it's the right solution.
推荐答案
先前接受的解决方案 有竞争条件,但它没有使用 map 和 async 函数.
The previously accepted solution has race conditions and it does not work with map and async functions.
multiprocessing.Pool处理Ctrl+C/SIGINT的正确方法是:
- 在创建进程 Pool 之前让进程忽略 SIGINT.这种方式创建的子进程继承 SIGINT 处理程序.
- 在创建 Pool 后,在父进程中恢复原始 SIGINT 处理程序.
- 使用 map_async 和 apply_async 而不是阻塞 map 和 apply.
- 使用超时等待结果,因为默认阻塞等待忽略所有信号.这是 Python 错误 https://bugs.python.org/issue8296.
- Make the process ignore SIGINT before a process Pool is created. This way created child processes inherit SIGINT handler.
- Restore the original SIGINT handler in the parent process after a Pool has been created.
- Use map_async and apply_async instead of blocking map and apply.
- Wait on the results with timeout because the default blocking waits to ignore all signals. This is Python bug https://bugs.python.org/issue8296.
<小时>
把它放在一起:
Putting it together:
#!/bin/env python from __future__ import print_function import multiprocessing import os import signal import time def run_worker(delay): print("In a worker process", os.getpid()) time.sleep(delay) def main(): print("Initializng 2 workers") original_sigint_handler = signal.signal(signal.SIGINT, signal.SIG_IGN) pool = multiprocessing.Pool(2) signal.signal(signal.SIGINT, original_sigint_handler) try: print("Starting 2 jobs of 5 seconds each") res = pool.map_async(run_worker, [5, 5]) print("Waiting for results") res.get(60) # Without the timeout this blocking call ignores all signals. except KeyboardInterrupt: print("Caught KeyboardInterrupt, terminating workers") pool.terminate() else: print("Normal termination") pool.close() pool.join() if __name__ == "__main__": main()
正如@YakovShklarov 所指出的,在父进程中忽略信号和取消忽略信号之间有一个时间窗口,在此期间信号可能会丢失.使用 pthread_sigmask 代替在父进程中临时阻止信号的传递可以防止信号丢失,但是在 Python-2 中不可用.
As @YakovShklarov noted, there is a window of time between ignoring the signal and unignoring it in the parent process, during which the signal can be lost. Using pthread_sigmask instead to temporarily block the delivery of the signal in the parent process would prevent the signal from being lost, however, it is not available in Python-2.