问题描述
我正在尝试创建一个使用 python 的多处理模块的脚本.该脚本(我们称之为 myscript.py)将从另一个带有管道的脚本中获取输入.
I'm trying to create a script which is using multiprocessing module with python. The script (lets call it myscript.py) will get the input from another script with pipe.
假设我这样调用脚本;
$ python writer.py | python myscript.py
这里是代码;
// writer.py import time, sys def main(): while True: print "test" sys.stdout.flush() time.sleep(1) main() //myscript.py def get_input(): while True: text = sys.stdin.readline() print "hello " + text time.sleep(3) if __name__ == '__main__': p1 = Process(target=get_input, args=()) p1.start()
这显然不起作用,因为主进程和 p1 的 sys.stdin 对象不同.所以我尝试了这个来解决它,
this is clearly not working, since the sys.stdin objects are different for main process and p1. So I have tried this to solve it,
//myscript.py def get_input(temp): while True: text = temp.readline() print "hello " + text time.sleep(3) if __name__ == '__main__': p1 = Process(target=get_input, args=(sys.stdin,)) p1.start()
但是我遇到了这个错误;
but I come across with this error;
Process Process-1: Traceback (most recent call last): File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run self._target(*self._args, **self._kwargs) File "in.py", line 12, in get_input text = temp.readline() ValueError: I/O operation on closed file
所以,我猜 main 的 stdin 文件已关闭,我无法从中读取.在这个结合处,我如何将 main 的 stdin 文件传递??给另一个进程?如果无法通过标准输入,我该如何使用来自另一个进程的 main 标准输入?
So, I guess that main's stdin file closed and I can't read from it. At this conjunction, how can I pass main's stdin file to another process? If passing stdin is not possible, how can I use main's stdin from another process?
更新:好的,我需要澄清我的问题,因为人们认为使用多处理并不是真正必要的.像这样考虑 myscript.py;
update: Okay, I need to clarify my question since people think using multiprocessing is not really necessary. consider myscript.py like this;
//myscript.py def get_input(): while True: text = sys.stdin.readline() print "hello " + text time.sleep(3) def do_more_things(): while True: #// some code here time.sleep(60*5) if __name__ == '__main__': p1 = Process(target=get_input, args=()) p1.start() do_more_things()
所以,我真的需要将 get_input() 函数与 main 函数(或其他子进程)并行运行.很抱歉发生冲突,我的英语不错,我想我对这个问题不太清楚.如果你们能告诉我我是否可以在另一个进程中使用主进程 STDIN 对象,我将不胜感激.
so, I really need to run get_input() function parallelly with main function (or other sub processes). Sorry for the conflicts, I have a decent English, and I guess I couldn't be clear on this question. I would appreciate if you guys can tell me if i can use the main processes STDIN object in another process.
提前致谢.
推荐答案
最简单的就是交换get_input()和do_more_things(),即读取sys.stdin 在父进程中:
The simplest thing is to swap get_input() and do_more_things() i.e., read sys.stdin in the parent process:
def get_input(stdin): for line in iter(stdin.readline, ''): print("hello", line, end='') stdin.close() if __name__ == '__main__': p1 = mp.Process(target=do_more_things) p1.start() get_input(sys.stdin)
下一个最好的方法是在 get_input() 中使用 Thread() 而不是 Process():
The next best thing is to use a Thread() instead of a Process() for get_input():
if __name__ == '__main__': t = Thread(target=get_input, args=(sys.stdin,)) t.start() do_more_things()
如果上述方法没有帮助你可以尝试 os.dup():
If the above doesn't help you could try os.dup():
newstdin = os.fdopen(os.dup(sys.stdin.fileno())) try: p = Process(target=get_input, args=(newstdin,)) p.start() finally: newstdin.close() # close in the parent do_more_things()