问题描述
我有一个相当简单的嵌套 for 循环,它遍历四个数组:
I have a fairly straightforward nested for loop that iterates over four arrays:
for a in a_grid: for b in b_grid: for c in c_grid: for d in d_grid: do_some_stuff(a,b,c,d) # perform calculations and write to file
也许这并不是在 4D 网格上执行计算的最有效方式.我知道 joblib 能够并行化两个嵌套的 for 循环,例如 this,但我无法将其推广到四个嵌套循环.有什么想法吗?
Maybe this isn't the most efficient way to perform calculations over a 4D grid to begin with. I know joblib is capable of parallelizing two nested for loops like this, but I'm having trouble generalizing it to four nested loops. Any ideas?
推荐答案
我通常使用这种形式的代码:
I usually use code of this form:
#!/usr/bin/env python3 import itertools import multiprocessing #Generate values for each parameter a = range(10) b = range(10) c = range(10) d = range(10) #Generate a list of tuples where each tuple is a combination of parameters. #The list will contain all possible combinations of parameters. paramlist = list(itertools.product(a,b,c,d)) #A function which will process a tuple of parameters def func(params): a = params[0] b = params[1] c = params[2] d = params[3] return a*b*c*d #Generate processes equal to the number of cores pool = multiprocessing.Pool() #Distribute the parameter sets evenly across the cores res = pool.map(func,paramlist)