基本的操作请参考:joblib多线程、多进程学习案例(一)——一步步写多进程任务
可以使用delayed
简化任务列表[ 函数名, [参数列表], {参数列表} ],案例如下:
首先是加法函数:
def add_func(x, y):
print('---------=============---------')
print("当前进程:", os.getpid(), " 父进程:", os.getppid())
t = threading.currentThread()
print('Thread id : %d' % t.ident)
return x + y
然后确定参数列表:
x_list = [1, 2, 3] # 第一个参数
y_list = [4, 5, 6] # 第二个参数
tasks = [delayed(add_func)(x, y) for x, y in zip(x_list, y_list)]
然后使用多进程、多线程:
multi_work = Parallel(n_jobs=2, backend='threading')
res = multi_work(tasks)
全部案例
import os
import threading
from joblib import Parallel, delayed
def add_func(x, y):
print('---------=============---------')
print("当前进程:", os.getpid(), " 父进程:", os.getppid())
t = threading.currentThread()
print('Thread id : %d' % t.ident)
return x + y
if __name__ == '__main__':
x_list = [1, 2, 3] # 第一个参数
y_list = [4, 5, 6] # 第二个参数
# tasks = [delayed(add_func)(x, y) for x, y in zip(x_list, y_list)]
tasks = []
for x, y in zip(x_list, y_list):
tasks.append(delayed(add_func)(x, y))
# print(tasks) # [(<function add_func at 0x7f95f0710680>, (1, 4), {}) ...]
multi_work = Parallel(n_jobs=2, backend='threading')
res = multi_work(tasks)
print(res)