在对fastreid进行容器化服务时,使用pytorch的Dataloader类加载数据,加载数据代码
读取数据过程中,出现如下错误:
Traceback (most recent call last):
File "/dl/python/python/lib/python3.6/multiprocessing/queues.py", line 234, in _feed
obj = _ForkingPickler.dumps(obj)
File "/dl/python/python/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
File "/dl/python/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 321, in reduce_storage
fd, size = storage.share_fd()
RuntimeError: unable to write to file </torch_108661_22634063>
Traceback (most recent call last):
File "/dl/python/python/lib/python3.6/multiprocessing/queues.py", line 234, in _feed
obj = _ForkingPickler.dumps(obj)
File "/dl/python/python/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
File "/dl/python/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 321, in reduce_storage
fd, size = storage.share_fd()
RuntimeError: unable to write to file </torch_108661_1392190158>
问题原因:
上述问题发生在容器中,可能是因为容器内的共享内存机制与宿主机有所不同,导致无法正确地传输数据。
解决方法:
使用 --ipc=host
选项启动容器,以便容器和宿主机可以共享进程间通信(IPC)资源,上述错误不再出现。