searchusermenu
  • 发布文章
  • 消息中心
点赞
收藏
评论
分享
原创

RuntimeError: unable to write to file </torch_1086

2023-06-29 08:33:33
195
0

在对fastreid进行容器化服务时,使用pytorch的Dataloader类加载数据,加载数据代码

test_loader = DataLoader(
       
        dataset=test_set,
        batch_sampler=batch_sampler,
        num_workers=num_workers,  # save some memory
        collate_fn=fast_batch_collator,
        # pin_memory=True,
    )

读取数据过程中,出现如下错误:

Traceback (most recent call last):
File "/dl/python/python/lib/python3.6/multiprocessing/queues.py", line 234, in _feed
obj = _ForkingPickler.dumps(obj)
File "/dl/python/python/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
File "/dl/python/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 321, in reduce_storage
fd, size = storage.share_fd()
RuntimeError: unable to write to file </torch_108661_22634063>
Traceback (most recent call last):
File "/dl/python/python/lib/python3.6/multiprocessing/queues.py", line 234, in _feed
obj = _ForkingPickler.dumps(obj)
File "/dl/python/python/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
File "/dl/python/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 321, in reduce_storage
fd, size = storage.share_fd()
RuntimeError: unable to write to file </torch_108661_1392190158>

问题原因:

上述问题发生在容器中,可能是因为容器内的共享内存机制与宿主机有所不同,导致无法正确地传输数据。

解决方法:

使用 --ipc=host 选项启动容器,以便容器和宿主机可以共享进程间通信(IPC)资源,上述错误不再出现。

 

0条评论
0 / 1000
张****佳
7文章数
1粉丝数
张****佳
7 文章 | 1 粉丝
原创

RuntimeError: unable to write to file </torch_1086

2023-06-29 08:33:33
195
0

在对fastreid进行容器化服务时,使用pytorch的Dataloader类加载数据,加载数据代码

test_loader = DataLoader(
       
        dataset=test_set,
        batch_sampler=batch_sampler,
        num_workers=num_workers,  # save some memory
        collate_fn=fast_batch_collator,
        # pin_memory=True,
    )

读取数据过程中,出现如下错误:

Traceback (most recent call last):
File "/dl/python/python/lib/python3.6/multiprocessing/queues.py", line 234, in _feed
obj = _ForkingPickler.dumps(obj)
File "/dl/python/python/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
File "/dl/python/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 321, in reduce_storage
fd, size = storage.share_fd()
RuntimeError: unable to write to file </torch_108661_22634063>
Traceback (most recent call last):
File "/dl/python/python/lib/python3.6/multiprocessing/queues.py", line 234, in _feed
obj = _ForkingPickler.dumps(obj)
File "/dl/python/python/lib/python3.6/multiprocessing/reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
File "/dl/python/lib/python3.6/site-packages/torch/multiprocessing/reductions.py", line 321, in reduce_storage
fd, size = storage.share_fd()
RuntimeError: unable to write to file </torch_108661_1392190158>

问题原因:

上述问题发生在容器中,可能是因为容器内的共享内存机制与宿主机有所不同,导致无法正确地传输数据。

解决方法:

使用 --ipc=host 选项启动容器,以便容器和宿主机可以共享进程间通信(IPC)资源,上述错误不再出现。

 

文章来自个人专栏
AI技术分享
7 文章 | 1 订阅
0条评论
0 / 1000
请输入你的评论
0
0