python - Multiprocessing IndexError out of range

Question

I'm having a issue with multiprocessing.pool on the following piece of code:

from multiprocessing import cpu_count
from multiprocessing.pool import Pool

if __name__ == '__main__':

    files = compile_paths()
    pool = Pool(processes=cpu_count())
    r = pool.map(compile_,files,chunksize=len(files))

That gives out:

 Traceback (most recent call last):

  File "<ipython-input-39-06f23d3cf8fa>", line 1, in <module>
    runfile('C:/Users/rafael/scripts/ETL/read.py', wdir='C:/Users/rafael/scripts/ETL')

  File "C:Users
afaelAnaconda3libsite-packagesspyder_kernelscustomizespydercustomize.py", line 827, in runfile
    execfile(filename, namespace)

  File "C:Users
afaelAnaconda3libsite-packagesspyder_kernelscustomizespydercustomize.py", line 110, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "C:/Users/rafael/scripts/ETL/read.py", line 185, in <module>
    r = pool.map(compile_,files,chunksize=len(files))

  File "C:Users
afaelAnaconda3libmultiprocessingpool.py", line 268, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()

  File "C:Users
afaelAnaconda3libmultiprocessingpool.py", line 657, in get
    raise self._value
IndexError: list index out of range

This compile_ is a function that reads the paths from files and saves then in a folder. The problem is that I put a print("Howdy") at the start of the function and it's never printed. Moreover, I made smaller files just to test it out and it never leaves the pool.map() function.

Answer 1

等待大神答复