Запуск простых трансформаторов для проверки NER вызывает ошибку. Какие-нибудь решения?

Question

Запуск простых трансформаторов для проверки NER вызывает ошибку. Какие-нибудь решения?

Я пытаюсь проверить этот https://github.com/ThilinaRajapakse/simpletransformers NER, используя простые трансформаторы, и столкнулся со следующей ошибкой. может кто-нибудь помочь мне решить эту проблему?

Я попытался написать свой собственный NER, используя библиотеку трансформаторов объятия лица, и все еще сталкиваюсь с той же проблемой.

Пробовал это: https://github.com/ThilinaRajapakse/simpletransformers/issues/370 все еще сталкивается с той же проблемой с python3.7

Ошибка:

File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\site-packages\simpletransformers\ner\ner_model.py", line 242, in train_model
    train_dataset = self.load_and_cache_examples(train_data)
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\site-packages\simpletransformers\ner\ner_model.py", line 986, in load_and_cache_examples
    features = convert_examples_to_features(
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\site-packages\simpletransformers\ner\ner_utils.py", line 242, in convert_examples_to_features
    with Pool(process_count) as p:
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
INFO:simpletransformers.ner.ner_model: Converting to features started.
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\pool.py", line 212, in __init__
Traceback (most recent call last):
    self._repopulate_pool()
  File "<string>", line 1, in <module>
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\pool.py", line 303, in _repopulate_pool
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 116, in spawn_main
    return self._repopulate_pool_static(self._ctx, self.Process,
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 125, in _main
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\pool.py", line 326, in _repopulate_pool_static
    prepare(preparation_data)
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 236, in prepare
    w.start()
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\process.py", line 121, in start
    self._popen = self._Popen(self)
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\context.py", line 326, in _Popen
    main_content = runpy.run_path(main_path,
    return Popen(process_obj)
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\runpy.py", line 265, in run_path
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__
    prep_data = spawn.get_preparation_data(process_obj._name)
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 154, in get_preparation_data
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
    _check_not_importing_main()
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\runpy.py", line 87, in _run_code
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 134, in _check_not_importing_main
    exec(code, run_globals)
    raise RuntimeError('''
  File "C:\Users\Harsha\Desktop\git_repos\bert-ner\src\inference_ner_simple_transformers.py", line 27, in <module>
RuntimeError:
        An attempt has been made to start a new process before the
        current process has finished its bootstrapping phase.

        This probably means that you are not using fork to start your
        child processes and you have forgotten to use the proper idiom
        in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce an executable.    model.train_model(train_df)

  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\site-packages\simpletransformers\ner\ner_model.py", line 242, in train_model
    train_dataset = self.load_and_cache_examples(train_data)
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\site-packages\simpletransformers\ner\ner_model.py", line 986, in load_and_cache_examples
INFO:simpletransformers.ner.ner_model: Converting to features started.
    features = convert_examples_to_features(
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\site-packages\simpletransformers\ner\ner_utils.py", line 242, in convert_examples_to_features
Traceback (most recent call last):
    with Pool(process_count) as p:
INFO:simpletransformers.ner.ner_model: Converting to features started.
  File "<string>", line 1, in <module>
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\context.py", line 119, in Pool
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 116, in spawn_main
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    return Pool(processes, initializer, initargs, maxtasksperchild,
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\pool.py", line 212, in __init__
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 125, in _main
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 116, in spawn_main
    self._repopulate_pool()
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\pool.py", line 303, in _repopulate_pool
    prepare(preparation_data)
    exitcode = _main(fd, parent_sentinel)
    return self._repopulate_pool_static(self._ctx, self.Process,
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 236, in prepare
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 125, in _main
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\pool.py", line 326, in _repopulate_pool_static
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    w.start()
    prepare(preparation_data)
INFO:simpletransformers.ner.ner_model: Converting to features started.
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\process.py", line 121, in start
Traceback (most recent call last):
    main_content = runpy.run_path(main_path,
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 236, in prepare
    self._popen = self._Popen(self)
  File "<string>", line 1, in <module>
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\runpy.py", line 265, in run_path
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\context.py", line 326, in _Popen
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 116, in spawn_main
    _fixup_main_from_path(data['init_main_from_path'])
    return _run_module_code(code, init_globals, run_name,
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
    return Popen(process_obj)
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\runpy.py", line 97, in _run_module_code
    exitcode = _main(fd, parent_sentinel)
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\popen_spawn_win32.py", line 45, in __init__
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 125, in _main
    main_content = runpy.run_path(main_path,
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\runpy.py", line 265, in run_path
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\runpy.py", line 87, in _run_code
    prep_data = spawn.get_preparation_data(process_obj._name)
    prepare(preparation_data)
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 154, in get_preparation_data
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 236, in prepare
    return _run_module_code(code, init_globals, run_name,
    exec(code, run_globals)
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\runpy.py", line 97, in _run_module_code
  File "C:\Users\Harsha\Desktop\git_repos\bert-ner\src\inference_ner_simple_transformers.py", line 27, in <module>
    _check_not_importing_main()
    _fixup_main_from_path(data['init_main_from_path'])
    _run_code(code, mod_globals, init_globals,
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 134, in _check_not_importing_main
    model.train_model(train_df)
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\multiprocessing\spawn.py", line 287, in _fixup_main_from_path
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\runpy.py", line 87, in _run_code
  File "C:\Users\Harsha\Anaconda3\envs\simpt\lib\site-packages\simpletransformers\ner\ner_model.py", line 242, in train_model

Конфигурация системы: Windows 10 GTX 1070 Python 3.8.3 transformers 2.1.1 pytorch 1.5.1 conda environment

0

python multiprocessing huggingface-transformers simpletransformers

Источник

user10656943 19 июл '20 в 13:28

1 ответ

Другие вопросы по тегам python multiprocessing huggingface-transformers simpletransformers

user6063706 05 авг '20 в 22:03 2020-08-05 22:03 · Answer 1 · 2020-08-05 22:03

У меня была такая же проблема, и я решил ее, поставив:

model.train_model

Внутри следующего основного оператора if:

if __name__ == '__main__':

1

Источник

user6063706 05 авг '20 в 22:03