scrapyd Exception Ожидаемое имя свойства в двойных кавычках

Question

Когда я выполню

curl http://localhost:6800/schedule.json -d project=default -d spider=spider1

Я иду к scrapyd консоли и вижу исключение:

[-] Unhandled Error
Traceback (most recent call last):
  File "/home/scrapyV1/venv/lib/python2.7/site-packages/twisted/internet/defer.py", line 150, in maybeDeferred
    result = f(*args, **kw)
  File "/home/scrapyV1/venv/lib/python2.7/site-packages/twisted/internet/defer.py", line 1274, in unwindGenerator
    return _inlineCallbacks(None, gen, Deferred())
  File "/home/scrapyV1/venv/lib/python2.7/site-packages/twisted/internet/defer.py", line 1128, in _inlineCallbacks
    result = g.send(result)
  File "/home/scrapyV1/venv/lib/python2.7/site-packages/scrapyd/poller.py", line 23, in poll
    msg = yield maybeDeferred(q.pop)
--- <exception caught here> ---
  File "/home/scrapyV1/venv/lib/python2.7/site-packages/twisted/internet/defer.py", line 150, in maybeDeferred
    result = f(*args, **kw)
  File "/home/scrapyV1/venv/lib/python2.7/site-packages/scrapyd/spiderqueue.py", line 21, in pop
    return self.q.pop()
  File "/home/scrapyV1/venv/lib/python2.7/site-packages/scrapyd/sqlite.py", line 119, in pop
    return self.decode(msg)
  File "/home/scrapyV1/venv/lib/python2.7/site-packages/scrapyd/sqlite.py", line 170, in decode
    return json.loads(text)
  File "/usr/local/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python2.7/json/decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
exceptions.ValueError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)nter code here

После этого я не могу запустить следующий паук, я получаю

{"status": "error", "message": "No JSON object could be decoded", "node_name": "spider-test2"}

Как определить, с чем связана эта ошибка?

Answer 1

Проблема была в sqlite3, установленном в моем Centos6.5. Решение: нужно добавить

self.conn.text_factory = bytes

классам SqliteDict и SqlitePriorityQueue после создания соединения в scrapyd/sqlite.py b/scrapyd/sqlite.py