Как заставить редистого работать на героку?

Question

Как заставить редистого работать на героку?

Мой сайт разработки Django с фоновой обработкой через redis работает локально, но не работает на heroku. Мой файл require.txt содержит 'redis==2.7.2' и 'rq==0.3.2', и я раскручиваю web=1 и worker=1 на heroku. У меня есть дополнение 'redistogo:nano'. Инструкции по проверке того, что это работает на консоли heroku, не работают: нет redis.set() или redis.get(). Поэтому я попробовал код из фоновых задач Запуск Heroku только с 1 веб-динамо и 0 рабочими динамо следующим образом. В settings.py:

import os
import os.path
import socket
import sys
import urlparse
from rq import Queue
from worker import conn

redis_q = Queue(connection=conn)
...

Вот работник.пи:

import os
import redis
from rq import Worker, Queue, Connection
listen = ['high', 'default', 'low']
redis_url = os.getenv('REDISTOGO_URL', 'redis://localhost:6379')
conn = redis.from_url(redis_url)
if __name__ == '__main__':
    with Connection(conn):
        worker = Worker(map(Queue, listen))
        worker.work()

Вот фрагменты представления, выполняющего фоновый вызов:

from upload import *  # where process_upload_file() lives
...
    catchall = {...}
    catchall_string = str(catchall)
    redis_q.enqueue(process_upload_file, catchall_string)
...

И upload.py имеет:

def process_upload_file(catchall_string):
    ...

Почему это не работает?

(Для дополнительной оценки, если кто-то может дать мне лучший способ передать аргументы в виде строки, это будет оценено.)

PROCFILE:

web: gunicorn exim.wsgi -b "0.0.0.0:$PORT"
worker: python worker.py

Вот логи герою с последней попытки после установки ведра S3 и указания на него героку. Не удалось найти ни " http://s3.amazonaws.com/mycompany/sample.csv" (показано здесь), ни " http://mycompany.s3.amazonaws.com/sample.csv".

    0:46:31+00:00 app[worker.1]: [2013-02-09 00:46] INFO: worker: 
    2013-02-09T00:46:44+00:00 heroku[web.1]: Starting process with command `gunicorn myapp.wsgi -b "0.0.0.0:27999"`
    2013-02-09T00:46:45+00:00 app[web.1]: 2013-02-09 00:46:45 [2] [INFO] Starting gunicorn 0.15.0
    2013-02-09T00:46:45+00:00 app[web.1]: 2013-02-09 00:46:45 [2] [INFO] Listening at: http://0.0.0.0:27999 (2)
    2013-02-09T00:46:45+00:00 app[web.1]: 2013-02-09 00:46:45 [2] [INFO] Using worker: sync
    2013-02-09T00:46:45+00:00 app[web.1]: 2013-02-09 00:46:45 [5] [INFO] Booting worker with pid: 5
    2013-02-09T00:46:50+00:00 heroku[web.1]: State changed from starting to up
    2013-02-09T00:47:15+00:00 heroku[router]: at=info method=GET path=/favicon.ico host=chewy-chiclet-3186.herokuapp.com fwd=71.184.228.19 dyno=web.1 queue=0 wait=0ms connect=3ms service=3132ms status=404 bytes=30
    2013-02-09T00:47:15+00:00 heroku[router]: at=info method=GET path=/upl/upload75/ host=chewy-chiclet-3186.herokuapp.com fwd=71.184.228.19 dyno=web.1 queue=0 wait=0ms connect=6ms service=980ms status=200 bytes=958
    2013-02-09T00:47:15+00:00 heroku[router]: at=info method=GET path=/favicon.ico host=chewy-chiclet-3186.herokuapp.com fwd=71.184.228.19 dyno=web.1 queue=0 wait=0ms connect=2ms service=16ms status=404 bytes=30
    2013-02-09T00:47:21+00:00 heroku[router]: at=info method=POST path=/upl/upload75/ host=chewy-chiclet-3186.herokuapp.com fwd=71.184.228.19 dyno=web.1 queue=0 wait=0ms connect=4ms service=244ms status=302 bytes=5
    2013-02-09T00:47:22+00:00 heroku[router]: at=info method=GET path=/upl/upload77/ host=chewy-chiclet-3186.herokuapp.com fwd=71.184.228.19 dyno=web.1 queue=0 wait=0ms connect=2ms service=162ms status=200 bytes=44024
    2013-02-09T00:47:22+00:00 heroku[router]: at=info method=GET path=/static/css/bootstrap.min.css host=chewy-chiclet-3186.herokuapp.com fwd=71.184.228.19 dyno=web.1 queue=0 wait=1ms connect=3ms service=26ms status=200 bytes=95438
    2013-02-09T00:47:22+00:00 heroku[router]: at=info method=GET path=/static/css/style.css host=chewy-chiclet-3186.herokuapp.com fwd=71.184.228.19 dyno=web.1 queue=0 wait=0ms connect=2ms service=15ms status=200 bytes=515
    2013-02-09T00:47:22+00:00 heroku[router]: at=info method=GET path=/static/js/jquery-1.7.2.min.js host=chewy-chiclet-3186.herokuapp.com fwd=71.184.228.19 dyno=web.1 queue=0 wait=0ms connect=3ms service=17ms status=200 bytes=94840
    2013-02-09T00:47:22+00:00 heroku[router]: at=info method=GET path=/static/js/scripts.js host=chewy-chiclet-3186.herokuapp.com fwd=71.184.228.19 dyno=web.1 queue=0 wait=1ms connect=6ms service=5ms status=200 bytes=509
    2013-02-09T00:47:22+00:00 heroku[router]: at=info method=GET path=/static/js/bootstrap.min.js host=chewy-chiclet-3186.herokuapp.com fwd=71.184.228.19 dyno=web.1 queue=0 wait=0ms connect=2ms service=7ms status=200 bytes=22323
    2013-02-09T00:47:22+00:00 heroku[router]: at=info method=GET path=/static/snippets/footer.html host=chewy-chiclet-3186.herokuapp.com fwd=71.184.228.19 dyno=web.1 queue=0 wait=0ms connect=2ms service=16ms status=200 bytes=537
    2013-02-09T00:47:22+00:00 heroku[router]: at=info method=GET path=/static/img/groovepaper.png host=chewy-chiclet-3186.herokuapp.com fwd=71.184.228.19 dyno=web.1 queue=0 wait=0ms connect=1ms service=11ms status=200 bytes=74596
    2013-02-09T00:47:22+00:00 heroku[router]: at=info method=GET path=/static/font/fontawesome-webfont.woff host=chewy-chiclet-3186.herokuapp.com fwd=71.184.228.19 dyno=web.1 queue=0 wait=0ms connect=3ms service=12ms status=200 bytes=41752
    2013-02-09T00:47:22+00:00 heroku[router]: at=info method=GET path=/favicon.ico host=chewy-chiclet-3186.herokuapp.com fwd=71.184.228.19 dyno=web.1 queue=0 wait=0ms connect=1ms service=15ms status=404 bytes=30
    2013-02-09T00:47:37+00:00 heroku[router]: at=info method=POST path=/upl/upload77/ host=chewy-chiclet-3186.herokuapp.com fwd=71.184.228.19 dyno=web.1 queue=0 wait=0ms connect=2ms service=182ms status=200 bytes=98
    2013-02-09T00:47:37+00:00 app[worker.1]: [2013-02-09 00:47] INFO: worker: default: upload.process_upload_file("{'server_file': u'/app/myapp/myapp/uploads/d27/org20130208194721.csv', 'client_file': u'aorg.csv', 'myco_id': 27, 'user_email': u'me.myself@mycompany.com', 'ignore_header_line': True, 'content_type': 'org', 'column_choices': {u'org_name': 0, u'addr1': 1, u'country_code': 8}}") (eb79ed7e-cff8-4071-a0c8-b1d4b1bd834f)
    2013-02-09T00:47:37+00:00 heroku[router]: at=info method=GET path=/favicon.ico host=chewy-chiclet-3186.herokuapp.com fwd=71.184.228.19 dyno=web.1 queue=0 wait=0ms connect=1ms service=9ms status=404 bytes=30
    2013-02-09T00:47:42+00:00 app[worker.1]: [2013-02-09 00:47] ERROR: horse: IOError: [Errno 2] No such file or directory: 'http://s3.amazonaws.com/mycompany/sample.csv'
    2013-02-09T00:47:42+00:00 app[worker.1]:     rv = job.perform()
    2013-02-09T00:47:42+00:00 app[worker.1]:   File "/app/refactor.py", line 72, in read_uploaded_file
    2013-02-09T00:47:42+00:00 app[worker.1]:     allrows = csv.reader(open(filepath, 'rU'), dialect='excel')
    2013-02-09T00:47:42+00:00 app[worker.1]: IOError: [Errno 2] No such file or directory: 'http://s3.amazonaws.com/mycompany/sample.csv'
    2013-02-09T00:47:42+00:00 app[worker.1]:   File "/app/refactor.py", line 41, in read_csv
    2013-02-09T00:47:42+00:00 app[worker.1]: [2013-02-09 00:47] WARNING: horse: Moving job to failed queue.
    2013-02-09T00:47:42+00:00 app[worker.1]: Traceback (most recent call last):
    2013-02-09T00:47:42+00:00 app[worker.1]:     output, nrows, error = acceptable_upload_filetypes[ext](filepath, **kwargs)
    2013-02-09T00:47:42+00:00 app[worker.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/rq/worker.py", line 393, in perform_job
    2013-02-09T00:47:42+00:00 app[worker.1]: 
    2013-02-09T00:47:42+00:00 app[worker.1]: [2013-02-09 00:47] DEBUG: horse: Invoking exception handler <bound method Worker.move_to_failed_queue of <rq.worker.Worker object at 0x12cb310>>
    2013-02-09T00:47:42+00:00 app[worker.1]:   File "/app/.heroku/python/lib/python2.7/site-packages/rq/job.py", line 328, in perform
    2013-02-09T00:47:42+00:00 app[worker.1]:     self._result = self.func(*self.args, **self.kwargs)
    2013-02-09T00:47:42+00:00 app[worker.1]:   File "/app/upload.py", line 248, in process_upload_file
    2013-02-09T00:47:42+00:00 app[worker.1]:     all_rows, nrows, error = read_uploaded_file(server_file, maxrows=maxrows)
    2013-02-09T00:47:42+00:00 app[worker.1]: [2013-02-09 00:47] INFO: worker: 
    2013-02-09T00:47:42+00:00 app[worker.1]: [2013-02-09 00:47] INFO: worker: *** Listening on high, default, low...
    (venv)me@host:~/myapp$

1

django python-2.7 heroku redis redistogo

Источник

user2044195 05 фев '13 в 18:57

0 ответов

Другие вопросы по тегам django python-2.7 heroku redis redistogo