JSON, прошедший через ввод html-формы, не декодирует
Я бы предпочел не обрабатывать все данные дважды для моих пользователей, поэтому я пытаюсь отправить данные в виде скрытой формы ввода, закодированной как JSON, чтобы сгенерировать экспорт CSV в следующую функцию:
<h3> <form action="/c/flex_csv/" method="post">
<input type="hidden" name="data" value='%s'>
<input type="hidden" name="questions" value='%s'>
<input type="submit" name="submit" value="Download a CSV" />
</form></h3>
И данные в источнике HTML выглядит хорошо. У меня было много проблем с кавычками и символами Unicode, поэтому я сейчас вручную перекодирую эти проблемные символы:
['data' - это список словарей]
def safe_listdict(data):
# makes a list of dict that will pass through HTML and CSV without character encoding problems or quotes breaking form input fields
import unicodedata
safe_data = []
for x in data:
line = {}
for k,v in x.items():
if type(v) == str:
line[k] = v.replace("'",''').replace('"','"').encode('ascii','xmlcharrefreplace')
elif type(v) == unicode:
# first normallizes the unicode, then converts to ascii, replacing xml as needed
line[k] = unicodedata.normalize('NFKD',v.replace("'",''').replace('"','"')).encode('ascii','xmlcharrefreplace')
else:
line[k] = str(v)
safe_data.append(line)
return safe_data
import json
safe_data = json.dumps(safe_listdict(data))
Вот как выглядит html-источник:
<input type="hidden" name="data" value='[{"q0": "65604", "q3": "HOW VAP HAD HELP ME", "q2": "Before three months back i had nothing like knowledge in my mind about HIV/AID prevention, but now i had something in my mind. I had teach my three friends about it and when the had been reped where to visit before 72 hours. And after that i had also learned how to use ordinary towels which i didn't know how to use before. And how to maintain proper Hygine. Through VAP program i had know how to relate with other people like my parents, friends and boys in school. The last one is that i had now high parepresure, how to relate with my teachers and other pupils in school. For now i had teach over twenty girls in our community.", "q5": "HIV/AIDS AND PROPER HYGINE", "q4": "VIJANA AMANI PAMOJA", "q7": "NAIROBI", "q6": "KENYA", "q9": "Less than 1 month ago", "q8": "MATHARE 4B", "q15": "", "q14": "0720374037", "q16": "0735775820 lizokin2000@gmail.com", "q11": "14", "q10": "Female", "q13": "The right people", "q12": "Heard about it happening", "q19": "Knowledge"}, ...more dicts of same format...]
Но когда я декодирую JSON в моей следующей функции в Python, я всегда получаю ошибки при разборе:
ERROR:cherrypy.error.18520784:[07/Jan/2014:19:18:18] HTTP Traceback (most recent call last):
File "/home/djotjog/webapps/cp2/power_drill_down_flex.py", line 620, in flex_csv
data = json.loads(data)
File "/usr/local/lib/python2.7/json/__init__.py", line 338, in loads
return _default_decoder.decode(s)
File "/usr/local/lib/python2.7/json/decoder.py", line 365, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/local/lib/python2.7/json/decoder.py", line 381, in raw_decode
obj, end = self.scan_once(s, idx)
**ValueError: Expecting ',' delimiter: line 1 column 3108 (char 3107)**
Данные выглядят так в журнале ошибок, ДО ТОГО, как я запускаю json.loads(data), чтобы преобразовать его из json обратно в список словарей python. Я не вижу ничего явно неправильного...
DEBUG:root:[{"q0": "65784", "q3": "Getting students to contribute to the environment", "q2": "We went to the schools to do education serious on the environment so they are more aware. We had evaluation and the student said they are more aware of what to do with their waste and on environmental issues. They said they are more willing to contribute to segregating their waste and stopping environmental issues like illegal logging. We suggested things like informing + warning people about the implication of what they are doing. They said they would do it", "q5": "how students were encouraged to take part in envir", "q4": "High school students", "q7": "Bohol, Pilar", "q6": "Philippines", "q9": "1-2 months ago", "q8": "Classrooms in Virgen del Pilar Academy", "q15": "Yes", "q14": "09156646213", "q17": "None", "q16": "09175442613", "q23": "70", "q10": "Male", "q13": "The right people", "q12": "Helped make it happen", "q11": "20", "q19": "Food and shelter, Knowledge, Self-esteem", "q18": "Inspired"}, ...more dictionaries...]
Примечание. Цель - дать пользователю возможность сохранять данные с помощью кнопки экспорта в CSV. Я должен отправить его на другую страницу, чтобы функции python могли организовать его в первую очередь.