Может ли Json.NET сериализовать / десериализовать в / из потока?
Я слышал, что Json.NET быстрее, чем DataContractJsonSerializer, и хотел попробовать...
Но я не смог найти никаких методов на JsonConvert, которые принимают поток, а не строку.
Например, для десериализации файла, содержащего JSON на WinPhone, я использую следующий код, чтобы прочитать содержимое файла в строку, а затем десериализовать в JSON. В моем (очень специальном) тестировании это примерно в 4 раза медленнее, чем при использовании DataContractJsonSerializer для десериализации прямо из потока...
// DCJS
DataContractJsonSerializer dc = new DataContractJsonSerializer(typeof(Constants));
Constants constants = (Constants)dc.ReadObject(stream);
// JSON.NET
string json = new StreamReader(stream).ReadToEnd();
Constants constants = JsonConvert.DeserializeObject<Constants>(json);
Я делаю это неправильно?
Спасибо,
Омри.
7 ответов
Решение
ОБНОВЛЕНИЕ: Это больше не работает в текущей версии, правильный ответ см. Ниже (нет необходимости голосовать, это верно для более старых версий).
Использовать JsonTextReader класс с StreamReader или используйте JsonSerializer перегрузка, которая занимает StreamReader непосредственно:
var serializer = new JsonSerializer();
serializer.Deserialize(streamReader);
Текущая версия Json.net не позволяет использовать принятый код ответа. Текущая альтернатива:
public static object DeserializeFromStream(Stream stream)
{
var serializer = new JsonSerializer();
using (var sr = new StreamReader(stream))
using (var jsonTextReader = new JsonTextReader(sr))
{
return serializer.Deserialize(jsonTextReader);
}
}
Документация: десериализация JSON из файлового потока
public static void Serialize(object value, Stream s)
{
using (StreamWriter writer = new StreamWriter(s))
using (JsonTextWriter jsonWriter = new JsonTextWriter(writer))
{
JsonSerializer ser = new JsonSerializer();
ser.Serialize(jsonWriter, value);
jsonWriter.Flush();
}
}
public static T Deserialize<T>(Stream s)
{
using (StreamReader reader = new StreamReader(s))
using (JsonTextReader jsonReader = new JsonTextReader(reader))
{
JsonSerializer ser = new JsonSerializer();
return ser.Deserialize<T>(jsonReader);
}
}
Я написал класс расширения, чтобы помочь мне десериализовать из источников JSON (строка, поток, файл).
public static class JsonHelpers
{
public static T CreateFromJsonStream<T>(this Stream stream)
{
JsonSerializer serializer = new JsonSerializer();
T data;
using (StreamReader streamReader = new StreamReader(stream))
{
data = (T)serializer.Deserialize(streamReader, typeof(T));
}
return data;
}
public static T CreateFromJsonString<T>(this String json)
{
T data;
using (MemoryStream stream = new MemoryStream(System.Text.Encoding.Default.GetBytes(json)))
{
data = CreateFromJsonStream<T>(stream);
}
return data;
}
public static T CreateFromJsonFile<T>(this String fileName)
{
T data;
using (FileStream fileStream = new FileStream(fileName, FileMode.Open))
{
data = CreateFromJsonStream<T>(fileStream);
}
return data;
}
}
Десериализацию теперь так же просто, как написать:
MyType obj1 = aStream.CreateFromJsonStream<MyType>();
MyType obj2 = "{\"key\":\"value\"}".CreateFromJsonString<MyType>();
MyType obj3 = "data.json".CreateFromJsonFile<MyType>();
Надеюсь, это поможет кому-то еще.
Я пришел к этому вопросу, ища способ для потоковой передачи открытого списка объектов на System.IO.Stream и прочитайте их с другого конца, не буферизуя весь список перед отправкой. (В частности, я передаю постоянные объекты из MongoDB через Web API.)
@Paul Tyng и @Rivers отлично ответили на первоначальный вопрос, и я использовал их ответы, чтобы создать доказательство концепции моей проблемы. Я решил опубликовать свое тестовое консольное приложение здесь на случай, если кто-то еще столкнется с той же проблемой.
using System;
using System.Diagnostics;
using System.IO;
using System.IO.Pipes;
using System.Threading;
using System.Threading.Tasks;
using Newtonsoft.Json;
namespace TestJsonStream {
class Program {
static void Main(string[] args) {
using(var writeStream = new AnonymousPipeServerStream(PipeDirection.Out, HandleInheritability.None)) {
string pipeHandle = writeStream.GetClientHandleAsString();
var writeTask = Task.Run(() => {
using(var sw = new StreamWriter(writeStream))
using(var writer = new JsonTextWriter(sw)) {
var ser = new JsonSerializer();
writer.WriteStartArray();
for(int i = 0; i < 25; i++) {
ser.Serialize(writer, new DataItem { Item = i });
writer.Flush();
Thread.Sleep(500);
}
writer.WriteEnd();
writer.Flush();
}
});
var readTask = Task.Run(() => {
var sw = new Stopwatch();
sw.Start();
using(var readStream = new AnonymousPipeClientStream(pipeHandle))
using(var sr = new StreamReader(readStream))
using(var reader = new JsonTextReader(sr)) {
var ser = new JsonSerializer();
if(!reader.Read() || reader.TokenType != JsonToken.StartArray) {
throw new Exception("Expected start of array");
}
while(reader.Read()) {
if(reader.TokenType == JsonToken.EndArray) break;
var item = ser.Deserialize<DataItem>(reader);
Console.WriteLine("[{0}] Received item: {1}", sw.Elapsed, item);
}
}
});
Task.WaitAll(writeTask, readTask);
writeStream.DisposeLocalCopyOfClientHandle();
}
}
class DataItem {
public int Item { get; set; }
public override string ToString() {
return string.Format("{{ Item = {0} }}", Item);
}
}
}
}
Обратите внимание, что вы можете получить исключение, когда AnonymousPipeServerStream Я проигнорировал это, поскольку это не имеет отношения к рассматриваемой проблеме.
Другой вариант, если вы читаете в Json, — использовать DeserializeObject в классе JsonConvert:
using (StreamReader streamReader = new StreamReader("example.json"))
{
string json = streamReader.ReadToEnd();
ObjectType object = JsonConvert.DeserializeObject<ObjectType>(json);
}
другой вариант, который удобен, когда вам не хватает памяти, - это периодически сбрасывать
/// <summary>serialize the value in the stream.</summary>
/// <typeparam name="T">the type to serialize</typeparam>
/// <param name="stream">The stream.</param>
/// <param name="value">The value.</param>
/// <param name="settings">The json settings to use.</param>
/// <param name="bufferSize"></param>
/// <param name="leaveOpen"></param>
public static void JsonSerialize<T>(this Stream stream,[DisallowNull] T value, [DisallowNull] JsonSerializerSettings settings, int bufferSize=1024, bool leaveOpen=false)
{
using (var writer = new StreamWriter(stream,encoding: System.Text.Encoding.UTF32,bufferSize,leaveOpen))
using (var jsonWriter = new JsonTextWriter(writer))
{
var ser = JsonSerializer.Create( settings );
ser.Serialize(jsonWriter, value);
jsonWriter.Flush();
}
}
/// <summary>serialize the value in the stream asynchronously.</summary>
/// <typeparam name="T"></typeparam>
/// <param name="stream">The stream.</param>
/// <param name="value">The value.</param>
/// <param name="settings">The settings.</param>
/// <param name="bufferSize">The buffer size, in bytes, set -1 to not flush till done</param>
/// <param name="leaveOpen"> true to leave the stream open </param>
/// <param name="cancellationToken">Propagates notification that operations should be canceled.</param>
public static Task JsonSerializeAsync<T>(this Stream stream,[DisallowNull] T value, [DisallowNull] JsonSerializerSettings settings, int bufferSize=1024, bool leaveOpen=false, CancellationToken cancellationToken=default)
{
using (var writer = new StreamWriter(stream,encoding: System.Text.Encoding.UTF32,bufferSize: bufferSize,leaveOpen: leaveOpen))
using (var jsonWriter = new JsonTextWriter(writer))
{
var ser = JsonSerializer.Create( settings );
ser.Serialize(jsonWriter, value);
return jsonWriter.Flush();
}
//jsonWriter.FlushAsnc with my version gives an error on the stream
return Task.CompletedTask;
}
Вы можете протестировать/использовать его так:
[TestMethod()]
public void WriteFileIntoJsonTest()
{
var file = new FileInfo(Path.GetTempFileName());
try
{
var list = new HashSet<Guid>();
for (int i = 0; i < 100; i++)
{
list.Add(Guid.NewGuid());
}
file.JsonSerialize(list);
var sr = file.IsValidJson<List<Guid>>(out var result);
Assert.IsTrue(sr);
Assert.AreEqual<int>(list.Count, result.Count);
foreach (var item in result)
{
Assert.IsFalse(list.Add(item), $"The GUID {item} should already exist in the hash set");
}
}
finally
{
file.Refresh();
file.Delete();
}
}
вам нужно будет создать методы расширения, вот весь набор: public static class JsonStreamReaderExt { static JsonSerializerSettings _settings; статический JsonStreamReaderExt() { _settings = JsonConvert.DefaultSettings?.Invoke() ?? новые настройки JsonSerializer(); _settings.ConstructorHandling = ConstructorHandling.AllowNonPublicDefaultConstructor; _settings.DateTimeZoneHandling = DateTimeZoneHandling.Utc; _settings.DateFormatHandling = DateFormatHandling.IsoDateFormat ; }
/// <summary>
/// serialize the value in the stream.
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="stream">The stream.</param>
/// <param name="value">The value.</param>
public static void JsonSerialize<T>(this Stream stream,[DisallowNull] T value)
{
stream.JsonSerialize(value,_settings);
}
/// <summary>
/// serialize the value in the file .
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="file">The file.</param>
/// <param name="value">The value.</param>
public static void JsonSerialize<T>(this FileInfo file,[DisallowNull] T value)
{
if (string.IsNullOrEmpty(file.DirectoryName)==true && Directory.Exists(file.DirectoryName) == false)
{
Directory.CreateDirectory(file.FullName);
}
using var s = file.OpenWrite();
s.JsonSerialize(value, _settings);
file.Refresh();
}
/// <summary>
/// serialize the value in the file .
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="file">The file.</param>
/// <param name="value">The value.</param>
/// <param name="settings">the json settings to use</param>
public static void JsonSerialize<T>(this FileInfo file, [DisallowNull] T value, [DisallowNull] JsonSerializerSettings settings)
{
if (string.IsNullOrEmpty(file.DirectoryName) == true && Directory.Exists(file.DirectoryName) == false)
{
Directory.CreateDirectory(file.FullName);
}
using var s = file.OpenWrite();
s.JsonSerialize(value, settings);
file.Refresh();
}
/// <summary>
/// serialize the value in the file .
/// </summary>
/// <remarks>File will be refreshed to contain the new meta data</remarks>
/// <typeparam name="T">the type to serialize</typeparam>
/// <param name="file">The file.</param>
/// <param name="value">The value.</param>
/// <param name="cancellationToken">Propagates notification that operations should be canceled.</param>
public static async Task JsonSerializeAsync<T>(this FileInfo file, [DisallowNull] T value, CancellationToken cancellationToken = default)
{
if (string.IsNullOrEmpty(file.DirectoryName) == true && Directory.Exists(file.DirectoryName) == false)
{
Directory.CreateDirectory(file.FullName);
}
using (var stream = file.OpenWrite())
{
await stream.JsonSerializeAsync(value, _settings,bufferSize:1024,leaveOpen:false, cancellationToken).ConfigureAwait(false);
}
file.Refresh();
}
/// <summary>
/// serialize the value in the file .
/// </summary>
/// <remarks>File will be refreshed to contain the new meta data</remarks>
/// <typeparam name="T">the type to serialize</typeparam>
/// <param name="file">The file to create or overwrite.</param>
/// <param name="value">The value.</param>
/// <param name="settings">the json settings to use</param>
/// <param name="cancellationToken">Propagates notification that operations should be canceled.</param>
public static async Task JsonSerializeAsync<T>(this FileInfo file, [DisallowNull] T value, [DisallowNull] JsonSerializerSettings settings, CancellationToken cancellationToken=default)
{
if (string.IsNullOrEmpty(file.DirectoryName) == true && Directory.Exists(file.DirectoryName) == false)
{
Directory.CreateDirectory(file.FullName);
}
using (var stream = file.OpenWrite())
{
await stream.JsonSerializeAsync(value, settings,bufferSize:1024,leaveOpen:false, cancellationToken).ConfigureAwait(false);
}
file.Refresh();
}
/// <summary>serialize the value in the stream.</summary>
/// <typeparam name="T">the type to serialize</typeparam>
/// <param name="stream">The stream.</param>
/// <param name="value">The value.</param>
/// <param name="settings">The json settings to use.</param>
/// <param name="bufferSize">The buffer size, in bytes, set -1 to not flush till done</param>
/// <param name="leaveOpen"> true to leave the stream open </param>
public static void JsonSerialize<T>(this Stream stream,[DisallowNull] T value, [DisallowNull] JsonSerializerSettings settings, int bufferSize=1024, bool leaveOpen=false)
{
using (var writer = new StreamWriter(stream,encoding: System.Text.Encoding.UTF32,bufferSize,leaveOpen))
using (var jsonWriter = new JsonTextWriter(writer))
{
var ser = JsonSerializer.Create( settings );
ser.Serialize(jsonWriter, value);
jsonWriter.Flush();
}
}
/// <summary>serialize the value in the stream asynchronously.</summary>
/// <typeparam name="T"></typeparam>
/// <param name="stream">The stream.</param>
/// <param name="value">The value.</param>
/// <param name="settings">The settings.</param>
/// <param name="bufferSize">The buffer size, in bytes, set -1 to not flush till done</param>
/// <param name="leaveOpen"> true to leave the stream open </param>
/// <param name="cancellationToken">Propagates notification that operations should be canceled.</param>
public static Task JsonSerializeAsync<T>(this Stream stream,[DisallowNull] T value, [DisallowNull] JsonSerializerSettings settings, int bufferSize=1024, bool leaveOpen=false, CancellationToken cancellationToken=default)
{
using (var writer = new StreamWriter(stream,encoding: System.Text.Encoding.UTF32,bufferSize: bufferSize,leaveOpen: leaveOpen))
using (var jsonWriter = new JsonTextWriter(writer))
{
var ser = JsonSerializer.Create( settings );
ser.Serialize(jsonWriter, value);
jsonWriter.Flush();
}
return Task.CompletedTask;
}
/// <summary>
/// Determines whether [is valid json] [the specified result].
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="stream">The stream.</param>
/// <param name="result">The result.</param>
/// <returns><c>true</c> if [is valid json] [the specified result]; otherwise, <c>false</c>.</returns>
public static bool IsValidJson<T>(this Stream stream, [NotNullWhen(true)] out T? result)
{
if (stream is null)
{
throw new ArgumentNullException(nameof(stream));
}
if (stream.Position != 0)
{
stream.Seek(0, SeekOrigin.Begin);
}
JsonSerializerSettings settings = (JsonConvert.DefaultSettings?.Invoke()) ?? new JsonSerializerSettings() { DateTimeZoneHandling = DateTimeZoneHandling.Utc, DateFormatHandling = DateFormatHandling.IsoDateFormat };
settings.ConstructorHandling = ConstructorHandling.AllowNonPublicDefaultConstructor;
using (var reader = new StreamReader(stream))
using (var jsonReader = new JsonTextReader(reader))
{
var ser = JsonSerializer.Create(settings);
try
{
result = ser.Deserialize<T>(jsonReader);
}
catch { result = default; }
}
return result is not null;
}
/// <summary>
/// Determines whether [is valid json] [the specified settings].
/// </summary>
/// <typeparam name="T"></typeparam>
/// <param name="stream">The stream.</param>
/// <param name="settings">The settings.</param>
/// <param name="result">The result.</param>
/// <returns><c>true</c> if [is valid json] [the specified settings]; otherwise, <c>false</c>.</returns>
public static bool IsValidJson<T>(this Stream stream, JsonSerializerSettings settings, [NotNullWhen(true)] out T? result)
{
if (stream is null)
{
throw new ArgumentNullException(nameof(stream));
}
if (settings is null)
{
throw new ArgumentNullException(nameof(settings));
}
if (stream.Position != 0)
{
stream.Seek(0, SeekOrigin.Begin);
}
using (var reader = new StreamReader(stream))
using (var jsonReader = new JsonTextReader(reader))
{
var ser = JsonSerializer.Create(settings);
try
{
result = ser.Deserialize<T>(jsonReader);
}
catch { result = default; }
}
return result is not null;
}
/// <summary>
/// Determines whether file contains valid json using the specified settings and reads it into the output.
/// </summary>
/// <typeparam name="T">Type to convert into</typeparam>
/// <param name="file">The file.</param>
/// <param name="settings">The settings.</param>
/// <param name="result">The result.</param>
/// <returns><c>true</c> if [is valid json] [the specified settings]; otherwise, <c>false</c>.</returns>
/// <exception cref="System.ArgumentNullException">file</exception>
/// <exception cref="System.ArgumentNullException"></exception>
/// <exception cref="System.ArgumentNullException">settings</exception>
/// <exception cref="System.IO.FileNotFoundException">File could not be accessed</exception>
public static bool IsValidJson<T>(this FileInfo file, JsonSerializerSettings settings, [NotNullWhen(true)] out T? result)
{
if (file is null)
{
throw new ArgumentNullException(nameof(file));
}
if (File.Exists(file.FullName) == false)
{
throw new FileNotFoundException("File could not be accessed",fileName: file.FullName);
}
using var stream = file.OpenRead();
if (stream is null)
{
throw new ArgumentNullException(message:"Could not open the file and access the underlying file stream",paramName: nameof(file));
}
if (settings is null)
{
throw new ArgumentNullException(nameof(settings));
}
using (var reader = new StreamReader(stream))
using (var jsonReader = new JsonTextReader(reader))
{
var ser = JsonSerializer.Create(settings);
try
{
result = ser.Deserialize<T>(jsonReader);
}
catch { result = default; }
}
return result is not null;
}
/// <summary>
/// Determines whether file contains valid json using the specified settings and reads it into the output.
/// </summary>
/// <typeparam name="T">Type to convert into</typeparam>
/// <param name="file">The file.</param>
/// <param name="result">The result.</param>
/// <returns><c>true</c> if [is valid json] [the specified result]; otherwise, <c>false</c>.</returns>
/// <exception cref="System.ArgumentNullException">file</exception>
/// <exception cref="System.IO.FileNotFoundException">File could not be accessed</exception>
public static bool IsValidJson<T>(this FileInfo file, [NotNullWhen(true)] out T? result)
{
if (file is null)
{
throw new ArgumentNullException(nameof(file));
}
if (File.Exists(file.FullName) == false)
{
throw new FileNotFoundException("File could not be accessed",fileName: file.FullName);
}
JsonSerializerSettings settings =( JsonConvert.DefaultSettings?.Invoke()) ?? new JsonSerializerSettings() { DateTimeZoneHandling= DateTimeZoneHandling.Utc, DateFormatHandling= DateFormatHandling.IsoDateFormat };
settings.ConstructorHandling = ConstructorHandling.AllowNonPublicDefaultConstructor;
return file.IsValidJson<T>(settings,out result);
}
}