Custom JSON serialization with Azure Cosmos DB SDK

Background

Azure Cosmos DB SDK 3+ for SQL API replaced Azure DocumentDB SDK a couple of years back. DocumentDB used Newtonsoft Json.NET for its serialization. However, with the growth of .NET Core and the introduction of shiny new System.Text.Json, the team wanted to reduce exposure to Newtonsoft. So the idea was in future, with CosmosDB SDK 4, System.Text.Json would replace Json.NET.

The unfortunate side-effect of this was that we do not have an easy way to supply custom JSON serializer settings. The SDK allows us to update a limited set of JSON settings such as IgnoreNullValues, Intend and PropertyNamePolicy through CosmosSerializationOptions. However, in the real world scenario, this is not always enough.

The CosmosSerializer class from the SDK is abstractand all its implementation, such as CosmosJsonDotNetSerializer, are internal sealed. I also raised an issue with the developer team to fix this limitation.

CUSTOM JSON Serializer for Cosmos

Fix for this issue is easy, albeit ugly. We can create our own CosmosJsonDotNetSerializer inspired from Cosmos DB SDK as below:

	using System;
	using System.IO;
	using System.Text;
	using Microsoft.Azure.Cosmos;
	using Newtonsoft.Json;
	/// <summary>
	/// Azure Cosmos DB does not expose a default implementation of CosmosSerializer that is required to set the custom JSON serializer settings.
	/// To fix this, we have to create our own implementation inspired internal implementation from SDK library.
	/// <remarks>
	/// See: https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos/src/Serializer/CosmosJsonDotNetSerializer.cs
	/// </remarks>
	/// </summary>
	public sealed class CosmosJsonDotNetSerializer : CosmosSerializer
	{
	private static readonly Encoding DefaultEncoding = new UTF8Encoding(false, true);
	private readonly JsonSerializerSettings _serializerSettings;
	/// <summary>
	/// Create a serializer that uses the JSON.net serializer
	/// </summary>
	public CosmosJsonDotNetSerializer(JsonSerializerSettings jsonSerializerSettings)
	{
	_serializerSettings = jsonSerializerSettings ??
	throw new ArgumentNullException(nameof(jsonSerializerSettings));
	}
	/// <summary>
	/// Convert a Stream to the passed in type.
	/// </summary>
	/// <typeparam name="T">The type of object that should be deserialized</typeparam>
	/// <param name="stream">An open stream that is readable that contains JSON</param>
	/// <returns>The object representing the deserialized stream</returns>
	public override T FromStream<T>(Stream stream)
	{
	using (stream)
	{
	if (typeof(Stream).IsAssignableFrom(typeof(T)))
	{
	return (T)(object)stream;
	}
	using (var sr = new StreamReader(stream))
	{
	using (var jsonTextReader = new JsonTextReader(sr))
	{
	var jsonSerializer = GetSerializer();
	return jsonSerializer.Deserialize<T>(jsonTextReader);
	}
	}
	}
	}
	/// <summary>
	/// Converts an object to a open readable stream
	/// </summary>
	/// <typeparam name="T">The type of object being serialized</typeparam>
	/// <param name="input">The object to be serialized</param>
	/// <returns>An open readable stream containing the JSON of the serialized object</returns>
	public override Stream ToStream<T>(T input)
	{
	var streamPayload = new MemoryStream();
	using (var streamWriter = new StreamWriter(streamPayload, encoding: DefaultEncoding, bufferSize: 1024, leaveOpen: true))
	{
	using (JsonWriter writer = new JsonTextWriter(streamWriter))
	{
	writer.Formatting = Formatting.None;
	var jsonSerializer = GetSerializer();
	jsonSerializer.Serialize(writer, input);
	writer.Flush();
	streamWriter.Flush();
	}
	}
	streamPayload.Position = 0;
	return streamPayload;
	}
	/// <summary>
	/// JsonSerializer has hit a race conditions with custom settings that cause null reference exception.
	/// To avoid the race condition a new JsonSerializer is created for each call
	/// </summary>
	private JsonSerializer GetSerializer()
	{
	return JsonSerializer.Create(_serializerSettings);
	}
	}

view raw CosmosJsonDotNetSerializer.cs hosted with ❤ by GitHub

As you can see in the above code CosmosJsonDotNetSerializer takes JsonSerializerSettings as a parameter in its Constructor.

This would allow us to create CosmosClient using CosmosJsonDotNetSerializer and pass our own JsonSerializerSettings as shown in the code snippet below:

	var cosmosClient = new CosmosClient("<cosmosDBConnectionString>",
	new CosmosClientOptions
	{
	Serializer = new CosmosJsonDotNetSerializer(new JsonSerializerSettings
	{
	// Update your JSON Serializer Settings here.
	TypeNameHandling = TypeNameHandling.Auto,
	ReferenceLoopHandling = ReferenceLoopHandling.Error,
	PreserveReferencesHandling = PreserveReferencesHandling.None,
	ConstructorHandling = ConstructorHandling.AllowNonPublicDefaultConstructor
	Converters = new JsonConverter[]
	{
	new StringEnumConverter()
	}
	})
	});

view raw CosmosClient.cs hosted with ❤ by GitHub

BUT WAIT… THERE IS MORE

I raised a question to the Cosmos DB team sometime back if there is a way to use replace Json.NET with System.Text.Json. Mark Brown, from the Cosmos DB team, responded that we can easily do that by extending CosmosSerializer like we did above.

@AzureCosmosDB trying to System.Text.Json instead of Newtonsoft with https://t.co/TkeBJwYYES.Cosmos SDK v3+
But it doesn't seem to support it. Am I missing something?

— Ankit Vijay (@vijayankit) April 6, 2021

Here is the System.Text.Json implementation of CosmosSerializer. The implementation is inspired by Azure Cosmos Samples.

	using System.IO;
	using System.Text.Json;
	using Azure.Core.Serialization;
	using Microsoft.Azure.Cosmos;
	/// <remarks>
	// See: https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos.Samples/Usage/SystemTextJson/CosmosSystemTextJsonSerializer.cs
	/// </remarks>
	public sealed class CosmosSystemTextJsonSerializer : CosmosSerializer
	{
	private readonly JsonObjectSerializer _systemTextJsonSerializer;
	public CosmosSystemTextJsonSerializer(JsonSerializerOptions jsonSerializerOptions)
	{
	_systemTextJsonSerializer = new JsonObjectSerializer(jsonSerializerOptions);
	}
	public override T FromStream<T>(Stream stream)
	{
	if (stream.CanSeek && stream.Length == 0)
	{
	return default;
	}
	if (typeof(Stream).IsAssignableFrom(typeof(T)))
	{
	return (T) (object) stream;
	}
	using (stream)
	{
	return (T) _systemTextJsonSerializer.Deserialize(stream, typeof(T), default);
	}
	}
	public override Stream ToStream<T>(T input)
	{
	var streamPayload = new MemoryStream();
	_systemTextJsonSerializer.Serialize(streamPayload, input, typeof(T), default);
	streamPayload.Position = 0;
	return streamPayload;
	}
	}

view raw CosmosSystemTextJsonSerializer.cs hosted with ❤ by GitHub

We can now use the serializer to create CosmosClient as below

	var cosmosClient = new CosmosClient("<cosmosDBConnectionString>",
	new CosmosClientOptions
	{
	Serializer = new CosmosSystemTextJsonSerializer(new JsonSerializerOptions
	{
	// Update your JSON Serializer options here.
	PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
	Converters =
	{
	new JsonStringEnumConverter()
	},
	IgnoreNullValues = true,
	IgnoreReadOnlyFields = true
	})
	});

view raw CosmosClient.cs hosted with ❤ by GitHub

Wrapping Up

This post explains a workaround to create custom serializer settings using Json.NET and System.Text.Json when working with Cosmos DB. Hopefully, this workaround is short-lived and with the release of v4 we get this option out of the box.

Feature Photo by Greg Rakozy on Unsplash