Running an ASP.NET Core application against multiple DB providers – Part 2

This is the second and last post in the series: Running an ASP.NET Core application against Azure SQL and Amazon Aurora database provider. In part 1 of this series, I talked about how we configured our ASP.NET application to run against both the DB providers. In this part, I will talk about how we set up our integration tests.

As mentioned in the previous post, a little disclaimer first:

Disclaimer: This entire exercise was a POC. The actual implementation turned out to be a lot different due to the organization’s Governance model, regional limitations, and cross-cutting concerns such as authentication, logging, build and deployment pipeline, etc.

Goal

We had set up the following goals for our integration tests:

• Minimum change to existing code base since we already had integration tests running against SQL Server before introducing the MySQL (Amazon Aurora) flavor.
• Maximum test code reuse to avoid waste and ensure we have the same tests running against both the DB providers to avoid any side effect.
• The integration tests should run against both the DB providers to ensure that any new change introduced by the developer in one provider does not break the other.
• There should still be some separation between two test suits so that they can be run independently. For example, we needed to have a separate build pipeline for each DB provider. Each build pipeline would run the integration tests for the respective DB provider.

I had raised this question on StackOverflow to get the answers from the community. And I did receive one good answer which ticked most of the requirement boxes. However, the solution fell a little short of solving our problem completely. Below, I have explained all the different approaches we considered.

Test Set up

Our integration tests are set up using Xunit. We use Microsoft.AspNetCore.TestHost to create the TestServer and TestClient. Here is the code snippet of our setup before the introduction of new DB provider (MySQL):

	public class TestStartup : IStartup
	{
	public IServiceProvider ConfigureServices(IServiceCollection services)
	{
	var configuration = new ConfigurationBuilder()
	.SetBasePath(Directory.GetCurrentDirectory())
	.AddJsonFile("appsettings.json", false)
	.AddEnvironmentVariables()
	.Build();
	services.AddMvc()
	.SetCompatibilityVersion(version: CompatibilityVersion.Version_2_2);
	// Code to add required services based on configuration
	return services.BuildServiceProvider();
	}
	public void Configure(IApplicationBuilder app)
	{
	app.UseMvc();
	// Code to configure test Startup
	}
	}

view raw 1_TestStartup.cs hosted with ❤ by GitHub

	public class TestServerFixture
	{
	public TestServerFixture()
	{
	var builder = new WebHostBuilder().ConfigureServices(services =>
	{
	services.AddSingleton<IStartup>(new TestStartup());
	});
	var server = new TestServer(builder);
	Client = server.CreateClient();
	}
	public HttpClient Client { get; private set; }
	}

view raw 2_TestServerFixture.cs hosted with ❤ by GitHub

	public class MyTest : IClassFixture<TestServerFixture>
	{
	private readonly TestServerFixture _fixture;
	public MyTest(TestServerFixture fixture)
	{
	_fixture = fixture;
	}
	[Fact]
	public async Task ShouldValidateTheGetApiResponse()
	{
	var result = await _fixture.Client.GetAsync("SOME_API_URL");
	// Code to Assert Result
	}
	}

view raw 3_MyTest.cs hosted with ❤ by GitHub

Approach 1

The first approach that we considered came from @Nkosi from StackOverflow. The idea was to use Xunit Theory attribute to pass on DB provider-specific settings to the test method. Then, use the setting to create a separate TestClient and TestServer for each setting.

	public class TestStartup : IStartup {
	private readonly string settings;
	public TestStartup(string settings) {
	this.settings = settings;
	}
	public void ConfigureServices(IServiceCollection services) {
	var configuration = new ConfigurationBuilder()
	.SetBasePath(Directory.GetCurrentDirectory())
	.AddJsonFile(settings, false) // Load appsettings.azure.json or appsettings.aws.json
	.AddEnvironmentVariables()
	.Build();
	services.AddMvc()
	.SetCompatibilityVersion(version: CompatibilityVersion.Version_2_2);
	//…Code to add required services based on configuration
	}
	public void Configure(IApplicationBuilder app) {
	app.UseMvc();
	//…Code to configure test Startup
	}
	}

view raw 1_TestStartup.cs hosted with ❤ by GitHub

	public class TestServerFixture {
	static readonly Dictionary<string, TestServer> cache =
	new Dictionary<string, TestServer>();
	public TestServerFixture() {
	//…
	}
	public HttpClient GetClient(string settings) {
	TestServer server = null;
	if(!cache.TryGetValue(settings, out server)) {
	var startup = new TestStartup(settings);
	var builder = new WebHostBuilder()
	.ConfigureServices(services => {
	services.AddSingleton<IStartup>(startup);
	});
	server = new TestServer(builder);
	cache.Add(settings, server);
	}
	return server.CreateClient();
	}
	}

view raw 2_TestServerFixture.cs hosted with ❤ by GitHub

	public class MyTest : IClassFixture<TestServerFixture> {
	private readonly TestServerFixture fixture;
	public MyTest(TestServerFixture fixture) {
	this.fixture = fixture;
	}
	[Theory]
	[InlineData("appsettings.aws.json")]
	[InlineData("appsettings.azure.json")]
	public async Task ShouldValidateTheGetApiResponse(string settings) {
	var client = fixture.CreateClient(settings);
	var result = await client.GetAsync("SOME_API_URL");
	// Code to Assert Result
	}
	}

view raw 3_MyTest.cs hosted with ❤ by GitHub

Here is the link to the original answer.

While this approach was a smart and simple way to solve our problem, it did not solve our problem completely. We had the following issues with the approach:

• We couldn’t run tests for aws or azure setting individually. The reason it was important for us as in the future, there could two different teams maintaining their specific implementation and deployment. With Theory, it becomes slightly difficult to run tests against a single DB provider.
• Our build and deployment for pipelines for each setting or DB provider were required to be different. That would mean the tests for Azure SQL and Amazon Aurora would run on a separate build pipeline. This approach made it difficult to achieve.
• While the API endpoints, Request, and Response are absolutely the same today, we do not know if it will continue to be the case as our development proceed.

Approach 2

In our second approach, we considered having a common class library with common Fixture and Tests as an abstract class.

• Here is what Common.IntegrationTests project looked like after the changes.

	public abstract class TestStartup : IStartup
	{
	public abstract IServiceProvider ConfigureServices(IServiceCollection services);
	public void Configure(IApplicationBuilder app)
	{
	app.UseMvc();
	// Code to configure test startup
	}
	}

view raw 1_TestStartup.cs hosted with ❤ by GitHub

	public abstract class TestServerFixture
	{
	protected TestServerFixture(IStartup startup)
	{
	var builder = new WebHostBuilder().ConfigureServices(services =>
	{
	services.AddSingleton<IStartup>(startup);
	});
	var server = new TestServer(builder);
	Client = server.CreateClient();
	}
	public HttpClient Client { get; private set; }
	}

view raw 2_TestServerFixture.cs hosted with ❤ by GitHub

	public abstract class MyTest
	{
	private readonly HttpClient _client;
	protected MyTest(TestServerFixture fixture)
	{
	_client = fixture.CreateClient();
	}
	[Fact]
	public async Task ShouldValidateTheGetApiResponse()
	{
	var result = await _client.GetAsync("SOME_API_URL");
	// Code to Assert Result
	}
	}

view raw 3_MyTest.cs hosted with ❤ by GitHub

• Project AWS.IntegrationTests

	public class TestStartup : Common.IntegrationTests.TestStartup
	{
	public override IServiceProvider ConfigureServices(IServiceCollection services)
	{
	var configuration = new ConfigurationBuilder()
	.SetBasePath(Directory.GetCurrentDirectory())
	.AddJsonFile("appsettings.aws.json", false)
	.AddEnvironmentVariables()
	.Build();
	services.AddMvc()
	.SetCompatibilityVersion(version: CompatibilityVersion.Version_2_2);
	// Code to add required services based on configuration
	return services.BuildServiceProvider();
	}
	}

view raw 1_TestStartup.cs hosted with ❤ by GitHub

	public class TestServerFixture : Fixtures.TestServerFixture
	{
	public TestServerFixture() : base(new TestStartup())
	{
	}
	}

view raw 2_TestServerFixture.cs hosted with ❤ by GitHub

	public class MyTest : Common.IntegrationTests.MyTests, IClassFixture<TestServerFixture>
	{
	public MyTest(TestServerFixture fixture) : base(fixture)
	{
	}
	}

view raw 3_MyTest.cs hosted with ❤ by GitHub

• Project Azure.IntegrationTests

A similar structure as AWS.IntegrationTests

This approach ticked all the boxes of our requirements. However, we were still not 100% convinced with the approach. There was still some waste with a lot of abstract classes and inheritance. That’s where approach 3 came into the picture.

Approach 3

Approach 3 more of a variant of approach 2 with a difference that instead of creating a common integration test project with abstract classes, we chose to create a Shared Project.

What is a Shared Project? From the Microsoft documentation:

Shared Projects let you write common code that is referenced by a number of different application projects. The code is compiled as part of each referencing project and can include compiler directives to help incorporate platform-specific functionality into the shared code base.
Unlike most other project types a shared project does not have any output (in DLL form), instead the code is compiled into each project that references it. 

Shared Project was introduced to solve a specific problem in Xamarin. However, I feel it is probably one of the most underrated and lesser-known features of .NET. Its usage goes beyond Xamarin.

With Shared Project we were able to simplify our integration tests set up quite a bit. We no longer needed unnecessary class inheritance or abstraction. Also, there was no ugliness of link files since the Shared Project is part of the Visual Studio solution just like a class library.

To conclude, we evaluated 3 different approaches to set up our integration tests and found the last approach to be the most suitable for our needs.

I hope this series helps you solve similar problems in your workplace.