Quick fix for integration testing with Selenium in ASP.NET Core 3.1

Quick fix for integration testing with Selenium in ASP.NET Core 3.1

For those who read Scott Hanselman about integration testing, you’ll have an issue while migrating to ASP.NET Core 3. You’ll end up with an error as the URI will be empty.

In this short post, I’ll show you how to fix it in a few lines!

Disclaimer: I’m still not very keen about the complexity of the original solution but I don’t think it has been improved in .NET Core so it’s a small trade-off to be able to do real integration testing!

At some point, you have created a class that inherits from WebApplicationFactory. Problem is, with .NET Core 3, the method CreateServer is not called, so the original workaround doesn’t work anymore.

The fix is to make an explicit call to CreateServer method with a new method to be added to the class. See the example below.

using System;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using Microsoft.AspNetCore;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Hosting.Server.Features;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.AspNetCore.TestHost;

namespace Withywoods.WebTesting.TestHost
{
    public class LocalServerFactory<TStartup> : WebApplicationFactory<TStartup>
        where TStartup : class
    {
        private const string _LocalhostBaseAddress = "https://localhost";

        private IWebHost _host;

        public LocalServerFactory()
        {
            ClientOptions.BaseAddress = new Uri(_LocalhostBaseAddress);

            // Breaking change while migrating from 2.2 to 3.1, TestServer was not called anymore
            CreateServer(CreateWebHostBuilder());
        }

        public string RootUri { get; private set; }

        protected override TestServer CreateServer(IWebHostBuilder builder)
        {
            _host = builder.Build();
            _host.Start();
            RootUri = _host.ServerFeatures.Get<IServerAddressesFeature>().Addresses.LastOrDefault();

            // not used but needed in the CreateServer method logic
            return new TestServer(new WebHostBuilder().UseStartup<TStartup>());
        }

        protected override IWebHostBuilder CreateWebHostBuilder()
        {
            var builder = WebHost.CreateDefaultBuilder(Array.Empty<string>());
            builder.UseStartup<TStartup>();
            return builder;
        }

        [ExcludeFromCodeCoverage]
        protected override void Dispose(bool disposing)
        {
            base.Dispose(disposing);
            if (disposing)
            {
                _host?.Dispose();
            }
        }
    }
}

And here is a simple example of a actual test class.

using System;
using System.Net.Http;
using FluentAssertions;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Remote;
using Withywoods.Selenium;
using Withywoods.WebTesting.TestHost;
using Xunit;

namespace Withywoods.AspNetCoreApiSample.IntegrationTests.Resources
{
    [Trait("Environment", "Localhost")]
    public class SwaggerResourceTest : IClassFixture<LocalServerFactory<Startup>>, IDisposable
    {
        private const string _ResourceEndpoint = "swagger";
        private const string _ChromeDriverEnvironmentVariableName = "ChromeWebDriver";

        private readonly HttpClient _httpClient;
        private readonly RemoteWebDriver _webDriver;
        private readonly LocalServerFactory<Startup> _server;

        public SwaggerResourceTest(LocalServerFactory<Startup> server)
        {
            _server = server;
            _httpClient = server.CreateClient();

            var chromeOptions = new ChromeOptions();
            // if there is an issue with the run in CI, comment the headless part
            chromeOptions.AddArguments("--headless", "--ignore-certificate-errors");
            // chrome driver is sensitive to chrome browser version, CI build should provide the path to driver
            // for Azure DevOps it's described here for example: https://github.com/actions/virtual-environments/blob/master/images/win/Windows2019-Readme.md
            var chromeDriverLocation = string.IsNullOrEmpty(Environment.GetEnvironmentVariable(_ChromeDriverEnvironmentVariableName)) ?
                System.IO.Path.GetDirectoryName(System.Reflection.Assembly.GetExecutingAssembly().Location) :
                Environment.GetEnvironmentVariable(_ChromeDriverEnvironmentVariableName);
            _webDriver = new ChromeDriver(chromeDriverLocation, chromeOptions);
        }

        [Fact]
        public void AspNetCoreApiSampleSwaggerResourceGet_ReturnsHttpOk()
        {
            _server.RootUri.Should().Be("https://localhost:5001");

            try
            {
                // Arrange & Act
                _webDriver.Navigate().GoToUrl($"{_server.RootUri}/{_ResourceEndpoint}");

                // Assert
                _webDriver.FindElement(By.ClassName("title"), 60);
                _webDriver.Title.Should().Be("Swagger UI");
                _webDriver.FindElementByClassName("title").Text.Should().Contain("My API");
            }
            catch
            {
                var screenshot = (_webDriver as ITakesScreenshot).GetScreenshot();
                screenshot.SaveAsFile("screenshot.png");
                throw;
            }
        }

        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }

        protected virtual void Dispose(bool disposing)
        {
            if (disposing)
            {
                _webDriver?.Dispose();
            }
        }
    }
}

Source code is available in GitHub: devpro/withywoods and you can look at the CI Azure pipeline that runs every night: withywoods-CI.

Don’t hesitate to contact me directly (on twitter) if you have any question or leave a comment on this page!

References:
Integration tests in ASP.NET Core
Real Browser Integration Testing with Selenium Standalone, Chrome, and ASP.NET Core 2.1

bertrand

9 thoughts on “Quick fix for integration testing with Selenium in ASP.NET Core 3.1

  1. Hi Bertrand,

    your blog post helped me a lot to get my test automation one step further, now using Selenium to automate Chrome out of regular xUnit tests hosting the website.

    Just some questions/comments on this:

    I noticed that, using some kind of factory like yours, the TStartup methods runs twice. It is possible to avoid that by replacing line 36
    “return new TestServer(new WebHostBuilder().UseStartup());”

    with

    “return null;”
    (Of course only if really nothing is done with the returned server value!)
    As I don’t know anything about the internals, there is a small chance, that this value is used inside the base factory and throws some NullRef exceptions, but I haven’t had any yet.

    Second one, as I’m doing everything via Chrome I don’t need a client like line 26. In the original blog post there is a comment that it’s needed to start everything up, but I found out that my tests run fine without it. Can you approve that?

    Regards
    Kai

  2. Hi Bertrand,
    Thanks for your blog post, Now I am doing integration test with Selenium in ASP.NET Core 5. I followed your advice, but my problem is that “WebHost starts twice” and throws the error in the below. I’m trying to figure it out, but I’m not having any success.

    Have you ever found this issue, and if so, how did you resolve it?

    Message:
    System.IO.IOException : Failed to bind to address http://127.0.0.1:5000: address already in use.
    —- Microsoft.AspNetCore.Connections.AddressInUseException : Only one usage of each socket address (protocol/network address/port) is normally permitted.
    ——– System.Net.Sockets.SocketException : Only one usage of each socket address (protocol/network address/port) is normally permitted.

    Regards
    Tarn Piyathida

    1. Hi Tarn,
      Thank you for your comment!
      It is working in a net5.0 test project I have on GitHub (tests run every night in Azure DevOps) :
      – Test class: SwaggerResourceTest.cs
      – Factory class: LocalServerFactory.cs
      I looked at the git history of the repository and didn’t see a specific change I may have made for .NET 5.
      Maybe the web app runs elsewhere on your machine? (from the command line maybe? or a container?)
      Regards,
      Bertrand

  3. Hi Bertrand,
    I know that this article is not for net 6, but did you make it work with net 6 and the minimal API (no StartUp)?
    Best,
    Marius

    1. Hi Marius,

      Yes! It needs a small update in Program.cs file.
      This is what I add to the file:


      // fix: make Program class public for tests
      // see: https://docs.microsoft.com/en-us/aspnet/core/test/integration-tests?view=aspnetcore-6.0#basic-tests-with-the-default-webapplicationfactory
      #pragma warning disable CA1050 // Declare types in namespaces
      public partial class Program { }
      #pragma warning restore CA1050 // Declare types in namespaces

      Regards,
      Bertrand

      1. Hi bertrand,
        Congrats for your article. The problem I’m facing is that for NET 6 we only have Program. So I have WebApplicationFactory.
        I have created a client and everything runs fine when I call my endpoints with this client, but I looks like the server created is not visible for an external client (like a browser).
        The code is the following:

        // this code runs ok in my integration test, but i need it to be accesible from an external client
        var application = new WebApplicationFactory();
        var client = application.CreateClient();
        var weatherResponse = await client.GetAsync(“/weatherforecast”);

        1. Hi ArielMax,

          Thank you for your feedback!

          What is the need for the test to be available for an external client? (like a web browser)

          WebApplicationFactory is part of
          Microsoft.AspNetCore.Mvc.Testing library which is supposed to be ran by a .NET test engine. It is a way to execute integration testing without having to care about the API execution. To have it available in a web browser you could use a Continuous Integration platform like Azure DevOps.

          Outside of a test execution context, the ASP.NET wep API will run (with Kestrel for instance) and the external client just has to make an HTTP call (for instance to “http://myapiurl/weatherforecast”)).

Leave a Reply

Your email address will not be published. Required fields are marked *