don't expose a localhost https endpoint when there is no developer certificate

[tmds]

By default kestrel exposes an HTTPS endpoint for localhost using a development certificate.

On Linux, the dotnet dev-certs https --trust doesn't work well leading to a bricked development experience.

It would be nice if there was a global way to opt-out of the HTTPS localhost bind.

One option may be to not bind localhost HTTPS when there is no development certificate.

I think the reason for having the HTTPS endpoint is to be secure by default, though not having it on localhost does not make things insecure by default.

@Tratcher @halter73 @jkotalik what do you think?

[Tratcher]

cc: @javiercn

leading to a bricked development experience.

Can you say more about the failures you hit?

One option may be to not bind localhost HTTPS when there is no development certificate.

This should already be the default behavior. However, there's a gap between providing a dev cert and having anything trust it. Can you confirm there is no default https endpoint if you remove the dev cert?

I think the reason for having the HTTPS endpoint is to be secure by default, though not having it on localhost does not make things insecure by default.

The main goal is to make the dev environment closely match the production environments where HTTPS should be enabled. Many features behave differently between http and https.

You can suppress the default https endpoint by providing an http endpoint when running them app.

[BrennanConroy]

However, there's a gap between providing a dev cert and having anything trust it. Can you confirm there is no default https endpoint if you remove the dev cert?

Triage: Maybe we should avoid installing the dev cert by default on Linux. If you want the dev cert it is easy to install (although harder to get it actually working).

cc @javiercn @blowdart @halter73

[tmds]

Can you say more about the failures you hit?

I've been clicking the ignore certificate in Firefox as long as I remember.

I'm now playing around with tye and there are some manual steps in the repo that are based on some stackoverflow: https://github.com/dotnet/tye/blob/main/docs/tutorials/hello-tye/00_run_locally.md#certificate-is-invalid-exception-on-linux.

Like with the ASP.NET apps, this breaks UX. The steps here are distro specific. Changes to the config files can't be checked into source control. And root is required to make things work.

This should already be the default behavior. However, there's a gap between providing a dev cert and having anything trust it. Can you confirm there is no default https endpoint if you remove the dev cert?

If the dev cert is removed, the server fails to start:

crit: Microsoft.AspNetCore.Server.Kestrel[0]
      Unable to start Kestrel.
      System.InvalidOperationException: Unable to configure HTTPS endpoint. No server certificate was specified, and the default developer certificate could not be found or is out of date.
      To generate a developer certificate run 'dotnet dev-certs https'. To trust the certificate (Windows and macOS only) run 'dotnet dev-certs https --trust'.
      For more information on configuring HTTPS see https://go.microsoft.com/fwlink/?linkid=848054.
         at Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(ListenOptions listenOptions, Action`1 configureOptions)
         at Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(ListenOptions listenOptions)
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.AddressesStrategy.BindAsync(AddressBindContext context)
         at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(IEnumerable`1 listenOptions, AddressBindContext context)
         at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.BindAsync(CancellationToken cancellationToken)
         at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)

You can suppress the default https endpoint by providing an http endpoint when running them app.

I'm looking for a way to opt out for all my apps at once, rather than have to change it per app.
It could be through removing the dev cert, but that doesn't currently work.

Maybe we should avoid installing the dev cert by default on Linux.

Yes, I think that makes sense.

[Tratcher]

I've been clicking the ignore certificate in Firefox as long as I remember.

Ok, that's a trust issue. We know that story needs work.

AddressBinder.AddressesStrategy means that an https address is being passed in manually, it's not using the defaults. Is it in the launchsettings.json?

[tmds]

Is it in the launchsettings.json?

Yes: "applicationUrl": "https://localhost:5001;http://localhost:5000".
So this is part of the template?
Is this file for Visual Studio to find out the url to launch?
What part of dotnet run ends up reading this file and configuring ASP.NET?

[Tratcher]

Yes it's in the template, and only includes https if that checkbox is marked.

aspnetcore/src/ProjectTemplates/Web.ProjectTemplates/content/EmptyWeb-CSharp/Properties/launchSettings.json

Lines 25 to 28 in 8cf9885

	//#if(NoHttps)
	"applicationUrl": "http://localhost:5000",
	//#else
	"applicationUrl": "https://localhost:5001;http://localhost:5000",

VS and dotnet run read this and set environment variables when starting the app. You only get the server default behavior if you remove this line.

[tmds]

@Tratcher can we special case this in Kestrel?

Something like: ignore the "https" binding for "localhost" when the default development certificate is missing and the environment is "Development"?

[javiercn]

@BrennanConroy we are not changing the HTTPS experience for Linux to make it different from other platforms. We already provide options to people who don't want to install the https dev cert or use https within their templates.

The certificate installation can be skipped on first run by setting the following environment variable DOTNET_GENERATE_ASPNET_CERTIFICATE=0, the templates all support the --no-https flag and we offer documentation on how to setup trust on Linux here for all browsers and for the system trust root.

[javiercn]

On Linux, the dotnet dev-certs https --trust doesn't work well leading to a bricked development experience.

You can still browse to the site, and you can add a permanent exception if Firefox for localhost:5001 if you decide not to trust the certificate using the instructions provided in the docs.

[tmds]

we are not changing the HTTPS experience for Linux to make it different from other platforms.

The development experience for Linux for HTTPS is not the same as for other platforms: on Windows and macOS it works out-of-the box, on Linux it does not.

You can still browse to the site, and you can add a permanent exception if Firefox for localhost:5001 if you decide not to trust the certificate using the instructions provided in the docs.

I created this issue when confronted with the steps in https://github.com/dotnet/tye/blob/main/docs/tutorials/hello-tye/00_run_locally.md#certificate-is-invalid-exception-on-linux. These steps describe a workaround and not a real solution. I've made a suggestion to improve the situation for tye here: dotnet/tye#1025.

the certificate installation can be skipped

For general ASP.NET I think it would be an interesting option to not perform the https bind when there is no developer cert. This now gives an error, which could become a warning.

That makes it possible for the individual developer to opt-out. So when he checks out some repo that used the template with the default https option, that code runs for him too.

[javiercn]

The development experience for Linux for HTTPS is not the same as for other platforms: on Windows and macOS it works out-of-the box, on Linux it does not.

That is not the case, you are complaining about the certificate not being trusted and what happens is:

• On all cases you need to run dotnet dev-certs https --trust
  • On windows and Mac OS we take the steps to add them to the machine trust store.
  • On Linux we point you to the docs article with instructions on how to make those changes.

dotnet uses the machine trust root for determining whether a certificate is trusted or not. On windows we use the machine store, on Mac OS we use the system keychain. On Linux there is no centralized trust root and the different trust roots change across distros and browsers.

Browsers on Mac and Windows rely on the system trust root for determining if the certificate is valid, Firefox does too if you provide the appropriate setting (which we cover in docs).

On Linux we provide instructions on how to trust the certificate in all three major browsers (Edge, Chrome, Firefox).

In all cases unless the user runs dotnet dev-certs https --trust the experience is the same, the app launches and runs, and the browser will prompt a warning pointing out that the certificate is not trusted and the developer can choose to bypass the warning or go to the command line, and run the command.

When they run the command we provide all that is necessary on all OS to work. On Windows and Mac we rely on the system trust roots, on Linux we point you to the steps you need to follow to achieve the same result. In all cases we tell you the manual steps to achieve a trusted dev-cert across the different browsers and OSs. The only difference is that some of the steps are automatic on Windows and Mac and require a few steps that you perform as a one time gesture on Linux to make it happen.

While I understand your frustration about the experience on Linux, I don't think that's a good enough reason to change the default experience. The experience is not as good as on Windows or Mac because the HTTPS experience is in general "harder" on Linux, however even in the Linux case you can be up and running in 5 minutes by following the instructions we provide.

As for this issue on tye, this is caused by a "long standing bug" that was fixed on Open SSL at least 6 month ago. Recent versions of most popular distros contain newer versions of Open SSL that contain this fix.

HTTPS by default is a fundamental development default for several years and is not something we have plans to change. While I understand it might make things more difficult in some more advanced scenarios and require manual steps to solve, that doesn't mean we should switch defaults at the cost of offering a different experience across OS flavors.

If the issue is with the certificate not being trusted in the container image, we can instead change how we build the image to add the certificate to the trust store for the distro as part of building the image, we don't have to change the experience for everyone not using containers on Linux.

If we ever see a convergence on Linux to provide a common system trust root, we will likely consider adopting that within the dev-certs tool itself. However, given that it is not the case today we provide docs for how you can do it yourself.

Our "limit" with the dev-certs tool for now is that we support the system trust root, and offer docs for how to setup trust in other circumstances.

I hope this helps bring some clarity about our reasoning.

[Tratcher]

@Tratcher can we special case this in Kestrel?

Something like: ignore the "https" binding for "localhost" when the default development certificate is missing and the environment is "Development"?

We want to avoid special cases like that in the runtime, they're hard to distinguish from a real issue that you need to know about. In this case the app is explicitly telling us to bind to https but we can't and tell you accordingly.

It works as expected when you remove the https address or the launchsettings file, right?

[tmds]

As for this issue on tye, this is caused by a "long standing bug" that was fixed on Open SSL at least 6 month ago. Recent versions of most popular distros contain newer versions of Open SSL that contain this fix.

Are you suggesting there is a better way to do this now?

The experience is not as good as on Windows or Mac because the HTTPS experience is in general "harder" on Linux, however even in the Linux case you can be up and running in 5 minutes by following the instructions we provide.

I spent more than 5 minutes and I wasn't able to make it work. I use Fedora, not Ubuntu:

The path in the preceding command is specific for Ubuntu. For other distributions, select an appropriate path or use the path for the Certificate Authorities (CAs).

It works as expected when you remove the https address or the launchsettings file, right?

Yes. Though I don't think I should push that as a change in version control if the HTTPS dev cert works fine for other collaborators.

HTTPS on localhost isn't providing any real security. Instead of causing me issues, I wish there was a way to opt-out.

[blowdart]

HTTPS on localhost isn't providing any real security. Instead of causing me issues, I wish there was a way to opt-out.

As @javiercn points out there is an opt-out, --no-https when create the project or unticking the box in Visual Studio

HTTPS on localhost is not there for security, it's there to emulate what a real deployment looks like, to allow you to use oauth and other federated identities, to enable samesite cookies to work in the same way they would in production. Removing it, even if it's "just" on linux makes that experience impossible, and gives you surprises on deployment, something we want to avoid.