I've been running a .NET6 API on Ubuntu 20.04 with NGINX, hosted in AWS EC2, since about March 2022. Today we did a routine deployment and the service running the API would not come back up. First time this has ever happened.
Output from systemctl restart followed by systemctl status:
Anovite.Api.service - .NET Web API App running on Ubuntu Loaded: loaded (/etc/systemd/system/Anovite.Api.service; enabled; vendor preset: enabled) Active: activating (auto-restart) (Result: core-dump) since Sun 2023-01-08 22:18:31 UTC; 8s ago Process: 1970 ExecStart=/usr/bin/dotnet /var/www/Anovite.Api/Anovite.Api.dll (code=dumped, signal=ABRT) Main PID: 1970 (code=dumped, signal=ABRT)I ran command journalctl -f and got the following output:
Unhandled exception. System.InvalidOperationException: Unable to configure HTTPS endpoint. No server certificate was specified, and the default developer certificate could not be found or is out of datTo generate a developer certificate run 'dotnet dev-certs https'. To trust the certificate (Windows and macOS only) run 'dotnet dev-certs https --trust'.For more information on configuring HTTPS see https://go.microsoft.com/fwlink/?linkid=848054. at Microsoft.AspNetCore.Server.Kestrel.KestrelConfigurationLoader.Reload() at Microsoft.AspNetCore.Server.Kestrel.KestrelConfigurationLoader.Load() at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.BindAsync(CancellationToken cancellationToken) at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerImpl.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken) at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken) at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken) at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token) at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token) at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host) at Microsoft.AspNetCore.Builder.WebApplication.Run(String url)At first I was concerned that our certificates had expired. They were created using letsencrypt. When I went to request a new certificate it alerted me that the existing certificate is nowhere near expiring.
Googling the error messages shown above leads me to numerous threads where a dev machine is being affected. In my case this is not a dev machine. We are not attempting to launch the API and browse it on the same machine. This is a production server. I'm confused why the .NET runtime all of a sudden thinks it needs a dev cert when we have valid, unexpired certificates already installed.
The default.conf file for NGINX specifies the path to the directory where the certificate is saved on the host. The config file has not been edited in months.
.NET versions currently running:
dotnet --list-sdks6.0.101 [/usr/share/dotnet/sdk]dotnet --list-runtimesMicrosoft.AspNetCore.App 6.0.1 [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]Microsoft.NETCore.App 6.0.1 [/usr/share/dotnet/shared/Microsoft.NETCore.App]The strangest thing of all is that rolling back to the previous version of the API had no effect resolving this problem. The problem started after the new version's files were copied up and the API service restarted. After reverting to the backup of the previous files the API service still would not restart. We're getting 503 Bad Gateway on Postman requests to the API regardless of which version we deploy.
What am I missing?