Ubuntu upgrading kicks the dotnet apps failed.

I manages many server using Ubuntu Server for hosting dotnet apps as service. Recently, they failed one by one. The dotnet services on them cannot start. When typing dotnet –version, it reports

A fatal error occurred. The folder [/usr/share/dotnet/host/fxr] does not exist.

The whole problem is we installed the dotnet very early. In that time, we need to install the packages-microsoft-prod due to lack of official support of dotnet from ubuntu. But now it’s changed. Recently, dotnet is listed in the Ubuntu package manager feeds. That leads into the conflict which cause the services cannot start.

The way to fix is easy:

First, lists all installed packages using

sudo apt list --installed | grep dotnet

You may see some like dotnet-host dotnet-hostfxr-6.0 dotnet-runtime-6.0.

Now, removes those package listed using

sudo apt purge dotnet-host dotnet-hostfxr-6.0 dotnet-runtime-6.0

Also, packages-microsoft-prod is need to be purged also. To be mentioned, it must be purged instead of removed, which will remove the config file which is very the reason cause the conflicts.

sudo dpkg -P packages-microsoft-prod

Now, we can install the dotnet runtime from ubuntu official feeds using some like

sudo apt update && \
  sudo apt install -y aspnetcore-runtime-6.0

Hope this can save your day.

MessageBus

Message Bus is a component provided from SecretNest.info, providing publishing subscription model support within one app.

Source code: https://github.com/SecretNest/MessageBus

License: MIT

Doc: https://messagebus.docs.secretnest.info/

Standard: netstandard 1.3

Packages:

  • SecretNest.MessageBus.Abstractions: Abstractions for Message Bus from SecretNest.info. Imports this package when composing a component intended to be used to connect to Message Bus but do not hold the instance of the Message Bus itself. When the project contains the instance of Message Bus itself, imports SecretNest.MessageBus directly.
  • SecretNest.MessageBus: Provides publishing subscription model support within one app. Imports this package when composing an app managing the instance of Message Bus. This package references SecretNest.MessageBus.Abstractions.

Copyright: SecretNest.info / Allen Cui

No way to stop FileStream.Read

Recently, I need to read the data from a code scanner, which is recognized as a keyboard, in the dotnet program development of Linux. Because the program is running background, hosted in systemd, there is no way to get the entered text from console. All devices are presented as files in Linux. Reading the event from keyboard device file /dev/input/eventX is a good choice.

To read from the device in Linux, FileStream need to be created on the device file. Using the method Read() form the instance of FileStream, all key events can be processed one by one. For make it easy, I post the code in GitHub. Nuget package is also presented.

The sad thing is, there is no way to stop the FileStream.Read method except quit the app. The Read method is designed to read at least one byte. If there is no data available currently but not reaching the end of file, it will block and wait. And there is no way to break the waiting. I also tried to use BinaryReader — “Stream was not readable” reported. ReadByte failed in the same way. Cancel the CancellationToken cannot break the ReadAsync either. Setting read timeout is not supported in FileStream. And finally, the Thread.Abort is obsoleted and no use any more.

Is there any approach to cancel the running Read method of FileStream without exiting application?

Use GZipStream as response in plain Asp.Net Core 5

Recently, I’m struggle with asp.net core 5 working with GZipStream. My request is easy: using GZipStream to compress a large text before sending it as response of asp.net content.

Here is my first guess code, which is not working:

//This code is not working.
public static async Task WriteWithGZipAndCompleteAsync(this HttpResponse response, string text)
{
    response.StatusCode = 200;
    response.ContentType = "application/gzip";

    await using var gzip = new GZipStream(response.Body, CompressionLevel.Optimal, true);
    await using var streamWriter = new StreamWriter(gzip, Encoding.UTF8, -1, true);

    await streamWriter.WriteAsync(text);
    await streamWriter.FlushAsync();
    streamWriter.Close();

    await gzip.FlushAsync();
    gzip.Close();

    await response.CompleteAsync();
}

It does send the text to the client but the gunzip reports “unexpected end of file” after decompressed all texts.

The second version is worse, it sends nothing at all:

//This code is not working either.
public static async Task WriteWithGZipAndCompleteAsync(this HttpResponse response, string text)
{
    response.StatusCode = 200;
    response.ContentType = "application/gzip";

    using (var compressed = new MemoryStream())
    {
        using (var gzip = new GZipStream(compressed, CompressionLevel.Optimal, true))
        {
            var uncompressed = Encoding.UTF8.GetBytes(text);
            gzip.Write(uncompressed);

            gzip.Flush();
            gzip.Close();
        }

        var result = compressed.ToArray();
        response.Body.Write(result);
    }

    await response.CompleteAsync();
}

Finally, I get it work by this code below:

public static async Task WriteWithGZipAndCompleteAsync(this HttpResponse response, string text)
{
    response.StatusCode = 200;
    response.ContentType = "application/gzip";

    await using var gzip = new GZipStream(response.Body, CompressionLevel.Optimal, true);
    await using var streamWriter = new StreamWriter(gzip, Encoding.UTF8, -1, true);

    await streamWriter.WriteAsync(text);
    await streamWriter.FlushAsync();
    await gzip.FlushAsync();
    await response.Body.FlushAsync();

    //WARNING: DO NOT CALL CompleteAsync, which will thrown an exception.
}

Conclusion:

  • When calling Close() on instances of StreamWriter and GZipStream, the underlying stream will be closed, NO MATTER the value of leaveOpen specified. Uhh? Weird? But it’s true.
  • Do not call response.CompleteAsync() after write data to response.Body.
    • If Close() called on streams before, calling response.CompleteAsync() will thrown an ObjectDisposedException: Cannot access a closed stream.
    • If Close() does not present, like the code above, calling response.CompleteAsync() will thrown an InvalidOperationException: Writing is not allowed after writer was completed.

Forward Client Certificate to .net Core App through Nginx

When dotnet core app is deployed in Linux with Kestrel, nginx works in front as a proxy. Usually, Nginx will handle all https related issue and forward a plain http request to core app. Some app may require clients to use certificate for authentication. In this case, client certificate need to be transferred to core app.

Core App

First, the core app need to be prepared to receive and check the client certificate.

public class StartUp
{
    ...
    public void ConfigureServices(IServiceCollection services)
    {
        ...
        //Add code here
        services.AddCertificateForwarding(options => "X-ARR-ClientCert");
        //PointA - for later reference
    }
    public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
    {
        ...
        //Add code here
        app.UseAuthorization();
        app.UseCertificateForwarding();
        ...
    }
}

Note: UseHttpsRedirection() cannot be used because the core app is set to use http only. UseCertificateForwarding() may expose a security issue, you could set a switch to open it only when required. The header name X-ARR-ClientCert can be changed as your wish.

By default, core app will validate client certificate with local trusted CA. For additional tuning, add this code to the PointA position above.

services.AddAuthentication(CertificateAuthenticationDefaults.AuthenticationScheme)
    .AddCertificate(options =>
    {
        options.Events = new CertificateAuthenticationEvents
        {
            OnCertificateValidated = aMethod,
            OnAuthenticationFailed = anotherMethod
        };

It is not required to present OnCertificateValidated and OnAuthenticationFailed at the same time. Check this doc for details.

Nginx

Now, in the Nginx setting, some lines need to be added.

Check Certificate with Nginx

When need to check the client certificate by Nginx,

ssl_client_certificate file;
ssl_verify_client on;

is required. The file should contain trusted CA certificates in PEM format. When using multiple CA certificates, write all of them into the same file. When client certificate is not forcible, change on to optional.

Not Check Certificate with Nginx

If we need Nginx to leave the certificate checking to core app, simply use the code

ssl_verify_client optional_no_ca;

in site file. This will let Nginx transfer the client certificate to proxy without touch.

Pass to Proxy

After processing one of those above, the client certificate is ready to be passed into the proxy app — core app. This code below will do that.

proxy_set_header X-ARR-ClientCert $ssl_client_escaped_cert;

If you changed the name X-ARR-ClientCert above, use the same value here. This code can be placed into location block too.

Now you can enjoy your dirty job by checking everything about client certificate in your core app. 😀

Thread-Safe calling support in Remote Agency 2

In the next release of Remote Agency, the thread safe calling support will be added.

In the current version, all accessing to assets is from the thread which sending message to the Remote Agency Manager. Due to network transportation, this may cause multithread calling on the target service object. Without special treatment, some error may be caused when accessing object without thread safe designing.

In the next release, a new attribute is introduced. User can specify the behavior of thread using for each interface or the class of service object: Free — like now, use SynchronizationContext — useful on form based program, one certain free thread or one task schedule. A new task scheduler is builtin with Remote Agency which always use one thread to execute all jobs one by one. The task scheduler also support user passing a thread in as the working one inside. Therefore, user code can use the same thread to initialize some object and then turn it into a task scheduler to run all accessing on the same object.