Implementation (#1)

* first commit * Added Artemis producer. * Added Kafka producer. * setting up * rework#1 * sendasync without topic * basing done * identifier, first run * Added Kafka services (almost all). * backgroundservice * Moved background worker. * Added parameters for Kafka. * Fixed backgroundservice. * Added producer factory. * Added Kafka health check. * verifiche sull'esistenza dei topic * merged * Fix. * Artemis connection lost healthcheck * file and folders * Fix error sentence * Added connection health check. * Added healthchecks in config. * Added identifier to messages. * Fix. * Added comments. * readme * dovrei aver commentato tutto * renaming * some naming * program * message action * test * Added KafkaBuilder tests. * Added tests. * git actions * OpenMessaging * fix readme --------- Co-authored-by: Squall-85 Co-authored-by: Henrich87
thesharpninjas · Aug 22, 2024 · 7715622 · 7715622
1 parent 3002aa0
commit 7715622
Show file tree

Hide file tree

Showing 75 changed files with 2,357 additions and 389 deletions.
diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml
@@ -0,0 +1,106 @@
+# yaml-language-server: $schema=https://json.schemastore.org/github-workflow.json
+
+name: publish
+on:
+  workflow_dispatch: # Allow running the workflow manually from the GitHub UI
+  push:
+    branches:
+      - 'main'       # Run the workflow when pushing to the main branch
+  pull_request:
+    branches:
+      - '*'          # Run the workflow for all pull requests
+  release:
+    types:
+      - published    # Run the workflow when a new GitHub release is published
+
+env:
+  DOTNET_SKIP_FIRST_TIME_EXPERIENCE: 1
+  DOTNET_NOLOGO: true
+  NuGetDirectory: ${{ github.workspace}}/nuget
+
+defaults:
+  run:
+    shell: pwsh
+
+jobs:
+  create_nuget:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v3
+      with:
+        fetch-depth: 0 # Get all history to allow automatic versioning using MinVer
+
+    # Install the .NET SDK indicated in the global.json file
+    - name: Setup .NET
+      uses: actions/setup-dotnet@v4
+
+    # Create the NuGet package in the folder from the environment variable NuGetDirectory
+    - run: dotnet pack --configuration Release --output ${{ env.NuGetDirectory }}
+
+    # Publish the NuGet package as an artifact, so they can be used in the following jobs
+    - uses: actions/upload-artifact@v3
+      with:
+        name: nuget
+        if-no-files-found: error
+        retention-days: 7
+        path: ${{ env.NuGetDirectory }}/*.nupkg
+
+  validate_nuget:
+    runs-on: ubuntu-latest
+    needs: [ create_nuget ]
+    steps:
+      # Install the .NET SDK indicated in the global.json file
+      - name: Setup .NET
+        uses: actions/setup-dotnet@v4
+
+      # Download the NuGet package created in the previous job
+      - uses: actions/download-artifact@v3
+        with:
+          name: nuget
+          path: ${{ env.NuGetDirectory }}
+
+      - name: Install nuget validator
+        run: dotnet tool update Meziantou.Framework.NuGetPackageValidation.Tool --global
+
+      # Validate metadata and content of the NuGet package
+      # https://www.nuget.org/packages/Meziantou.Framework.NuGetPackageValidation.Tool#readme-body-tab
+      # If some rules are not applicable, you can disable them
+      # using the --excluded-rules or --excluded-rule-ids option
+      - name: Validate package
+        run: meziantou.validate-nuget-package (Get-ChildItem "${{ env.NuGetDirectory }}/*.nupkg")
+
+  run_test:
+    runs-on: ubuntu-latest
+    steps:
+    - uses: actions/checkout@v3
+    - name: Setup .NET
+      uses: actions/setup-dotnet@v4
+    - name: Run tests
+      run: dotnet test --configuration Release
+
+  deploy:
+    # Publish only when creating a GitHub Release
+    # https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository
+    # You can update this logic if you want to manage releases differently
+    if: github.event_name == 'release'
+    runs-on: ubuntu-latest
+    needs: [ validate_nuget, run_test ]
+    steps:
+      # Download the NuGet package created in the previous job
+      - uses: actions/download-artifact@v3
+        with:
+          name: nuget
+          path: ${{ env.NuGetDirectory }}
+
+      # Install the .NET SDK indicated in the global.json file
+      - name: Setup .NET Core
+        uses: actions/setup-dotnet@v4
+
+      # Publish all NuGet packages to NuGet.org
+      # Use --skip-duplicate to prevent errors if a package with the same version already exists.
+      # If you retry a failed workflow, already published packages will be skipped without error.
+      - name: Publish NuGet package
+        run: |
+          foreach($file in (Get-ChildItem "${{ env.NuGetDirectory }}" -Recurse -Include *.nupkg)) {
+              dotnet nuget push $file --api-key "${{ secrets.NUGET_APIKEY }}" --source https://api.nuget.org/v3/index.json --skip-duplicate
+          }
diff --git a/.whitesource b/.whitesource
@@ -0,0 +1,14 @@
+{
+  "scanSettings": {
+    "baseBranches": []
+  },
+  "checkRunSettings": {
+    "vulnerableCheckRunConclusionLevel": "failure",
+    "displayMode": "diff",
+    "useMendCheckNames": true
+  },
+  "issueSettings": {
+    "minSeverityLevel": "LOW",
+    "issueType": "DEPENDENCY"
+  }
+}
diff --git a/LICENSE → LICENSE.txt b/LICENSE → LICENSE.txt
diff --git a/README.md b/README.md
@@ -1,4 +1,182 @@
-# Ninja.Sharp.OpenMessagingMiddleware
-[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
-A .NET library that abstracts messaging middleware using unified interfaces, with built-in connection recovery, failover, and resource optimization for reliable messaging in distributed systems.
+OpenMessaging - a .Net wrapper for message brokers
+========================================
+Release Notes
+-------------
+First release has been created!
+
+Packages
+--------
+
+| Package | NuGet Stable | 
+| ------- | ------------ | 
+| [OpenMessaging](https://github.com/thesharpninjas/Ninja.Sharp.OpenMessaging/) | [![OpenMessaging](https://img.shields.io/badge/nuget-v0.0.1-blue)](https://www.nuget.org/packages/Ninja.Sharp.OpenMessaging/)
+
+Features
+--------
+OpenMessaging is a [NuGet library](https://www.nuget.org/packages/Ninja.Sharp.OpenMessaging) that aims in simplifying usage for the most common messaging framework.
+The library encapsulates some behavior and common options for most frameworks, providing a simple management to rule them all via configuration.
+
+Right now, it allows integration with these framework:
+ - [**Apache ArtemisMQ**](https://activemq.apache.org/components/artemis/): a next-generation message broker, based on the HornetQ code-base. The implementation is based on [ArtemisNetClient](https://havret.github.io/dotnet-activemq-artemis-client/docs/getting-started/) 
+ - [**Apache Kafka**]: Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications. The implementation is based on [Confluent .NET client](https://docs.confluent.io/kafka-clients/dotnet/current/overview.html)
+
+Next steps:
+ we'll try to provide abstraction for most popular message brokers, such as
+  - [**Azure ServiceBus**]
+  - [**Azure StorageQueue**]
+  - [**Azure EventGrid**]
+  - [**Amazon MQ**]
+  - [**Amazon SQS**]
+  - [**Amazon SNS**]
+  - [**GCloud Pub/Sub**]
+  - [**RabbitMQ**]
+
+## Limitations
+OpenMessaging allows to use in the same application several message brokers at a time. However
+ - you cannot use multiple instances of the same message broker; you can use ArtemisMQ AND Kafka, but you cannot connect multiple instances of ArtemisMQ.
+ - you cannot use the same topic name twice, even if the topic resides in different message brokers.
+
+## Configuration
+OpenMessaging can be configured manually or via appsettings.json. Configuration differs for each message broker, while the subsequent usage will be hidden by OpenMessaging framework. 
+
+If you're using appsetting.json, then you just need to add configurations under the 'Messaging' tag:
+
+``` json
+{
+  "Messaging": {
+    "Kafka": {
+        "Identifier": "myIdentifier",
+        "BootstrapServers": [
+          {
+            "Host": "myHost", 
+            "Port": 9092
+          }
+        ],
+        "SecurityProtocol": "SaslSsl", 
+        "UserName": "myUsername",
+        "Password": "myPassword",
+        "GroupId": "myGroupId",
+        "SaslMechanism": "ScramSha256"
+    },
+    "Artemis": {
+        "Identifier": "myIdentifier",
+        "Retries": 2,
+        "RetryWaitTime": 500,
+        "Endpoints": [
+          {
+            "Host": "myHost",
+            "Port": 61616,
+            "Username": "myUsername",
+            "Password": "myUsername",
+          }
+        ]
+    },
+  }
+}
+```
+otherwise, if you're using manual configuration, you just need to add configurations when you add the requested services 
+``` csharp
+builder.Services.AddArtemisServices(new ArtemisConfig()
+    {
+        //...
+    });
+```
+
+## Choose and add a provider
+OpenMessaging manages allows you to add several message brokers, and simplify the message management.
+You just need to provide, for each message broker you are configuring
+ - the topics/queues where you need a Producer (the object that *sends* messages)
+ - the topics/queues where you need a Consumer (the object that *receive* messages), and the the class that will manage those messages. These class **must** implement IMessageConsumer.
+
+``` csharp
+builder.Services
+    .AddArtemisServices(builder.Configuration)
+    .AddProducer("myArtemisProducerTopic")
+    .AddConsumer<LoggerMqConsumer>("myArtemisConsumerTopic")
+    .Build();
+
+builder.Services
+    .AddKafkaServices(builder.Configuration)
+    .AddProducer("myKafkaProducerTopic1")
+    .AddProducer("myKafkaProducerTopic2")
+    .AddProducer("myKafkaProducerTopic3")
+    .AddConsumer<LoggerMqConsumer>("myKafkaConsumerTopic1")
+    .AddConsumer<LoggerMqConsumer>("myKafkaConsumerTopic2")
+    .Build();
+```
+
+## Sending messages
+Once configured, sending a message is quite easy. You do not need to know how the broker work, or which broker you need - you just need the destination topic/queue and the message you have to send.
+Once configured, you can inject the IMessageProducerFactory instance that you'll use to send a single message
+
+``` csharp
+    [ApiController]
+    [Route("[controller]")]
+    public class MyApiController(IMessageProducerFactory producerFactory) : ControllerBase
+    {
+        private readonly IMessageProducerFactory producerFactory = producerFactory;
+
+        [HttpPost("Send/{topic}")]
+        public async Task<string> Get([FromRoute]string topic, [FromBody]Tester payload)
+        {
+            var producer = producerFactory.Producer(topic);
+            var msgId = await producer.SendAsync(payload);
+            return msgId;
+        }
+    }
+```
+## Receiving messages
+Receive a message could be quite a pain, depending on the specific broker implementation. 
+OpenMessaging simplifies message management, you just need to provide, while adding a Consumer, a class, implementing IMessageConsumer. 
+Whenever a message is available for the specified topic, the method ConsumeAsync will be triggered, providing you basic info about the message.
+You just need to specify if the message has been correctly processed (returning MessageAction.Complete), if it needs to be reprocessed (MessageAction.Requeue), or it should be discarded (MessageAction.Reject).
+
+``` csharp
+    public class LoggerMqConsumer(ILogger<LoggerMqConsumer> logger) : IMessageConsumer
+    {
+        public Task<MessageAction> ConsumeAsync(MqMessage message)
+        {
+            logger.LogWarning("Message consumed: {0}", message.Body);
+            return Task.FromResult(MessageAction.Complete);
+        }
+    }
+```
+
+## Bonus: infrastructure healthcheck
+Sometimes, can happen that you message broker can fail, disconnected, crash, or whatever. When this happens, your services need to be restarted. 
+If you're using a k8s cluster, you'll need to implement healthcheck for readiness and liveness, to tell your cluster when pods needs to be started again.
+OpenMessaging exploits the amazing features provided by Microsoft [HealthChecks](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.diagnostics.healthchecks?view=net-8.0) and adds some checks for the provided brokers and topics. You just need to build a simple liveness/readiness probe to use them
+
+``` csharp
+    [ApiController]
+    [Route("[controller]")]
+    public class MyApiController(HealthCheckService healthCheckService) : ControllerBase
+    {
+        private readonly HealthCheckService healthCheckService = healthCheckService;
+
+        [HttpGet]
+        [Route("liveness")]
+        [Route("readiness")]
+        public async Task<IActionResult> Get()
+        {
+            HealthReport report = await healthCheckService.CheckHealthAsync();
+            return report.Status == HealthStatus.Healthy ? Ok(report) : StatusCode((int)HttpStatusCode.ServiceUnavailable, report);
+        }
+    }
+```
+
+
+
+
+ ## Contributing
+Thank you for considering to help out with the source code!
+If you'd like to contribute, please fork, fix, commit and send a pull request for the maintainers to review and merge into the main code base.
+
+**Getting started with Git and GitHub**
+
+* [Setting up Git for Windows and connecting to GitHub](http://help.github.com/win-set-up-git/)
+* [Forking a GitHub repository](http://help.github.com/fork-a-repo/)
+* [The simple guide to GIT guide](http://rogerdudler.github.com/git-guide/)
+* [Open an issue](https://github.com/thesharpninjas/Ninja.Sharp.OpenSODA/issues) if you encounter a bug or have a suggestion for improvements/features
+****
diff --git a/icon.png b/icon.png
diff --git a/nuget.config b/nuget.config
@@ -0,0 +1,7 @@
+<?xml version="1.0" encoding="utf-8"?>
+<configuration>
+  <packageSources>
+    <clear />
+    <add key="NuGet" value="https://api.nuget.org/v3/index.json" />
+  </packageSources>
+</configuration>
diff --git a/src/Ninja.Sharp.OpenMessaging.sln b/src/Ninja.Sharp.OpenMessaging.sln
@@ -0,0 +1,49 @@
+
+Microsoft Visual Studio Solution File, Format Version 12.00
+# Visual Studio Version 17
+VisualStudioVersion = 17.10.35201.131
+MinimumVisualStudioVersion = 10.0.40219.1
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Ninja.Sharp.OpenMessaging", "Ninja.Sharp.OpenMessaging\Ninja.Sharp.OpenMessaging.csproj", "{9F44959E-C0D6-4BC0-BC7E-E483A5AFA18E}"
+EndProject
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Ninja.Sharp.OpenMessaging.Tests", "tests\Ninja.Sharp.OpenMessaging.Tests\Ninja.Sharp.OpenMessaging.Tests.csproj", "{8F029041-8553-46F1-8F43-DEB16065E44B}"
+EndProject
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Ninja.Sharp.OpenMessaging.App", "samples\Ninja.Sharp.OpenMessaging.App\Ninja.Sharp.OpenMessaging.App.csproj", "{22963DE6-ADBC-413C-BFB2-6BD6BA9D3758}"
+EndProject
+Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Sample", "Sample", "{02714B11-0C17-4F2A-9FBD-5E4053229C20}"
+EndProject
+Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Ninja.Sharp.OpenMessaging.Api", "samples\Ninja.Sharp.OpenMessaging.Api\Ninja.Sharp.OpenMessaging.Api.csproj", "{2FEEE2B0-3919-4EAF-B376-AE7C4ED0E7E6}"
+EndProject
+Global
+	GlobalSection(SolutionConfigurationPlatforms) = preSolution
+		Debug|Any CPU = Debug|Any CPU
+		Release|Any CPU = Release|Any CPU
+	EndGlobalSection
+	GlobalSection(ProjectConfigurationPlatforms) = postSolution
+		{9F44959E-C0D6-4BC0-BC7E-E483A5AFA18E}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{9F44959E-C0D6-4BC0-BC7E-E483A5AFA18E}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{9F44959E-C0D6-4BC0-BC7E-E483A5AFA18E}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{9F44959E-C0D6-4BC0-BC7E-E483A5AFA18E}.Release|Any CPU.Build.0 = Release|Any CPU
+		{8F029041-8553-46F1-8F43-DEB16065E44B}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{8F029041-8553-46F1-8F43-DEB16065E44B}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{8F029041-8553-46F1-8F43-DEB16065E44B}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{8F029041-8553-46F1-8F43-DEB16065E44B}.Release|Any CPU.Build.0 = Release|Any CPU
+		{22963DE6-ADBC-413C-BFB2-6BD6BA9D3758}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{22963DE6-ADBC-413C-BFB2-6BD6BA9D3758}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{22963DE6-ADBC-413C-BFB2-6BD6BA9D3758}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{22963DE6-ADBC-413C-BFB2-6BD6BA9D3758}.Release|Any CPU.Build.0 = Release|Any CPU
+		{2FEEE2B0-3919-4EAF-B376-AE7C4ED0E7E6}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{2FEEE2B0-3919-4EAF-B376-AE7C4ED0E7E6}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{2FEEE2B0-3919-4EAF-B376-AE7C4ED0E7E6}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{2FEEE2B0-3919-4EAF-B376-AE7C4ED0E7E6}.Release|Any CPU.Build.0 = Release|Any CPU
+	EndGlobalSection
+	GlobalSection(SolutionProperties) = preSolution
+		HideSolutionNode = FALSE
+	EndGlobalSection
+	GlobalSection(NestedProjects) = preSolution
+		{22963DE6-ADBC-413C-BFB2-6BD6BA9D3758} = {02714B11-0C17-4F2A-9FBD-5E4053229C20}
+		{2FEEE2B0-3919-4EAF-B376-AE7C4ED0E7E6} = {02714B11-0C17-4F2A-9FBD-5E4053229C20}
+	EndGlobalSection
+	GlobalSection(ExtensibilityGlobals) = postSolution
+		SolutionGuid = {345FC9C4-7C9E-46DC-A28D-0723B33D7F63}
+	EndGlobalSection
+EndGlobal