Porting Apache Avro into .NET Core

We are from Geekseat has been used Avro extensively. Currently our project needs to be done on .NET core so .As i post previously i’m working on porting Apache Avro into .NET Core. Basically there are two Avro library available. One from Apache and one from Microsoft. I want to port both of library as that has been used a lot in big data application.

Let’s get back to Apache Avro.

There are two main problem that made headache when porting this one. One is AppDomain and IL Generator / Dynamic method is missing.

Both problem exists in the ObjectCreator.cs. This class is responsible for dynamically creating object on the fly based on the avro type.

ILGenerator has been known as the fastest object creator by Ayende Research. So i think this can be replaced with Expression tree but not sure how. Then i remember that Zeddy Iskandar has been create a forum post on how to create object with expression tree with constructor. You can see the method from this forum post. Here’s the complete source code of the ctor delegate.

One down, one more to go.

AppDomain has also been solved by Michael Whelan. Here’s the blog post around how to replace AppDomain.

I use the polyfill approach from that blog post. You can see that in action here.

After fixing both problem i’ve got 382 test passed and 6 failing. I’m quite happy with this result, declare victory and push that to repository. Thanks Zeddy and Michael.

Cheers

 

 

Advertisement

Publishing to Maven Central

You can follow the tutorial for publishing your library from this following series of youtube videos. You will need to request via Sonatype JIRA and wait for approval. It’s explained in the series.

However if you want the easiest way you can leverage using bitbucket pipeline. You can see the step by step on how to do that from this tutorial. It’s still in beta phase but looks promising. You should give it a try and ask for the trial. Yes, you need to request the trial first and wait for the approval.

The missing part on the tutorial above is on how to generate the key. Here’s how you can do that in Windows machine.

1. Download the gnugpg for windows here
2. Setup the key for signing
   1. Generate the key
      gpg –gen-key
   2. Find the list of the key to find the id
      gpg2 –list-keys
      gpg2 –list-secret-keys

      Let’s say that the id is 4C7CF393. We will use this key in the subsequent step.

   3. Publish the public key
      gpg2 –keyserver hkp://pool.sks-keyservers.net –send-keys 4C7CF393
   4. Export the private key
      gpg -a –export-secret-key 4C7CF393 > private-key.gpg
   5. Donlot http://gnuwin32.sourceforge.net/packages/openssl.htm for encryption
   6. Add the binaries to your PATH environment variables
   7. Encrypt the key
      openssl aes-256-cbc -in private-key.gpg -out private-key.gpg.enc -pass pass:somesecretpwd
3. Upload that key to your repository. You can base your repository by copying this repository and replacing the private key there.
4. Copy your source code and etc there.
5. Replace the configuration for username and password by looking at README from this repository
6. Edit the pom.xml so it publish your library
7. You can follow the rest of the tutorial from this

Happy Publishing !

Cheers

Energistics Transfer Protocol (ETP) available in Maven Central

Finally ETP java library available in maven central. You could use this one in your pom.xml

<dependency>
    <groupId>org.energistics</groupId>
    <artifactId>etp</artifactId>
    <version>1.4.1</version>
</dependency>

Here’s the sample of my pom.xml

It will automatically import avro as dependency. Now you can use it from your project.

You can also find the various way to add that to your build system from this link

All the source code and pom.xml that required to generate the java source using maven plugin can be seen from ETP Java Bitbucket

And you can find the pom example in my Github

Enjoy !
Cheers

 

MsTest, xunit and nunit configuration in .NET Core

.NET core supports three unit test framework in this version. We can also use Test Explorer in Visual Studio. Here’s the configuration in project.json

MSTest

nunit

xunit

 

Here’s three project with each test framework and a couple of test class

Open the test explorer from Menu Test -> Window -> Test Explorer

Please build the test projects first and the test will be shown in Test Explorer.

 

You can access the source code and configuration in my Github

Happy TDD-ing !!

 

Cheers

 

 

Porting to .NET Core – My Little Project