Zookeeper C# Client with IKVM .NET

Why ?

A lot of big data technology created with JVM. The language are vary from plain old Java, Scala and Clojure. There are a huge number of Apache project for this kind of stuff. The infrastructure for building a serious data product is amazingly mature in Java world.

The server is in Java . Ok. fine. But do we have to code in Java too ? At least as a client using the server ?. I know that JVM is awesome, but the language is not so well compare to C#. Especially the latest one C# 5. But for now we have to admit that .NET world is very lack of Server solution.

I always try to find a client library in C#/.NET but seems that it’s not really mature or has been already dead before birth. I will give you an example for this one.

 

Zookeeper is for the distributed zoo

Zookeeper has been used as a distributed process control. If you still remember the operating system 101 about process, threads, inter-process communication so this one should be easy for you to learn. If you familiar using lock or any synchronization mechanism like mutex, semaphore etc, just think this as way to do to coordinate multiple server synchronization. And it’s only expose a simple primitive like a file system. This is amazing and flexible. At the same time also make you do a lot of works to get the stuff working.

So we know that Zookeeper is good, it used by a lot of big data technology like Storm, Kafka, etc. So we need to find a client library to use it right ? Good luck finding it for .NET. There are a couple of library for doing that stuff but it’s either old or not up to date.

A couple of sample of that kind of library is :

https://www.nuget.org/packages/ZooKeeper.Net/

https://www.nuget.org/packages/ZooKeeperNet/

https://github.com/devhong/Zookeeper.Net

Curator

If you try to find the similar library in Java you will be found Curator. It’s really a nice stuff. It implement a couple of recipes on Zookeeper. So you don’t need to scratch your head with an odd zookeeper API. It’s really make your eyes hurts. Especially if you are a C# programmer.

 

So how about Curator in .NET ? Is that really a dream. Actually no. You can see the similar stuff build in .NET with the previous nuget link I gave you. Especially this one .

https://github.com/ewhauser/zookeeper/tree/trunk/src/dotnet

But if you look at the last update time it’s really sad.

 

IKVM

After searching and searching. I found an interesting stuff about how to convert the java library into .NET library. I know this a long time ago. I seems to forget this one. It’s doing bytecode to IL translation. Isn’t that awesome ? It’s open source and free.

Please enter IKVM .NET

http://www.ikvm.net/

You can find the latest binary usually posted on the website. The sourceforge and the nuget seems haven’t been updated quite sometimes.

So go to this website http://weblog.ikvm.net/ and then find the latest ikvmbin. It also covers the lambda stuff from Java 8. At the time this blog post created I can see the ikvmbin 8.xx. Go download and extract.

Here’s a couple of video that discuss about IKVM.

Jeroen Frijters live in Action

http://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2012/IKVM-NET-Building-a-Java-VM-on-the-NET-Framework

And here’s an interesting conversation between two Dutch man.

http://channel9.msdn.com/Blogs/Charles/E2E-Erik-Meijer-and-Jeroens-Frijters-IKVMNET-Java-VM-implemented-in-NET

 

Enough said show me the code..

Before we are working IKVM I think we need to install maven in our system. This will make us easier to maintain and download java library and the dependencies so we don’t need to download it ourselves.

Please download maven first from this website.

http://maven.apache.org/download.cgi

Also find the step by step to install that stuff under below the download section of the previous page.

To test your Maven installation you can follow this tutorial.

http://maven.apache.org/guides/getting-started/maven-in-five-minutes.html.

 

IKVM in Action

Make sure that you have already extract the IKVM and set your environment path variable to it’s bin directory. Also set up the environment path for Maven.

Now we need to download the Curator library with maven as that is easier because it has already contains some recipes.

Create a folder and pom.xml in that folder. Enter this config to pom.xml

Go to that folder and execute this command

mvn install dependency:copy-dependencies

It will download Curator and the direct dependencies.

Ok don’t be afraid to the bloated text/log in the console. If you come from .NET maybe you think that’s an error. Don’t worry. It’s Not. That’s just how it’s work. You will get use to it.

You will find all the library/jar in target/dependencies folder

 

Ok now you are about to enter the parallel universe. Java and .NET with IKVM.

It’s just one line of command.

ikvmc -lib:target\dependency -recurse:target\dependency -target:library -out:Curator.dll

That command will convert all library in target\dependency and also figure out the dependency itself. The result will be output as Curator.dll.

You may see a couple of warning. You can fix that warning by adding the required dependencies into pom.xml.

 

Now you can find your familiar .NET dll library.

 

Let’s try that on your project. Don’t forget to start your Zookeeper server first. Find the step by step to start here

Create a config first in conf/zoo.cfg. Just copy the zoo_sample.cfg for now to get start quickly.

Then start your zookeeper server.

You can try to connect with commandline to your zookeeper server

 

 

Now create a new project on your beloved Visual Studio and include the Curator.dll. Also don’t forget to include all IKVM library. This will be required for Curator.dl

 

Create a console application and create a class to demonstrate the distributed lock. You can take a look this page on how to get there.

http://curator.apache.org/getting-started.html

 

On your main program enter this code.

Run two console application for this same project. You will see something like this. The first console will acquire the lock and the other one will wait.

 

After the first one finish with it’s work then it will release the lock and give another console to proceed with it’s task.

 

Curator is really Awesome right ? IKVM also great.

 

Conclusion

Very easy right ? Meh. I lied.

 

Converting the library is not so easy. You sometimes have to figure out the dependency that missing, add to pom.xml and download and try again.

Also not all library can be converted with IKVM especially the one that use a lot of magic stuff and undocumented library. For example IMDG GridGain. It use Unsafe class from sun.misc. for changing the value of public final field in interface. How crazy this is right ?

Seems that Unsafe class hasn’t been implemented completely in IKVM. Another class that I notice missing is the implementation of JMX on monitoring java system. Some method is not implemented.  So sometimes it works and sometimes it don’t so take your poison.

 

A little bit of advice

Another thing to consider is to wrap your java library implementation into wrapper so it will feel natural to .NET developer, you may need to convert a callback stuff using Task from TPL, also using async await and even Reactive Extensions.

Don’t let the java class implementation leak into your .NET code. Tha’s just not right. It hurts my feeling. So don’t use it directly , always use anti corruption layer on top of the converted library.

You maybe find this article is interesting too on how to working with converted library.

http://www.palladiumconsulting.com/2014/06/using-zookeeper-from-net-ikvm-task/

 

Bring that library to .NET !

 

Cheers

Advertisement