Home > Uncategorized > Zookeeper C# Client with IKVM .NET

Zookeeper C# Client with IKVM .NET

Why ?

A lot of big data technology created with JVM. The language are vary from plain old Java, Scala and Clojure. There are a huge number of Apache project for this kind of stuff. The infrastructure for building a serious data product is amazingly mature in Java world.

The server is in Java . Ok. fine. But do we have to code in Java too ? At least as a client using the server ?. I know that JVM is awesome, but the language is not so well compare to C#. Especially the latest one C# 5. But for now we have to admit that .NET world is very lack of Server solution.

I always try to find a client library in C#/.NET but seems that it’s not really mature or has been already dead before birth. I will give you an example for this one.

 

Zookeeper is for the distributed zoo

Zookeeper has been used as a distributed process control. If you still remember the operating system 101 about process, threads, inter-process communication so this one should be easy for you to learn. If you familiar using lock or any synchronization mechanism like mutex, semaphore etc, just think this as way to do to coordinate multiple server synchronization. And it’s only expose a simple primitive like a file system. This is amazing and flexible. At the same time also make you do a lot of works to get the stuff working.

image

So we know that Zookeeper is good, it used by a lot of big data technology like Storm, Kafka, etc. So we need to find a client library to use it right ? Good luck finding it for .NET. There are a couple of library for doing that stuff but it’s either old or not up to date.

A couple of sample of that kind of library is :

https://www.nuget.org/packages/ZooKeeper.Net/

https://www.nuget.org/packages/ZooKeeperNet/

https://github.com/devhong/Zookeeper.Net

Curator

If you try to find the similar library in Java you will be found Curator. It’s really a nice stuff. It implement a couple of recipes on Zookeeper. So you don’t need to scratch your head with an odd zookeeper API. It’s really make your eyes hurts. Especially if you are a C# programmer.

image

 

So how about Curator in .NET ? Is that really a dream. Actually no. You can see the similar stuff build in .NET with the previous nuget link I gave you. Especially this one .

https://github.com/ewhauser/zookeeper/tree/trunk/src/dotnet

image

But if you look at the last update time it’s really sad.

 

IKVM

After searching and searching. I found an interesting stuff about how to convert the java library into .NET library. I know this a long time ago. I seems to forget this one. It’s doing bytecode to IL translation. Isn’t that awesome ? It’s open source and free.

Please enter IKVM .NET

http://www.ikvm.net/

You can find the latest binary usually posted on the website. The sourceforge and the nuget seems haven’t been updated quite sometimes.

So go to this website http://weblog.ikvm.net/ and then find the latest ikvmbin. It also covers the lambda stuff from Java 8. At the time this blog post created I can see the ikvmbin 8.xx. Go download and extract.

image

Here’s a couple of video that discuss about IKVM.

Jeroen Frijters live in Action

http://channel9.msdn.com/Events/Lang-NEXT/Lang-NEXT-2012/IKVM-NET-Building-a-Java-VM-on-the-NET-Framework

And here’s an interesting conversation between two Dutch man. Smile

http://channel9.msdn.com/Blogs/Charles/E2E-Erik-Meijer-and-Jeroens-Frijters-IKVMNET-Java-VM-implemented-in-NET

 

Enough said show me the code..

Before we are working IKVM I think we need to install maven in our system. This will make us easier to maintain and download java library and the dependencies so we don’t need to download it ourselves.

Please download maven first from this website.

http://maven.apache.org/download.cgi

image

Also find the step by step to install that stuff under below the download section of the previous page.

To test your Maven installation you can follow this tutorial.

http://maven.apache.org/guides/getting-started/maven-in-five-minutes.html.

 

IKVM in Action

Make sure that you have already extract the IKVM and set your environment path variable to it’s bin directory. Also set up the environment path for Maven.

Now we need to download the Curator library with maven as that is easier because it has already contains some recipes.

Create a folder and pom.xml in that folder. Enter this config to pom.xml

image

Go to that folder and execute this command

mvn install dependency:copy-dependencies

It will download Curator and the direct dependencies.

Ok don’t be afraid to the bloated text/log in the console. If you come from .NET maybe you think that’s an error. Don’t worry. It’s Not. That’s just how it’s work. You will get use to it. Open-mouthed smile

You will find all the library/jar in target/dependencies folder

image

 

Ok now you are about to enter the parallel universe. Java and .NET with IKVM.

It’s just one line of command.

ikvmc -lib:target\dependency -recurse:target\dependency -target:library -out:Curator.dll

That command will convert all library in target\dependency and also figure out the dependency itself. The result will be output as Curator.dll.

You may see a couple of warning. You can fix that warning by adding the required dependencies into pom.xml.

image

 

Now you can find your familiar .NET dll library.

image

 

Let’s try that on your project. Don’t forget to start your Zookeeper server first. Find the step by step to start here

Create a config first in conf/zoo.cfg. Just copy the zoo_sample.cfg for now to get start quickly.

image

Then start your zookeeper server.

image

You can try to connect with commandline to your zookeeper server

image

 

 

Now create a new project on your beloved Visual Studio and include the Curator.dll. Also don’t forget to include all IKVM library. This will be required for Curator.dl

image

 

Create a console application and create a class to demonstrate the distributed lock. You can take a look this page on how to get there.

http://curator.apache.org/getting-started.html

 

image

On your main program enter this code.

image

Run two console application for this same project. You will see something like this. The first console will acquire the lock and the other one will wait.

image

 

After the first one finish with it’s work then it will release the lock and give another console to proceed with it’s task.

image

 

Curator is really Awesome right ? IKVM also great.

 

Conclusion

Very easy right ? Meh. I lied.

 

Converting the library is not so easy. You sometimes have to figure out the dependency that missing, add to pom.xml and download and try again.

Also not all library can be converted with IKVM especially the one that use a lot of magic stuff and undocumented library. For example IMDG GridGain. It use Unsafe class from sun.misc. for changing the value of public final field in interface. How crazy this is right ?

Seems that Unsafe class hasn’t been implemented completely in IKVM. Another class that I notice missing is the implementation of JMX on monitoring java system. Some method is not implemented.  So sometimes it works and sometimes it don’t so take your poison.

 

A little bit of advice

Another thing to consider is to wrap your java library implementation into wrapper so it will feel natural to .NET developer, you may need to convert a callback stuff using Task from TPL, also using async await and even Reactive Extensions.

Don’t let the java class implementation leak into your .NET code. Tha’s just not right. It hurts my feeling. So don’t use it directly , always use anti corruption layer on top of the converted library.

You maybe find this article is interesting too on how to working with converted library.

http://www.palladiumconsulting.com/2014/06/using-zookeeper-from-net-ikvm-task/

 

Bring that library to .NET !

 

Cheers

  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: