Sector 0

November 15, 2008

Is IronErlang (or Erlang.NET) a Good Idea?

I am kind of between projects right now so I am wondering what I should do next. I really like functional programming, and in particular languages like Erlang, Haskell and F#. So I started to wonder if I should make an Erlang compiler for the .NET runtime.

I like Erlang for a number of reasons: It is simple, it is a functional programming language, the language is designed around the actor-model for interprocess (Erlang processes that is) communication, a system written in Erlang is very robust, and creating computing clusters and high-availability servers is almost too easy.

So if I am about to start making an implementation of Erlang for the .NET platform there are a few obstacles and questions that needs to be addressed first:

  1. Should it be pure Erlang on the .NET runtime, or should there be interoperability with the rest of .NET? While the first is easier than the latter, I am a big fan of programmers having several tools available – all of which have their justification in different parts of a software project. Giving programmers more tools will only improve their performance as well as have an impact on the number of errors in software – something that is skyrocketing in these times where software systems grow more and more complex, and development teams become larger and larger. And we will see a rise in different technologies in the near future; The DLR (Dynamic Language Runtime) is right around the corner, and F# is becoming more and more mainstream and will be included in the official Microsoft .NET language stack.
  2. In Erlang a process (abstractly speaking the equivalent of a thread) is extremely light weight. As a consequence a system written in Erlang can easily have 10.000 processes running concurrently with no discernible impact on performance. This is not possible with regular .NET threads. I have been experimenting with making such light weight threads and have been successful as a proof-of-concept as well as some running code. So that is no big problem – it just needs some hard work.
  3. The actor model that Erlang uses to communicate between processes is wonderfully simple to use and understand, and the same goes for implementing the framework behind it; I have written a framework for message passing in C# on distributed nodes, and I use the actor model heavily in F# so I understand it thouroughly.
  4. Should a compiler compile directly to IL code? Writing a compiler that outputs IL code is hard work, and the complexity is perhaps too much to handle. But I have also been experimenting with compiling from one (simple) functional language to F# and then using the F# compiler itself to produce IL code. Much easier.

So it seems that I have at least some of the obstacles covered. What I need to address is how to bridge Erlang and .NET since the former is strictly functional and the latter focuses on object orientation.

I have also been searching the net for something similar, but I have only found some propositions and small articles about the benefits of having Erlang for the .NET platform. I think I will start this project as an academic exercise and see where it takes me. Maybe I can even find someone who are willing to help on such a project..? If you have any input whatsoever you are welcome to comment below or send me an email.

Is Erlang on the .NET platform desirable and feasible? Are there any Erlang programmers out there with comments or ideas?

16 Comments »

  1. I think the biggest hurdle would be a) implementing the ‘no side effects’ model where variables cannot change, and dealing with calling into other code in C# or another language. Also I can’t wrap my mind around implementing optimized tail recursion that doesn’t grow the stack in the CLR. I’m sure it can be done I just can’t see how to do it. I don’t know enough about F# though – perhaps they already do this? If so, it does make sense to try to not reinvent the wheel and compile to F# then let someone else do the heavy lifting/optimization. The thing with Erlang is that the VM is truly optimized for the language at level it would be tough to duplicate by spitting out your own IL. The CLR is incredibly efficient, but it would be a lot of work to generate IL that would take advantage of the CLR… Interesting to think about.

    Comment by Chad DePue — November 15, 2008 @ 18:21

  2. No. The single major advantage that Erlang has is its existing runtime. Erlang has a unique set of performance / reliability trade offs that .NET isn’t going to replicate. For example, the VM directly supports large numbers software threads with immutable variables and separate address spaces and optimizes accordingly.

    But all that ignores the key feature of Erlang: The runtime is rock solid, and guarantees that no-one can break that with extension modules. The main thing that you gain with .NET – the ability to load in existing .NET libraries – actually breaks exactly Erlang’s biggest feature.

    If you want to implement something on .NET, try this: http://www.scala-lang.org/ – no need to implement “crashy Erlang” (which would be right up there with eager Haskell or a stand-alone C# interpreter in usefulness).

    Comment by Chandon — November 15, 2008 @ 20:26

  3. I am mainly a .Net programmer, dabbling in Erlang on the side at the moment as a guerilla activity; and I’d say it was a desirable goal (or rather set of goals).

    Having the syntax available (along with the immutable data and easy concurrency) the sort of thing that would allow a lot of Erlang code to “just work” would be nice. The stretch form of the goal would have to include all the high-availability high-performance qualities of the existing VM — approaching the performance of YAWS in the YAWS vs Apache face-off; allowing code hot-swapping — would make it compelling.

    How feasible the stronger forms of the goal are, I don’t know — for that I would have to defer to folk with more appropriate expertise. I would just love to be able to use it.

    Comment by Steve — November 15, 2008 @ 22:26

  4. [...] Is IronErlang (or Erlang.NET) a Good Idea? (Frank Thomsen) [...]

    Pingback by Dew Drop - Weekend Edition - November 15-16, 2008 | Alvin Ashcraft's Morning Dew — November 16, 2008 @ 4:42

  5. @Chad DePue

    I agree that the “no side effects” is the major problem with implementing Erlang with interoperability with .NET. However, I am still looking for good ideas as to how this might be implemented. I’m not saying this _can_ be done in a nice manner, but if we stretch the “no side effects” definition a bit then Erlang also has side effects. This is – as I mentioned – the main problem at the moment.

    With regards to the tail call optimization, Michel Schinz and Martin Odersky wrote a nice paper on doing just that on the Java Virtual Machine. Besides, F# implements tail call recursion, so I am sure that can be accomplished too; Especially if I compile from Erlang to F# as is my plan – at least for the first version. As you say, I won’t have to reinvent the wheel. Making a compiler that outputs IL is just too complicated, and would just slow me down. Besides, I am not convinced I can do that so why even start? :-) But maybe I will bump into someone down the road who can do it… no one knows.

    Comment by frank — November 16, 2008 @ 12:00

  6. @Chandon

    I know that the Erlang VM is both stable as h***. But what about the HipE compiler? As far as I know they rewrote the entire thing to fit into an executable. Besides, with the compiler I have in mind I can generate code that does not – in theory – break the CLR. But of course I cannot guarantee that the VM itself has bugs or odd behaviours – but the CLR has been tested by so many people so many times that I am quite confident about the reliability factor.

    However, this is not a quest to put erlang.org out of business. It is just meant to open all the good stuff from Erlang to the .NET developers, which I personally think would be a good thing. Agreed, one of the key features of Erlang is the “unbreakability”, but that is also due to the fact that you can divide your application onto several physical machines.
    The features I am interested in – besides the availability – is the simple syntax, the simplicity of a funtional language, and the simplicity of the actor model. This all combines to make developers write software with fewer bugs and thus contributes to “unbreakability”.

    Regarding the Erlang VM’s ability to juggle a large number of threads (processes in Erlang lingo) is not a problem; Using continuations I have successfully run a large number of tasks concurrently in a single thread, so it can definitely be done on the .NET platform. The Erlang VM is itself dependent on the operating system threads so it does something similar.

    Implementing Scala on .NET does not make much sense right now – the F# language is pretty much Scala on .NET, although I know there are a number of differences.

    But thanks for your inputs – they are duly noted :-)

    Comment by frank — November 16, 2008 @ 12:16

  7. Frank,

    What about the concurrency model only? I think that the biggest advantage of Erlang (the language, I’m not considering OTP) today is its concurrency model based on the actor-model. I think that the actor-model is the best programming model for the current multicore crisis (the definitive thread killer). Of course it is not the universal panacea that will solve all the issues, but it would fit for most of the problems that concern hobbysts/business developers. Scala has it on top of the JVM. Kilim (library) has it also on top of the JVM (via instrumentation) but in Java. But I don’t know any proponent for the .NET. It is true that its success, in Erlang, relies on the characteristics of functional languages (e.g. shared-nothing) but it is a start for a small paradigm change again.

    I would like to be aware of your steps on this regard, as I’m doing a masters in computer science and the actor-model is my main focus.

    Best Regards,
    FABIO CORREA

    Comment by Fábio Corrêa — November 16, 2008 @ 13:08

  8. F# does tail call optimization via a “tail” opcode present in IL; you don’t have to go through the transformation to a goto to re-use the stack frame. See here — http://blogs.msdn.com/jaredpar/archive/2008/11/10/comparing-continuations-in-f-and-c.aspx — for a sample.

    By comparison with Erlang, Scala is a rather cumbersome language (despite its support for actor-based concurrency). It has local type inferencing, rather than Erlang’s dynamic types, and lacks the pattern-matching everywhere that makes Erlang such a joy to use.

    Scala’s primary virtue, in my eyes, is as a functional language that can _already_ be compiled to both JVM and CLR — see http://www.saager.org/2007/10/09/scala-on-net.html — though you have to abstract out the runtime-associated libraries if you want to use a common codebase across both.

    Comment by Steve — November 16, 2008 @ 13:34

  9. @Fábio Corrêa

    I also think the actor model for concurrency is the key factor in programming multicores. For that same reason I have been toying with it in both F# and C# (I wrote a framework as a testbed : http://www.codeplex.com/MPAPI ) and they simplicity is just what is needed. However, in both C# and F# there is the possibility of sharing state between workers since the languages are not geared towards this type of concurrency.

    Erlang is another matter altogether. What I was thinking about implementing first was a subset of the language, light weight processes (with or without OTP) and the actor model, but no distribution. As you mention – this fits most hobbyists needs.

    Now, Erlang with no interoperability with the .NET platform does not justify the entire exercise (except having some fun doing it) so how that will be accomplished is something I am still pondering… and hoping people might give me some good ideas for.

    Comment by frank — November 16, 2008 @ 13:40

  10. I agree with others in the implementing the actor model in f# sounds more feasible. and to do it as erlang is going to need some pretty hefty work, erlang is pretty much an os in itself.

    .net already has all the libraries most erlang people want, the thing people like about it is the stable vm and the concurrency orientated programming.

    right now it sounds like your trying to put a syntax not very many people like onto a vm that not many other people like, and there is almost no intersection in those sets

    .net already has all the libraries most erlang people want, the thing people like about erlang is the stable vm and the concurrency orientated programming.

    Comment by Dale Harvey — November 19, 2008 @ 16:08

  11. What is important in Elang is not just the separate features but the way they interact and work together. For example pattern matching is used everywhere and is the same everywhere. So to build robust systems you not only need light weight processes and actor model of communication, but also share nothing semantics, immutable data and Erlang’s error handling mechanism.

    Interoperability with .NET is, of course, the main reason to want to do this, but it could also cause problems as other parts of .NET could violate some of the basic features of Erlang. Another point of course is the extreme reliability of the Erlang VM.

    In ny opinion if you seriously wanted to integrate Erlang and .NET you are probably better off to keep them separate and build a good interface between them.

    Of course the fun of doing an implementation is not to be denied. :-)

    Comment by Robert Virding — November 19, 2008 @ 18:11

  12. I am inclined to agree with all opponents in their reservations against implementing Erlang for the .NET platform. After some digging into the problem I have identified four major problematic areas:

    1)
    The Erlang processes. However, as I have previously stated this is absolutely solvable (although some hard work is needed); I have made light weight “threads” running inside a number of regular OS threads. This was just a proof-of-concept and still needs some work.

    2)
    The robustness of the Erlang runtime. I’m not saying that I want to write something that rivals Erlangs VM in robustness, but the idea is to bring the great language Erlang to the .NET community.

    3)
    Integration with the rest of .NET and immutable state. The way I figured I would go about this would enable any .NET language to use components written in Erlang, but the other way around is more tricky. I started wondering if allowing mutable state ONLY inside objects (from outside the Erlang part) would be a viable solution, but I haven’t pursued this trail of thought much yet. Ofcourse sharing objects (regarless of where they are defined) would not be allowed between processes.

    4)
    Erlangs module-concept. In Erlang circular references in modules is allowed, and currently I can’t see how this could be implemented without using some ugly hacks.

    What I want to do with this is – as previously mentioned – to extend the toolbox available to .NET programmers. I firmly believe that the more tools we as programmers have, the more error free (or less buggy, depending on how you view the world) we will be able to write; Using the correct tool for the given task at hand instead of using some mediocre general-purpose language like C# for ALL tasks.
    Dale Harvey mentioned in a previous comment on this blog entry that I might be trying to put a syntax not many people like onto a VM. But already I see a move with f# and the syntax is not much different. What I think you are missing is that the minute people realize (as I myself did back then) the benefits of for example Erlang, functional programming and the actor model, the minute they will start programming it. But it’s an uphill battle – trust me; I am currently fighting to make people at work realize how they may benefit from learning F# or Scala. I trust that it will happen – people are already using dynamic languages and are looking forward to the DLR (Dynamic Language Runtime). Besides, Anders Hejlsberg (chief architect on the C# language) – amongst others – advocated funtional programming and immutable state at this years JAOO conference in Aarhus. In the near future developers will be both imperative, dynamic and functional.

    And let’s not forget that Erlangs concurrency model is currently the chief solution to the multicore problem – developers must start to think and program differently.

    Comment by frank — November 19, 2008 @ 19:38

  13. Regarding the comments above about the Erlang syntax. I’m reading a little bit about F# for a couple of weeks already and, IMHO, Erlang’s syntax is very simple and easier to read compared to F#. By the way, #light is the one that helps it not to be too verbose, otherwise, sometimes, F# code can be cryptic. Anyway, both are great languages.

    Comment by Fábio Corrêa — November 21, 2008 @ 7:08

  14. I urge you to take a look at CCR, a Microsoft research lab project. It has a similar concurrency model (message-passing system) as Erlang.

    Comment by jim — November 21, 2008 @ 8:53

  15. I’m a .net programmer and I’m now learnning erlang.

    The CLR is native OO. And the DLR for dynamic language is well.

    I don’t know how they implement f#, just delegate and library or more expression tree?

    If the erlang can run on .net, so do f#.

    The benefit of erlang is fast and scalable. I don’t think the erlang.net run as fast as erlang. Except MS update the CLR.

    You should think how to improve the communication between .Net or java or c++ with erlang. Find a better way than the jinterface.

    Comment by IL — February 18, 2009 @ 11:01

  16. Hi friend, why you wanna do this? are you a fan of erlang syntax? If I use IronErlang, I just wanna use other .NET lib smoothly, or I will use F# + actor lib.
    Tail recursion is not a problem for .NET. see my post: http://iron9light.wordpress.com/2009/07/23/good-f-may-2009-support-tail-recursion-very-well/
    I just thought about how to implement erlang process. Is there any way blocking the running code without suspend the thread? I wish your MPAPI or CCR could answer me.

    Comment by IL — July 29, 2009 @ 10:58

RSS feed for comments on this post. TrackBack URL

Leave a comment

Powered by WordPress