Archive

Archive for the ‘Java’ Category

Spring vs Dagger

Spring contains configurable factory classes called contexts. These contexts can read description (specified in XML, for example) about
creating objects and satisfying dependecies. Finally, your app can obtain instances of objects from context and start using
them.

Dagger requires you to describe if class requires depdendency to be injected via @Inject annotation (right, that’s intruzive).
If dependency is concrete class, then it may be created in default way, and no more work required for it. Otherwise, if you
have to inject an instance of interface, or you need to create injected class in some specific way, then you use special
factory class and annotate it with @Module annotation and annotate each factory method with @Provides annotation. Finally, you
can generate a factory class for any classes which need injected dependency. This class will have factory methods identified
by return types, and they need all necessary @Modules to be supplied.

Compare:

Spring Dagger
Classes which don’t have dependencies, but on which other classes depend Describe them in XML, init them via values If they require some specific way to create them, then you need a @Provides factory method in some @Module class. Otherwize they will be created automatically
Classes which have dependencies, and on which other classes depend Describe them in XML, inject them with references to other beans If they require some specific way to create them, then you need a @Provides factory method in some @Module class. Otherwize they will be created automatically. In both cases their dependencies will be satisfied
Classes which have dependencies, but which are not needed to be injected See previous These classes will be needed externally, so you need to generate factory methods for them
A way to make it work Load description, call refresh(), obtain objects which you need. All beans will be created and injected in each other in a way which you describe Invoke generated factory methods for objects which you need and supply specific @Module factories to customize the result

Building OpenJDK 8 for Windows using MSYS

August 3, 2015 Leave a comment

This article will describe how to build OpenJDK 8 on Windows using MSYS. Since the building itself is performed by build scripts, we will focus on two things: installation of necessary libraries and compilers, and fixing build scripts, since they don’t work out of the box. As most of my articles, this one is written for my future self, because I’m sure I’ll get back to this task in future, and I don’t like solving problems I know I’ve solved before and I don’t remember how. Also this article is not a simple list of steps, I’ve tried to explain the reasons behind the actions. Readme file of OpenJDK says that “building the source code for the OpenJDK requires a certain degree of technical expertise”, so let’s make ourselves this expertise by learning while doing.

Getting the source.

The first step is to get the source. OpenJDK developers use Mercurial version control system as a code storage. You can open this URL: http://hg.openjdk.java.net/ in browser to see a list of projects hosted by OpenJDK. The project you need is jdk8. If you click on jdk8, you’ll see a list of repositories which jdk8 consists of. The top one is called jdk8, which makes a full URL: http://hg.openjdk.java.net/jdk8/jdk8/. You may wonder why there are two jdk8 directories in the URL? This remained from some old times when there were so called “gate” repositories to which changes were pushed for integration, and once those changes were verified, they were merged into read-only main repositories. So, jdk8/jdk8 is a read-only repository. Gate repositories approach was abandoned, but for OpenJDK 8 the path to read-only repository remains the same. If you are curious, you can read more about OpenJDK Mercurial repositories here.

So, let’s get ourselves this respository. You will need Mercurial tools for this. I like GUI tools, so I’ve downloaded SmartGit/Hg. It took me a while to figure out why there are no Mercurial option when you try to clone a remote repository. To make this work, you need to download and install official Mercurial command-line tools, and then go to settings of SmartGit and point it to hg.exe tool. This will make Mercurial to appear in a list of VCSes. Thus, GUI tools are not a full replacement for command-line tools, they just make life a little easier. If you don’t like GUIs, you can skip them and use command-line Mercurial tools, that’s quite easy. So go ahead and clone a repository http://hg.openjdk.java.net/jdk8/jdk8/ to some local directory.

Structure of OpenJDK build

The top repository jdk8/jdk8 contains only build infrastructure, it doesn’t contain any real source code, which is sorted into several additional other repositories. So, from this point we can either download those repositories, or we can do that later, when we will prepare everything. Let’s take a second approach, and start with preparing for a build. Take a look at a repository we just cloned. There are two readme files: a short text file and a bigger HTML file. Both are worth reading. Also there are two directories: common and make, and three scripts. The script named get_source.sh will download all remaining sources using Mercurial command-line tools, and we will postpone this until later. Two remaining scripts are the core of build process.

C is not a Java, there are many aspects of the language which are not defined and compiler-specific, like, for example, size of int value. So C programmers achieve portability by having special cases for compiler-dependent things. This is usually done “on source level”: a compiler-specific information is moved to dedicated header file. So to port a C program to another compiler requires changing compiler-dependent info and recompilation. To simplify this task programs used scripts which probe the compiler they are running on and generate a compiler-dependent header files. By convention these scripts are called configure. And OpenJDK has this script. We need to run it at least once. After that we have to use make tool to build everything, becase we have a script for it, called Makefile. Such two-stage comfigure/make approach is standard in Unix world for open-source software.

Let’s take a look at configure file. It is a unix shell script which prepares a build. It is very small, all it does is executing another configure script, located in common/autoconf. This second configure does a little more, like parsing command-line parameters, of which you can read more in readme.html. The main job is done by big script called generated-configure.sh. So, in order to run these scripts we need some Unix-like environment on Windows. There are two options: Cygwin and MSYS. Both environments are quite similar: each provides a shared library (dll) which implements some set of POSIX functions on Windows, and a set of Unix tools compiled as Windows executables, which rely on that dll. Cygwin is bigger, provides a larger set of POSIX calls and includes more Unix tools, so it’s like a complete unix-like environment. MSYS (which means “minimal system”) supports a smaller set of POSIX calls and provides a set of Unix tools just enough to be able to run typical configure scripts. I like everything minimal, so I prefer MSYS.

Installing MSYS and dealing with configure.

MSYS itself is not an independent project, it is a part of another project called MinGW (Minimalist Gnu for Windows), which is a quite interesting story worth telling. Most of the application programs written in C use standard library, and there are many reasons for that. On Unix systems it’s a convenient and portable way to do system calls. Standard library also includes lots of useful functions, like string manipulation. Since standard library relies on OS services, the OS kernel itself cannot use standard library. Windows provides it’s own set of services for applications, called Win32 API, but their compiler suite provides a standard library for compatibility and convenience. Some standard libraries are tied to specific compilers, but there are independent libraries: newlib, uClibc, dietlibc, mucl. When choosing a standard library one has to consider its features, performance, size, support of particular OS/CPU, and also the licence. For example, using library released with GPL requires you to release your program under GPL. The licence terms may be different depending on how you link against a library. There are two options: static linking (library will be included into executable) and dynamic linking. Licensing terms for dynamic linking are usually less restrictive then for static linking. However, if you choose dynamic linking you should somehow ensure that library is installed on computers where your program will run. So, knowing all this we can now get to MingGW. It is a version of GCC compiler which produces Windows executables dynamically linked with standard library supplied with Microsoft Visual C v 6.0 (msvcrt.dll). The license allows any code to dynamically link against it, and practically this library is present in all Windows systems (used by Microsoft’s own applications), so you don’t need to distribute it yourself. Thus MinGW produces executables which can be released under any license and distributed in a very simple way. Technically MinGW consists of a set of header files for standard library, an import library for mscvrt.dll and a version of GCC which produces Windows executables linked with import library. Later some additional libraries were ported to MinGW and now are provided as a part of it. Also MinGW was extended with include files and import libraries for Windows API, so now you can use it to write native Windows software. MinGW made it easier to port software written in C from Unix to Windows, but that was not enough. Thus MSYS was born, it is an environment for running configure scripts.

OK, back to building OpenJDK. Go to MinGW site and download installer. Run it. It will show a list of packages you can install. You don’t actually need MinGW compilers, since they are not used by OpenJDK built, but I advice you to install them. You’ll definitely need make and autoconf. Also you’ll need basic MSYS, and several specific MSYS packages: bsd cpio, mktemp, zip, unzip.

Now, as you have installed MSYS, you can start it’s shell (bash). You can use your windows paths in a special way, for example “C:\projects\openjdk” should be used as “/c/projects/openjdk”. You can try to run configure script right away. At the beginning this script will check availability of required tools, so if you forgot to install abovementioned cpio, mktemp, zip and unzip, then configure will complain (that’s how I learned that I need them). So here we will encounter a first problem with OpenJDK build environment which requires manual intervention. The script will fail finding cpio.

Learning autoconf

The script will fail finding cpio, since it is called bsdcpio. If you’ll try to track the problem (either by looking at source code or by reading log file) you’ll get to a script generated-configure.sh. To fix our problem, we need to modify this generated-configure.sh script. However, editing it directly is a wrong way. This script is generated (hence the name) by a tool called autoconf from sources located in OpenJDK folder common/autoconf. So, let’s get there and edit the sources. The actual change should be made in file basics.m4. Replace cpio with bsdcpio.

To generate new generated-configure.sh you should execute autogen.sh. But attempt to do it will fail, autogen.sh will complain that it can’t find autoconf. The reason is simple: autoconf was installed into MinGW location which is not available for MSYS by default. So, you should go to MSYS installation directory and find “etc” directory (on my machine it is located at c:\tools\mingw\msys\1.0\etc). Here you should create a file called fstab which will configure mounting of windows directories to msys filesystem. Take a look at fstab.sample to see how to do it, you may even copy it as fstab and edit it. Your task is to map root MinGW folder as /mingw. To apply changes in fstab you should restart MSYS bash. There is another file in etc called profile, which configures bash. By default this profile will add /mingw/bin into search path. So, if you did everything right, the result of “which autoconf” should be something like “/mingw/bin/autoconf”. Now you can get back and use autogen.sh to generate build script. Do it. Oops, another error.

This time autogen will complain that autoconf 2.69 or higher is required. However, MinGW includes version 2.68. When I encountered this error I’ve decided to try with 2.68, and believe me, it works perfectly fine. So, let’s hack OpenJDK build scripts and fix the required version. It is specified in file configure.ac. Again execute autogen.sh. This time it should work. Ignore output about no custom hook found.

We just fixed our first configure-stage error, and there will be more. To simplify troubleshooting, you should take a look at file called config.log, which contains output produced by conifugure script. If this log is not verbose enough, you can start the configure with command-line argument –debug-configure. It will make the script to produce additional log called debug-configure.log which is very verbose.

Installing bootstrap JDK.

Large part of JDK is written in Java,including the compiler. So building JDK requires you to have some bootstrap JDK. I’ve never got any problems installing it. You can even install it into default directory, and at any path, even the one which includes spaces.

Having fun with Microsoft Windows 7 SDK.

MinGW provides a C and C++ compilers for Windows, but the only officially supported by OpenJDK is Microsoft Visual C++ compiler, and we are going to use it. Otherwise configure will complain that it cannot find Visual Studio and quit. If you own Visual Studio, that’s great, and you can skip this part. However, in this article I’ll describe how to use minimalist development tools. So, we will use Microsoft Windows 7 SDK, which includes command-line C and C++ compilers from Visual Studio 2010. And it is free! You should download it from official site of Microsoft. There are web installer and several ISO images: for 32-bit systems, for Itanium and for 64-bit systems (amd-64). During the installation you can select which components to install, and I suggest to keep default settings, which include all necessary libraries and the compiler. If you will encounter some problems during the installation, check installation logs for exact description of the failure. I’ve got an error saying that SDK can’t install redistributable runtime libraries. Even de-selecting these libraries in a list of installed components doesn’t help. This happens because you already have a more recent version of those libraries installed (I had version 10.0.40219, and SDK will install 10.0.30319). It’s a shame for Microsoft to keep such bugs in installer. The only workaround is to uninstall your current redistributable of Microsoft Visual C runtime libraries, then install Windows SDK, and then download and install latest version of runtime library.

Now let’s check if compilers are working. If you will skip this part, you may get nasty errors much later. So, go to “c:\Program files (x86)\Microsoft Visual Studio 10.0\VC\bin\amd64” and launch cvtres.exe. If it has started successfully, that’s good. But on some systems it fails with application error. In fact you can skip this error, since it will not manifest at configure stage, but you’ll get strange error messages later on make stage, so let’s fix it now. Careful investigation with Dependency Walker tool shows that cvtres.exe imports a bunch of functions from msvcr100_clr0400.dll, and this dll doesn’t have any exported functions. Actually a version of this library included in SDK is OK, but some update for Microsoft .Net framework overwrites it with no-export version. Nice. In order to fix this, you need to download a patch from Microsoft called Microsoft Visual C++ 2010 Service Pack 1 Compiler Update for the Windows SDK 7.1. It will fix dependency problem for cvtres.exe, it will use another version of runtime dll. Download the update, install it and check that cvtres.exe works.

No, that’s not all. The update we just applied broke another thing. Unbelievable. I’ve created an empty file called ammintrin.h just to get around this annoying thing.

Patching the build scripts

Having Windows SDK will let you get further with configure, but eventually it will fail. That happens because scripts for building OpenJDK 8 using MSYS have errors. These errors were fixed in scripts for OpenJDK 9. The link to fixes could be found in this mail thread. Initial letter from Volker Simonis contains change request, and in subsequent messages Eric Joelsson extended it. Here is a list of changes:

  1. Fix for bsdcpio in basics.m4, which we have already applied
  2. Change in basics_windows.m4, which fixes AC_DEFUN([BASIC_FIXUP_EXECUTABLE_MSYS] problem with configure cannot find set_env.cmd file of Windows SDK
  3. Two fixes in toolchain_windows.m4: one for architecture type, and another with quotes for grep
  4. Fixes in platform.m4 for correct environment value. It’s a supplementary fix for other fixes to work.
  5. Fixes in NativeCompilation.gmk and specs.gmk.in will help if you’ll have an error during make. Without those fixes you’ll have to clean everything and re-make again from scratch, which takes a lot of time

So we should manually apply those fixes for OpenJDK 8. There are also change in generated_configure.sh, but you don’t need to apply it. Instead, generate it via autogen.

FreeType.

OpenJDK requires FreeType library. You can build it yourself from sources, I’ve downloaded a pre-built version. However, this pre-build version was strange: it included import library freetype.lib with all functions prefixed with underscore (“_”). To fix this, I’ve created an import library manually from dll using lib tool included in Microsoft Visual C command-line compiler suite (lib.exe /def:freetype6.def). This will produce a file freetype6.lib, which you should rename to freetype.lib, overwriting existing file (I’ve made a backup copy of it called _freetype.lib). You also need to copy freetype6.dll from bin directory in to lib directory and rename it to freetype.dll. And, finally, you need to explicitly specify path to the location where you’ve installed FreeType. A corresponding command-line argument for configure script is called –with-freetype.

Completing the configure

If you’ve done everything right, the configure step will successfully finish. The result will be stored in build directory of OpenJDK. The main item here is specs.gmk. Now you should download modules with source code.

Compilation

Launch make all. If make hangs edit specs.gmk and set JOBS=1. As a result you’ll get directory called j2sdk-image, that’s your JDK!

Categories: Java, Windows Tags: , , ,

The last difference between OpenJDK and Oracle JDK

January 18, 2013 6 comments

Recently I’ve spent a lot of time investigating font rasterization (a great topic which deserves a separate post). Most applications use font engine which is built into graphics library or widget toolkit. Only few cross-platform applications which badly need to provide consistent text layout (Acrobat Reader, for example) are using their own font engines (like Adobe CoolType). Java platform is one of such applications, since it has its own graphics library. If you are curious take a look at this article comparing font engines, including one from Java platform. From publically available information I understood that OpenJDK uses FreeType library. I thought: “That’s great, I have JDK 1.7 installed so this library must be there, let’s take a look”. But I could not find any traces of freetype.dll in JDK. I was puzzled and tried to find some answers in sources of OpenJDK. Imagine my surprize then I’ve found that Oracle JDK still uses proprietary T2K font library (located in jre/bin/t2k.dll)! Both Oracle JDK and OpenJDK are built from the same sources, and linking to external libraries happens in runtime. There is a logic which checks if JDK is running in “OpenJDK” mode or “Oracle JDK” mode, and, depending on that, loads either FreeType or T2K (see sun.font.FontScaler.java). I thought: “I was always curious about remaining differences between OpenJDK and Oracle JDK, and eventually I’ve found one!”. An interesting thing is how JDK determines in runtime if it is OpenJDK or Oracle JDK. It checks if there is a file for Lucida font in JRE. A long time ago Sun had lots of complains that “write once run anywhere” promise doesn’t actually hold, especially for look and feel. Different systems had different fonts, and sometimes even fonts with same names had different glyph sizes, leading to inconsistent text layout. To fix this Sun have licensed Lucida font for distribution with JRE and made this font default. This font is absent from OpenJDK distribution, and, as I said, this fact causes JDK to link in runtime with FreeType, and to link with T2K otherwise. I was surprized, since I’ve expected a global configuration flag, something like “isOpenJDK”. I thought: “OK, let’s take a look what other specific hacks other sub-systems use to distinguish between OpenJDK and Oracle JDK”. It turned out that there are no other parts where this difference matters. Font subsystem is a last one. So I was a lucky guy, getting directly into it.

Categories: Java Tags: , ,

How RMI works

July 17, 2012 Leave a comment

Sometimes the best way to teach someone how to use something is to explain how it works inside. This small explanation on Java RMI was written especially for me so I could quickly restore this knowledge in case I forget.

If you have some object and you want to make it accessible for remote parties, then you have to “export” it into RMI subsystem. RMI will generate some sort of identifier for your object and will store a binding between your object and this identifier inside some storage. When remote party wants to make a call to your object, it will make a connection to your JVM, and will send a protocol message containing object identifier, name of method and parameters in serialized form. RMI subsystem will find an object by identifier, will deserialize parameters and then will perform method invocation by using reflection.

Serialized form of parameters contain their exact class. So even if parameters are declared in method as something abstract, a server first creates instances of their exact class and only then performs upcast. This means that exact classes of parameters should be in server’s classpath.

To perform a communication a remote party should somehow obtain an identifier for exported object. This is solved by making additional lookup. A specific object named “Registry” is bound to some static identifier (let’s call it “1”). This object has his own storage and it allows mapping of other objects to strings. So to obtain a reference to “registered” remote object a client should know a string key which was used during object’s registration. A client  constructs a reference to registry using static identifier “1”, then asks it to return an identifier of registered object.

This double-referencing seems complex. However, it provides some level of protection. Registered objects are “public” and anyone who knows a name can call them. Names by which objects are registered are not secret, and you can query a registry for a list of all names. A method call to a public object may return a remote reference to some “private” object, with randomly generated id which is hard to guess, so it will be available only to method’s caller.

If a server wishes to return a remote reference to an object instead of a serialized copy, then it should export this object to RMI subsystem. The same is true for a client if it provides a callback parameter. UnicastRemoteObject is an object which automatically exports itself in a constructor.

Let’s check if we understand everything by describing a process of registering an object. Registry is often started as a standalone process. If a server wants to register an object, it should first construct a remote interface for registry. Interface itself is known (“java.rmi.Registry”) and located in runtime library. Object identifier is also known, it is static. So server should provide only host and port where RMI registry is running. A server exports his object, then invokes bind() method. RMI understands that argument to remote call was exported, so it sends object identifier and names of classes which are required for remote interface (interface itself, all super-interfaces, all declared parameter classes and all declared return classes). String key is serialized. Now serialized string, identifier of registry object and info about registered object will be sent to registry process. RMI subsystem in registry will create a remote reference with object’s identifier which implements object’s remote interface. Now RMI will locate registry object using registry’s identifier, and will invoke a method bind() to store remote reference together with key. When a client invokes lookup() it connects to registry in a same way as server, and server transfers stored remote reference to client. Now client can connect directly to server and make a call.

The bad thing with RMI is that because of serialization a server should be able to create exact classes of parameter objects, and client should be able to create exact classes of return values. Registry also should know a lot about all registered interfaces. This makes systems build on top of RMI not very flexible. However, there is a way how one side can tell to RMI classloader on the other side about location of classfiles. It is a system property “java.rmi.server.codebase”. To make things easy I’ve written a simple HTTP server which could be deployed in any application which uses RMI, so you will be sure that if it compiles, then it works.

Categories: Java, Technology Tags: ,

To SLEE or not to SLEE?

July 5, 2012 Leave a comment

I was working for a company which is making a product based on JAIN SLEE specification. I’ve developed several protocol stacks, several protocol frameworks and applications. Sometimes I’ve even studied code of SLEE implementation. However, I should admit that I had never read a specification myself. Most of my former collegues too. It is big and boring. I always had an impression that it is based on some very simple techinal ideas, and whole complexity is artificial. I felt that this specification was making life of developers harder, while it should have made it simpler. However, I understood that producing a JAIN SLEE was more a political and market-oriented decision.

A widespread approach to systems design is to compose solutions from components with well-defined interfaces. This means that each component should not know any details about other components it interacts with. Unfortunatelly a component design also often means that implementation difficulties of each component are considered to be specific to this component, and should be solved on component level. Architects are tasked in defining functional requirements to components and often don’t care about implementation difficulties. Sometimes implementations could be very much simplified if someone knows internals of communicating components. This problem is called “a cost of abstraction”. Since I have an advantage of understanding both whole picture and lots of individual parts, I’ve making this article. I claim that JAIN SLEE is a too complex solution to simple technical problems, and complexity makes it hard for people to see these simple technical problems. I claim that simple technical problems should have simple technical solutions. I also claim that SLEE is simply not enough.

The idea of this article is to show that JAIN SLEE should not be a framework or application server. Instead it should be an implementation principle of application server, covering all protocol stacks and protocol frameworks. Instead of being a layer of a solution stack it should be an approach applied to all layers of a solution stack, thus fixing problems for a whole solution.

The article is big and is split into several parts. First part is an introduction in multithreaded design. It explains how threads should be used. Second part compares synchronous and asynchronous protocol APIs. Third part introduces SLEE by combining ideas from first and second parts. Fourth part explains layered design to networking software and shows that protocol layers suffer from same problems as applications. Fifth part explains a new solution which is better than JAIN SLEE. Sixth part explains why JAIN SLEE became such a monster.

JAIN SLEE is implemented in Java, but most of the things I’ll say are not specific to Java, and applicable to any networking systems. All examples are written in Java-like pseudocode.

Remote debugging C programs with Eclipse and GDB

December 27, 2011 3 comments

On my daily job I write in Java. My last assignment, however, was to patch existing networking program written in C. Last time when I wrote something in C it was 10 years ago.

My IDE of choice is Intellij IDEA, but I’ve tried Eclipse several times, so it’s my backup IDE. I also knew from my collegues who actually practicing C that Eclipse CDT is a one of the best free IDEs for C development available. So I chose  Eclipse CDT it as my C IDE. On my desktop PC I use Windows, but the program I was modifying is Unix-only. When I’ve finished writing code I’ve uploaded it using FTP on Linux host (more on that later), and launched make. Of course I had compilation errors, but after several iterations of editing and uploading I finally got binary executable. The program is a network proxy. I’ve started it, sent some messages through it, and, as I expected, it was not working in a right way. I had to debug it to know why.

With Java it is very easy to debug programs remotelly. You ask IDE to do a remote debug, IDE will display to you which command-line options you should provide to java executable. You run your program, then you just ask IDE to attach to your program by specifying remote host and port. All debugging takes place in JVM, and your IDE is a debugging front-end. Easy and straightforward.

Not like this with Eclipse CDT. I’ve opened “Debug configurations” window, and saw “C/C++ Remote Application” option. I chose this option, but I found myself confused with fields I had to specify: location of executable and location of debugger. Even after I’ve downloaded executable from Linux to Windows, I couldn’t start remote debugging. So I decided to abandon this attempt for a while and try manual debugging with GDB.

This debugger was surprisingly easy for me to learn. It is much less cryptic than other Unix tools. Just a dozen of simple and memorable commands which you can print on one small sheet of paper for reference. GDB helped me to quickly found my errors. The only inconvenience is that you have to switch between terminal window where GDB is running and Eclipse CDT window where the code is. However, I still had an impression that remote debugging is actually possible from Eclpse CDT, so I decided to get it working, or at least to have an explanation why it is not possible.

And that’s that I’ve discovered after some long investigation. When C guys say “remote debug” they mean something very-very different from what Java guys mean. To remotelly debug C program you should link it with small chunk of code, which is able to do basic control of program execution (setting breakpoints, pausing-resuming, …) and basic program state examination (memory, stack frames, registers). This chunk of code is called “target side“. It will communicate with main part of a debugger which will be located on another machine. This main part  should be able to access a binary image of a program with all it’s symbols. This symbolic info allows debugger to perform all high-level debugging job, like showing contents of local variables, execution of code in line-by-line mode, setting breakpoint on a function. This part of a debugger is called “symbol side+UI“. When you perform a local debug both sides are in a same gdb process. You can also avoid linking with target size code by launching gdbserver which will attach to any process on a host. There is no official name for a protocol between target side and symbol side, so I’ll call it Remote Serial Protocol. It can work over serial cable or over TCP/IP.

So, difference between remote debugging in Java and remote debugging in C is that in case of Java target side and symbol side work on the same machine as program being debugged, and UI is on remote machine. In case of C only target side is located with program being debugged, and symbol side together with UI is on remote machine. Such approach allows debugging programs in cases where normal gdb will not work in local mode. For example, if you have a program for embedded device, which doesn’t have enough memory for full GDB.

But I want to remotelly debug programs with GDB in Java style. C guys will call it “remote UI for GDB”. And I believe that this is possible. Then you debug a program locally with Eclipse CDT, it creates a process for gdb, and intercepts it’s input and output. Then it says to GDB that instead of human-oriented command language it will use for communication a special machine-oriented language called MI (“machine interface”), which is easier to parse. So in order to use Eclipse CDT as a remote UI for GDB I should just launch GDB remotelly and remotelly attach to it’s input and output. I’ll try to make a plugin for Eclipse using my knowledge of Java, because a bunch of questions on StackOverflow shows that such capability is demanded.

Some links:

http://sourceware.org/gdb/current/onlinedocs/gdb/Remote-Protocol.html#Remote-Protocol

http://sourceware.org/gdb/current/onlinedocs/gdb/GDB_002fMI.html#GDB_002fMI

http://sourceware.org/gdb/current/onlinedocs/gdbint/Overall-Structure.html#Overall-Structure

http://dev.eclipse.org/mhonarc/lists/dsdp-tm-dev/msg01369.html

http://wiki.eclipse.org/TM_and_RSE_FAQ#How_can_I_do_Remote_Debugging_with_CDT.3F

http://www.ibm.com/developerworks/library/os-eclipse-cdt-debug2/index.html

Categories: Java, Technology

Is network stack a framework ?

July 13, 2009 1 comment

My recent articles about JAIN SIP API and SIP Servlets API often mention a term “framework”. I’ve planned to discuss this term in regard to SIP stack in depth, but forgot. Of course, this caused some questions, thus I’m doing it now.

I’m calling software module a “framework” if it is built with “Inversion of control” pattern. You provide callbacks, and framework invokes them according to it’s specification. Sometimes you can manage frameworks, but you cannot customize it beyond certain degree. Your code is not active, it is either passive or reactive.

Event-driven programming model is a one example of framework architecture. Software containers is another example.

Now you can see why I call SIP stacks which implement JAIN SIP API or SIP Servlets API as frameworks. They read data from network, handle them and then invoke a supplied listener (either SipServlet or SipListener). This invocation takes place in a thread of a SIP stack, so if you should not block it. JAIN SIP API has dispatch of incoming traffic based on local endpoint, and SIP Servlets API has method-based dispatch, but this is not a very significant difference.

Why SIP stacks are implemented as frameworks? To answer this, let’s imagine a stack which is implemented as a library. So, you create a socket, read from it, and then pass a byte array to stack for handling. SIP stack will return a result to you in functional style:

ByteBuffer data = ByteBuffer.allocate(MAX_MESSAGE);

SocketAddress remote = channel.receive(data);

SipResult result = stack.handle(data, remote);

Since there are many possible results of handling SIP messages, now you should analyze the result and dispatch according to it: was the message parsed correctly or not, was it request or response, was it a retransmission or not, and many other choices. If request was parsed correctly but has some mandatory headers missing, then result should contain error response which you can send through stack object. Such dispatch code is large, and should be written once because it’s behaviour is well specified in RFC 3261. This is a first reason why stacks are implemented as frameworks: they include common dispach code.

A second reason is that application programmers often afraid of working with threads and sockets directly. They consider that to be “system-level” code, which should be hidden from them. Developers of SIP stacks should bother about performance, race conditions and other complex stuff.

Thus, SIP stacks are frameworks, and I think that this is a right way. By the way, most HTTP stacks are also frameworks.

Now I will explain why I think that JAIN SIP API and SIP Servlets API are not perfect frameworks.

JAIN SIP API has a single callback called SipListener. It has only two methods for processing incoming messages: processRequest() and processResponse(). Thus, SIP stack does very little dispatch for you. If you are doing a stateless proxy, you’ll have very simple logic there. But for UA and statefull proxy there will be one large “if” statement. It could be implemented in different ways. One way is to map transactions and dialog on your application contexts. In this case you’ll have to look up into maps. Another way is to bind application contexts to transactions and dialogs using setApplicationData() method. In this case you’ll need to invoke getApplicationData() then cast it to your application context. When you have your application context you have additional dispatch. JAIN SIP API is flexible here, but this dispatching code is reusable, thus it should be written once. This dispatch code makes a better protocol framework on top of framework provided by SipListener.

A better protocol framework should have the following capabilities:

  • ServerTransactionListener, which can be provided to specific server transaction. This listener will be notified when transaction terminates, when final response retransmission timeout happens, and when CANCEL is received
  • ClientTransactionListener, which can be provided to specific client transaction. This listener will be notified when response is received
  • DialogListener, which can be provided to specific dialog. This listener will be notified when dialog has been forked and when dialog has been terminated
  • ServerListener, which is invoked for incoming requests. There should be one “global”  listener, and there could be specific listener for each dialog.

Such protocol framework will allow you to write applications with much less dispatch code.

Most of that is also true for SIP Servlets API. You have to extract attributes from SipSession and dispatch your execution based on them. However, they have some things better:

  • You can specify which servlet will handle which SipSession. Unfortunatelly, servlets are stateless.
  • Method-based dispatch is provided by SipServlet class

Thus, SIP Servlets API doesn’t provide a powerful protocol framework. Instead, they provide application framework: you can compose servlets, you have listener for various things such as binding attributes to sessions.

I hope I have explained why I consider SIP stacks to implement “framework” architecture. I also hope I have explaided why I think that it could be a better frameworks.

And, finally, what I’m calling an “application server” ? An application server:

  • Is a server for some protocol
  • Is implemented as a framework for this protocol
  • Is implemented as component container

Thus, SIP Servlets API and JAIN SLEE are describing application server, but JAIN SIP API is not.

Categories: Java, SIP, Telecom