Archive

Archive for the ‘Uncategorized’ Category

Practical essentials of Java web stack

October 12, 2017 Leave a comment

Prerequisites

Common prerequisites are listed here

  1. Maven is used as a build tool

Standalone Jetty 9 serving static files

  1. Download Jetty 9 distribution
  2. Generate Jetty base at some location, like this:
    C:\projects\my-jetty-instance>%JAVA_HOME%bin\java -jar c:\tools\jetty-distribution-9.4.3\start.jar --add-to-start=http,deploy,logging-jetty 
  3. In webapp dirs, create subdir with any name. Put your files there
  4. Start Jetty
  5. Your files will be available in URL
    http://localhost:8080/_subdir_/

Plain servlet with standalone Jetty 9

  1. Download Jetty 9 distribution
  2. Generate Jetty base at some location, like this:
    C:\projects\my-jetty-instance>%JAVA_HOME%\bin\java -jar c:\tools\jetty-distribution-9.4.3\start.jar --add-to-start=http,deploy,logging-jetty
    
  3. From IDE or from command line generate a project using maven-archetype-webapp. It will generate a pom.xml and some project dirs, including almost empty web.xml.
  4. Maven, however, will not generate a source dir for your Java code. Do it yourself in /src/main/java.
  5. Also Maven will not add Servlet API jar to your project’s dependencies. Add something like this:
     <dependency>
    <groupId>javax.servlet</groupId>
     <artifactId>javax.servlet-api</artifactId>
     <version>3.1.0</version>
     </dependency>
    
  6. Write your basic servlet class which will extend javax.servlet.http.HttpServletRequest. Note that to generated HTML should go to response’s body obtained via getOutputStream(). Use print() method.
  7. Extend web.xml by adding servlet descriptor and URI mappings:
    
    <servlet>
     <servlet-name>TestServlet</servlet-name>
     <servlet-class>MyServlet</servlet-class>
     </servlet>
    
    <servlet-mapping>
     <servlet-name>TestServlet</servlet-name>
     <url-pattern>/*</url-pattern>
     </servlet-mapping>
    
  8. Use mvn package to compile and package your project into war file. Note that this file will be named as artefactId from pom.xml
  9. Put file to webapps dir of Jetty. Run Jetty.
  10. Test your app by opening URL (place artefactId of your app)
    http://localhost:8080/_artefactIdHere_/
    
Advertisements
Categories: Uncategorized

Q programming language tutorial

Basic interpreter actions

Installing: put following files to c:\tools\kdb: q.k, w32\q.exe

setx QHOME c:\tools\kdb

Then run q.exe.
Exiting:

q)\\

or:

q)exit 0

Switching between q and K languages:

q)\

Load script:

q)\l script.q

Start as server:

q -p 5001

Connect to server and assign a connection to variable hdl:

q)hdl: hopen `:localhost:5001

Communicate from client to server (synchronous message):

q)hdl "put command here"

Closing connection:

q)hclose hdl

Language basics

Atoms:

q)5 / this is comment. Before was an expression, immediatelly evaluated. Integer atom evaluates to itself
5
q)`symbol / symbol atom
`symbol

Simple arithmetic:

q)2+2
4
q)2-3
-1
q)3%4 / division
0.75

Variables:

q)x:5 / define x
q)x / get value of variable
5
q)x+x / use arithmetic with variable
10

Simple functions:

q){x*x}[4] /square function, simple expression,x is an implicit parameter, applied to atom 4.
16
q){x*x} 4 /if applied to one parameter, square brackets can be omitted
16
q)f:{[a;b]b+a*a} /function with explicit parameters
q)f [4;5] /apply to parameters. This is full syntax which must be used if params more than 1
21
q)f /will display function definition
{[a;b]b+a*a}
q)+ /built-in functions will be printed like this, it's not a symbol!
+
q)+[4;5] /can apply 2-arg functions in this way
9
q)g:{[a;b]c:a*a;c+b} /statements in function are separated by semicolon ;
q)g [4;5]
21
q)k:{7} /no-arg function, returns a constant
q)k[] /application
7
q)g[4][5] /partial application, or currying: first apply g to 4, then apply result to 5
21
q)p:{x*x} /just one-arg function
q)p 4
16
q)p@4 /@ is a two-arg 'apply' function which applies one-arg function to arg
16
q)@[p;4] /can to be called this way
16
q)`p 4 /you can address a function via symbol (only in application!)
16
q)u: `p /variable u holds symbol `p
q)u 4 /this will apply 4 to p, since u references p via symbol
16

Lists:

q)x: 1 2 3 / define a list of items of same type using simplified syntax
q)x
1 2 3
q)b: (9 9 9) /more verbose syntax
q)b
9 9 9
q)v:(9;1;8;2;7;3;6;4;5) /most verbose syntax
q)v
9 1 8 2 7 3 6 4 5
q)n:() /empty list
q)n
q)m:(7) /attempt to create a single-item list will not work, brackets are ignored and atom is evaluated
q)m
7
q)enlist 7 /create a single-list element. Displayed starting with comma to distinguish from atoms
,7
q)z: 7 6 5 4 3 2 1
q)z[0] /access by integer index. Result is atom. Syntax is like function application
7
q)z[5 4 1] /access by list of indexes. Result is list
2 3 6
q)z[x] /works this way too
6 5 4
q)z 0 /also works without brackets
7
q)z 5 4 1 /and this way too
2 3 6
q)z x /and this way too
6 5 4
q)z[] /empty indexing returns whole list
7 6 5 4 3 2 1
q)z[20] /indexing outside of list bounds returns empty value for list item type
0N
q)z[4]:9 /assignment of list item, can't assign to non-existing index
q)z
7 6 5 4 9 2 1
q)z[4]:3
q)z
7 6 5 4 3 2 1
q)z@1 / another way of indexing, since lists are like functions
6

q)"mystring" 0 1 /strings are just list of chars
"my"
q)y:4 4 4

q)5#z /take first 5 elements, # is a two-arg function, you can use spaces around it
7 6 5 4 3
q)-4#z /take last 2 elements
4 3 2 1
q)8#x /if list is shorter than left arg, then items will repeat
1 2 3 1 2 3 1 2
q)2 _ z /drop first 2 elements, _ is a two-arg function
5 4 3 2 1

q)x,y /joining lists, also works in joining list to atom
1 2 3 4 4 4
q)count 4 5 6 7 /function, returns length of list
4
q)count x
3
q)count y
3
q)#:[1 2 3 4 5] / in fact, function # is overloaded and one-arg is count. Use : to explicitly invoke one-arg
5
q)z ? 2 /search value in list from left, returns index of first found value
5
q)y ? 4 /will return 0, since it is an index of first item in list
0
q)z ? 12 /if value is not found, then returns index of element beyound last (it is equal to size of list)
7

q)x+5 / add an atom to each element in list
6 7 8
q)x+y /add corresponding elements of lists, works only on lists of same length
5 6 7
q)z = 4 /returns a binary value with each bit in 1 if element equals to atom
00010000b
q)(x+1) = y /also works on lists of the same size, compares corresponding elements
001b
q)where 2 3 0 1 /where works for lists of integers. It repeats value of index as many times as value of corresponding list item
0 0 1 1 1 3
q)&amp;:[2 3 0 1] /where is an alias for one-arg version of &amp;. Use : to explicitly invoke one-arg
0 0 1 1 1 3
q)where 0 0 1 0 1 1 0 /interesting fact: applied to list of booleans (0 or 1), where will return indicies of 1's
2 4 5
q)where z&gt;4 /so, where could be used as filter: returns list of indexes which match the condition
0 1 2
q)z where z&gt;4 /you can apply a list of indexes back to list to get values
7 6 5

q)mixed:("string";1;`mysymbol;1 2 3) /mixed list, must be enclosed in () and separated with ;
q)mixed[0] /indexing works in a same way
"string"
q)mixed[3]
1 2 3
q)mixed[0 3;2] /semicolon allows to use indexing in depth, for example: take items 0 and 3, and from each take sub-item 2
"r"
3

q)p each x /apply function p to each value in list x and return a list of results
1 4 9

Functions and Lists:

q)count 4 5 6 7 /application syntax: count is one-arg function applied to single arg of list type
4
q)count[4 5 6 7] /the same, using explicit application syntax
4
q)f 4 5 /apply single argument of type list to two-arg function, it's a partial application
{[a;b]b+a*a}[4 5]
q)f[4 5] /the same
{[a;b]b+a*a}[4 5]
q)f[4 5;1 3] /explicit syntax is used to apply two args, each is a list, both have same size
17 28
q)p @ 4 /apply function to single arg of type atom
16
q)p @ 4 5 /apply function to single arg of type list
16 25
q)f @ 4 5 /since it's like f[4 5], it's a partial application
{[a;b]b+a*a}[4 5]
q)f . 4 5 /curry-style application: function . accepts a single arg of type list, and puts elements into args
21

Dictionaries:

q)monthdays: `Jan`Feb`Mar`Apr`May ! 31 28 31 30 31 / composed from two lists of same length using !
q)monthdays
Jan| 31
Feb| 28
Mar| 31
Apr| 30
May| 31

q)monthdays[`Jan] / obtain value by key using function application syntax
31
q)monthdays[`Jan`Mar`May] /obtain list of values from lists of keys
31 31 31

q)key monthdays / function key returns list of keys
`Jan`Feb`Mar`Apr`May
q)value monthdays / function value returns list of values
31 28 31 30 31
q)count monthdays
5
q)monthdays ? 31 / search value, returns key. Searches from left to right
`Jan

q)monthdays[`Jan]:1  / assignment syntax allows to map existing key to new value
q)monthdays
Jan| 1
Feb| 28
Mar| 31
Apr| 30
May| 31
q)monthdays[`Jan]:31 / change back
q)monthdays
Jan| 31
Feb| 28
Mar| 31
Apr| 30
May| 31
q)monthdays[`Jun]:30 / it is possible to add new key-&gt;value to dictionary
q)monthdays
Jan| 31
Feb| 28
Mar| 31
Apr| 30
May| 31
Jun| 30

q)monthdays - 1 / built-in functions for atoms are overloaded to work over dictionary values
Jan| 30
Feb| 27
Mar| 30
Apr| 29
May| 30

q)monthdays = 31 / compares each value with constant, results in dict of booleans
Jan| 1
Feb| 0
Mar| 1
Apr| 0
May| 1

q)where `A`B`C!3 4 2 / where works with dicts of integer values. It makes a list, repeating each key as many times as corresponding value
`A`A`A`B`B`B`B`C`C
q)where `A`B`C`D! 0 1 0 1 / interesting fact: if applied to dict with boolean values, returns only keys which values are 1
`B`D
q)where monthdays = 31 / so, where could be used as filter: return list of keys for which condition of value holds
`Jan`Mar`May
q)monthdays where monthdays = 31 /we can apply those keys back to dictionary to obtain expected result
31 31 31

q)2#monthdays / take first 2 key-value pairs as dictionary (sub-dictionary)
Jan| 31
Feb| 28
q)`Jan`Feb`Mar#monthdays / take function with list of keys returns sub-dictionary for provided keys
Jan| 31
Feb| 28
Mar| 31
q)`Feb`Mar _ monthdays /drop function will accept a list of keys, and will return dictionary without these keys
Jan| 31
Apr| 30
May| 31
Jun| 30

Tables:

q)dictX: `month`days!(`Jan`Feb`Mar`Apr`May;31 28 31 30 31) / this is called a column dictionary: dictionary which values are lists of same length
q)dictX
month| Jan Feb Mar Apr May
days | 31 28 31 30 31

q)tableX: flip dictX / this is a table. Table is a list of dictionaries with same keys for all list elements
q)tableX
month days
----------
Jan 31
Feb 28
Mar 31
Apr 30
May 31

q)tableY:((`month`days!(`Jan;31));(`month`days!(`Feb;28));(`month`days!(`Mar;31));(`month`days!(`Apr;30));(`month`days!(`May;31)))
q)tableY
month days
----------
Jan 31
Feb 28
Mar 31
Apr 30
May 31

q)tableX[1] / get table row by index, returns dictionary
month| `Feb
days | 28
q)tableX[1][`month] / you can first get row by index, then obtain values by column names
`Feb
q)3#tableX / take function returns table containing first 3 rows
month days
----------
Jan 31
Feb 28
Mar 31

q)select from tableX where month = `Mar
month days
----------
Mar 31
q)select days from tableX where month = `Mar
days
----
31
q)select days from tableX
days
----
31
28
31
30
31
q)`tableX insert (`Jun;30) / adding rows to table, accepts untyped list, returns list of inserted row numbers. Table must be referenced by symbol
,5
q)tableX
month days
----------
Jan 31
Feb 28
Mar 31
Apr 30
May 31
Jun 30

q)meta tableX / shows meta info about table
c | t f a
-----| -----
month| s
days | j

 

Categories: Uncategorized

API vs DSL

February 13, 2013 Leave a comment

Had I ever designed any domain-specific languages? Sure, many times. Like many people, I did it accidentially. As soon as I notice a function which uses one of its arguments only to dispatch control flow, something like this:

process(action, data) {
 if (action == Open) {
   open(data);
 } else if (action == Close) {
   close(data);
 } else if (action == New) {
   new();
 }
}

I know I’ve just got an interpreter. And action+data is a DSL. I don’t like dispatch code, because every branch means analysis complexity: you either need to keep a value of action in your head or track back to find out what is the value. I think that having an interpreter means that you are exposing a narrow generic interface. User of your API doesn’t have any clues on usage, and he cannot rely on compiler to check for errors. That’s why I consider interpreters as a code smell and avoid them. Good API is better than DSL.

Categories: Uncategorized

More compact format for Java classes

October 9, 2012 Leave a comment

I’ve noticed that compiled Java code takes more place then source code. For example, a simple equation solver written by me takes 15 kbytes in source and 30 kbytes in compiled form. Source code is archived to 3 kbytes, and compiled code  to 17 kbytes. Since I have written a library for reading and manipulating .class files, I’ve decided to find out why these files are so big.

For equation solver I have following results: contstant pool takes 61% of total space, and methods take 32%. So, majority of space is taken by constants. Maybe this could be optimized? There are several types of constants which contain references to other constants. These are MethodReference, ClassReference, StringReference, FieldReference, NameTypePair. Each reference takes two bytes. My program is composed of small classes, with typical constant pools consisting of 50-80 entries. So, for such classes it is possible to store references using only one byte. I’ve invented a new file format, named “slass”, which uses such technique for saving space, and implemented it in my library. Testing with abovementioned equation solver gives 7% improvement on size of constant pool, which makes a 4% improvement on size of .class file! No, it was not worth inventing.

Categories: Uncategorized

Tracing facilities and tools for Unix

May 28, 2009 3 comments

I’ve become confused with amount of tools and facilities ending with “trace” which exist on different Unix-like OSes. This small article lists them together will small description

ptrace

A system call of System V and BSD (also exists in Linux) which may be used by one process “to observe and control the execution of another process”. Available commands include reading and writing a memory of the process (used for setting breakpoints), reading CPU registers, intercepting signals sent to the process, intercepting system calls made by the process. Intercepting is done by stopping a traced process and sending a signal SIGCHILD to process which requested the tracing. Observer then may request information about stopped process and may continue its execution.

strace

A Linux utility which is used to trace system calls and signals. Uses ptrace() system call

ktrace/ktruss/kdump

A system call of BSD unixes which traces system calls and signals of specified process and writes the data in specified file. Since there is no context switch between processes, a tracing is faster. Trace data are not human-readable. A ktrace is also a name of command-line utility for unixes of BSD family which does a process tracing by invoking ktrace(). kdump is an utility which reads tracing info written into file by ktrace() and prints this info in human-readable form. ktruss is another command-line utility for BSD-derived unixes which traces system calls and signals for some process by using ktrace() system call. It prints tracing info in human-readable form on console.

systrace

A facility of some BSD unixes which is intended to restrict certain system calls done by some process. In simplest case can be used just to trace those calls. Implemented as pseudo-device “/dev/systrace”. System call ioctl() is used to perform actions such as attaching to process or permit to do a system call. System call read() on systrace device will block until some traced process will perform a system call. Systrace is also a name of command-line utility which uses systrace facility.

ltrace

A command-line utility which traces system calls and calls to functions of dynamically linked libraries. System calls are traced using ptrace(). Library calls are traced by analyzing symbol table of ELF file, calculating addresses and placing breakpoints for every library call using ptrace(). When breakpoint hits a stack trace of the process is obtained (using ptrace()) and compared with breakpoint table to get function name.

dtrace

Tracing facility originally developed for Solaris and later moved to other Unixes. It consists of core, “probe providers” which register with core, and “probe consumers” which are user-level processes like ‘dtrace’ command-line tool. Consumers can express an interest in some probe by asking a core to enable it. Core, in turn, uses provider to obtain and address.

Categories: Uncategorized

Book review: “Internet multimedia communications using SIP”

July 21, 2008 1 comment

For a long time my only source of information about SIP was RFC 3261 and others. However, recently we got a new book in a library Book is called “Internet multimedia communications using SIP” by Rogelio Martinez Perea. Now I’ve finished about 90% of the book, so I already have an opinion which I would like to share.

This book is interesting in it’s approach. It tells about lots of things, often oversimplifying them just to show only essential features.

First part is called “Fundamentals”. It explains principles of signaling/media approach, then moves to explanation of TCP/IP protocol family basics. Then on abstract level (without syntax!) it explains functionality of SIP. This part is intended for people who know absolutely nothing about networks.

Second part is called “core protocols”. First it explains basics of SIP syntax and interaction. Then it contains Java practice with free SIP stack. Next it moves to SDP, also with small Java practice. Next it moves to RTP, also with Java practice. And finally, all pieces are assembled together in soft-phone application and proxy application.

For me, such composition of the book is not very good. If somebody needs explanation of TCP/IP principles and never used sockets, then he probably will not be able to troubleshoot an application. On the other hand, advanced people like me will find code too amateurish, and most explanations as oversimplified.

Third part is called “Advanced topics” and tries to cover in small degree many (I’ve found 10) areas of SIP usage. There are no programming here, just concepts. This chapter, in my opinion, is the best in the book.

This book is quite good for students: it contains “main facts” about technology, so person will at least know that is it, and he can get deeper knowledge, if necessary. Other people who can benefit from the book are curious engineers, programmers and managers. I suspect that nobody will actually use practical part of the book.

To prove that I actually read the book and that my opinion has a base, I’m posting here a list of small annoyances, errors and mistakes I’ve found so far. Unfortunately, there are no site or e-mail address where I could send this list, so I’m posting it here in hope that author finds it useful.

  • Figure 6.15 on page 91 has an error. ACK request after 200 OK response always follows the same path as BYE request.
  • In chapter 8.6.7, an Example 4 doesn’t explain at which moment a “Dialog” object was created. There are no code, compared with Example 3, which would create a dialog. I perfectly understand that dialog is created automatically by stack, because default value of property “javax.sip.AUTOMATIC_DIALOG_SUPPORT” is “on”, so dialogs in fact are created in both cases. I believe it is worth mentioning, or replacing with creation of dialog explicitly. I believe that in this case oversimplification is bad.
  • In chapter 12.5.5 author explains that he uses TimerTask to close a dialog in case ACK is not received. Such approach is totally wrong. Actually a timeout for ACK is handled by stack, which will notify SipListener through processTimeout() method. Also, if TimerTask is intended only for ACK timeout, then a task should be cancelled if ACK is received. I believe that TimerTask should be removed at all.
  • In chapter 13.2.1, an exception #4 contains wrong information about CANCEL processing. In fact, proxy doesn’t send “487 terminated” response on INVITE. Such response will be sent by UA. Chapter 13.3.3 contains correct information about processing a CANCEL. I believe that author made this mistake because he wanted to make simpler proxy example.
  • In chapter 13.6.5 author explains proxy example. He uses ArrayList for context storage. A Map should be used instead, because code will be simpler and will work faster.

I hope that author will accept this list as a sign of respect from another professional. I believe that going to the matter is better then empty praise and stupid testimonials.

SIP gets more and more mindshare, and I fill that demand for professionals will grow. That puts me in a good position.

Categories: Uncategorized Tags: ,

Container approach in Java. Part 2: instances (version 2.0)

June 2, 2008 Leave a comment

In a previous article I’ve defined a component/container architecture and explained the reasons why such architecture is used. This article will cover real examples of containers which are known to me. in historical perspective.

Servlet containers

Earliest containers known to me are servlet containers. These are typical IoC containers, used on servers communicating asymmetric request/response protocols, like HTTP. This allows applying IoC approach to handling of protocol logic: servlets must implement “Servlet” interface containing method “service()”. This method is invoked by container for each incoming request. For HTTP, IoC principle also applies to creation and sending of responses. 

Interface “Servlet” also contains methods “init()” and “destroy()”, which are used for lifecycle management. Method “init()” also used for “context” injection. From this “context” a servlet can extract references to all components it depends on. 

Servlets are developed according to special convention, so they are collections of classes and XML descriptor.

Well-known examples of servlet containers are Apache Tomcat, Jetty, Resin.

EJB containers

EJB spec is also quite old. It is a “generic” component/container architecture for complex data-processing logic used inn enterprize IT systems. Curioulsy enough, it was developed as distributed architecture, meaning that each component will be located on dedicated machine, and container will provide inter-component communication. IoC principle is applied to lifecycle and persistence. Dependencies are resolved using lookup (in JNDI facility), however binding in JNDI is done automatically.

Later EJB spec changed to support “local” access between components. Anyway, it was and still criticised for being complex and slow, and many other frameworks emerged to fix its flaws.

Well-known examples of EJB containers are: Glassfish, Apache OpenEJB, JBoss, JOnAs, Bea WebLogic, IBM websphere, and lots of others.

Microcontainers

Microcontainers are “generic” component containers focusing only on local access and providing just lifecycle and late binding. All other features could be realized “on top” of container, by implementing them as a components. 

Apache Avalon was first known to me attempt to build “lightweight” container. Later its developers divided, but they have tried to support common framework for containers. This framework follows “interface injection” approach, which means that dependency on something is declared by implementing certain interface, and the same interface is used for injecting this dependency. Thus, Avalon framework contains lots of interfaces, for example for lifecycle, logging, configuration. However, injection is used only for dependencies which are part of framework. For resolving other dependencies there is also lookup facility.

Apache Hivemind is another minimalistic container, with IoC principles applied to lifecycle, configuration and automatic dependency injection, although lookup is also supported. This container is best classified as “declarative de-centralized”. It uses a special format for its components (code + XML-based descriptor). Dependency injection is supported through components following a naming convention.

Other examples of microcontainers are: Picocontainer, Butterfly, Guice.

Spring

Spring framework is a set of components aimed to be simple, lightweight and cheap alternatives to  all parts of JEE. As a a replacement of EJB it provides a much simpler IoC container. However, this container is often used not just for binding user components together, but also for binding them with system components. This was new at the time, because JEE application servers didn’t allow to customize the “system” part. Spring container is not just “business logic integration point”, but whole “application integration point”.

IoC principle in Spring could be applied to lots of concerns, including lifecycle, dependency resolving via injection, configuration. However, usage of IoC is not mandatory and could be avoided, but it will make whole arcitecture less consistent.

One interesting application of IoC principle used Spring is “aspect-oriented programming”: a container “wraps” modules with its own “proxy”, and injects this proxy into dependent modules. This “proxy” allows inserting some functionality before invocation and after invocation, so some component can affect interaction between two other modules without modifying them.

Spring is very popular, because it provides a large base for building custom server-side software, either complex or simple.

 JEE application servers

After success of Spring vendors of many JEE application servers understood that their products should be more customizable, so those servers were re-designed as microcontainers. Apache Geronimo is a microcontainer allows deployment of components called “GBeans”. JBoss also implements microcontainer architecture. Glassfish and JOnAs stated that they will move to OSGi.

Apache Geronimo is an example of “embedded container” architecture: EJB container (OpenEJB) is itself a component in another container (Geronimo microcontainer).

OSGi

The distinguishing feature of OSGi framework is a complex classloading. This allows “hot upgrade” through dynamic loading and unloading of classes, and other interesting capabilities. IoC principle is applied only to lifecycle. As a separate component there is a “service facility” which could be used for basic dependency lookup, and also allows subscription to notifications about lifecycle events in other services. There is also component for dependency injection called “Service Binder” based on XML descriptors.

There are several implementations of OSGi framework, including Eclipse Equinox and Apache Felix.

JSLEE

JSLEE is another non-generic component architecture for Java. It is similar to EJB in a way in which IoC principle is applied to persistence and lifecycle. JSLEE includes scalable event-delivery facility, and IoC principle is appled to some aspects of interfaction between components and event-delivery facility. could be viewed as a combination of Servlet Container and EJB container. Dependencies are resolved using lookup (in JNDI).