Venish Joe

Dynamic Columns in JSP Model with Struts Framework

2010-04-18T21:41:00.002-04:00

While developing web applications, we will come across a lot of scenarios where we should use dynamic columns depending on the entitlement of the user or from the result set depending on the data itself. Dynamic columns in web applications give control to the user on what he wants to see (specially when we have a huge amount of data to work with). For applications with sensitive data, this can be even extended to act as a security layer where the access to specific data can be controlled with high precision.

In this article, I will explain one of the methods to implement this in any J2EE application with little or no code change.

High Level Architecture

In a nut shell, this design uses the Application Context of the container to maintain the values pertaining to a particular user, if the requirement demands not to maintain the preference after a user session is terminated, then it can be achieved by destroying the object stored in the context.

We will start by creating a singleton class, which will be used to store the user preference about the columns. The user preference object can be mapped against the user id or any other primary key, so that different preferences are maintained for different users. While the container starts, the instance for the singleton will be created. The default preferences can be loaded from a property / xml file or from a data store (DB). This object will contain the preferences of different pages with the different key names so that the same object can be used to maintain the preference across the application. This will be read during the logon operation and if the object in the application context doesn't contain any values (if the user logs in for the first time or in an application where the preference is specific to the session) then the default values are loaded. Once the page loads, the preference can be read from the application context and can be presented to the user. If the user edits his preferences, it will be updated in the application context. Note that the application context is not persistent between container restarts, so appropriate mechanisms should be taken to store the data.

Implementation

Let’s go through the implementation now. The following steps describe how to integrate this component to an existing Struts application.

Start-up servlet and initializing the Singleton Class

Create an Initializer Servlet and make an entry for the same in the web.xml file so that the Servlet starts when the container is initialized. Make sure the load-on-startup is set to 1, which ensures that the application server loads the servlet while startup.

<servlet> 
  <servlet-name> InitializerServlet </servlet-name> 
  <servlet-class>com.startup.common.InitializerServlet</servlet-class> 
  <load-on-startup> 1 </load-on-startup> 
</servlet>

Next, create a Singleton class, which contains getter and setter methods for dynamic column preference, the object can be any collection, we are using Hash Map in this example which will be used to store the primary key against the list containing the preference. The set & get methods in the Singleton should be synchronized so that the simultaneous access is restricted. Also override the clone() method in your singleton.

public class AppSingleton implements Serializable {
	private Hashtable cusomizeViewValues = null; 
	private static AppSingleton appSingleton = null; 
	
	private AppSingleton (){ }
	
	public synchronized void setCusomizeViewValues (Hashtable cusomizeViewValues){ 
		this.cusomizeViewValues = cusomizeViewValues; 
	}
	
	public static synchronized AppSingleton getInstance () throws Exception { 
		try { 
			if (appSingleton == null)
				return new AppSingleton (); 
          } catch (Exception e) { 
			throw new Exception(); 
		} 
	}

	public Object clone() throws CloneNotSupportedException{ 
		throw new CloneNotSupportedException(); 
	} 
}

In the startup servlet, create an instance of the singleton class. When created, the object will be available in the application context of the container, and no one will be able to create another instance, until the object created in startup is destroyed. Since we have overridden the clone method, no one will be able to clone the particular object. These measures are to ensure the integrity of the user preference stored in the singleton. A sample Initializer servlet will look like the following code.

public class InitializerServlet extends HttpServlet { 
	public void init () throws ServletException { 
		AppSingleton appSingleton = AppSingleton.getInstance (); 
	} 

	public void destroy (){} 
	public void service (HttpServletRequest request, HttpServletResponse response) 
		throws ServletException, IOException {} 
}

Now, create a Data Transfer Object (DTO) / Value Object (VO) for storing the values. The VO/DTO will contain just two getters and setters, one for the Column Display name and the other for the bean property. This will be a POJO.

Populate the Application context

When the container starts, populate the list in the application context, from the property file or from the data source. If you are having a separate page to choose the columns displayed, you can use the same list to render the values initially. Similarly if the user has changed his preference then update the application context accordingly. This can be done during the Login Action, once the user is authorized and authenticated. You can use your own logic to get all the user preference and then update list with DTO’s/VO’s containing the display name and the property name. This list is updated in the application context against the primary key. Before updating the application context check if the PK is already present in the Hash Table if yes, update or create a new entry.

A sample property file will look like the one given below. By using different keys, we can have entries for different pages. Also the columns to be displayed to the user irrespective of the individual preference can also be marked here under a different key. The columns users are not allowed to modify are added to the rendering list once the request is got from the particular page and not during the logon time. The values are appended to the modifiable columns list and rendered to the user.

Validations$Optional=Plan #, Plan Name, Administrator  
Validations$Core= Plan Val Description, Plan Val Status 
 
# Optional represents the Columns users can modify 
# Core represents the Columns users can’t modify 
 
Validations$Plan#=strClientExtEntityId 
Validations$PlanName=strPlanName 
Validations$Administrator=strAdministrator 
Validations$PlanValDescription=strDescription 
Validations$PlanValStatus=strStatus

Rendering Logic

Once the values are available in the session, using JSP, logic iterate, render the Column names. Then to display the values from the result set, use the logic iterate with the list containing the values for the page, which is used to render the <tr> tag and inside that logic iterate, use another logic iterate, which is used to render the columns and use a bean define tag to get the column name properties in a scriplet variable and then use a bean define tag to display the value of the property. This logic is highly dynamic.

To display the column names,

<logic:iterate name="<Form Bean Name>" id="testId" 
		property="<Name of the List>" > 
	<td> 
		<bean:write name=" testId " property="<Col Disp Name>"/> 
	</td> 
</logic:iterate>

To display the result set,

<logic:iterate name="<Form Name>" id="outerId"
		 property="<Property of the Hitlist>"> 
	<tr> 
		<logic:iterate name="<Form Name>" id="innerId" 
			property="<Name of the List>" > 
		<bean:define name="innerId" id="propId"
			 property="<Col Property>" type="String"/> 
			<td> 
				<bean:write name="outerId" property="<%= propId %>" /> 
			</td> 
		</logic:iterate> 
	</tr> 
</logic:iterate>

We also will have situations to display hyperlinks, textboxes etc. in the result set, the same logic can be used to display the different objects in the JSP. Just before the bean write tag, have a logic equal tag to check for specific types and render the display.

This architecture is highly customizable and can be easily plugged in into any existing J2EE application. Also this can be easily enhanced to incorporate new functionalities.

Java String Concatenation and Performance

2009-11-08T18:01:00.002-05:00

The quick and dirty way to concatenate strings in Java is to use the concatenation operator (+). This will yield a reasonable performance if you need to combine two or three strings (fixed-size). But if you want to concatenate n strings in a loop, the performance degrades in multiples of n. Given that String is immutable, for large number of string concatenation operations, using (+) will give us a worst performance. But how bad ? How StringBuffer, StringBuilder or String.concat() performs if we put them on a performance test ?. This article will try to answer those questions.

We will be using Perf4J to calculate the performance, since this library will give us aggregated performance statistics like mean, minimum, maximum, standard deviation over a set time span. In the code, we will concatenate a string (*) repeatedly 50,000 times and this iteration will be performed 21 times so that we can get a good standard deviation. The following methods will be used to concatenate strings.

Concatenation Operator (+)
String concat method - concat(String str)
StringBuffer append method - append(String str)
StringBuilder append method - append(String str)

And finally we will look at the byte code to see how each of these operations perform. Let’s start building the class. Note that each of the block in the code should be wrapped around the Perf4J library to calculate the performance in each iteration. Let’s define the outer and inner iterations first.

private static final int OUTER_ITERATION=20;
private static final int INNER_ITERATION=50000;

Now let’s implement each of the four methods mentioned in the article. Nothing fancy here, plain implementations of (+), String.concat(), StringBuffer.append() & StringBuilder.append().

String addTestStr = "";
String concatTestStr = "";
StringBuffer concatTestSb = null;
StringBuilder concatTestSbu = null;

for (int outerIndex=0;outerIndex<=OUTER_ITERATION;outerIndex++) {
    StopWatch stopWatch = new LoggingStopWatch("StringAddConcat");
    addTestStr = "";
    for (int innerIndex=0;innerIndex<=INNER_ITERATION;innerIndex++)
 addTestStr += "*";
    stopWatch.stop();
}        

for (int outerIndex=0;outerIndex<=OUTER_ITERATION;outerIndex++) {
    StopWatch stopWatch = new LoggingStopWatch("StringConcat");
    concatTestStr = "";
    for (int innerIndex=0;innerIndex<=INNER_ITERATION;innerIndex++)
 concatTestStr = concatTestStr.concat("*");
    stopWatch.stop();
}

for (int outerIndex=0;outerIndex<=OUTER_ITERATION;outerIndex++) {
    StopWatch stopWatch = new LoggingStopWatch("StringBufferConcat");
    concatTestSb = new StringBuffer();
    for (int innerIndex=0;innerIndex<=INNER_ITERATION;innerIndex++)
 concatTestSb.append("*");
    stopWatch.stop();
}

for (int outerIndex=0;outerIndex<=OUTER_ITERATION;outerIndex++) {
    StopWatch stopWatch = new LoggingStopWatch("StringBuilderConcat");
    concatTestSbu = new StringBuilder();
    for (int innerIndex=0;innerIndex<=INNER_ITERATION;innerIndex++)
 concatTestSbu.append("*");
    stopWatch.stop();
}

Let’s run this program and generate the performance metrics. I ran this program in a 64-bit OS (Windows 7), 32-bit JVM (7-ea), Core 2 Quad CPU (2.00 GHz) with 4 GB RAM.

The output from the 21 iterations of the program is plotted below.

Well, the results are pretty conclusive and as expected. One interesting point to notice is how better String.concat performs. We all know String is immutable, then how the performance of concat is better. To answer the question we should look at the byte code. I have included the whole byte code in the download package, but let’s have a look at the below snippet.

45: new #7; //class java/lang/StringBuilder
48: dup
49: invokespecial #8; //Method java/lang/StringBuilder."<init>":()V
52: aload_1
53: invokevirtual #9; //Method java/lang/StringBuilder.append:
    (Ljava/lang/String;)Ljava/lang/StringBuilder;
56: ldc #10; //String *
58: invokevirtual #9; //Method java/lang/StringBuilder.append:
    (Ljava/lang/String;)Ljava/lang/StringBuilder;
61: invokevirtual #11; //Method java/lang/StringBuilder.toString:()
    Ljava/lang/String;
64: astore_1

This is the byte code for String.concat(), and its clear from this that the String.concat is using StringBuilder for concatenation and the performance should be as good as String Builder. But given that the source object being used is String, we do have some performance loss in String.concat.

So for the simple operations we should use String.concat compared to (+), if we don’t want to create a new instance of StringBuffer/Builder. But for huge operations, we shouldn’t be using the concat operator, as seen in the performance results it will bring down the application to its knees and spike up the CPU utilization. To have the best performance, the clear choice is StringBuilder as long as you do not need thread-safety or synchronization.

The full source code, compiled class & the byte code is available for download in the below link.

Download Source, Class & Byte Code: String_Concatenation _Performance.zip

Recursive File Tree Traversing in Java using NIO.2

2009-10-26T11:45:00.001-04:00

In my previous article about NIO.2, we have seen how to implement a service which monitors a directory recursively for any changes. In this article we will look at another improvement in JDK7 (NIO.2) called FileWatcher. This will allow us to implement a search or index. For example, we can find all the *some_pattern* files in a given directory recursively and (or) delete / copy all the all the *some_pattern* files in a file system. In a nutshell FileWatcher will get us a list of files from a file system based on a pattern which can be processed based on our requirement.

The FileVistor is an interface and our class should implement it. We have two methods before the traversal starts at the directory level & file level, and one method after the traversal is complete, which can be used for clean up or post processing. The important points from the interface is given in the below diagram.

While I think FileVistor is the best way to handle this, JDK7 NIO.2 has given another option to achieve the same, a class named SimpleFileVistor (which implements FileVisitor). It should be self explanatory, a simplified version of FileVisitor. We can extend the SimpileFileVisitor into our class and then traverse the directory with overriding only the methods we need, and if any step fails we will get an IOException.

According to me, FileVisitor is better because it forces you to implement the methods (sure, you can leave them blank) since these methods are really important if you plan to implement recursive delete / copy or work with symbolic links. For example, if you are copying some files to a directory you should make sure that the directory should be created first before copying which can be done in the preVisitDirectory().

The other area of concern is symbolic links and how this will be handled by FileVisitor. This can be achieved using FileVisitOption enums. By default, the symbolic links are not followed so that we are not accidentally deleting any directories in a recursive delete. If you want to handle manually, there are two options FOLLOW_LINKS (follow the links) & DETECT_CYCLES (catch circular references).

If you want to exclude some directory from FileVisitor or if you are looking for a directory or a file in the file system and once you find it you want to stop searching that can be implemented by using the return type of FileVisitor, called FileVisitResult. SKIP_SUBTREE allows us to skip directories & subdirectories. TERMINATE stops the traversing.

The search can be initiated by the walkFileTree() method in Files class. This will take the starting directory (or root directory in your search) as a parameter. You can also define Integer.MAX_VALUE if you want to manually specify the depth. And as mentioned in the above diagram, define FileVisitOption for symbolic links if needed.

Enough with the API description, let's write some sample code to implement what we discussed. We will be using the SimpleFileVisitor so that in our demo we don’t need to implement all the methods.

Let’s start with defining the pattern which needs to be searched for. In this example, we will search for all the *txt file / directory names recursively in any given directory. This can be done with getPathMatcher() in FileSystems

PathMatcher pathMatcher = FileSystems.getDefault().getPathMatcher("glob:" + "*txt*");

Now, let’s initiate the search by calling walkFileTree() as mentioned below. We are not defining anything specific for symbolic links so, by default its NO_FOLLOW.

Files.walkFileTree(Paths.get("D://Search"), fileVisitor);

Let’s go through the implementations of class SimpleFileVisitor, we will be overriding only visitFile() & preVisitDirectory() in this example, but its a good practice to override all the five methods so that we have more control over the search. The implementation is pretty simple, based on the pattern the below methods will search for a directory or file and print the path.

@Override
public FileVisitResult visitFile(Path filePath, BasicFileAttributes basicFileAttributes) {        
if (filePath.getName() != null && pathMatcher.matches(filePath.getName()))
    System.out.println("FILE: " + filePath);
return FileVisitResult.CONTINUE;
}

@Override
public FileVisitResult preVisitDirectory(Path directoryPath) {
if (directoryPath.getName() != null && pathMatcher.matches(directoryPath.getName()))
    System.out.println("DIR: " + directoryPath);
return FileVisitResult.CONTINUE;
}

Once this is completed, we can use the postVisitDirectory() to perform additional tasks or any cleanup if needed. A sample output from my machine is given below.

The complete source code is given below. Please note that you need JDK7 to run this code. I have also given a link to download the compiled class along with source.

Download Source & Class: NIO2_FileVisitor.zip

Complete Source Code.

public class NIO2_FileVisitor extends SimpleFileVisitor<Path> {
    private PathMatcher pathMatcher;
    
    @Override
    public FileVisitResult visitFile(Path filePath, 
		BasicFileAttributes basicFileAttributes) {        
        if (filePath.getName() != null && 
		pathMatcher.matches(filePath.getName()))
            System.out.println("FILE: " + filePath);
        return FileVisitResult.CONTINUE;
    }

    @Override
    public FileVisitResult preVisitDirectory(Path directoryPath) {
        if (directoryPath.getName() != null && 
		pathMatcher.matches(directoryPath.getName()))
            System.out.println("DIR: " + directoryPath);
        return FileVisitResult.CONTINUE;
    }

    public static void main(String[] args) throws IOException {
        NIO2_FileVisitor fileVisitor = new NIO2_FileVisitor();
        fileVisitor.pathMatcher = FileSystems.getDefault().
		getPathMatcher("glob:" + "*txt*");
        Files.walkFileTree(Paths.get("D://Search"), fileVisitor);
    }
}

Dynamically Load Compiled Java Class as a Byte Array and Execute

2009-10-21T21:20:00.004-04:00

As we know, all the compiled java classes runs inside the JVM. The default class loader from Sun loads the classes into JVM and executes it. This class loader is a part of JVM which loads the compiled byte code to memory. In this article, I will show how to convert a compiled java class to a array of bytes and then load these array of bytes into another class (which can be over the network) and execute the array of bytes.

So the question arises, why should we write a custom class loader ? There are some distinct advantages. Some of them below

We can load a class over any network protocol. Since the java class can be converted to a series of numbers (array of bytes), we can use most of the protocols.
Load Dynamic classes based on the type of user, especially useful when you want to validate the license of your software over the web and if you are paranoid about the security.
More flexible and secure, you can encrypt the byte stream (asymmetric or symmetric) ensuring safer delivery.

For this article we will be creating three classes

JavaClassLoader – The custom class loader which will load the array of bytes and execute. In other words, the client program.
Class2Byte – The Java class which converts any compiled class / object to a array of bytes
ClassLoaderInput – The class which will be converted to array of bytes and transferred

Let’s divide this article into two sections, in the fist section we will convert the java class to array of bytes and in the second section, we will load that array.

Create & Convert the Java class to array of bytes

Let’s write a simple class (ClassLoaderInput) which just prints a line. This is the class which will be converted to a byte array.

public class ClassLoaderInput {
	public void printString() {
		System.out.println("Hello World!");
	}
}

Now, let’s write another class (Class2Byte) which will convert the ClassLoaderInput to a byte of array. The concept to convert the file is simple, compile the above file and load the class file through input stream and with an offset read and convert the class to bytes and write the output in to another out stream. We need these bytes as a comma separated value, so we will use StringBuffer to add comma between the bytes.

int _offset=0;
int _read=0;

File fileName = new File(args [0]);
InputStream fileInputStream = new FileInputStream(fileName);
FileOutputStream fileOutputStream = new FileOutputStream(args[1]);
PrintStream printStream = new PrintStream(fileOutputStream);
StringBuffer bytesStringBuffer = new StringBuffer();

byte[] byteArray = new byte[(int)fileName.length()];
while (_offset < byteArray.length && 
	(_read=fileInputStream.read(byteArray, _offset, 
	byteArray.length-_offset)) >= 0)
    _offset += _read;    

fileInputStream.close();
for (int index = 0; index < byteArray.length; index++)
    bytesStringBuffer.append(byteArray[index]+",");

printStream.print(bytesStringBuffer.length()==0 ? "" : 
	 bytesStringBuffer.substring(0, bytesStringBuffer.length()-1));

Now let’s run this file and generate the output. A sample output from my machine is below.

Now,we have the sample class (ClassLoaderInput) file as a bunch of numbers. Now this bunch of numbers can be transferred over any protocol to our custom class loader which will “reconstruct” the class from these bytes and run it, without any physical trace in the client machine (the array of bytes will be on memory).

Load the array of bytes and execute

Now, to the important part of this article, we are going to write a custom class loader which will load those bunch of numbers (array) and execute them. The array of bytes can be transferred over the network but in this example, we will define it as a string in the class loader for demonstration purpose.

Let’s start by defining the array of bytes.

private int[] data = {-54,-2,-70,-66,0,0,0,51,0,31,10,0,6,0,17,9,0,18,0,19,8,
0,20,10,0,21,0,22,7,0,23,7,0,24,1,0,6,60,105,110,105,116,62,1,0,3,40,41,86,1,
0,4,67,111,100,101,1,0,15,76,105,110,101,78,117,109,98,101,114,84,97,98,108,
101,1,0,18,76,111,99,97,108,86,97,114,105,97,98,108,101,84,97,98,108,101,1,0,
4,116,104,105,115,1,0,18,76,67,108,97,115,115,76,111,97,100,101,114,73,110,
112,117,116,59,1,0,11,112,114,105,110,116,83,116,114,105,110,103,1,0,10,83,
111,117,114,99,101,70,105,108,101,1,0,21,67,108,97,115,115,76,111,97,100,101,
114,73,110,112,117,116,46,106,97,118,97,12,0,7,0,8,7,0,25,12,0,26,0,27,1,0,
12,72,101,108,108,111,32,87,111,114,108,100,33,7,0,28,12,0,29,0,30,1,0,16,67,
108,97,115,115,76,111,97,100,101,114,73,110,112,117,116,1,0,16,106,97,118,97,
47,108,97,110,103,47,79,98,106,101,99,116,1,0,16,106,97,118,97,47,108,97,110,
103,47,83,121,115,116,101,109,1,0,3,111,117,116,1,0,21,76,106,97,118,97,47,105,
111,47,80,114,105,110,116,83,116,114,101,97,109,59,1,0,19,106,97,118,97,47,105,
111,47,80,114,105,110,116,83,116,114,101,97,109,1,0,7,112,114,105,110,116,108,
110,1,0,21,40,76,106,97,118,97,47,108,97,110,103,47,83,116,114,105,110,103,59,
41,86,0,33,0,5,0,6,0,0,0,0,0,2,0,1,0,7,0,8,0,1,0,9,0,0,0,47,0,1,0,1,0,0,0,5,42,
-73,0,1,-79,0,0,0,2,0,10,0,0,0,6,0,1,0,0,0,1,0,11,0,0,0,12,0,1,0,0,0,5,0,12,0,
13,0,0,0,1,0,14,0,8,0,1,0,9,0,0,0,55,0,2,0,1,0,0,0,9,-78,0,2,18,3,-74,0,4,-79,
0,0,0,2,0,10,0,0,0,10,0,2,0,0,0,3,0,8,0,4,0,11,0,0,0,12,0,1,0,0,0,9,0,12,0,13,
0,0,0,1,0,15,0,0,0,2,0,16};

The conversion of these bytes to class is done by the ClassLoader.defineClass() method We should supply the stream of bytes that make up the class data. The bytes in positions off through off+len-1 should have the format of a valid class file as defined by the Java Virtual Machine Specification. The offset and length will be the additional parameters. Once the defineClass converts the array to class, then we can use reflection to execute the methods in the class.

JavaClassLoader _classLoader = new JavaClassLoader();        
byte[] rawBytes = new byte[_classLoader.data.length];
for (int index = 0; index < rawBytes.length; index++)
    rawBytes[index] = (byte) _classLoader.data[index];
Class regeneratedClass = _classLoader.defineClass(args[0], 
	rawBytes, 0, rawBytes.length);
regeneratedClass.getMethod(args[1], null).invoke(null, new Object[] { args });

Now, let’s compile the class loader and run. The the class file name & method name should be passed as a run time argument. If you have done everything right, you should see the output from the input class which we created (ClassLoaderInput) initially. Sample output from my machine below.

The precompiled classes and the source code can be downloaded from the below location.

Download Source & Class: Java_Dynamic_Class_Byte_Array.zip

Monitor a Directory for Changes using Java

2009-10-18T02:23:00.002-04:00

Many applications which we use on a day to day basis like a music organizer, file editors monitor the directory for any changes in the files/directories and take appropriate action in the application if there are any changes detected on the fly. Since Java do not have direct access to the system level calls (unless we use JNI, which will make the code platform specific) the only way to monitor any directory is to use a separate thread which will be using a lot of resources (memory & disk I/O) to monitor the changes inside the directory. If we have sub-directories and need a recursive monitor, then the thread becomes more resource intensive.

There was a JSR (Java Specification Request) requested to add / rewrite more I/O APIs for Java platform. This was implemented in JDK 7 as JSR 203 with support for APIs like file system access, scalable asynchronous I/O operations, socket-channel binding and configuration, and multicast datagram's.

JSR 203 is one of the big feature for JDK 7 (Developer Preview is available in java.sun.com) and its been implemented as the second I/O package is java, called as NIO.2. I will be looking into more of these packages in future posts, but in this, I will show how to monitor a directory and its sub-directories for any changes using NIO.2 (JDK 7).

The APIs which we will be using WatchService (A watch service that watches registered objects for changes and events), WatchKey (A token representing the registration of a watchable object with a WatchService) & WatchEvent (An event or a repeated event for an object that is registered with a WatchService) to monitor a directory. So, without further explanation, let’s start working on the code.

Please note that you need JDK 7 to run this program. While writing this post, JDK 7 is available as a EA (Early Access) in Java Early Access Downloads page. Download the JDK and install it.

The first step is to get a directory to monitor. Path is one of the new I/O API as a part of NIO.2 which gives us more control over the I/O. So let’s get the directory to watch, if you want to watch the directory recursively then there should be another boolean flag defined, but in this example we will watch only the parent directory.

Path _directotyToWatch = Paths.get(args[0]);

Now let’s create a Watch service to the above directory and add a key to the service. In the watch key we can define what are all the events we need to look for. In this example we will monitor Create, Delete & Rename/Modify of the files or directories in the path.

WatchService watcherSvc = FileSystems.getDefault().newWatchService();
WatchKey watchKey = _directotyToWatch.register(watcherSvc, 
	ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);

Now we have all the variables defined. Let’s start a infinite loop to monitor the directory for any changes using WatchEvent. We will poll events in the directory and once some event is triggered (based on the WatchKey definition) we will print the type of event occurred and the name of the file/directory on which the event occurred. Once done, we will reset the watch key.

while (true) {
    watchKey=watcherSvc.take();
    for (WatchEvent<?> event: watchKey.pollEvents()) {
        WatchEvent<Path> watchEvent = castEvent(event);
        System.out.println(event.kind().name().toString() + " " 
		+ _directotyToWatch.resolve(watchEvent.context()));
        watchKey.reset();
    }
}

Now to make the WatchEvent <Path> work, we should create a small utility as below ( this is the castEvent which is used in the above code).

static <T> WatchEvent<T> castEvent(WatchEvent<?> event) {
    return (WatchEvent<T>)event;
}

Now compile the file and give a directory as a runtime parameter while running it. Once the program starts running, start creating some directories/files or modify/rename some files in the directory which you gave as a parameter, the program will start triggering the event and you should be able to watch the modifications in the console. A sample output from my machine is below.

The full source code of the application is given below. You can also download the compiled class and code.

Download Source & Class: JSR203_NIO2_WatchFolder.zip

import java.nio.file.*;
import static java.nio.file.StandardWatchEventKind.*;

static <T> WatchEvent<T> castEvent(WatchEvent<?> event) {
    return (WatchEvent<T>)event;
}

public static void main (String args[]) throws Exception {
    Path _directotyToWatch = Paths.get(args[0]);
    WatchService watcherSvc = FileSystems.getDefault().newWatchService();
    WatchKey watchKey = _directotyToWatch.register(watcherSvc, 
	ENTRY_CREATE, ENTRY_DELETE, ENTRY_MODIFY);

    while (true) {
        watchKey=watcherSvc.take();
        for (WatchEvent<?> event: watchKey.pollEvents()) {
            WatchEvent<Path> watchEvent = castEvent(event);
            System.out.println(event.kind().name().toString() + " " 
		+ _directotyToWatch.resolve(watchEvent.context()));
            watchKey.reset();
        }
    }
}

Signing Java Objects for Secure Transfer

2009-10-13T22:45:00.004-04:00

In distributed J2EE applications or in any application where you need to transfer Java objects to another system then there is always a security risk where the object can be intercepted which can result in data theft/loss. Especially in Serialization, (where the object is a physical file in the native file system) when the serialized Java objects are sent through the network, whoever knows the type of the object can always read it.

In this article, we will build two simple applications, one which generates the object, the keys (public & private) and signs the object with the private key. Other application which verifies the signed object in other end over the network or another application in the same machine. Both these apps can run independently in different machines. For signing the object we will be using Public-Key cryptography. This is one of the most widely used standards to sign data along with DSA & SHA1PRNG (cryptographically strong pseudo-random number generator (PRNG)). Public-Key cryptography is a asymmetric key algorithm, where the key used to encrypt a message is not the same as the key used to decrypt it.

This is the class diagram of the applications which we will be building. This article will be divided into two parts, the first part we will sign the object (serialized) and in the second part, we will verify it.

1. Sign the Java Object

First of all we need a class which will generate a public & private key. We will create a class named SecurityUtil which will generate those based on DSA (we can use RSA or any other algorithm as long as its available) and we will generate a cryptographically strong pseudo-random number generator (PRNG) which can be clubbed along with DSA (SHA1PRNG). The strength of the key will be 1024.

  protected KeyPair generateKey () throws Exception {
    KeyPairGenerator keyPairGen = KeyPairGenerator.getInstance("DSA");
    SecureRandom secureRandom = SecureRandom.getInstance("SHA1PRNG");

    keyPairGen.initialize(1024,secureRandom);
    KeyPair keyPair = keyPairGen.generateKeyPair();

    return keyPair;
  }

Next we will create a class named EmployeeValueObject which is nothing but a POJO with a HashMap getter/setter. This will be the object which we will be transferring over the network/application. Since we serialize the object before transferring, this class should implement Serializable.

public class EmployeeValueObject implements Serializable {
    HashMap employeeSalary = new HashMap();

    public void setSalary (HashMap employeeSalary){
        this.employeeSalary = employeeSalary;
    }

    public HashMap getSalary () {
        return employeeSalary;
    }
}

Now we have all the supporting classes which we need and let’s start building the main application. Let’s call this class EmployeeDetails and this will create an object for the POJO which we created in our previous step and populate with some data. In addition to that, we will sign the POJO object and then serialize to a file. In this example we will be also serializing the public key to transfer to the other end. (Note: In production implementations, both these objects shouldn’t be sent at the same time. The application at the other end should already have the public key)

Let’s create the POJO and populate with some data in the HashMap.

EmployeeValueObject employeeVO = new EmployeeValueObject();
employeeVO.setSalary(populateData());

private static HashMap populateData (){
    HashMap employeeSalary = new HashMap ();
    employeeSalary.put("3", "Johns, Galvin D. --> $18,000");
    employeeSalary.put("4", "Weber, Murphy I. --> $5,000");

    return employeeSalary;
}

Now let’s generate the public & private keys from SecutityUtil and sign the POJO which we created in the above step.

KeyPair keyPair = new SecurityUtil().generateKey();
PrivateKey privateKey = keyPair.getPrivate();
PublicKey publicKey = keyPair.getPublic();

Signature digitalSignature = Signature.getInstance(privateKey.getAlgorithm());
SignedObject digitalSignedObj = 
	new SignedObject(employeeVO, privateKey, digitalSignature);

Now digitalSignedObj is a digitally signed data with the private key which we generated. Now let’s serialize this object for the secure transfer.

FileOutputStream serializedFileOutput = new FileOutputStream("employee.ser");
ObjectOutputStream serializedObjOutput = new ObjectOutputStream(serializedFileOutput);
serializedObjOutput.writeObject(digitalSignedObj);
serializedObjOutput.close();
serializedFileOutput.close();

We will also serialize the public key so that for this example we can send both of them to another machine to verify. (Note: In production implementations, both these objects shouldn’t be sent at the same time. The application at the other end should already have the public key)

serializedFileOutput = new FileOutputStream("publickey.ser");
serializedObjOutput = new ObjectOutputStream(serializedFileOutput);
serializedObjOutput.writeObject(publicKey);
serializedObjOutput.close();
serializedFileOutput.close();

This will complete the creation of application one. When you run this application, it will create two new files in the same directory. employee.ser – which is the signed & serialized POJO (Salary details) & publickey.ser – public key to verify the POJO. Now using the appropriate protocol send these files to the other application (remote or local) and let’s start building the verification part.

2. Verification & De-Serializing the Java Object

As a start we have the files employee.ser & publickey.ser. Let’s start building up the class to verify & de-serialize these files. Let’s name this class DecryptEmployee. The following code should de-serialize the objects.

FileInputStream serializedPublicKeyIn = new FileInputStream("publicKey.ser");
ObjectInputStream serializedPublicKey = new ObjectInputStream(serializedPublicKeyIn);
PublicKey publicKey = (PublicKey) serializedPublicKey.readObject();

FileInputStream serializedEmployeeIn = new FileInputStream("employee.ser");
ObjectInputStream serializedEmployee = new ObjectInputStream(serializedEmployeeIn);
SignedObject digitalSignedObj = (SignedObject) serializedEmployee.readObject();

Since the public key was not signed, publicKey variable will be readable. But the employee POJO was signed, so we are reading the object as a SignedObject. Let’s move forward and verify this.

Signature digitalSignature = Signature.getInstance(publicKey.getAlgorithm());

boolean decryptFlag = digitalSignedObj.verify(publicKey, digitalSignature);

The decryptFlag contains the status of the verification. If the public key is incorrect or if the object was tampered, then this will return false and we won’t be able to verify the object. If its true then everything looks good and we can successfully verify the POJO and print the values from HashMap.

if(decryptFlag) {
    EmployeeValueObject employeeVO = (EmployeeValueObject) digitalSignedObj.getObject();
    HashMap employeeSalary = (HashMap) employeeVO.getSalary();
    Collection collHashMap = employeeSalary.values();
    Iterator collectionIterator = collHashMap.iterator();
     while (collectionIterator.hasNext()) {
	System.out.println(collectionIterator.next());
    }
} else {
    System.out.println ("Decryption Failed. Please check the Keys.");
}

If you run this application, we will get an output similar to below.

This can be used in any sensitive application to make sure that the objects which are transferred over the network are safe.

UPDATE: As mentioned by Salman A in the comments, the SignedObject signs the object, it doesn't encrypt it. So if you need encryption, you can use the Cipher class in Java.

Recover from Out of Memory Errors in Java

2009-10-11T01:15:00.003-04:00

In Java, as any other programming language there is a restriction of amount of memory any program can use. In languages like C the memory is limited to amount of RAM the operating system allocates to the applications or the user space. Since Java applications are technically running in the Java Virtual Machine (JVM), the applications have memory allocated by the JVM. Due to that we can start java applications with the amount of memory we need by using the –Xms & –Xmx command line parameters (Given the JVM has the memory to spare).

In Java, the memory allocation is handled by JVM. We create the objects and JVM decides where to keep those objects in heap. In languages like C we can use methods like malloc to dynamically allocate memory for objects (variables) and when we are done, the appropriate clean up methods should be called to release the memory space. There are advantages in each method, and discussing them will take another separate post.

In this post, I am going to explain how to recover from the Out of Memory errors. When a Java class faces out of memory errors, we should try to recover from those errors by reducing or blocking the service rather than crashing the application. In many cases when the application crashes due to out of memory, JVM is also impacted (which may be running other applications).

You can ask why can’t we catch the exception. First of all, catching the exception means that the error has already occurred and in complex systems letting the error happen will be costly. This method will prevent you from the error itself. When the memory is lower than the defined threshold, the code is not even executed and we are taking recovery measures to free up some memory. Also if your application is using all the memory allocated to JVM, when out of memory error occurs it will end up crashing JVM so there is no point catching the exception since your application would have been crashed.

To recover from the out of memory errors, we need to simulate the error first and then find a solution to recover. So this post consists of two major parts.

1. Simulate Out of Memory Error

Simulating the error is pretty simple, we will create a lot of array objects in a loop for which JVM will allocate memory spaces. Once the objects reaches a threshold, the JVM throws out of memory errors. But with the current desktops having at least 1GB of RAM (with at least 100M as the max limit to JVM) we need to create thousands of objects to simulate the error. So as I mentioned in the start of this post we will use the –Xms & –Xmx options to start the program with a maximum of 2M so that we can easily simulate the error.

Create a Java Class with the following code and start with the parameter java -Xms2m –Xmx3m

public void fillMemory() throws Exception {
	int voidSpace=20;
	for (int outerIterator=1;outerIterator<50;outerIterator++) {
		System.out.println ("Iteration " + outerIterator + " Free Mem: "
				+ Runtime.getRuntime().freeMemory());
		int innerIterator=10;
		int[] memoryFillIntVar=new int[voidSpace];
		do {
			memoryFillIntVar[innerIterator]=0;
			innerIterator--;
		}
		while(innerIterator>0);
		voidSpace = voidSpace * 10;
	}
}

When you run this class (MemoryTest.java), you will get an output which will be similar to below.

D:\temp\>java -Xms2m -Xmx3m MemoryTest
Iteration 1 Free Mem: 1826368
Iteration 2 Free Mem: 1826368
Iteration 3 Free Mem: 1826368
Iteration 4 Free Mem: 1818352
Iteration 5 Free Mem: 1738336
Iteration 6 Free Mem: 1100952
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at MemoryTest.fillMemory(MemoryTest.java:22)
at MemoryTest.main(MemoryTest.java:12)

Now we have successfully simulated the out of memory error. Let’s continue to the next part where I will show how to recover from these errors.

2. Recover from the error

To recover from the error we will be using one of the utility classes from Apache Derby V10.6 Internals called LowMemory. The class has the below 2 methods drawing our intrest.

void setLowMemory() - Sets a low memory watermark where the owner of this object just hit an OutOfMemoryError.

boolean isLowMemory() - Returns true if a low memory water mark has been set and the current free memory is lower than it.

So we should use setLowMemory to set a threshold and once set, whenever we call the isLowMemory it will return true/false based on the available memory. The isLowMemory will allow an low memory watermark to be valid for five seconds after it was set. This stops an incorrect limit being set for ever. This could occur if other threads were freeing memory when we called Runtime.getRuntime().freeMemory().

Now let’s rewrite our earlier MemoryTest program to use these methods to recover from the error. The modification is simple, we just need to encapsulate the object creation in a if condition which checks for the memory before creating the objects. The modified code is given below.

public class MemoryTest {
    public static void main(String[] args) throws Exception {
		MemoryTest memoryTest = new MemoryTest ();
		LowMemory lowMemory = new LowMemory();
		long[] memoryFillLongVar = new long[70000];
		lowMemory.setLowMemory();
		memoryTest.fillMemory(lowMemory);
    }
    public void fillMemory(LowMemory lowMemory) throws Exception {
		int voidSpace=20;
		for (int outerIterator=1;outerIterator<50;outerIterator++) {
			System.out.println ("Iteration " + outerIterator + " Free Mem: " 
					+ Runtime.getRuntime().freeMemory());
			int innerIterator=10;
			if (!lowMemory.isLowMemory()) {
				int[] memoryFillIntVar=new int[voidSpace];
				do {
					memoryFillIntVar[innerIterator]=0;
					innerIterator--;
				}
				while(innerIterator>0);
				voidSpace = voidSpace * 10;

			} else {
				System.out.println ("Memory lower than threshold to 
							continue. Exiting the loop.");
				break;
			}
		}
	}
}

Let’s run the modified code with the same command line arguments. The output is given below. Note that the class file for LowMemory.java is present in the same directory as of MemoryTest.

D:\temp>java -Xms2m -Xmx3m MemoryTest
Iteration 1 Free Mem: 1349576
Iteration 2 Free Mem: 1349576
Iteration 3 Free Mem: 1349576
Iteration 4 Free Mem: 1341560
Iteration 5 Free Mem: 1261544
Iteration 6 Free Mem: 461528
Memory lower than threshold to continue. Exiting the loop.

This will be very useful while writing distributed J2EE applications which involves huge databases and processing of a large amount of data within a class. This will ensure there will be no data corruption and ensure the JVM (which may host multiple applications) is able to recover from the memory errors. You can always call the GC & runFinalization once this error occurs to free some memory.

Just as an additional note, in C these kind of errors can be handled as the example code given below.

int *pointer = malloc(3 * sizeof(int));
if(pointer == NULL) {
   fprintf(stderr, "Out of memory");
   exit(1);
}

Simple Proxy Server in Java

2009-09-27T00:55:00.006-04:00

In this post, I am going to show how a simple Proxy server can be designed in Java which will provide the core functionalities of a proxy. From Wikipedia, “a proxy server is a server (a computer system or an application program) that acts as an intermediary for requests from clients seeking resources from other servers.” The proxy server generally has some resources, or has access to some resources and when it receives request from a client, it access the appropriate resource and sends back the response.

So it serves two purposes, giving access to a resource which the client do not have access and it hides the real client who needs the resource. The second part is used for the anonymous browsing in Internet using anonymous proxy servers. Although this may not be as “anonymous” as it sounds because the proxy server which you connect to can maintain logs about the clients who are connecting to the server, and also the bigger risk being the data theft. Since the “anonymous” proxy server has access to all the request you send, there is risk of that data being exposed unless there is some kind of encryption in place between the client and the proxy server.

If you are in a network, and the machine which you are using don’t have access to Internet but another machine in the same subnet has access to Internet, then you can run this proxy application in the machine which has Internet access and in your local machine, you can configure your web browser to use the IP & Port number of the machine where the proxy is running in the LAN/Network settings. Once done, you should be able to access the Internet. This is one of the practical use for proxy servers.

The proxy server which we will be designing will be a simple HTTP proxy which gets web requests based on HTTP protocol and sends back response in the same protocol. We will start by declaring some constants.

int proxyServerPortNumber = 7879;    
int webServerPortNumber = 80;     
String webServerIpAddress = "192.168.1.1"     
final int PROXY_SERVER_TIME_OUT = 5000;     
final int SOCKET_TIME_OUT = 2000;

proxyServerPortNumber defines the port where the proxy server will be listening. You can use any port number which is not used by other process / standard port.

webServerIpAddress & webServerPortNumber defines the IP/port where the server can access the resources. Since we are designing this to access websites over http, the port is defined as 80 and the IP should be a machine which has Internet access (can be localhost too)

PROXY_SERVER_TIME_OUT & SOCKET_TIME_OUT are standard parameters defined for Server & network socket timeout. You can adjust this based on your needs.

Now we need to start the Java class to bind to the port number defined and to run in a infinite loop or as a daemon thread, so that it keeps on listening for requests. In this example, we will be using the infinite loop.

try {
 serverSocket = new ServerSocket(proxyServerPortNumber);
}
catch(Exception exc) {
 System.exit(0);
}

Now the process will be listening in the port 7879 for requests from clients. Now, let’s open an infinte loop and make a connection to the resource, in our case the web server.

socketProxy = new Socket(InetAddress.getByName(webServerIpAddress),webServerPortNumber);
socketProxy.setSoTimeout(PROXY_SERVER_TIME_OUT);    
socket = serverSocket.accept();    
socket.setSoTimeout(SOCKET_TIME_OUT);
inputStream = socket.getInputStream();

The request from the client will be received by Input Stream, so once we are connected, we will be sending this Input Stream which is nothing but a normal HTTP REQ object to the server which has the resource or as in our case the machine connected to Internet. The server will respond with the response which will be a HTTP RES over the Java Output Stream. We will be forwarding this output stream to the client which will be rendered in the application which the client requested in. That is if the request was sent through command line, the entire HTML will be printed, if its a browser the page will be rendered.

try {
 while( (intCounter = in.read()) != -1) {
  if(intCounter == -1) break;
  if(!bREQ)
   bREQ = true;
  pOutputStream.write(intCounter);
  if(byteCheckFl > 0) {
   break;
  }
  if(intCounter == 13 && intPrevCounter == 10)
   byteCheckFl++;
  intPrevCounter = intCounter;
 } 
}
catch(Exception e) {   
 if(!bREQ) { 
  continue;
 }     
}

pOutputStream.flush();
pIutputStream = socketProxy.getInputStream();
outputStream = socket.getOutputStream();
try {
 while( (intCounter = pIutputStream.read()) != -1) {
  outputStream.write(intCounter);
 }
}
catch(Exception e) {
}
outputStream.flush();

Make sure to close all the opened sockets and connections once the response is sent. Since this whole code is running in an infinite loop, the server will keep on waiting for the requests and sending the responses back. This is a crude implementation of the concept and there is room for a lot more improvements (like supporting SSL), but this should serve as a foundation to make the proxy a robust one.

We can easily make this Java class as a Win32 service, so that it will always run in background and starts along with the OS. That will need some work with JNI. I will explain that in a later post.

Remainder about ERROR: Invalid row index

2009-09-26T12:32:00.002-04:00

Apparently I asked one of the top airlines to remind me about any SQL errors in their system. I got this in my inbox yesterday.

Recover Database Passwords from Weblogic Server

2009-09-20T17:24:00.003-04:00

In this post, I am going to explain how to Decrypt or recover the passwords which are encrypted (or hashed) in Weblogic 8.1 especially the database passwords. If you ever forgot the database password which is already configured with Weblogic or the password for the user which is used to start Weblogic, this will be handy. The hashed passwords can be normally found in config.xml & boot.properties inside the application domain. The database passwords will be in config.xml under the JDBC configuration and will look something like below.

PasswordEncrypted=”{3DES}bDcllidskanDsaIsnaiG==”

To recover the passwords, we need some prerequisites which are listed below.

weblogic.jar & jsafeFIPS.jar from the Weblogic server. (You can find this under WL_HOME/server/lib directory)
SerializedSystemIni.dat from the application domain (You can find this under the application domain root)
The encrypted password from config.xml / boot.properties including {3DES}

SerializedSystemIni.dat has the key to recover the passwords so this is absolutely essential and it should be from the same server since the key depends on the machine. If you are not able to find this file, then probably the server administrator hasn’t given read access to this file to all the users. For Production systems that should be a practice, only the id which will be starting the weblogic server should have read permissions to the SerializedSystemIni.dat file. If not, this will be a security risk, its like leaving the key for your Home on the street. Coming back to the subject, if you got the file then you are all set for the next step.

Assuming you have all the files needed, let’s proceed to the next step. You can copy all the prerequisite files to your workstation or if needed you can run the utility from the server too. As long as the machines have JDK installed it should be fine.

To recover the passwords we will be using a utility from Apache Geronimo 2.0.1 API (Weblogic81Utils.java). Although this file is a part of a package, this can be run as a standalone class with little modification. All you need to do is to remove the package reference and add a main() method to the class. The code for main() method is below.

public static void main(String args[]) {
 try{
  String beaDir = "- Server/lib or the Directory which has the requried JAR files";
  String appDir = "- App Domain or the Directory which has SerializedSystemIni.dat";
  String hashedPassword = "{3DES}Vdsds76nGsfdsfKJbg54ss==";
  Weblogic81Utils weblogic81Utils = new Weblogic81Utils(beaDir, appDir);
  String plainTextPassword = weblogic81Utils.decryptString(hashedPassword);
  String configXML = weblogic81Utils.getConfigXML();
  Properties bootProperties = (Properties) weblogic81Utils.getBootProperties();
  System.out.println("---------------------------------------------------------------------");
  System.out.println("hashedPassword” + " == " + plainTextPassword);
  System.out.println("boot.properties" + " <username> " + bootProperties.getProperty("username"));
  System.out.println("boot.properties" + " <password> " + bootProperties.getProperty("password"));
  System.out.println("---------------------------------------------------------------------");
 }
 catch (Exception e) {
  throw (RuntimeException)new IllegalArgumentException("Unable to initialize encryption routines from provided arguments").initCause(e);
 }
}

beaDir - If you are running this on the server, it will be the WL_HOME/server/lib directory. If you are running elsewhere it will be the path which has weblogic.jar & jsafeFIPS.jar.

appDir - If you are running this on the server, it will be your application domain directory. If you are running elsewhere, it will be the path which contains SerializedSystemIni.dat, config.xml & boot.properties.

hashedPassword – In this example, I am not reading config.xml directly instead I give the encrypted password as a input. So this will be the encrypted database password from your config,xml.

Save the class file and we are all set to run the program. Make sure you removed the package reference since we are running this as a standalone program. No other change is required. Compile the class and run the class. I tested this using JDK 1.6, but any JDK higher than 1.4 should work.

Sample output is given below.

This was tested with Weblogic Server 8.1 & JDK 1.6. If you encounter any issues or improved this, please drop a comment.

Serial Key Generation and Validation in Java

2009-08-31T23:54:00.007-04:00

In this post, I am going to show how to write a very basic serial key generation module for any Java based application - Same algorithms can be used for any programming language. The module consists of three parts.

Algorithm for Serial Generation
Generating the Serial
Validating the Serial

1. Algorithm for Serial Generation

The following can be used as a simple algorithm for generating serial keys with 18 digits. In this method we are generating the serials based on the input the user gives, which can be the name of the user, company name etc. So most of the times the serial will be unique (based on the input). We get the name of the user as the input and generate the MD2, MD5 & SHA1 hashes for the string and concatenate together. This will generate a total of 104 digits, since we need only 18 of them, we can use a set of pre defined numbers to select the 18 digits.

The below figure explains the algorithm. I have selected some random numbers to pick the 18 digits from 104 character hash, you can use any number you like which makes the serial unique.

2. Generating the Serial

To generate the serial, we need a input string and based on the input string we will be generating MD2, MD5 and SHA1 hashes. The method calculateSecurityHash takes the input string and the hashing method as input and generates the hash based on the method.

String serialNumberEncoded = calculateSecurityHash(fullNameString,"MD2") +  calculateSecurityHash(fullNameString,"MD5") +
 calculateSecurityHash(fullNameString,"SHA1");

Generating the Hash for the input string based on the type.

private String calculateSecurityHash(String stringInput, String algorithmName)
 throws java.security.NoSuchAlgorithmException {
 String hexMessageEncode = "";
 byte[] buffer = stringInput.getBytes();
 java.security.MessageDigest messageDigest =
  java.security.MessageDigest.getInstance(algorithmName);
 messageDigest.update(buffer);
 byte[] messageDigestBytes = messageDigest.digest();
 for (int index=0; index < messageDigestBytes.length ; index ++) {
  int countEncode = messageDigestBytes[index] & 0xff;
  if (Integer.toHexString(countEncode).length() == 1) hexMessageEncode = hexMessageEncode + "0";
  hexMessageEncode = hexMessageEncode + Integer.toHexString(countEncode);
 }
 return hexMessageEncode;
}

Once all the three types of hashes are combined, we will have a total of 104 characters. Since we need only 18 for our serial key we can pick any random 18 digits from the combined hash. We cannot use random number generation to pick up the 18 digits since we need a valid exit strategy for validating the key.

String serialNumber = ""
    + serialNumberEncoded.charAt(32)
    + serialNumberEncoded.charAt(76)
    + serialNumberEncoded.charAt(100)
    + serialNumberEncoded.charAt(50)
    + "-"
    + serialNumberEncoded.charAt(2)
    + serialNumberEncoded.charAt(91)
    + serialNumberEncoded.charAt(73)
    + serialNumberEncoded.charAt(72)
    + serialNumberEncoded.charAt(98)
    + "-"
    + serialNumberEncoded.charAt(47)
    + serialNumberEncoded.charAt(65)
    + serialNumberEncoded.charAt(18)
    + serialNumberEncoded.charAt(85)
    + "-"
    + serialNumberEncoded.charAt(27)
    + serialNumberEncoded.charAt(53)
    + serialNumberEncoded.charAt(102)
    + serialNumberEncoded.charAt(15)
    + serialNumberEncoded.charAt(99);

You can replace the numbers with anything between 0 & 103 so that the key is unique and based on the input string.

3. Validating the Serial

Now, whenever we get a user name & serial combination, we should be able to validate that. Since we already know the algorithm used to generate the serial, the validation part is pretty easier. We cannot follow the steps which we did while generating the serial in opposite direction because we will end up with a bunch of characters without any valid lead. Since we have the serial and user name to validate, we take the user name and generate the serial for the user name as per our algorithm. Once we have the serial number, we compare this against the serial which we got for validation, if both matches then we have a valid key.

String serialNumberEncoded = registrationAppSerialGenerationReversal.calculateSecurityHash(fullNameString,"MD2")
    + registrationAppSerialGenerationReversal.calculateSecurityHash(fullNameString,"MD5")
    + registrationAppSerialGenerationReversal.calculateSecurityHash(fullNameString,"SHA1");

String serialNumberCalc = ""
    + serialNumberEncoded.charAt(32)
    + serialNumberEncoded.charAt(76)
    + serialNumberEncoded.charAt(100)
    + serialNumberEncoded.charAt(50)
    + "-"
    + serialNumberEncoded.charAt(2)
    + serialNumberEncoded.charAt(91)
    + serialNumberEncoded.charAt(73)
    + serialNumberEncoded.charAt(72)
    + serialNumberEncoded.charAt(98)
    + "-"
    + serialNumberEncoded.charAt(47)
    + serialNumberEncoded.charAt(65)
    + serialNumberEncoded.charAt(18)
    + serialNumberEncoded.charAt(85)
    + "-"
    + serialNumberEncoded.charAt(27)
    + serialNumberEncoded.charAt(53)
    + serialNumberEncoded.charAt(102)
    + serialNumberEncoded.charAt(15)
    + serialNumberEncoded.charAt(99);

if (serialNumber.equals(serialNumberCalc))
    System.out.println("Serial MATCH");
else
    System.out.println("Serial MIS-MATCH");

Sample output from this program is demonstrated below.

This is a very basic implementation and used to demonstrate the underlying concept of serial generation and validation, the algorithm can be improved by adding more variables.

Windows Vs Linux..

2006-11-01T06:08:00.001-05:00

Once upon a time Windows and Linux were studying Maths in a school. After 30 minutes of teaching, the Maths teacher decided to spend the last 10 minutes in evaluating how well her students have learnt. So, she started asking questions. Lets see how it went.

Question 1

Teacher: What is 5 ^ 2 ? ( ^ stands for _raised to the power_ )

Windows said:

Linux said: Huh! Thats too easy! Are you insulting me? Heres the answer. Now ask more difficult ones.

Question 2

Teacher: What is 123 ^ 123 ?

Windows said:

Teacher: Windows? What is this e+257 shit? Who asked you to express the result in an exponential form? I want the complete result, accurate up to the last units place.

Windows: Sorry Madam! I dont know how to calculate so accurately.

Teacher: Linux! Can you answer it the way I want?

Linux said: Sure Maam! Have a look at my answer.

Question 3

Teacher: Calculate 12345 ^ 12345 ?

Windows said:

Teacher: You are calling my question invalid? Ridiculous!! You should rather be saying, “Sorry! This is too much for me to calculate. I cant do such huge computations.” Linux, I hope you wont let me down.

Linux said: Certainly not maam! You give me anything to do! I will always give you the answer and that too the way you want, correct up to the units place. J Heres my answer.

(The output was too big to capture in one screen shot! The output had to be redirected to a file. Then it was copied from there and pasted here.)

2867865225003669442826455604983179963965839852965721255676827662
8156962612431723477893199417600869568419659727004886423070459986
7544518984960996737862117995807379142674907192989339718880065284
8434847092204652299777212519222211211855775790000672091949348154
9573810933428036891651229008201570886042147105025448769826910495
4167718679483699298907974081747060546208718375389204314445497215
8817946948946846873753596268760733357119146627102186168829460473
4302375739696328810431827027914295640120520459141195071770134660
1768336189641061593767921159648617986564739287063543667898753671
5133198284973093222583966794178115970438760485533505646234130050
2451888485722096720490458750302708255415692912617746692782441064
8482899463256210162754988594184123945689704625414614675309447803
4086813219933215422175735000653202662472156102866134886087009703
6396420962265777237753176891113921898740033781948909201938522845
7594888993610777036265042885729123287057627529857659854753706471
2932903036903959252994976552148022399172352350540836780723068926
2655994309118396576775576229858397973422818527494520944224575106
1264613938797571409489345760864837045291116629302573267309856371
1917350608799043711114596917413088573518842105072077291635994948
2442828489295515838431465428588309246468375610915776153064128259
3001975189535624972396865938632057141395418068212648082693455899
5094745140923185080150305914142566631321515964797235381322579264
6093971709334252485118207692987578175381758834567692417702683915
7485031835718592770185137943960423121737989877523059100260589246
7013739904633668695753609626491937049172768000677034310548400845
5830266621412606683370296081407125414130325746403742668032098897
6966965732925723019138016837737425965970978383763604705910173033
4292478246111073565105294973384523707786032452448340167890292244
3133453473704044620831390365093180452220239899382558452418373301
7214551676592065501606988370221275398914553994703310806348147428
5686348570645162072862861624249448943577045380466719996250693879
6338737747456434184812376133557373483919671173360193355900323743
3993471077246037736125510726491908306195047380322512168872355392
4370265448433268904799607285835097258954763495613423971231691404
2005614666391355315656704012622738733038749285567934907227408143
2506514063347399057432992918094088735591555552834152038182481186
4964806989793259622419798380018188348968598861956700554403395580
4730559479460281716921176104443752506865322696572169324514585832
9141427806976222400783791875882267856383901198052748592637013696
3793180184536359401966813639646085351278121903035969684857607519
9184028649504590667201638800692182052399772376149674386368328987
6701682822803806302152224658223120230557516788814904381772226549
19137327485225801467623119588746..........................

Sorry I have to truncate the output since its running for pages.

Maam the length of this result is seven hundred and forty lines (740 lines!!!) and I have calculated all the fifty thousand, five hundred ten digits (50510 digits!!!) accurately.

Teacher: Mind blowing Linux! Very well done! You will do great in your life. Windows, you might earn a lot of money in your life but you will never earn respect. You might find your way to every home and children might love to play games on you, but computer scientists and hackers would stay away from you. They would choose Linux instead. You will only manage to become an idiot-friendly operating system but never a useful operating system.

Hosts File - Boon or Bane

2006-01-16T01:32:00.001-05:00

Have you ever wondered..
If some web server is installed in the machine and navigating to the URL localhost points to the Web server?
Do you know how its mapped?
Do you know how most hackers redirect a website?
Do you know how the ads are blocked while you browse the Internet by your Antispyware / Antivirus Software?

The answer for all the above questions is "Hosts File". A hosts file is used to look the Internet Protocol (IP) addresses of a machine or device connected to a computer network. It has a mappping of Domain/Device Names to the corresponding IP Addresses. When we try to access a device or website by name, the system will try to locate the name in the hosts file. The hosts file is used as a first means to search for a network resource before accessing the Domain Name System. The reason behind this is to make the access to the network resource faster, since looking-up in the system takes much less time than looking up in Domain Name System.

The Hosts file is generally named as "hosts" [with no extension] and will be located in the following directories.

Linux/Unix ==> /etc
Windows 9x/Me ==> %windir%
Windows 2000/XP == > %SystemRoot%\system32\drivers\etc
Windows NT ==> winnt\system32\drivers\etc
Mac OS ==> System Folder

Is hosts file a boon? Yes! It is. The hosts file can be used to filter the content in website, or more precisely, we can filter out the ads in the websites. Take the Google Ads, normally all the Google Ads arise from the same location, so mapping the Ad location aganist a non-existing / local IP Address will make the ad disabled in the site. This concept is widely used by the softwares to block unwanted ads in the site. And this has a flaw too.. The softwares gather most of the spyware, viruses, ads urls and they will update the url aganist a local IP address which makes a big sized hosts file. So when we try to access a spyware-free site, the system searches the address in the whole list hosts file and then the Domain Name System is looked-upon. This makes site to load a little slower. So the basic purpose of the hosts file is throwed upon. So we have to make sure that the entries in the hosts file is optimal.

An Example entry in the hosts file to block the ads from google is given below.

127.0.0.1 pagead.googlesyndication.com
127.0.0.1 pagead2.googlesyndication.com
127.0.0.1 adservices.google.com

The 127.0.0.1 is the local system address, this can be replaced with any non-existing address too.
For example 0, 0.0.0.0

Is hosts file a bane? Yes! It is. The above explained concept can be tweaked a little to hack the other systems. If an entry is made in the hosts file with the url of google aganist a spyware or virus site, whenever the user is trying to visit google, the user will be redirected to the spyware/virus site. Normally hackers hijack the hosts file and they use it according to there will.
This is still a serious security issue. Softwares like AntiSpyware can be used to monitor the changes to the hosts file.

If a hosts file is hijacked, to restore the file just open the file in any plain-text editor such as notepad/vi editor, remove everything in the file other than the line

127.0.0.1 localhost

Save the updated file. A restart is requried for Windows Machines.

So what will be the conclusion? The hosts file is a boon, which should be used optimally and securely to attain the result.

Anonymous Browsing

2006-01-12T00:51:00.001-05:00

Most of the Internet users want to browse the web anonymously i.e., browsing by hiding their machines IP Address, there are some services which does this but we have to pay them. After a quick ruthrough in net I found some free services which does the same.

A good and best way is to use Google Transltor as a Proxy Server. The details are avaliable in the this link.

Now the above way wont hide your IP Address, so to hide even the IP Address, we have to try out other services. The following page has listed a around 11 anonymizing services under a single roof. We have the options of disabling ads, cookies, scripts too. Look the service in this url.

Anonymouse is a website which allows us to browse anonymously via their site. There is no proxy set up or any other tech work, just go to their site, enter the url and we can browse through the pages. Also in addition to this they offer a Anonymous Email system called AnonEmail. A pretty impressive service. Although the security can be less if you are browsing through this site, but most of us wont need that and this is pretty faster than other services. Have a vist.

There comes the impressive way of browsing anonymously. The product is called JAP (Java Anonymous Proxy). We need to download and intstall JAP in our machines to avail the service, but its worth !. A single static address is used by JAb, which is shared by many users. This way no one can determine who visited the site. This is a pretty impressive service. Have a checkout.

There is another service similar to JAP, called TOR I have not tested it. The details can be foud in Tor

Thats all for now guys !!

Getting the RGB/Hex code for colors

2006-01-11T02:10:00.001-05:00

Sometimes while working with designing web pages or some logos, we would face the problem to choose a particular color, we will be having the color in some page and we wont know the Hex or RGB of the color. But to code a page with a specific color we should know the RGB or the Hex code (Although mostly used colors such as Red, Green, Yellow etc. can be addressed by text). It is possible to find out the color using Dreamweaver or some other software, but just to find a RGB or Hex we cant install a coslty software. I found out a way, which makes the job easier, it even wont need any other tools. Follow the steps to get the RGB or HEx code.

Press [Ctrl + Print Scrn] in the page where the color which we need resides. Open Microsoft Paint (Start-->Programs-->Accessories-->Paint) and Paste the image [Ctrl+V]. Now Click the icon called "Pick Color" and click on the color which we need. Then in the menu Click Colors-->Edit Colors and then in the window which opens, click "Define Custom Colors", There you can find the RGB value of the color which we have chosen. Now to get the Hex Value, visit the site Click here and give the RGB value and get the HEX code.

Access to restricted sites using Google Translator

2006-01-10T00:26:00.001-05:00

Can Google Translator act as a Proxy? The answer will be it can. Say you dont have access to the site called accessrestricted.com. You can use some free proxies avaliable on the net to browse the site, but if you dont have access to some sites in your network then there is a higher probability that most of the proxies will also be access restricted. So you should try of another way. This trick exactly does that.

Almost in all networks, Google wont be blocked (Although GMail is blocked in some networks, you can always access google), so we can use the Language Translator to browse the access restricted site. Just Type the following URL in your Address Bar

http://www.google.com/translate?langpair=enen&u=www.accessrestricted.com

Wow ! we are able to view the page now. What happened is, we are now accessing the access restricted site using google.com as a proxy. And an added feature is now we can see the access
restricted site in the language what we need. Change the language pair enen in the URL to the language code which we need and you can translate the same.

But there is a catch, even though you are able to access the site, our IP Address wont be hidden. The log will record that our IP Address visitied the particualr site. And also we cant access any pages which requries authentication or uses SSH port, which means we cant check web mails by using this.

I started blogging

2006-01-09T04:40:00.001-05:00

And Here I come as one among the millions. At last I have started to Blog. I don't have a specific topic to blog, let me give you what I think about the current happenings. There will be technical resources, news, some fun etc. etc. So In this 2006 Let me start my blog....