Friday, May 25, 2007

A perceptron in Haskell

After reading Michael Negnevitsky book on Artifical Intelligence, I started playing with some of the algorithms he gives in Haskell. So my first endeavour is writing a simple perceptron (a one neuron neural network) to perform AND or OR logical operations. Next I'll start a full multi layer network to do much more complex operations like exclusive OR (-:.
Once again I was struck with the simplicity of the Haskell code necessary to implement such functionality (and I'm no Haskell expert yet). You think about how you going to implement it, the type checker tells you when you're doing something stupid, and by the time everything is written and compiles it kinda works.
The core method is calculating the output of the neuron, given the inputs, the weight of each input and the threshold value for that neuron:

neuronOutput :: [Float] -> [Float] -> Float -> Float
neuronOutput inputs weights threshold =
let
tot=(foldl (+) 0 (zipWith (*) inputs weights)) - threshold
in
if tot >= 0
then 1
else 0

Using zipWith we multiply each input with its weigth, calculate the sum with fold and remove the threshold. A negative result yields 0, else it yields 1.

Adjusting the weights to adapt to the error of a test case is no more difficult:

defaultLearningRate::Float
defaultLearningRate=0.1

adjustWeights :: [Float] -> [Float] -> Float -> Float -> [Float]
adjustWeights inputs origWeights expected actual =
let
e=expected - actual
delta (i,w) = w + (defaultLearningRate * i * e)
in
map delta (zip inputs origWeights)

We modify the weight by applying to each weight a delta that is dependant on the input weight, the error ratio and the learning rat

A single step is made of calculating the output of the neuron and adapting the weights, given the expected value:

defaultThreshold::Float
defaultThreshold=0.2

step :: [Float] -> [Float] -> Float -> [Float]
step inputs weights expected=
let
o = neuronOutput inputs weights defaultThreshold
in
adjustWeights inputs weights expected o

Then an epoch applies the step function to each different input sets. The hardest part is calculating the average of all deltas to find if the weights have changed at all during the evaluation of all the test cases

epoch :: [([Float],Float)] -> [Float] -> ([Float],Float)
epoch allInputs weights=
let
f w (inputs,expected) = step inputs w expected
newWeights=foldl f weights allInputs
delta= (foldl (+) 0 (map abs (zipWith (-) newWeights weights))) / (fromIntegral $ length weights)
in (newWeights,delta)

A training run is only launching the epoch function repeatedly till the delta is zero:

run :: [([Float],Float)] -> [Float] -> Int -> ([Float],Int)
run allInputs weights epochNb=
let (newWeights,delta) = epoch allInputs weights
in if delta == 0
then
(newWeights, epochNb)
else
run allInputs newWeights (epochNb+1)

We generate the initial weights in the IO monad, passing the number of weights needed:

initialWeights :: Int -> IO [Float]
initialWeights nb = do
let interval= randomR (-0.5,0.5)
(replicateM nb (getStdRandom interval))

Putting it all together:

test :: IO()
test = do
w<-initialWeights 2
let (ws,i)=run [([0,0],0),([0,1],0),([1,0],0),([1,1],1)] w 1
print (ws,i)
return ()

test find out the proper weights to implement AND and print them out, with the number of epochs needed to reach the result. You can then call neuronOutput with the resulting weights and your input: you have a AND logical gate!

As usual, any pointer to how to improve the code is welcome!

Wednesday, May 16, 2007

Javascript and Hibernate, hand in hand

Some time ago I had written a little prototype that was quite interesting, but I never pursued it. Maybe I should try to get some people interested and develop it further...
The idea was to use Rhino to have a Javascript-based framework on the server, thus using the same language on the server and on the client, like ASP did in the good old days. I was also thinking of having the source code for custom applications stored in the database, instead of having it in source files, but that's another story. That explains why the examples below are about "applications" and "components"
The first component is Hibernate connectivity. My framework lets you:
1. Dynamically configure Hibernate mapping for Javascript defined classes. For example, if you wanted to define a Web Application with several attributes and a reference to the components that it defines:

function Application(){
this.displayName='';
this.urlName='';
this.defaultUrl='';
this.id=-1;
this.components=[];
}


Application.storage(
{
id:
{name:'id',column:'ID',type:'long'},
properties:
[
{name:'displayName','not-null':true},
{name:'urlName','not-null':true,unique:true,'unique-key':'APPLICATION_URL'},
{name:'defaultUrl'}
],
references:
[
{'list':
{name:'components',table:'APPLICATION_COMPONENT',
details:[
{'key':{column:'APPLICATIONID'}},
{'list-index':{column:'idx',base:'0'}},
{'many-to-many':{'column':'COMPONENTID','class':'Component'}}
]
}
}
]
}
);

Note that storage() is a function of the Function object which takes a Javascript structure that mimicks the Hibernate XML.

2. Access Hibernate functionality through a "hibernate" host object. For example, a JSUnit method testing the addition of a new "Application" object:

var appId;
var appUrlName;

function testAdd(){
assertNotUndefined(hibernate);
var app=new Application();
assertNotUndefined(app);
app.displayName="TestAppJS"+new Date();
app.urlName=app.displayName;
appUrlName=app.urlName;
hibernate.save(app);
hibernate.commit();
assertNotNaN(app.id);
assertTrue(app.id>0);
appId=app.id;
}

The hibernate object lets you list objects, get a particular object, modify or create instances, etc. I have implemented a Hibernate EntityTuplizer and an subclass of Rhino's NativeArray that encapsulate a PersistentList, so that multiples references can be seen as Javascript arrays.

3. Query the database either through the HQL language:

var apps=hibernate.query("from Application where urlName=?",["TestAppJS"]);

or through a pure Javascript function based mechanism:

var apps=hibernate.query("Application",function(app){
return app.urlName=="TestAppJS";}
);

That I think is very cool. In Rhino I can access the source code of the function that is passed as the second argument to query, and transform it into HQL. I haven't implemented as much as I could, of course, but it works with simple queries. At least there is no other syntax to learn to do a query, it's basically the equivalent of a filter function in functional language.

Then I had started to develop a web framework (probably the millioneth Java web framework by now) with Javascript objects to generate HTML and a mapping between urls and javascript functions, and where one URL could chain several functions, but I haven't gone very far.

Now if somebody feels there's some value in any of that I can sure post the code and we can move it forward. Anything that can show that Javascript is a cool language to run in a JVM in a worthwhile enterprise (-:

Friday, May 11, 2007

Poor man's JavaFX Script is, er... Javascript

Sun has release its own RIA framework called Java FX. It provide JavaFX Script, which is a language based on Java whose goal is to develop quickly graphical applications with the power of Java. Is it touted as competition to Flex that I have mentioned before. It is safe to say that it hasn't been received with a great deal of enthusiasm. The fact is, I don't want to learn another language again to do my Web 2.0 application. And I do believe that Fielding has a point when he reflects that applets break visibility and raise the barrier of entry for web developers. Also, I'm usually all in favor of static typing and compilation, but for GUIs I'm not too sure. When you want to tweak your borders or your layouts you probably happy enough with a quick cycle of change code/refresh page. So why don't we use Javascript and Java Applets together instead of inventing yet another technology?
Ok, it's time you get over your irrational distrust of Javascript. A multi paradigm (object, functional) language available for free in every browser!! And no, I don't suggest we go on using the HTML DOM or things like that. That's why we need applets.
Applets? I must be mad! Applets have not worked on the web. But what I suggest is not to develop one applet for each RIA application, but only develop one Applet that everybody could use, much like a plugin, that would bridge the gap between the JavaScript world and the Java/Swing world. If it has been done, please let me know. As for me, I've put together a prototype in an hour that works fine:
  1. The JavaScript code constructs an object tree that represents the GUI.
  2. The object tree is serialized to JSON and passed on to the applet
  3. The applet deserializes the JSON and constructs a Swing GUI by translating the object tree
  4. The object tree can contain handlers that are the names of Javascript functions
  5. The applet calls the javascript functions on Swing events
  6. The JavaScript event handlers update the GUI tree and notify the applet with the changes.
A very simple example:


<html>
<head><title>JSApplet</title>

<script language="Javascript">

var helloButton={type:"button","id":"helloButton","text":"hello","action":onClickHello}
var frame={"contents":
[
helloButton
]
};


function onClickHello(){
helloButton.text=helloButton.text.toUpperCase();
updateJS([helloButton])
}

function startJS(){
var msg=document.getElementById("JSApplet").setContent(toJSON(frame));
if (msg){
alert(msg);
}
}

function updateJS(objs){
var msg=document.getElementById("JSApplet").updateContent(toJSON(objs));
if (msg){
alert(msg);
}
}


toJSON=function(arg) {

switch (typeof arg) {
case 'object':
if (arg) {
if (arg.constructor == Array) {
var o = '';
for (var i = 0; i < arg.length; ++i) {
var v = toJSON(arg[i]);
if (o) {
o += ',';
}
if (v != null) {
o += v;
} else {
o += 'null,';
}
}
return '[' + o + ']';
} else if (typeof arg.toString != 'undefined') {
var o = '';
for (var i in arg) {
var v = toJSON(arg[i]);
if (v != null) {
if (o) {
o += ',';
}
o += toJSON(i) + ':' + v;
}
}
return '{' + o + '}';
} else {
return;
}
}
return 'null';
case 'unknown':
case 'undefined':
case 'function':
return arg.name;
case 'string':
return '"' + arg.replace(/(["\\])/g, '\\$1') + '"';
default:
return String(arg);
}
}

</script>
</head>
<body onload="startJS();">
<applet id="JSApplet" alt="JSApplet" code="fr/moresmau/jp/jsapplet/JSApplet.class" codebase="bin" mayscript="mayscript" height="100" width="100"></applet>



</body>
</html>


This web page creates a simple "Hello" button that when clicked changed its text to upperCase(). It demonstrate the two ways communication between the Javascript code and the applet (note how the function is serialized in JSON just with its name, that's the only thing we need). So the Javascript can deal with a model that is a slightly abstracted Swing model (no more hair pulling between different browsers). I stress again, the applet is a generic applet that deals with the link between the JSON objects and the Swing layer. Of course the applet could offer other services: simple API to call back the server (AJAX from the applet), etc...
Then the applet could be packaged with a cut down version of the JRE (only the classes really needed) as a plugin to do rich Swing application from JavaScript...
And voilĂ ! A scriptable RIA platform!