Posts on Xavier Llorà

Bluesky

Thu, 28 Nov 2024 08:21:04 +0200

I created a Bluesky account a while back. I was curious about the differences between ActivityPub and ATProto. Mastodon has been a bit of a slow burner, but in the last two weeks, lots of old evolutionary computing friends have shown up in Bluesky. Still not sure what I’ll post. Will I crosspost to the increasing number of networks? Will bridges like skybridge enable better experiences across all? I guess we’ll learn soon.

Vector Symbolic Architectures in Clojure

Sun, 08 Sep 2024 10:40:04 -0700

A few days ago I run into Carin Meier post titled Vector Symbolic Architectures in Clojure. Beside this point that I can really relate to

[…] I don’t have a lot of free time. When I do get a few precious hours to do some coding just for me, I want it it to be small enough for me to fire up and play with it in a REPL on my local laptop and get a result back in under two minutes. […]

Mastodon

Sat, 10 Dec 2022 18:29:01 +0200

I was curious, so I finally created an account. I’m still not sure what is the best server to join, and I’m not going to run my own instance, but I ended with @xllora@mastodon.social.

A More Connected Approach to Notes

Sun, 11 Jul 2021 16:35:51 +0200

I have recently been using tools like Athens research and LogSeq for connected note taking (see my previous post). A key thing you may want to check before choosing the one you will use is the lincese of each. As I write these lines Athens is friendlier if you plan to use it for work. LogSeq one, unfortunately, not so much. However if you plant to use it for fun, LogSeq is feature packed, the team keeps delivering and outstanding product, and the new publishing capability is great.

Taking Notes

Mon, 03 May 2021 12:23:55 -0700

Personal knowledge management has been getting popularity as a term in the recent years. However, there has been quite a bit of research across different fields on how to organize, manage and grow your knowledge. The presentation below is a quick intro with some examples.

You can download the presentation above as a PDF here.

Genetic Programming, The Multiplexer and DEAP

Sun, 29 Mar 2020 00:23:55 -0700

Learning classifier systems (LCS) and Genetics-Based Machine Learning (GBML) have been using the multiplexer problem as a toy problem because of its properties; among others, its dynamic dependencies based on the input values and the exponential nature of the required solution as a function of the number of inputs. There is a great introduction why the multiplexer is interesting for LCS and GBML systems at Wilson’s XCS field-defining paper.

A simple definition [1] of the multiplexer is as follows:

Updated Site and Updated Looks

Sat, 28 Mar 2020 21:22:55 -0700

Here we go! It has been a while. I finally got around getting rid of WordPress for this site. The amount of functionality and machinery required to just render a simple blog started to show its age. I just wanted a simple, easy to update, website. Eventually, I settled for Hugo, a fast framework to create websites.

Moving out of WordPress required the following steps:

Go to the managed WordPress installation and get an export of all posts and pages.
Download all the files uploaded that are part of the site content.
After multiple failed attempts, blog2md turned out to be the tool that got me the farther. It converted the export into a collection of markdown files for each post and page. It also output files for contents, but I discarded those since most of them were old ping backs.
Then, quite a bit of manual clean up. Shortcodes for embedded content etc. needed to be converted to Hugo’s shortcodes. I could have automated, but the numbers were small and I also wanted to reread all the content.
Move the uploads to the right place and make sure all markdown correctly links to it.

At this point, I was able to create and bring up the Hugo development server locally using some random theme so I could see a working version of how the site would work. After this milestone, yes, a few more hours cleaning up content but now everything was quick to iterate. Once all the content was ready, time to move to the looks.

Large Format Photography

Mon, 20 Oct 2014 03:06:02 +0000

I finally put some of the large format photography work I’ve been doing up online at http://photography.xavierllora.net/

Action in 4x5 bites

Mon, 14 Apr 2014 02:54:28 +0000

As I am getting more comfortable with film photography, it seems it helps me to write some experiences and thoughts down. I just wrote down the experience of taking a 4x5 view camera out for the first time. It is more than 7 months using mainly film and being able to touch every part of the process with my hands is more rewarding than digital so far… You can read more about it at “Action in 4x5 bites”.

Reversing Course

Sun, 06 Apr 2014 22:47:59 +0000

I could not remember when was the last time I loaded a roll of film into a camera. As time moves forward, I walk backwards. That is what I thought when I enrolled in the photography course series where film was king. I had to scramble to get a film camera. The good news, black and white. The bad news, film, chemicals, dark rooms, lots of papers, and huge amounts of time invested. So I have decided to star writing about it a bit. I just hope that eventually it will make sense. I am writing these and more thoughts over at Medium.

Yet Another cGA Implementation, Now in Haskell.

Sat, 22 Mar 2014 22:48:57 +0000

A year ago, I mentioned that I always write a cGA implementation when I learn a new language. Then, I was trying to get back to fluent in Haskell. A couple of days ago, Martin Pelikan just did the same and wanted to compare implementations. So, what did I do? I looked for my implementation to post it here.

I took a look at the code and change a couple of things, but I can say that the Haskell implementation is the shortest working implementation that I have ever written in any language. It is shorter than the versions I wrote in Scala and Erlang. Python could get awkwardly compressed using some functional flavor to get close to this, but dynamic typing… C, C++, Java, Go and other friends, are far away when you look in the rear Haskell mirror. Anyway, the code below implements cGA for binary strings. You chose the population size, the number of bits, and the evaluation function. Also, some of the constructs are simple and elegant that do not need much explanation (not to mention maintainability…)

Assignment 8: New Looks of Noir Textures

Tue, 29 Oct 2013 06:09:50 +0000

Catching up with the list of assignments proposed by Harold Davis article series Becoming a More Creative Photographer. The eight assignment is all about new looks.

Starting with a place you know very well, find a way to “slip through the cracks” so you are looking around you with new eyes. Create a photographic image that conveys what you are now seeing.

Assignment 7: Improving Mistakes

Sun, 06 Oct 2013 00:48:14 +0000

Another week, another assignment down the list extracted from Harold Davis article series Becoming a More Creative Photographer, seventh assignment, improving mistakes.

Your assignment: The next time something goes wrong with a shoot, grab the problem, turn it into a possibility, and make it the basis of a photograph.

Auto-focus, that friend that gets it right most of the time, kind of. Yes, I am guilty of usually relying on autofocus most of the time, mainly as a way to remove one var out of the equation. You can chose where you want the auto-focus easily and the camera does the rest. Mostly. There is a scenario where it always goes wrong. You have a composition where you have objects in the far back and an object in front which you look through. Most of the time, your friend auto-focus will decide to focus on the background, likely because it covers most of the frame. So how would the moment in time if I choose the other option? You get something intriguing. A fence to something uncertain. A fence that dilutes the clear-cut and turns it into an image of interpretation. The changed focus trap an eluding reality giving it a transcending quality. It makes the dirt and unfinished lines into textures. It also turns non-exciting lights into puzzling flares of unknown dusk. I am wondering if it could improved further with another mistake, over/under exposure. Maybe for the next round of experiments. Endless possibilities for improving mistakes.

Assignment 6: Making The Ordinary Visually Appealing

Mon, 23 Sep 2013 04:28:20 +0000

I had the assignment done last week, but I was a bit busy and I could not put it out. Enough excuses, the assignment from the list extracted from Harold Davis article series Becoming a More Creative Photographer, sixth assignment, or making the ordinary visually appealing

Your assignment: Pick something that you’ve looked at often, and that you think is visually boring. Now, let go of your preconceptions about your subject. Study it carefully. Find something you haven’t observed before about it, and make an interesting image using this new aspect of your subject.

Assignment 5: Taking Pictures without Your Camera

Tue, 03 Sep 2013 04:23:53 +0000

This week assignment from list extracted from Harold Davis article series Becoming a More Creative Photographer, is the fifth assignment, or taking pictures without your camera.

Your assignment: Without a camera, observe a scene closely. What abstract pattern or patterns can the scene be boiled down to visually?

Staring up, light pours in trough the window softly fading away. The ceiling beams gently guide it through the room space. I am trying to think what are the basic visual patterns in which I could decompose the scene. Maybe the first one would be the window acting as the vanishing point for the main beam and the angled ceiling. It looks pretty simple as a visual construct. However, if I had to choose one abstract pattern that remains consistent and powers the image across the scene, that pattern would be line repetition. It starts on the window. Repeating lines define it. Beam lines emanate from it. They define the basic resting guidelines for the parallel and perpendicular lines to rest and repeat. Even when the scene gets abruptly interrupted, the guiding and recursively repeating lines emerge again with slanted generative angles. In case you wonder, yes, I just took the camera after I finished the assignment so you could see what I was staring at :)

Assignment 4: Post-process It Like You Mean It

Sun, 25 Aug 2013 03:07:24 +0000

Going down the list of assignments extracted from Harold Davis article series Becoming a More Creative Photographer, the fourth assignment challenges you to post-process it like you mean it.

Your assignment: Shoot a commonplace object with digital post-processing specifically in mind. Using your favorite image editing software, transform the photo of the commonplace object into something new and abstract.

This was a tough one. I am still not sure I got there, but I took it as a change to get out of the usual comfort zone. Yes, color. No, monochrome. Yes, post-processed till you cannot tweak any more knobs to get the image you want. If the previous assignments got me warmed up, this could be soul crushing if I took it too seriously. Choosing the picture was actually easy. After a week of scratching my head, I remembered an itch. A snap I took while passing by Zürich back in June. I knew I wanted to do something with it, but never got my head around it. Monochrome did not make justice to the moment I released the shutter. The shoot of a bridal store front got my eye, memories, and imagination. It was bold. It was eye-catching. It was full of color. It was sitting there, ignoring and defying the sobriety that surrounded it. It also had another ethereal property: if you discretely check the people walking by it, almost every street-walker tried to hide their furtive glances. There was no option. This moment was the image I had to post-process. Monochrome was not going to cut it. It will lose the magnetism that challenged your gaze. Also, it was way out there on the left field of the comfort zone. So there I went and start post-processing trying to turn it into that giant magnet that trapped people’s imagination while walking by. Yes, I have to admit that I may have failed turning it into an abstract image, but this was exciting new territory and I guess I could not cover it in a single shoot. So, I just sat in front of the digital darkroom and post-process that moment like I mean it.

Little Corners

Thu, 22 Aug 2013 05:15:31 +0000

Summer 2013. Roaming around Girona. Visiting corners long forgotten. Taking déjà vu turns. Permanent ephemeral moments.

Assignment 3: Out of the Rut

Mon, 19 Aug 2013 02:52:33 +0000

Another week, another photography assignment extracted from Harold Davis article series Becoming a More Creative Photographer. The third assignment reads as follows:

Your assignment: Wait until you are feeling no inspiration and stuck in a rut. Choose a single lens, set it at a single focal length, and use aperture-preferred metering to choose one f-stop. Start taking photos. I guarantee that you will be surprised with what you come up with. This exercise is a great way to reinvigorate your photography generally and get out of a rut.

Assignment 1: Photograph a Reflection

Sun, 04 Aug 2013 18:44:40 +0000

[Halsman defines creativity and imagination as Une Tournure D’Esprit in his book The Creation of Photographic Ideas.]({{ ref “/posts/une-tournure-desprit.md” >}} “Una Tournure D’esperit”) He also proposed a set of rules to help developed your photographic creativity. Some target your logic thinking, some target your unconscious. I was tempted to start practicing Halsman rules, but they felt a bit daunting and I was a bit lost on where to start. Where they written in order? Should I just focus on one of the rules? Should I stop thinking about it or just stop thinking about it and let it permeate for a while? Hence, I decided to dig a bit more around just to see if I could make my mind about what path to take.

cGA, Parallelism, Processes, and Erlang

Wed, 24 Jul 2013 05:22:13 +0000

Back in Fall 2006 I was lucky to be at the right place, at the right time. Kumara Sastry and David E. Goldberg were working to pulverize some preconceptions about how far you could scale genetic algorithms. As I said, I was lucky I could help the best I could. It turned out that the answer was pretty simple, as far as you want. The key to that result was, again, built on Georges Harik’s compact genetic algorithm. The results were published on a paper titled Toward routine billion-variable optimization using genetic algorithms if you are curious. Anyway, back on track. A few days ago, I was playing with Erlang and I coded, just for fun, [yet another cGA implementation, now in Erlang]({{ ref “/posts/yet-another-cga-implementation-now-in-erlang.md” >}} yet another cGA implementation, now in Erlang"). The code was pretty straight forward, so why not take another crack at it and write an Erlang version that uses some of the ideas we used on that paper. The idea we used on the paper was simple. Slice the probabilistic model into smaller segments and update all those model fragments in parallel. The only caveat, if you go over the cGA model, is that you need the evaluation of two individuals to decide which way to update the model. Also, you need to know when to stop, or when your global model has converged. The flow is pretty simple:

Une Tournure D'Esprit

Mon, 15 Jul 2013 02:44:34 +0000

I still do not own a copy of Henri Cartier-Bresson’s The Decisive Moment. It seems hard to get a decent copy at a reasonable price. However, Philippe Halsman’s The Creation of Photographic Ideas has been on my shelves for quite a while now. In the book opening, Halsman defines creativity and imagination as une tournure d’esprit-–or a mental attitude and ability which can be directed developed. Although this resonated strongly the first time I read the book, successive readings have let me to a different crossroad. A crossroad I am not sure how to approach. Philippe writes bluntly

Jacques Henri Lartigue

Tue, 09 Jul 2013 21:09:45 +0000

I always thought that if I wrote about photography I would likely be about some of Henri Cartier-Bresson’s photographs that I cannot shake off. Instead, I am writing about the first picture that got me puzzled to the point I needed to know how. How Jacques Henri Lartigue took such a grasping photo. The photo you can see below.

Spectators on the side of the road appear angled towards the left. The rear wheel of car number 6 deformed into an ellipse with a semi-major axis leaning on the opposite direction, away from the spectators. The construct gives the frame and incredible sense of speed and urgency. Having said that, this is not what got me staring at it again and again. What keep me intrigued was how could he shoot such a photograph and get such opposing lines on moving objects forming such a surreal V. I sketched a few theories. I read a bit more, hard task when you try to avoid the answer. I barely knew anything about taking pictures then, not that now I know any better now. Yes, panning while shooting may definitely have something to do with it. So I took my DSLR to a street corner and started taking pictures of cars driving by while panning. Frames showed the blurry background, the cars were crisply focused, but I could not reproduce the opposing angles between the background and the moving objects. Now I needed to know, there was not way I was going to give up now. Then, it hit me. Maybe his and my camera were not close relatives at all. And yes, I went and I read some more about the cameras and hardware used when Jacques was taking his pictures. Eventually, I found the missing piece. Suddenly, everything felt into place. I was so painfully obvious now. I am not going to spoil the joy of figuring it out on your own. You should definitely do the exercise. Definitely, it is much more rewarding than getting the answer. On another note, a while back I got an awesome gift in form of a nice compact collection of Jacques’ pictures in Thames & Hudson’s Photofile series book. Worth checking it out if you can get a copy. Be careful though, some of those pictures may grab your thoughts for a while.

Yet Another cGA Implementation, Now in Erlang.

Mon, 08 Jul 2013 06:03:27 +0000

Wanna have some Sunday afternoon fun? Just refresh your Erlang skills. Since this is me having fun, what better way to do so than to write yet another implementation of the compact Genetic Algorithm originally (cGA) proposed by Georges Harik?

I am going to skip describing the original algorithm and focus a bit on how to implement it in Erlang instead. You can find some nice books elsewhere and more information on the Erlang site. Erlang is an interesting mix of functional and logic programming languages. If you ever wrote code in ProLog, Erlang is going to look familiar. It will also look familiar if you are coming from Haskell, although, being Erlang a dynamically typed language, you will miss the type system and inference. Nevertheless, give it a chance. It concurrent model is worthwhile reading about. I will it for further posts thought.

Pacific Views

Sun, 07 Jul 2013 23:27:39 +0000

Winter of 2013, strolling down the Pacific cost. From Half Moon Bay to San Luis Obispo everything is crisp. Too crisp to let it pass by. Everything is oddly energized by the wind, rocked by the waves, supported by drift wood, embraced by warm low sunlight. Everything is crisp, too crisp, like a kite strolling down the Pacific cost.

Revamping My Blog

Fri, 05 Jul 2013 19:24:43 +0000

I have been away from my blog for quite a long time. I have barely posted anything compelling in the last three years. Most of the updates were the sporadic announcements to ACM SigEvo’s GECCO conference, but event that was spotty at best. Yes, like everybody else, I gravitated toward social media (pick your favorite poison here). I spend quite a bit of time thinking what I wanted to use my blog for. Should it be the same kind of blog? Should I change it under the hood? Should I give it a golden retirement since it seems I have no stories to share anymore? Then in the mist of all this unanswered questions, I realized I wanted my blog to be what it has been all along. It is whatever I need it to be. Yes, some thoughts are faster to share on ephemeral social media outlets, but there are things you want to keep around longer. Hence, I decided to start a face lift as part of this renewed path. Talking about look and feel, I kept it pretty similar as you may have realized. No big changes, mostly layout updates, removing as much clutter as possible, a bit of font sprinkling here and there, but eventually trying to keep it pretty much the same. I guess that I like the cozy feeling of it feeling familiar. However, one thing I decided to change, after people I care deeply kept insisting that I should, was to build a more permanent home, as I mentioned earlier, for those moments you want to keep around long after the social media rapid timing has digested them into oblivion. Curating photos into gift wrapped packages you find while window shopping was one of those itches that help drove change. As a result of it, you may now see a photo stream top main menu entry. It is a running stream of some of the photos I share on my G+ profile. Under this running photo stream, you will find soon some of those gift-wrapped packages containing some of the photos I cannot shake away. Today, I am adding one. Make sure you check it out. Will this revamping of the blog make me post more often? That is another story.

Night Meditations

Fri, 05 Jul 2013 19:23:11 +0000

On June 2013, I roamed the streets of Girona at night. It started feeling as an ephemeral compulsive need, a needed that could be easily tamed. Its recurrent nature ruled otherwise. I just kept going out at night. I just kept snapping away. I just kept walking over the old cobblestones. I just could not shake the itch. The transmutation of the eye kept charming me back. On June 2013, I roamed the streets of Girona at night. [pe2-gallery album=“http://picasaweb.google.com/data/feed/base/user/101892163699515371535/albumid/5891687846951531553?alt=rss&hl=en_US&kind=photo” ]

GECCO 2012 Deadline Extended to January 27

Sun, 08 Jan 2012 19:39:15 +0000

Is that time of year. Rushing to get your papers ready for GECCO 2012? Here are some good news. The deadline has been pushed back to January 27 to help those last minute pushes :). You can find more infomaion at the conference website http://www.sigevo.org/gecco-2012/ or you can follow GECCO 2012 updatse on Twitter (http://www.twitter.com/gecco_2012), Google+ (+GECCO http://goo.gl/F4ZTM), or Facebook (http://goo.gl/IqEbW).

A Little Functionality and Face Lift

Sat, 04 Jun 2011 06:19:51 +0000

It has been a while since the last face lift to this blog. No, I was not planning any major revamp, but a simple one. Since it was released, I had the little green ShareThis button hanging around. I just wanted to balance a bit the elements on the page. I decided to reposition the button on the top right of the excerpts and on the post themselves. While doing the repositioning I decided to simplify a bit its functionality and replace the button with something a bit lighter and still with purpose. After giving it a bit of a thought I replaced it with Google’s +1 button for publishers. It looks a little best balanced and it does not clutter the layout. You can find more information on how to add the +1 button to your site can be found at the +1 button page for webmasters.

GECCO 2011 Healthier Than Ever

Fri, 18 Feb 2011 04:32:51 +0000

I just got a note from Pier Luca Lanzi about some raw number on this year’s GECCO 2011 conference. The numbers are stagering. I will start with the numbers from the previous editions since GECCO become an ACM conference, just to build up suspense. The numbers of paper submissions to the main conference for the previous editions are listed below.

Year	Submissions
2006	446
2007	577
2008	451
2009	531
2010	373

And after this prelude, here comes the number of submission for this year: 686. Yes you read it correctly 686 papers submitted to the conference. The number just brakes the record of submissions since GECCO joined ACM. 686 papers is 109 papers more than the previous 2007 record. But, do you want to know what is even better than that? That you can still participate by submitting workshop and late breaking papers. Do you want to learn more about it, just check the conference calendar.

GECCO 2011 Submission Deadline: January 26, 2011

Fri, 19 Nov 2010 04:47:24 +0000

The Genetic and Evolutionary Computation Conference (GECCO-2011) is now inviting paper submissions. GECCO 2011 will be held in Dublin, Ireland, from July 12th till July 16th. The full text of the call for paper can be found here. More information about GECCO 2011 can be found on the conference website, Twitter, and Facebook. The paper submission deadline is January 26, 2011.

GECCO 2011 Competitions

Wed, 10 Nov 2010 06:57:56 +0000

The GECCO 2011 website has been updated with detailed information of this year’s competitions. Interested? Here is the list of open competitions:

Demolition derby
Evolutionary art
GPUs for genetic and evolutionary computation
Simulated car racing championship
Visualizing evolution

GECCO 2011: Call for Workshop Proposals

Mon, 27 Sep 2010 05:39:49 +0000

The call for workshop proposals for GECCO 2011 is out. The GECCO-2011 program committee invites proposals for workshops to be held in conjunction with the 2011 Genetic and Evolutionary Computation Conference (GECCO-2011) in Dublin, Ireland, July 12-16. The deadline for proposals is November 8th. More info at is available at the online call for workshop proposals document.

Parallel and Distributed Computational Intelligence book is out for pre-order

Tue, 14 Sep 2010 04:17:35 +0000

“Parallel and Distributed Computational Intelligence” edited by Francisco Fernández de Vega & Erick Cantú-Paz and published by Springer is out for pre-order. The first chapter “When Huge is Routine: Scaling Genetic Algorithms and Estimation of Distribution Algorithms via Data-Intensive Computing” of the book was written together with coauthors Abhishek Verma, Roy Campbell, and David E. Goldberg describing how data-intensive computing can help push the size of problems that GAs and EDAs can address. You may find the abstact of the book below.

Meandre 2.0 Alpha Preview = Scala + MongoDB

Thu, 15 Jul 2010 19:45:00 +0000

A lot of water under the bridge has gone by since the first release of Meandre 1.4.X series. In January I went back to the drawing board and start sketching what was going to be 1.5.X series. The slide deck embedded above is a extended list of the thoughts during the process. As usual, I started collecting feedback from people using 1.4.X in production, things that worked, things that needed improvement, things that were just plain over complicated. The hot recurrent topics that people using 1.4.X could be mainly summarized as:

Minor Update

Fri, 25 Jun 2010 05:04:12 +0000

If you used to check my Twitter stream in the center column at the bottom of my blog, you would have noticed I have just replaced it with my Google Buzz profile stream instead. Yes, you can still see the my Twitter activity, but you will also be able to (1) see aggregated content/activity coming from other sources in one place, (2) subscribe to the stream, and (3) comment on each of the entries. As, I said, a minor update to improve its functionality a bit more.

IWLCS 2010 - Discussion session on LCS / XCS(F)

Mon, 21 Jun 2010 22:04:08 +0000

I just got an email from Martin Butz about a discussion session being planned for IWLCS 2010 and his request to pass it along.

Hope all is well and you are going to attend GECCO this year. Regardless if you attend or not: Jaume asked me to lead a discussion session on “LCS representations, operators, and scalability – what is next?” … or similar during IWLCS… Basically everything besides datamining, because there will be another session on that topic. So, I am sure you all have some issues in mind that you think should be tackled / addressed / discussed at the workshop and in the near future. Thus, I would be very happy to receive a few suggestions from your side – anything is welcome – I will then compile the points raised in a few slides to try and get the discussion going at the workshop. Thank you for any feedback you can provide. Looking forward to seeing you soon! Martin P.S.: Please feel free to also forward this message or tell me, if you think this Email should be still sent to other people… —- PD Dr. Martin V. Butz <butz@psychologie.uni-wuerzburg.de> Department of Psychology III (Cognitive Psychology) Roentgenring 11 97070 Wuerzburg, Germany http://www.coboslab.psychologie.uni-wuerzburg.de/people/martin_v_butz/ http://www.coboslab.psychologie.uni-wuerzburg.de Phone: +49 (0)931 31 82808 Fax: +49 (0)931 31 82815

LCS and Software Development

Fri, 18 Jun 2010 16:43:25 +0000

“On the Road to Competence” is a slide deck by Jurgen Appelo with interesting analogies between learning classifier systems and software development. Definitely worth taking a look at it.

GAssist and GALE Now Available in Python

Fri, 11 Jun 2010 17:06:35 +0000

Ryan Urbanowicz has released Python versions of GAssits and GALE!!! Yup, so excited to see a new incarnation of GALE doing the rounds. I cannot wait to get my hands on it. Ryan has also done an excellent job porting UCS, XCS, and MCS to Python and making those implementations available via “LCS & GBML central” for people to use. I think Ryan’s efforts deserve recognition. His code is helping others to have an easier entry to the LCS and GBML. More information about Ryan’s implementations can found below

GECCO 2010 Around the Bend, GECCO 2011 Gearing Up

Wed, 09 Jun 2010 04:35:01 +0000

GECCO 2010 is getting ready to open July 7th at Portland (Oregon). But GECCO does not sleep and the 2011 edition is gearing up to take the torch after Portland. There is currently an announcement “GECCO 2011: Call for New Frontiers Track Proposals”. The deadline is June 30th, and you can get more information at http://goo.gl/m4my or at the public GECCO 2011 public announcements Wave (embedded below). You can follow GECCO 2011 on Twitter at @GECCO2011, join the group at GECCO 2011 on FaceBook or just Wave with GECCO 2011.

LCS & GBML Central Gets a New Home

Fri, 04 Jun 2010 04:32:38 +0000

Today I finished migrating the LCS & GBML Central site from its original URL (http://lcs-gbml.ncsa.uiuc.edu) to a more permanent and stable home located at http://gbml.org. The original site is already currently redirecting the trafic to the new site, and it will be doing so for a while to help people transition and update bookmarks and feed readers. I have introduced a few changes to the functionality of the original site. Functional changes can be mostly summarized by (1) dropping the forums section and (2) closing comments on posts and pages. Both functionalities, rarely used in their current form, have been replaced by a simpler public embedded Wave reachable at http://gbml.org/wave. The goal, provide people in the LCS & GBML community a simpler way to discuss, share, and hang out. About the feeds being aggregated, I have revised the list and added the feeds now available of the table of contents from

Scaling eCGA Model Building via Data-Intensive Computing

Thu, 08 Apr 2010 16:17:39 +0000

I just uploaded the technical report of the paper we put together for CEC 2010 on how we can scale up eCGA using a MapReduce approach. The paper, besides exploring the Hadoop implementation, it also presents some very compelling results obtained with MongoDB (a document based store able to perform parallel MapReduce tasks via sharding). The paper is available as PDF. Technical report

Abstract:

This paper shows how the extended compact genetic algorithm can be scaled using data-intensive computing techniques such as MapReduce. Two different frameworks (Hadoop and MongoDB) are used to deploy MapReduce implementations of the compact and extended com- pact genetic algorithms. Results show that both are good choices to deal with large-scale problems as they can scale with the number of commodity machines, as opposed to previous ef- forts with other techniques that either required specialized high-performance hardware or shared memory environments.

Soaring the Clouds with Meandre

Mon, 15 Mar 2010 22:55:11 +0000

You may find the slide deck and the abstract for the presentation we delivered today at the “Data-Intensive Research: how should we improve our ability to use data” workshop in Edinburgh.

Abstract

This talk will focus a highly scalable data intensive infrastructure being developed at the National Center for Supercomputing Application (NCSA) at the University of Illinois and will introduce current research efforts to tackle the challenges presented by big-data. Research efforts include exploring potential ways of integration between cloud computing concepts—such as Hadoop or Meandre—and traditional HPC technologies and assets. These architecture models contrast significantly, but can be leveraged by building cloud conduits that connect these resources to provide even greater flexibility and scalability on demand. Orchestrating the physical computational environment requires innovative and sophisticated software infrastructure that can transparently take advantage of the functional features and to negotiate the constraints imposed by this diversity of computational resources. Research conducted during the development of the Meandre infrastructure has lead to the production of an agile conductor able to leverage the particular advantages in the physical diversity. It can also be implemented as services and/or in the context of another application benefitting from it reusability, flexibility, and high-scalability. Some example applications and an introduction to the data intensive infrastructure architecture will be presented to provide an overview of the diverse scope of Meandre usages. Finally, a case will be presented showing how software developers and system designers can easily transition to these new paradigms to address the primary data-deluge challenges and to soar to new heights with extreme application scalability using cloud computing concepts.

Fast REST API prototyping with Crochet and Scala

Thu, 21 Jan 2010 14:59:59 +0000

I just finished committing the last changes to Crochet and tagged version 0.1.4vcli now publicly available on GitHub (http://github.com/xllora/Crochet). Also feel free to visit the issues page in case you run into question/problems/bugs.

Motivation

Crochet is a light weight web framework oriented to rapid prototyping of REST APIs. If you are looking for a Rails like framework written in Scala, please take a look at Lift at http://liftweb.net/ instead. Crochet targets quick prototyping of REST APIs relying on the flexibility of the Scala language. The initial ideas for Crochet were inspired while reading Gabriele Renzi post on creating the STEP picoframework with Scala and the need for quickly prototyping APIs for pilot projects. Crochet also provides mechanisms to hide repetitive tasks involved with default responses and authentication/authorization piggybacking on the mechanics provided by application servers.

GECCO 2010 Submission Deadline (Extended)

Sat, 19 Dec 2009 12:11:34 +0000

If you are planning to submit a paper for the 2010 Genetic and Evolutionary Computation Conference, the deadline is January 13, 2010 (and now extended to January 27th). You can find more information at the GECCO 2010 calendar site.

Meandre is going Scala

Tue, 01 Dec 2009 20:01:47 +0000

After quite a bit of experimenting with different alternatives, Meandre is moving into Scala. Scala is a general purpose programming language designed to express common programming patterns in a concise, elegant, and type-safe way. This is not a radical process, but a gradual one while I am starting to revisit the infrastructure for the next major release. Scala also generates code for the JVM making mix and match trivial. I started fuzzing around with Scala back when I started the development of Meandre during the summer of 2007, however I did fall back to Java since that was what most of the people in the group was comfortable with. I was fascinated with Scala fusion of object oriented programming and functional programming. Time went by and the codebase has grown to a point that I cannot stand anymore cutting through the weeds of Java when I have to extend the infrastructure or do bug fixing—not to mention its verbosity even for writing trivial code. This summer I decided to go on a quest to get me out of the woods. I do not mind relying on the JVM and the large collection of libraries available, but I would also like to get my sanity back. Yes, I tested some of the usual suspects for the JVM (Jython, JRuby, Clojure, and Groovy) but not quite what I wanted. For instance, I wrote most of the Meandre infrastructure services using Jython (much more concise than Java), but still not quite happy to jump on that boat. Clojure is also interesting (functional programming) but it would be hard to justify for the group to move into it since not everybody may feel comfortable with a pure functional language. I also toyed with some not-so-usual ones like Erlang and Haskell, but again, I ended up with no real argument that could justify such a decision. So, as I started doing back in 2007, I went back to my original idea of using Scala and its mixed object-oriented- and functional-programming- paradigm. To test it seriously, I started developing the distributed execution engine for Meandre in Scala using its Earlang-inspired actors. And, boom, suddenly I found myself spending more time thinking that writing/debugging threaded/networking code :D. Yes, I regret my 2007 decision instead of running with my original intuition, but better late than never. With a working seed of the distributed engine working and tested (did I mention that scalacheck and specs are really powerful tools for behavior driven development?), I finally decided to start gravitating the Meandre infrastructure development effort from Java to Scala-–did I mention that Scala is Martin Odersky’s child? Yes, such a decision has some impact on my colleagues, but I envision that the benefits will eventually weight out the initial resistance and step learning curve. At least, the last two group meetings nobody jumped off the window while presenting the key elements of Scala, and demonstrating how concise and elegant it made the first working seed of the distributed execution engine :D. We even got in discussions about the benefits of using Scala if it delivered everything I showed. I am lucky to work with such smart guys. If you want to take a peek at the distributed execution engine (a.k.a. Snowfield) at SEASR’s Fisheye. Oh, one last thing. Are you using Atlassian’s Fisheye? Do you want syntax highlighting for Scala? I tweaked the Java definitions to make it highlight Scala code. Remember to drop the scala.def file on $FISHEYE_HOME/syntax directory add an entry on the filename.map to make it highlight anything with extension .scala.

Scaling Genetic Algorithms using MapReduce

Fri, 09 Oct 2009 15:51:19 +0000

Below you may find the abstract to and the link to the technical report of the paper entitled “Scaling Genetic Algorithms using MapReduce” that will be presented at the Ninth International Conference on Intelligent Systems Design and Applications (ISDA) 2009 by Verma, A., Llorà, X., Campbell, R.H., Goldberg, D.E. next month.

Abstract:

Genetic algorithms(GAs) are increasingly being applied to large scale problems. The traditional MPI-based parallel GAs do not scale very well. MapReduce is a powerful abstraction developed by Google for making scalable and fault tolerant applications. In this paper, we mould genetic algorithms into the the MapReduce model. We describe the algorithm design and implementation of GAs on Hadoop, the open source implementation of MapReduce. Our experiments demonstrate the convergence and scalability upto 105 variable problems. Adding more resources would enable us to solve even larger problems without any changes in the algorithms and implementation. The draft of the paper can be downloaded as IlliGAL TR. No. 2009007.

Temporary storage for Meandre's distributed flow execution

Tue, 29 Sep 2009 15:14:28 +0000

Designing the distributed execution of a generic Meandre flow involves several moving pieces. One of those is the temporary storage required by the computing nodes (think of it as one node as one isolated component of a flow) to keep up with the data generated by a component, and also be able to replicate such storage to the node containing the consumer to be fed. Such storage, local to each node, must guarantee at least three basic properties.

Liquid: RDF endpoint for FluidDB

Thu, 24 Sep 2009 20:45:19 +0000

A while ago I wrote some thoughts about how to map RDF to and from FluidDB. There I explored how you could map RDF onto FluidDB, and how to get it back. That got me thinking about how to get a simple endpoint you could query for RDF. Imagine that you could pull FluidDB data in RDF, then I could just get all the flexibility of SPARQL for free. With this idea in my mind I just went and grabbed Meandre, the JFLuidDB library started by Ross Jones, and build a few components. The main goal was to be able to get an object, list of the tags, and express the result in RDF. FluidDB helps the mapping since objects are uniquely identified by URIs. For instance, the unique object 5ff74371-455b-4299-83f9-ba13ae898ad1 (FluidDB relies on UUID version four with the form xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx) is uniquely identified by http://sandbox.fluidinfo.com/objects/5ff74371-455b-4299-83f9-ba13ae898ad1 (or a url of the form http://sandbox.fluidinfo.com/objects/xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx), in case you are using the sandbox or http://fluiddb.fluidinfo.com/objects/5ff74371-455b-4299-83f9-ba13ae898ad1 if you are using the main instance. Same story for tags. The tag fluiddb/about can be uniquely identified by the URI http://sandbox.fluidinfo.com/tags/fluiddb/about, or http://fluiddb.fluidinfo.com/tags/fluiddb/about.

From Galapagos to Twitter: Darwin, Natural Selection, and Web 2.0

Fri, 18 Sep 2009 18:14:05 +0000

Yesterday I was visiting Monmouth College to participate on the Darwinpalooza which commemorates the 200th anniversary of Charles Darwin’s birth and the 150th anniversary of the publication of On the Origin of Species. After scratching my head about about what to present, I came out with quite a mix. You will find the abstract of the talk below, as well as the slides I used.

Abstract:

One hundred and fifty years have passed since the publication of Darwin’s world-changing manuscript “The Origins of Species by Means of Natural Selection”. Darwin’s ideas have proven their power to reach beyond the biology realm, and their ability to define a conceptual framework which allows us to model and understand complex systems. In the mid 1950s and 60s the efforts of a scattered group of engineers proved the benefits of adopting an evolutionary paradigm to solve complex real-world problems. In the 70s, the emerging presence of computers brought us a new collection of artificial evolution paradigms, among which genetic algorithms rapidly gained widespread adoption. Currently, the Internet has propitiated an exponential growth of information and computational resources that are clearly disrupting our perception and forcing us to reevaluate the boundaries between technology and social interaction. Darwin’s ideas can, once again, help us understand such disruptive change. In this talk, I will review the origin of artificial evolution ideas and techniques. I will also show how these techniques are, nowadays, helping to solve a wide range of applications, from life science problems to twitter puzzles, and how high performance computing can make Darwin ideas a routinary tool to help us model and understand complex systems.

Liquid: RDF meandering in FluidDB

Tue, 25 Aug 2009 18:04:20 +0000

Meandre (NCSA pushed data-intensive computing infrastructure) relies on RDF to describe components, flows, locations and repositories. RDF has become the central piece that makes possible Meandre’s flexibility and reusability. However, one piece still remains largely sketchy and still has no clear optimal solution: How can we facilitate to anybody sharing, publishing and annotating flows, components, locations and repositories? More importantly, how can that be done in the cloud in an open-ended fashion and allow anybody to annotate and comment on each of the afore mentioned pieces?

Easy, reliable, and flexible storage for Python

Thu, 13 Aug 2009 22:01:32 +0000

A while ago I wrote a little post about alternative column stores. One that I mentioned was Tokyo Cabinet (and its associated server Tokyo Tyrant. Tokyo Cabinet it is a key-value store written in C and with bindings for multiple languages (including Python and Java). It can maintain data bases in memory or spin them to disk (you can pick between hash or B-tree based stores). Having heard a bunch of good things, I finally gave it a try. I just installed both Cabinet and Tyrant (you may find useful installation instructions here using the usual configure, make, make install cycle). Another nice feature of Tyrant is that it also supports HTTP gets and puts. So having all this said, I just wanted to check how easy it was to use it from Python. And the answer was very simple. Joseph Turian’s examples got me running in less than 2 minutes—see the piece of code below—when dealing with a particular data base. Using Tyrant over HTTP is quite simple too—see PeteSearch blog post.

Large Scale Data Mining using Genetics-Based Machine Learning

Wed, 15 Jul 2009 21:56:17 +0000

Below you may find the slides of the GECCO 2009 tutorial that Jaume Bacardit and I put together. Hope you enjoy it.

Slides

Abstract

We are living in the peta-byte era.We have larger and larger data to analyze, process and transform into useful answers for the domain experts. Robust data mining tools, able to cope with petascale volumes and/or high dimensionality producing human-understandable solutions are key on several domain areas. Genetics-based machine learning (GBML) techniques are perfect candidates for this task, among others, due to the recent advances in representations, learning paradigms, and theoretical modeling. If evolutionary learning techniques aspire to be a relevant player in this context, they need to have the capacity of processing these vast amounts of data and they need to process this data within reasonable time. Moreover, massive computation cycles are getting cheaper and cheaper every day, allowing researchers to have access to unprecedented parallelization degrees. Several topics are interlaced in these two requirements: (1) having the proper learning paradigms and knowledge representations, (2) understanding them and knowing when are they suitable for the problem at hand, (3) using efficiency enhancement techniques, and (4) transforming and visualizing the produced solutions to give back as much insight as possible to the domain experts are few of them. This tutorial will try to answer this question, following a roadmap that starts with the questions of what large means, and why large is a challenge for GBML methods. Afterwards, we will discuss different facets in which we can overcome this challenge: Efficiency enhancement techniques, representations able to cope with large dimensionality spaces, scalability of learning paradigms. We will also review a topic interlaced with all of them: how can we model the scalability of the components of our GBML systems to better engineer them to get the best performance out of them for large datasets. The roadmap continues with examples of real applications of GBML systems and finishes with an analysis of further directions.

Data-Intensive Computing for Competent Genetic Algorithms: A Pilot Study using Meandre

Tue, 14 Jul 2009 04:15:51 +0000

Below you may find the slides I used during GECCO 2009 to present the paper titled “Data-Intensive Computing for Competent Genetic Algorithms: A Pilot Study using Meandre”. An early preprint in form of technical report can be found as an IlliGAL TR No. 2009001 or the full paper at the ACM digital library

NIGEL 2006 Part VI: Bacardit

Mon, 13 Jul 2009 23:54:27 +0000

After coming back from GECCO I just uploaded the last of the NIGEL 2006 talks at LCS & GBML Central. This last talk was by Jaume Bacardit and GBML for protein structure prediction.

NIGEL 2006 Part V: Bernardó vs. Lanzi

Mon, 29 Jun 2009 18:30:14 +0000

After the vacation break, two more NIGEL 2006 talks are available at LCS & GBML Central. This week Ester Bernardó presents how LCS can perform in the presence of class imbalance, whereas Lanzi continues his quest on computed predictions.

NIGEL 2006 Part IV: Llorà vs. Casillas

Mon, 08 Jun 2009 21:20:02 +0000

Two more NIGEL 2006 talks are available at LCS & GBML Central. This week Xavier Llorà presents how linkage learning can be achieve in Pittsburgh LCS, whereas Jorge Casillas reviews his work using XCS and Fuzzy LCS.

NIGEL 2006 Part III: Butz vs. Barry

Tue, 19 May 2009 14:38:05 +0000

NIGEL 2006 talks is available at LCS & GBML Central. This week Martin Butz review reviews the state of the union of XCS, where as Alwyn Barry introduces the theoretical framework for LCS that he and Jan Drugowitsch worked on.

LCS & GBML Central back to production

Thu, 14 May 2009 01:33:49 +0000

LCS & GBML Central exploit has been fixed. The site is restored and back to fully functional. Please do not hesitate to ping me if you see something missing.

LCS & GBML Central under inspection

Wed, 13 May 2009 15:07:21 +0000

It is very common that NCSA machines get continual attacks over the net. Today I got a note from the security team that LCS & GBML Central may have been compromised. We are looking into it. As a precaution the automatic measures has been fired, that means, that the box is currently unroutable. Hope it can be fixed soon. I’ll keep you posted about the progress.

NIGEL 2006 Part II: Dasgupta vs. Booker

Mon, 11 May 2009 16:08:38 +0000

The second weekly release of NIGEL 2006 talks is available at LCS & GBML Central. This week Dipankar Dasgupta reviews the negative selection algorithm, where as Lashon Booker travels in time to the past and future of learning classifier systems.

Transcoding NIGEL 2006 videos

Mon, 04 May 2009 20:13:23 +0000

Last week Pier Luca Lanzi was visiting IlliGAL. Yesterday, before he left for Chicago, we went for one last brunch. He mentioned that he liked a lot the videos we shot during NIGEL 2006. Thinking about it we agreed would be useful to recover the videos and upload them into some of the usual video sharing site suspects. Currently they are hosted, for long term storage purposes, at NCSA’s web archive. I spent sometime retrieving them from the archive (they are pretty fat and encoded in wmv) and I stated transcoding it in m4a. My plan? Make them available via Vimeo and LCS & GBML Central. Also, I will be uploading the presentation slides to SlideShare and also make them available via LCS & GBML Central.

Deadline extended for special issue on Metaheuristics for Large Scale Data Mining

Tue, 07 Apr 2009 15:32:03 +0000

The deadline for the special issue on Metaheuristics for Large Scale Data Mining to be published by Springer’s Memetic Computing Journal has been extended till May 31, 2009. More information can be found in this post at LCS & GBML Central.

Meandre overview slides

Thu, 02 Apr 2009 18:53:25 +0000

On May 26th I gave a seminar about Meandre’s basics at the Computer Science department at University of Illinois . The talk was part of the Cloud Computing Seminars. I merged together slides I have been using to talk about Meandre, and tried to give it an easy to grasp overview flavor. You view them below.

Squeezing for cycles

Thu, 02 Apr 2009 01:55:24 +0000

Sometimes thinking a bit helps to rush decisions that may lead to weird places. Today I was going over a simple genetic algorithm for numeric optimization written in C. The code is nothing special, tournament selection without replacement, SBX crossover operator, and polynomial mutation. To the point, I was running a simple OneMax-like problem (in this case, minimize the value of the sum of all the genes), and I was quite surprised the guy was taking so long for.

LCSweb + GBML blog = LCS & GBML Central

Fri, 27 Mar 2009 16:27:02 +0000

LCSweb was designed to allow researchers and those seeking to use Learning Classifier Systems within applications access to material on LCS and discussion between members of the LCS community. The site served this community since its was started by Alwyn Barry in 1997. Enhanced and maintained later by Jan Drugowitsch, LCSweb became a valuable community resource. The site was completely community-driven and allowed members to contribute to the content of the site and keeping it up to date. Later on in 2005, I started “LCS and other GBML” Blog to cover a gap providing information information regarding the International Workshop on Learning Classifier Systems (IWLCS), the collection of LCS Books available, and GBML related news. Some of you may have realized that after Jan’s move to Rochester and Alwyn’s retirement from research activities, LCSweb has vanished. Will Browne took on himself to take LCSweb to Reading, but technical circumstances have made that move rocky despite his best efforts. Jan and Will however still have a local copy of LCSweb contents. After talking to Jan and Will, I proposed to merge LCSweb with the LCS and other GBML blog, and host the new site at NCSA where dedicated resources has been made available. Jan and Will agreed with the idea. We are happy to announce that the merged site (still under the update cycle) can be reached at http://lcs-gbml.ncsa.uiuc.edu. More information about the process can be found here or at there LCS & GBML Central site.

Presentation Zen

Mon, 23 Mar 2009 15:32:45 +0000

Lately I have been thinking about how to effectively communicate ideas over presentations. This has turned to be a key element when trying to convey the benefits of jumping on the Meandre wagon. Presentation Zen is a very interesting resource for methods, techniques, and example on how to convey and communicate ideas. I am working on revamping some of my Meandre presentations trying to be able to get the points across easily.

Usages of R

Mon, 23 Feb 2009 15:47:06 +0000

R has gained a lot of traction on the scientific community for data analysis, modeling, and exploratory work. I just run into a post by Michael E. Driscoll in his Data Evolution blog about how R is used in Google and Facebook. Nothing new, but what got my attention was ParallelR. If you have been using R for large problems, I am pretty sure you have been wishing that there was some parallelization capabilities. ParallelR targets the problem, and it definitely an option to check out.

GroupTweet or getting groups in Twitter

Wed, 18 Feb 2009 19:34:31 +0000

One of the main handicaps I keep running over and over with Twitter is that there is no concept of groups. The usual story goes along these lines: You get forced to create accounts that behave as groups, make their updates private, and then ask the members to request to follow, and once they follow, that’s it. Yes, a bit convoluted. Yesterday, I ran into GroupTweet that basically automates this process. It still creates a new user account that works as a group fan but, at least it makes the process easier.

Revamping my Twitter accounts

Mon, 16 Feb 2009 20:50:08 +0000

Since summer 2007 I have been twittering. It started as way to have a conversation with a bunch of friends scattered all over. Since we were not discussing any world-changing topic, my updates have been kept private. Lately, I have been receiving requests to follow me. So, instead of polluting my original intent, @xllora is now my public Twitter. Hope this gets things a bit simplified. I still wish that Twitter would one day allow you to have a better access control, but, oh well, it is what it is right now. Also, I added my @xllora tweets below :D

Efficient serialization for Java (and beyond)

Wed, 04 Feb 2009 15:10:38 +0000

I am currently working on the distributed execution of flows as part of the Meandre infrastructure—as a part of the SEASR project. One of the pieces to explore is how to push data between machines. No, I am not going to talk about network protocols and the like here, but how you can pass the data around. If you have ever programmed MPI using C/C++ you remember the tedious efforts that requires passing complex data structures around between processes. Serialization is a way to take those complex structures into a form that can be easily stored/transmitted, and then retrieved/received and regenerate the original complex data structure. Some languages/platforms support this functionality (e.g. Java, Python), allowing to easily use the serialized representation for persistency or transmission purposes.

Data-Intensive Computing for Competent Genetic Algorithms: A Pilot Study using Meandre

Thu, 29 Jan 2009 19:36:25 +0000

by Llorà, X.

IlliGAL technical report 2009001. You can download the pdf here.

Abstract:

Data-intensive computing has positioned itself as a valuable programming paradigm to efficiently approach problems requiring processing very large volumes of data. This paper presents a pilot study about how to apply the data-intensive computing paradigm to evolutionary computation algorithms. Two representative cases—selectorecombinative genetic algorithms and estimation of distribution algorithms—are presented, analyzed, discussed. This study shows that equivalent data-intensive computing evolutionary computation algorithms can be easily developed, providing robust and scalable algorithms for the multicore-computing era. Experimental results show how such algorithms scale with the number of available cores without further modification.

Need a quick form on your WordPress site?

Tue, 27 Jan 2009 13:25:07 +0000

cForms is a WordPress plugin that allows you to quickly add forms on your site. The plugin will allow you to collect information from visitors easily. Besides mail notifications, it also provides database back-ended tracking. Pretty convenient if you have to boost the usefulness of your site. Two examples I have more or less been involved is the SEASR 2009 workshop registration form and the iFoundry application form. On both cases, cForms worked like a champ.

FireStats: Statistics on fire for Wordpress sites

Sun, 25 Jan 2009 13:51:36 +0000

I have been using FireStats for gathering statistics on WordPress sites for more than a couple of years now. I mainly use FireStats combined with WordPress Stats, and Google Analytics. Each of them give you different views into traffic, but FireStats is by far quick and fast and give you a good overall picture you can dig down using WordPress Stats and Google Analytics. I just installed the new version 1.6.0 on this blog and found a new interesting goodie. Now FireStats also tracks the number of RSS readers coming to your site :D

Protect yourself from genetic algorithms surprises

Fri, 23 Jan 2009 13:01:16 +0000

I just ran into the comic strip below at xkcd. I am still laughing now. Fitness functions are tricky. Once somebody told me that genetic algorithms always get their target; the main problem is to explain what the target is. If you want to learn a “fountain pen”, you better be accurate defining it or you may end up getting an unexpected all-terrain “pencil”. Yes, I know the example is quite solution free, but still has some truth to it. How many times have you end up getting something you did not expect, only because evolution find a better fitted crack in your description? Anyway, it is fun what you end running into on the Internet ;)

WP Cumulus: Exiting 3D tag clouds

Fri, 23 Jan 2009 00:29:08 +0000

Tag clouds, despite of being useful, can look a bit dull. My office mate, Bernie Acs, pointed me to a more aesthetic appealing one written in Flash for Wordpress. Roy Tank has released the WP-Cumulus plugin that makes the new 3D tag cloud available as a widget. I couldn’t help myself getting it installed. If you scroll down you will see it in action. Roy, kudos for you!

Meandre 1.4.0 released, 1.4.1 coming short after

Thu, 15 Jan 2009 17:22:28 +0000

Today, Meandre 1.4.0 has been released. The final release includes the infrastructure, tools (Workbench, ZigZag, etc.), and a set of component and flow repositories. Also, 1.4.1 will be also released later today adding the latest feature updates and few bug fixes on the tools.

Meandre 1.4.0 final release candidate tagged

Mon, 12 Jan 2009 03:01:49 +0000

Today I just tagged the final release candidate RC6 for the Meandre Infrastructure. The list of improvement from the previous 1.3.2 release is long, but I just want to highlight a few of them.

Standardization of web services and their response information. Also added new services to help manage users, repositories, and jobs.
The infrastructure is now able to provide a single system image cluster of Meandre servers.
It also provides a new light weight HTML interface for easy administration.
Location addition has been speeded up minimizing the network usage.
Component contexts are locally cached, drastically reducing the preparation time of a flow before executing.

Also, all the reported bugs reported so far has been fixed. Hopefully, the release will be pushed out this coming week. Stay tuned :D. Also, a few documentation for developers interested in the infrastructure can be find on the development tools Confluence wiki.

GECCO 2009 paper submission deadline extended till January 28

Fri, 09 Jan 2009 14:05:19 +0000

Martin Pelikan just posted on the MEDAL and IlliGAL blogs that the GECCO 2009 paper submission deadline has been extended till January 28. Always, great news to finish polishing last minute changes :D It seems that the website is not yet updated, neither emails sent to the usual mailing lists. Just a matter or time I guess ;) And since I am talking about GECCO 2009, you may also want to take a look at the competitions announcement video that Pier Luca Lanzi put together below. httpv://www.youtube.com/watch?v=i7cbgBOkn-s

Promoting creative interdisciplinary in a disciplinary world

Wed, 07 Jan 2009 17:32:22 +0000

David E. Goldberg has posted a serie of three short talks about how to promote creative interdisciplinary work in places where highly disciplinary work is conducted. This talks are part of the iFoundry initiative at the University of Illinois. You may find the sequence below and the slides on SlideShare. httpv://www.youtube.com/watch?v=GM-ASQ00pGA httpv://www.youtube.com/watch?v=lwt4hzo857Q httpv://www.youtube.com/watch?v=gqX-2ttcHbM

Dusting my Ph.D. thesis off

Tue, 30 Dec 2008 10:11:00 +0000

After attending Albert Orriols’s Ph.D. thesis defense, I ended wondering how many of the question I posted in mine have not been solved. The answer, quite a bit. So, I just decided to dig it up, and put it up here. Yes, the thesis was not written in English (in those days my fellowship had some strings attached), but math formulation, graphs, and results are readable in any language ;) Also, GALE was written and documented in english, and is available here.

Join me congratulating Albert Orriols, Ph.D.

Fri, 12 Dec 2008 23:35:30 +0000

Albert Orriols (for those who do not know him, a brilliant learning classifier systems researcher) defended his thesis today. The outcome: Excellent Cum Laude. Albert Orriols Ph.D. defense started at 11am at Enginyeria i Arquitectura La Salle in Barcelona. The thesis panel was presided by Prof. David E. Goldberg, and formed by members Prof. Francisco Herrera, myself, Dr. Martin Butz, and secretary Prof. Xavier Vilasís). I must say that it has been a great pleasure to read his remarkable thesis and great contributions to the Learning Classifier System field. I hope he will make it available soon, and encourage you to take a look at it.

Flexible LaTeX references with Natbib

Sat, 06 Dec 2008 13:37:53 +0000

LaTeX reference management is superb. However, some publisher want to have the citations arranged in some predefined manner. Yes, they are usually all different. However, if your are using LaTeX and BibTex no much need to be done to format your references right. The natbib package provides you with the tools to satisfy almost any publisher requirements.

Lulu and LaTeX

Fri, 05 Dec 2008 15:46:58 +0000

A while ago, I wrote about “Publishing yourself online”. Among other solutions, Lulu, seemed to be a more traditional publishing method, with the twist you do it your self, and you can decide what kind of book you want (e-book, paperback, hardcover, etc.). The other day I was spending some cycles trying to find out if they provide LaTeX templates, and no, they are a Microsoft shop (with the option of dealing with Open Office). They, however allow you to upload PDFs, so you can also prepare manuscripts in LaTeX. I searched a bit, and run into and very useful post by Veta Project describing how to take your LaTeX document and make it Lulu friendly. Worth taking a look at it…

Meandre Infrastructure 1.4 RC1 tagged

Tue, 02 Dec 2008 06:24:40 +0000

Last monday I tagged on the development SVN the first release candidate for the 1.4 version of the Meandre infrastructure. The new 1.4 version brings a pile of bug fixes, improves web service mechanics, provides a new HTML lightweight interface to the infrastructure, and implements a first version of the coordination facility (Meandre Distributed eXchange) used to assemble configurable single-image Meandre clusters based on Meandre servers—just to list a few of the 1.4 goodies. So far there is no firm release date, but it will definitely be before the SEASR 2009 Workshop on January 15th, 2009.

GECCO 2009 submission deadline

Mon, 17 Nov 2008 12:01:24 +0000

Yup, that time of year is coming around again. The 2009 Genetic and Evolutionary Computing Conference (GECCO 2009) is going to be held in Montreal, Canada. The paper submission dead line is January 14, 2009.

Meandre: Semantic-Driven Data-Intensive Flows in the Clouds

Sat, 15 Nov 2008 23:34:02 +0000

by Llorà, X., Ács, B., Auvil, L., Capitanu, B., Welge, M.E., Goldberg, D.E. (2008). This paper has been accepted at the 4th IEEE International Conference on e-Science. An early draft of the paper can be found as IlliGAL technical report 2008013. You can download the pdf here. More information is also available at the Meandre website as part of the SEASR project.

Abstract:

Data-intensive flow computing allows efficient processing of large volumes of data otherwise unapproachable. This paper introduces a new semantic-driven data-intensive flow infrastructure which: (1) provides a robust and transparent scalable solution from a laptop to large-scale clusters,(2) creates an unified solution for batch and interactive tasks in high-performance computing environments, and (3) encourages reusing and sharing components. Banking on virtualization and cloud computing techniques the Meandre infrastructure is able to create and dispose Meandre clusters on demand, being transparent to the final user. This paper also presents a prototype of such clustered infrastructure and some results obtained using it.

Free online survey service

Fri, 14 Nov 2008 12:01:54 +0000

Pier Luca Lanzi sent an email the other about help SigEvolution newsletter by taking an on-line survey. The survey was host at SurveyMonkey.com. I had never run into this guys before, but after digging a bit, the idea is pretty sweet. Need to run a survey? Just register to their site, create the survey, graph the link to it, and spread it around. Also, they allow you to upload surveys to their servers. As I said, pretty interesting option if you want to run a survey and do not want to stand up your own version of it.

Fast mutation implementation for genetic algorithms in Python

Thu, 13 Nov 2008 05:31:52 +0000

The other day I was playing to see how much I could squeeze out of a genetic algorithm written in Python. The code below shows the example I used. The first part implements a simple two loop version of a traditional allele random mutation. The second part is coded using numpy 2D arrays. The code also measures the time spent on both implementations using cProfile.

from numpy import *

pop_size = 2000
l = 200
z = zeros((pop_size,l))

def mutate () :
        for i in xrange(pop_size):
                for j in xrange(l) :
                        if random.random()<0.5 :
                                z[i,j] = random.random()

import cProfile
cProfile.run('mutate()')

def mutate_matrix () :
        r = random.random(size=(pop_size,l))<0.5
        v = random.random(size=(pop_size,l))
        k = r*v + logical_not(r)*z
        
cProfile.run('mutate_matrix()')

If you run the code listed above you may get something similar to

Synchronizing Mac OS X Mail rules across machines

Thu, 23 Oct 2008 20:53:58 +0000

If you are switching between mac and do not use mobile.me, but you want to carry your mail rule’s around you can play a little trick. The file that contains the mail rules is located at

~/Library/Mail/MessageRules.plist

You can just manually copy it from one box to another (you may want to make sure that the target machine does not run Mail), or you can create a little cron entry that uses rsync to keep them in sync.

On the road again for Internet2 and Bamboo

Mon, 13 Oct 2008 14:19:12 +0000

Yesterday I just got to New Orleans for the Internet2 fall meeting. I was invited to give a talk about work we are doing on the SEASR project at NCSA. SEASR fosters collaboration through empowering scholars to share data and research in virtual work environments. The SEASR project is funded by the Andrew W. Mellon Foundation. The last part of the week I will be at San Francisco joining the Project Bamboo workshop, again representing the SEASR project seeking a better understanding of possible synergies.

May be someone got an eReader right?

Thu, 11 Sep 2008 14:32:26 +0000

I have been looking over eReaders for a while, but I never found one that look quite right and cosy like good’old paper. Yes, Sony has his own eReader and Amazon launched Kindle, but still feel they are too small—not to mention aesthetics criteria. However, Plastic Logic is getting ready to release their take to eReaders based on their plastic electronics technology. Slim, light weight, days of battery life, and the ability to visualize all sorts of document formats (not like the Sony or Amazon ones) are some of the claims about the device. There is a presentation video on the Plastic Logic site, and a life demo on TGDaily—found via Engadget.

Who does your intranet link to?

Wed, 10 Sep 2008 05:22:32 +0000

Have you ever wondered who does your intranet link to? I was sitting the other day in a meeting (yes, I know, breaking news) and I was wondering what would be a fast way to be able to answer the question. The basic sketch I did in my mind was simple:

Set up a web crawler to the domain I want to analyze
Run the crawling job
Get the links collected on the web map
Process the links to only keep the site they refer to
Remove duplicates
Visualize the graph

Simple isn’t it? So, what do I need to get it to work? Basically three pieces of software (a web crawler, a graph manipulation library, and a visualization package) and some glue. Going over the things I been playing for the last year I draw three candidates: Nutch, RDFlib, and prefuse. Oh, the glue will be just two Python scripts.

New Semester and IACAT

Wed, 03 Sep 2008 20:52:39 +0000

Every new semester has its quickoff overhead. Besides trying to get back on track after travelling, and having meeting after meeting, now I am sitting at the inagural act of the Institute for Advanced Computing Applications and Technologies at the University of Illinois at Urbana-Champaign. A list and description of the three kicking off projects of the center is also available. These three themes are:

Quite and interesting mix to follow.

Having your notes always around

Wed, 13 Aug 2008 22:00:15 +0000

Remember everything. is the tagline of Evernote Corporation. Evernote concept is simple, your notes with you everywhere and always accessible. They version may seem a bit more accurate:

Evernote allows you to easily capture information in any environment using whatever device or platform you find most convenient, and makes this information accessible and searchable at any time, from anywhere.

I was a bit skeptical at the beginning. I have tried several times to use the Google notebook, and I never felt they got it quite right. I had to force myself into it. So far Evernote is looking promissing. Evernote, besides the web app interface to your notes provide desktop apps (for Mac and Windows), cellphone apps, clippers, and more. Definetely worthed to take for a spin.

Load balancing and high availability

Thu, 31 Jul 2008 04:56:45 +0000

One of the benefits of working with people smarter than you is the chance to never stop learning. My office mate, Bernie Ács, is an extraordinary source of experience and wisdom on production systems. Just right after coming back from vacation, he started unfolding a virtualized solution for high availability and load balancing of services. It is not me the one who should disclose the solution, but I just want to mention a piece of the puzzle that I found very interesting: ldirectord. In the words of the ldirectord developers:

Penrose, conciousness, and delayed homework

Wed, 30 Jul 2008 03:56:16 +0000

Long time delayed, I finally got a copy of The Emperor’s New Mind by Roger Penrose. The book was originally published in 1989. Yes, I never read it from cover to back, and finally yesterday I run out of self imposed excuses and got me a copy. I guess that it was the result of listening to the Teaching Company lecture series The Phylosophy of Mind by John Searl, also falling on the same alley. Recreative reading, may be, but I am still fascinated to keep running into belivers and detractors of strong and soft AI almost 20 years after Penrose’s book was published.

Yes, GECCO 2008 was intense!

Mon, 28 Jul 2008 14:35:08 +0000

Yes, I disappeared from my blog during GECCO 2008. Yes, I started blogging about the International Workshop on Learning Classifier System 2008 held on Sunday 13 at GECCO 2008. The workshop was terrific. Lots of new ideas were presented, but more importantly, lots of new ideas end sparking in participants head. As usual, the big workshop family went to dinner together where we all end having a blast. Some good traditions do not change, fortunately. Then, GECCO started. Monday, Tuesday, and Wednesday have been the most unusual GECCO days in my 10 years of attendance. Yes, I presented our paper and attended some interesting talks, but what made it different was that I spend a lot of time on hallways talking to people. There was an unusual concentrations of very interesting people to talk to, not to mention friends I have not seen in a long time. So, overall was really fun, and now I am trying to catch up with the papers published on the proceedings…

ILWCS 2008 live

Sun, 13 Jul 2008 15:32:17 +0000

The 11th edition of the International Workshop on Learning Classifier System 2008 is hot. So far a lot of idea exchange and interesting discussions. So far Gilles Enee, myself, Natalio Krasnogor, Albert Oriols, Thyago Duque covering map problems, encoding language and model building, TSP and metaheuristics, learning association rules, and multi class labeling. Had to rush back to the next one :D

Leaving for GECCO 2008

Fri, 11 Jul 2008 13:56:19 +0000

Yup, I am just packing on the run. Should get to Atlanta late afternoon or early evening. Wish me luck :D A bit of information about the city provided by Wikipedia

Atlanta (pronounced /ətˈlæntə/ or /ætˈlæntə/) is the capital and the most populous city in the state of Georgia, and the core city of the ninth most populous metropolitan area in the United States at 5,278,904. It is the county seat of Fulton County, although a small portion of the city extends into DeKalb County. As of July 2007, the city of Atlanta had a population of 519,145[4], and a combined statistical area population of 5,626,400[5]. Residents of the city are known as Atlantans.[6]

A simple skeleton plugin for Wordpress

Fri, 04 Jul 2008 04:02:14 +0000

Preparing for the upcoming Meandre release, as part of the larger SEASR on, I have been exploring a bit the inner machinery of WordPress to create a simple plugin for the SEASR site that could render Meandre component and flow descriptions straight out of their RDF desciptors. Actually, it turned out to be a breeze.

After being a heavy user of WordPress in several project for few years now, its extensibility keeps sparking new possibilities in my mind every time I look into any of its facets. Anyway, I am rambling. Back to the point. To hit the ground running, I started by creating a simple generic plugin that would replace a tag with parameters for something else. I looked into several freely available plugins, the one that got me going was a very simple one, the SlideShare plugin by Joost de Valk.

Efficient storage for Python

Tue, 01 Jul 2008 21:44:57 +0000

Did you ever run into the situation that your analysis/simulation data is too large to fit it in memory? Does the flat file format you use for storing your data sets become to big that renders it slow to a crawl? If you answered yes, you may want to give a spin to the HDF5 library. HDF5 file are not replacement for relational data bases. They are catered for storing complex data objects and a wide variety of metadata. It is also optimize for efficiency of storage and retrieval. The underlying library is written in C. If you are a Python user, PyTables provides a very efficient wrapper for HDF5 files. It gives you access to all the HDF5 api, plus it is nicely integrated with NumPy and provides natural naming conventions. In another words, you can quickly store and retrieve your arrays/matrix to HDF5 files, giving you a very interesting persistence layer. For instance you can do a simple table scan by:

Back to Urbana despite Continental Airlines

Thu, 26 Jun 2008 04:17:38 +0000

Yes, I am back. Almost 28 hours later than expected, but I am back to Urbana. The trip started with a very tiny little delay of 8 hours in Barcelona. Yes, COA121 was the flight to Newark. Yes, COA121 was the first leg toward Urbana. Yes, the plane had technical problems on the way to Barcelona and had to land at the Azores islands to get it “fixed”. Yes, we did not dare to ask. Yes, we were tired of waiting for a plane that seemed never to come; but you should have seen the faces of the passengers deplaning at Barcelona, exhausted is not a word descriptive enough. Yes, we all filled a complaint form asking for money back according to the European Union bill of right for air passengers. Yes, we know that it was a bit of wishful thinking and the lack of anything better to do while waiting.

Managing your digital library of research papers

Thu, 19 Jun 2008 11:28:38 +0000

A while a go I wrote about tools for managing your LaTeX bibliography. Despite the fact that the tools I described help managing your LaTeX bibliography collection, it still did not help much with managing the tons of PDFs files you end pilling up when doing research on a particular topic. BibDesk has now the ability to attach files to entries, Zotero with its ability to store snapshots is still the closest thing I have found so far. However, a friend just pointed me to Papers, a Mac tool—yes it is just available for Mac—for managing your digital library of papers. Very much like iTunes, it allows to streamline your search, reading, organizing, and writing—there is a very interesting webcast by the creators of the software. If you have a Mac, it is worthwhile to give it a spin.

ICEIS 2008: Blogging summary and final strings

Wed, 18 Jun 2008 15:28:37 +0000

If you are looking for a list of the related blogging done during ICEIS 2008 just follow this link. During Sunday morning I run into Angel A. Juan, an assistant professor at Open University of Catalonia (UOC), interested on analyzing online teaching efforts and how tools to assist professors monitoring students performance on online media. I visited him yesterday at his office and we got and interesting exchange of ideas. Most of them revolved around the work we have conducted under the DISCUS project, and how similar is our efforts on marketing focus groups and their online teaching environment. His group, Distributed, Parallel and Collaborative Systems, was also interested on the work done under the SEASR project, mostly focusing on the Meadre infrastructure for data-intensive flow computing we are getting close to release.

ICEIS 2008: Final sprint and Ricardo Baeza-Yates

Mon, 16 Jun 2008 11:10:46 +0000

This is the final sprint for ICEIS. I have been mostly focusing on posters this morning. It his hard to pick one up. I would just say that there was some interesting work on personalized recommender systems—paper 219. But as I said, there were a bunch of interesting ones and quite a few interesting by-the-poster conversations. Actually, I am having a very interesting time after the mix of attendees’ profiles. The morning finally meandered into Ricardo Baeza-Yates’s keynote talk. After the initial technical problems—presentation mode of OpenOffice running on Ubuntu 8.0.4 got up just 75% of the slide area—they finally succeeded on getting something up and get the talk started. This was a pretty technical talk about Yahoo! research effort on caching to improve the performance and also help scalability and contain cost on the coming years. Besides several cache techniques, he also presented a bunch of possible paralyzation models based on document/term partitions. A thing he breezed over was the machine learning model for classifying queries. That surfaced several places, from predicting common and rare content, to frequent, unfrequent, and rare queries. I was glad that the technical problems were solve and we could enjoy it. And the conference is finally close. Next year, Milan.

ICEIS 2008: A cloudy Sunday in Barcelona

Sun, 15 Jun 2008 15:14:45 +0000

The morning started with Jean-Marie Favre and his invited speech about “Software languages through the ages”. Quite an eclectic presentation that was quite a bit thought provoking. Some excerpts out of his amalgam of concepts:

Human kind is defined by language
Civilization is defined by writing
Languages expand across a vast period of history, computer science for just a few millimeters in such history line

You may agree or dispute his claims, but you cannot dispute that it was a thought provoking talk. His bottom line, the next research frontier software language engineering and software linguistics. You can find more information at planet-sl.org. Then I run to attend the presentation of paper 263 and 344. I would mention Texas Tech’s SORCER effort and their effort on service-oriented infrastructure to approach programming large-scale networked systems—their approach also take from the get go issues like availability and fault tolerance. Unfortunately I could not finish the presentation because I had to run because it was my turn. The afternoon started with papers 331, 346, and 723. The last one was presenting IBM’s work on modeling life cycles, and their effort of making it a generic model. The last round of papers 87, 614, and 655 where rather eclectics, RDF, RFID tags, and Wikipedia. Quite an interesting combinations. Also the afternoon was crowded with interesting hallway conversations, despite the gray day outside :)

ICEIS 2008: Slow afternoon and Moira Norrie

Sat, 14 Jun 2008 16:31:50 +0000

This afternoon has been quite slow. My path 340, 348, 495, 612, 193, and 467. The key slower of the afternoon has been the fact that for each of the session there has always been a missing presenter. May be because it is a Saturday afternoon in Barcelona, but then I check the rest of the parallel tracks and it seams quite endemic it. Anyway, I run into an interesting talk about how to map workflows onto multicore architectures—paper 340. It was just a talk that lead straight to scheduling problems of direct acyclic graphs. I ask about the cyclic cases and their approach was to break the cycle and use the acyclic schedule. Mmh. I also found myself how they plain to make those schedulers a reality without messing with the underlying OS one. Another paper that cough my attention was paper 193, where they were focusing to fraud risk reduction. The interesting twists there were that (1) it is, by nature, an unsupervised problem, and (2) they were trying to do both detection and prevention. Unfortunately the only evolutionary computation related paper— paper 306-–had no presenter… A total different story has been Moira Norrie keynote speech. She has presented their research efforts on paper & the digital world. A very interesting presentation covering from basic technologies—Anoto-–and latest digital paper products—Livescribe-–to their research on interactive paper. The basic idea, paper that allows you to interact with the content—for instance multimedia one. Their research involves from clicking on paper, to gestures, to the infrastructure (iServer) their cross-media information server, and their plugins—such as iPaper. Active components (no, not the Microsoft ones) are the bridges they build to control media from a paper and digital pen. Of course, they can also record annotations, whiteboards, audio, or also support collaboration via multiple pens usage. Other interesting usages of the digital paper was information search and proofreading to mention a few. You can find more information on Moira’s home page.

ICEIS 2008: Saturday morning jam

Sat, 14 Jun 2008 10:32:53 +0000

My morning jam involved papers 102, 363, 395, 450 709, 234, 392 and 499-–that included a poster session too. There were some puzzling questions running around my head. Just one example out of paper 450; there are always many terminologies meaning the same and rewrite or revolve around the same problem: where is the difference between multicriteria optimization instead of multiobjective optimization. As I said, puzzling. I also ran into a poster during the jam by Intel folks (paper 234) worth to mention. It was not proposing new technology but building on the creation of distributed data centers via virtualization technologies. They coined the term SVG’s to describe virtualization + data isolation + services providing. More on the level of exercise to show how people can move to virtual data centers hosted in the cloud, than on proposing new technology—they said they run VMWare for all the virtualization. The question still buzzes in my head unanswered: How is this new or different from Amazon’s EC2? The answer was that they focusses on the infrastructure not on hosting—which puzzled me more after claiming they rely on VMWare-–I guess that I will need to dig deeper to get a better picture of their work. Just getting close to the lunch break, paper 195 presented some work on an engineering approach to determine the emotional signature on clothing website. Built around the Kansei method, the paper resonate in my head very close to the Gladwell’s Blink book.

ICEIS 2008: Jorge Cardoso keynote speech

Fri, 13 Jun 2008 16:55:04 +0000

Jorge Cardoso (SAP Research @ Dressden) presented their efforts on Thetsus/Texo. They are using the SOA paradigm as a way to reengineer business processes, pushing the good’old functional design decomposition to the next level. The building stone of their work is the ISE methodology that proposes to streamline as follows.

Innovation
Requirements
Design
Implementation
Preparation to market
Market lunch

The interesting twist of the methodology is that it also tries to treat each of these steps as services, raising again the buzz word of the conferences “business services”.

ICEIS 2008: Second Friday afternoon session

Fri, 13 Jun 2008 15:35:21 +0000

My second afternoon session was formed by paper 506, 553, and 582. Machine learning was the pervasive fabric for this session. From bayesian network modeling for network intrusion detection and machine learning for process mining, to music information retrieval and similarity measures. All of them were interesting on their own, but I would like to write down some of the thoughts about paper 553. The paper focussed on process mining. Basically from a log of a process they reconstructed the events and graph and tried to induce the business rules governing the process. They authors took a divide and conquer approach to simplify the modeling inside smaller regions of the problem—very similar to the GALE approach back in 1999, or the mixed decision trees paper. After the talk was over I kept wondering where is the connection between process mining and provenance mining—provenance can be defined as the execution history of computer processes which were utilized to compute a final piece of data. Quite an intriguing thought.

ICEIS 2008: First Friday afternoon session

Fri, 13 Jun 2008 14:44:48 +0000

I attended four papers (you can check the abstracts of Friday papers here): paper 304, 323, 538, and 434. On of the paper has nobody to present it. There were quite a bunch of interesting ideas. I just want to mention one of them. Two papers presented several methodologies to model SOA. Another interesting approach was the claim (papers 304 and 434) that SOA can be used for modeling business processes, and have little to do with information technologies. They claimed that such a connection to information technologies should be done once the SOA business processes are well defined. This may sound quite estrange if you are coming from a technical background. However, it is not far fetch. It just sound like a reincarnation of the late nineties hype on business reengineering via functional designs—which SAP rided so successful. Interestingly Jorge Cardoso also hinted this direction during the morning panel.

ICEIS 2008: Blogging from Barcelona (Friday Morning)

Fri, 13 Jun 2008 14:26:52 +0000

The hotel finally got the wireless system rebooted, so I am finally back up. I found quite unusual to open the ICEIS 2008 conference with a panel form by the four invited speakers: Moira Norrie (Global Information System Group, ETH), Jorge Cardoso (SAP), Jean-Marie Favre (UFR, IMA), Ricardo Baeza-Yates (Yahoo! Research Barcelona). Each of them did a short introduction of their invited talks, and just three questions were asked. Since the conference started at 11am, after the panel, lunch break :D

Crash course on threading in Python

Mon, 09 Jun 2008 11:35:38 +0000

Are you familiar with threading in Java and looking for a crash course in Python? If the answer is yes, I just found an article written by Peyton McCullough that may help you. The “Basic Threading in Python” article can take you from no clue to writing your threaded Python code in a few minutes. See the example below, extracted from his article, to see if that rings a bell :D

Journal d'Hirondelle

Fri, 06 Jun 2008 11:31:08 +0000

Walking the streets of Barcelona—oops didn’t I mention I am on vacation?—I run into a bookstore near las Ramblas. Meandering around the narrow stacks, I found the newly released Catalan translation of Amélie Nothomb’s “Jornal d’Hirondelle” (Diary de l’oreneta/Diary of the swallow). The first book I read from her was “Méthaphysique des tubes”, and she got me at the first line. I need to speed up with Ulam’s autobiography—extremely addictive—so I can move on to this one.

The next generation of data bases

Thu, 05 Jun 2008 11:39:34 +0000

Yesterday I was reading an interview to Brian Aker (MySQL director of technology) I found via Slashdot when something caught my attention.

On the second side of this which may actually be more exciting is the issue of–instead of the structured data world of the relational database but the semi–the semi-structured world. You look at what is being done today with CouchDB, you look at Amazon ScaleDB, to a lesser extent but to a similar extent you–not ScaleDB, SimpleDB–to a lesser extent or a similar extent Tokyo Cabinet, those databases are really kind of fascinating because those databases are redefining really how we access data and how we are going to be searching and using data. So there’s a whole world out there that’s just starting to open up in that direction.

iPhone and your Wordpress

Wed, 04 Jun 2008 12:57:04 +0000

A while back I wrote about a Wordpress theme specially designed for you iPhone. Yesterday, a colleague make aware of another one. This one is called WPhone Admin Plugin and below you can find its description in their own terms.

WPhone allows you to use a custom admin interface while interacting with your WordPress install via your phone. It contains two versions of the mobile admin interface:

Making blogging easy

Tue, 03 Jun 2008 13:08:01 +0000

ScribeFire is a Firefox extension that allows you to blog from the browser itself—as shown for this blog on the above screenshot. ScribeFire allows you to post to different blogs, edit past entries, select the category, and modify other post information such as the publication post date. The ScribeFire extension also allows you to blog about the current page you are browsing or to add it to your delicious bookmarks.

Looking for code examples?

Mon, 02 Jun 2008 14:13:03 +0000

Are you looking for some code sniped? Are you looking for some freely available code available on the net written on a specific language? If the answer is yes, you may want to take a look at Google Code Search. I just canned a few examples below.

Managing your LaTeX bibliography

Fri, 30 May 2008 13:24:13 +0000

There are a lot of tools out there to help you manage your LaTeX bibliography collection. I just want to mention three of them: BibDesk, Zotero, and JabRef. I am a heavy user for the first one. I run into BibDesk the first time I installed LaTeX on a Mac. BibDesk was the tool that stopped me from writing my .bib files by hand—as I did for a long time. Besides allowing to manage your .bib files easily, it provides a nice service integration with TeX. BibDesk also come with a great intuitive graphic interface, the ability to hook you to libraries on the net, and it allows you to export entries to EndNote, Word, RTF, Atom, RSS, etc. Zotero is another great tool for managing your bibliography if you do not always have a Mac at hand. Zotero is a plugin for Firefox. It provides the same functionalities of BibDesk but with a twist. When browsing over the net, if you run into a site which is COinS aware (libraries, newspapers, blogs, etc.), with a simple click you can store that bibliography entry straight to the Zotero bibliography data base. The bibliographic entries are also exportable to BibTeX, RDF, and RIS—to mention a few. The last one is JabRef. Written in Java, you can easily use it on any platform that supports Java. They also provide a WebStart version of JabRef. As BibDesk and Zotero, you can import/export bibliography entries in a wide variety of formats, add annotations, etc. Also, the ability to use JabRef across different platforms may be a plus if your are moving between Windows, Linux, Mac, Solaris, etc.

Adventures of a mathematician

Thu, 29 May 2008 12:31:16 +0000

Preparing the list of reading for the summer vacation, I finally got a copy of Ulam’s autobiography. Mainly know for his contribution to the Manhattan project, Stanislaw Ulam may have been one of the most influential mathematicians of the 20th century. Close friend to another giant—John von Newman—, Stanislaw contributions have affected a wide range of scientific disciplines. A friend recommended me this book a while ago. The book went into the must-read queue, but till yesterday it did not make it to the head :)

On writing well

Wed, 28 May 2008 13:14:07 +0000

Writing can be a fun exercise. However, like many other skills, you have to do your best to develop the craft. Regularly I reread two books trying to remind me that I should never give up on trying to improve such an important skill. The two books are:

“Elements of Style” by Strunk & White
“On Writing Well” by Zinsser

Another one to also throw to the mix is “Writing with power: Techniques for mastering the writing process” by Elbow. However, If you are looking for something short and concise “Technical Writing for Fun & Profit” by David E. Goldberg is a must-read recipe for engineers.

Merging separate SVN repositories

Tue, 27 May 2008 12:18:35 +0000

I needed to merge together several SVN repositories together. After browsing a bit, I found a simple perl script named svn-merge-repos.pl published on December 2007 by Fabien Coelho. This script allows you to merge multiple repositories, properly interleaving the revisions by date. It also allows you to set specific target paths for the repositories being merged. The only issue I ran into was the limitation that you cannot target the same new directory as the destination for multiple repository sources on the same run. You can get around it using the usual svnadmin dump and svnadmin load sequence, but it is not as beautiful and it does not interleave the revisions…

Getting started with LaTeX

Mon, 26 May 2008 14:17:36 +0000

Yesterday I was looking for a quick start guide for LaTeX that could help someone who never used it before to get going. After a bit of browsing I ran into a quick guide to LaTeX by Joost Winne. It may not be the most exhaustive and detailed one, but it is well organized and definitely a good place to start. If you are looking for more nothing can beat “The LaTeX Companion”.

Doctor Who?

Fri, 23 May 2008 12:54:26 +0000

Rose Tyler and The Doctor traveling away in the Tardis again. Long long time since last time we met. I am just catching up with the good old Doctor. The new BBC version is pretty interesting, although I still miss some of the old low budget versions. Started season three yesterday while typing some stuff away :)

ZooKeeper and distributed applications

Thu, 22 May 2008 13:39:36 +0000

Lately I have been exploring different alternatives for coordinating the execution of distributed applications. Yes, you guessed it right, I am working on the distribution of the execution of Meandre flows. Chopping the data-intensive flow and mapping the chunks onto a set of distributed processors requires several elements (graph analysis, resource management, etc.). However, the basic element that needs to be solved first is the need for a reliable and scalable coordination system.

Publish yourself online

Wed, 21 May 2008 14:14:46 +0000

A while ago Kumara Sastry introduced me to the beauty of SlideShare. I have been using SlideShare services for a while in the little blogging I do to upload, share, embed, and display some of my presentations. The other day David E. Goldberg introduced me to Scribd. The site takes the SlideShare idea further and allows you to use their services as a publish-it-yourself site. It also allows you to embed the document on your page, as SlideShare does. A slightly different flavor of the publish-it-yourself idea is Lulu. Here, the gears are more oriented to have an online cheap publisher where you are your own editor. As I said, a different flavor for a different purpose.

SVNKit or analyzing SVN content in Java

Tue, 20 May 2008 14:47:52 +0000

The other day I was looking for a piece of software that could help me pull data out of SVN repository so I can do a little analysis on it. Browsing over the net, I run into several tools, but the one that finally caught my eye was SNVKit. The package does a lot of stuff, but for instance, if you are only interested on pulling information out, it makes your like quite easy. The code below is just a simple example of how you can use it to pull information out of a SVN repository.

Google analytics as a website optimization tool

Mon, 19 May 2008 13:35:14 +0000

Interesting twist using Google Analytics to do site optimization. The interesting part is the shift of mentality. Instead of scratching your head going for the best, why not put a bunch out there and using as a row data for the next generation. Sounds familiar? Yes, they will not mention it, but it heavily uses a simple evolutionary model. Using they words: you start with a basic set of variations, then you measure, oh you may want to try another set of variants based on the best ones. Yes, definitely evolutionary-based flavor. After going over the presentation, you will be wondering why they do not called interactive genetic algorithm? Well, I do not have the answer, but what I can say is that process can be modeled using an interactive genetic algorithm. The DISCUS project has shown that for the last five years. An introductory paper about how interactive genetic algorithms can help with such kind of modeling can be found here.

"Yo soy aquel" beating "Perrea, perrea"

Sun, 18 May 2008 17:53:47 +0000

Eurovision Song Contest is not dead after all. A simple proof is the amount of ink generated after the “ChikiChiki” (Spain entry to the contest), not to mention all the other curious participants from all over Europe (including the singing duck…). Thinking about it, I still feel that the ChikiChiki is dull and a bit borring when compared to the master of all the editions: Raphael. The above video was his performance on the 1966 edition.

Consolidating my blogging

Sun, 18 May 2008 06:42:05 +0000

Finally! I have been able to consolidate two of my blogs into this new domain :D.

I just moved over my old IlliGAL blog and my newer DITA one. I started blogging after the push at IlliGAL, and when I moved to NCSA I took the habit with me into DITA. There are also a few more satellite blogs around (that I may or may not bring over here), but the final goal is to regroup energies and restart the blogging in a more unified manner. Let’s see how it goes…

Meandre: Semantic-Driven Data-Intensive Flow Engine

Sat, 19 Apr 2008 00:30:22 +0000

Finally we have finished setting up the website for Meandre a semantic-driven data-intensive flow engine. Meandre provides basic infrastructure for data-intensive computation. It provides, among others, tools for creating components and flows, a high-level language to describe flows, and multicore and distributed execution environment based on a service-oriented paradigm. We are currently working on getting gear up for a first alpha release. You can visit the Meandre site here. I will be posting in the Meandre blog about our current steps toward getting the release out of the door. The Meandre infrastructure is being build to support the SEASR project

[BDCSG2008] Summary of BDCSG2008 blogging

Thu, 27 Mar 2008 04:16:48 +0000

It has been a greet meeting. Lots of interesting ideas and a lot to explore from now on. Just what I like :D. I summarized below the list of post I make related to the meeting.

Blogging from the Big Data Computing Study Group 2008

Thu, 27 Mar 2008 04:05:25 +0000

I was lucky to attend the Big Data Computing Study Group 2008. The line of speaker is impressive. The event was held at Yahoo! Sunnyvale, and Thomas Kwan (UIUC alumni know at Yahoo!) helped organize it. I blogged about it on my DITA blog where you can find links to all the related posts.

[BDCSG2008] NSF Plans for Supporting Data Intensive Computing (Jeannette Wing and Christophe Bisciglia)

Thu, 27 Mar 2008 00:39:06 +0000

NSF listens at you academics. Jeannete opens the floor with this claim. Questions: What are the limitations of this modeling paradigm (data-intensive one)? What are meaningful metrics of performance here? What about security processes and data on a shared resource? How can we reduce power consumption? Can this parading problem not possible otherwise, or simplify them, or open the door to new applications? NSF rolling out cluster exploratory program, also going to roll out a new solicitation for Data-Intensive Computing. Also emphasizing from data to knowledge, since scientist are throwing it away. This is a great opportunity for collaborative efforts between CS and scientist. NSF goal: provide access to cluster resource and access to massive data sets. Google and IBM rolling out the cluster (for academics). NSF will roll out a cluster exploratory will be the solicitation program announced yesterday to distribute access to the cluster and research grants. Review of Christophe experience on teaching a class about clustering, and he realized that providing away computer cycles is more valuable than plain grant money. It runs on Hadoop. The cluster will be allocate by rack weeks, 5 Terabytes and priority on 80 processes (but still people there and lower priority and large data sets). And since the reviewing was not Google expertise they reach to NSF to use it. Googler to start collaborations and IBM will also help providing support for it. Jeannette claiming this is a new model, but NSF is open for new model and other partners.

[BDCSG2008] Data-Rich computing: Where It’s All (Phil Gibbons)

Thu, 27 Mar 2008 00:18:08 +0000

The next speaker of the afternoon is Phil Gibbons from Intel Research. Intel has created a research theme on data-rich computing for the next few years (same as the other one presented on the Hadoop summit about ground modeling). An approach, bring the computation to the data (cluster approach), but there are also two elements in the picture: (1) memory hierarchy issues, and (2) pervasive multimedia sensing. The first one is in important because for pure performance, the second one keeps forcing pushing the computation closer to the sensors. The memory hierarchy implies that multi cores share a common L2 cache, and the farther we move the bandwidth drops, and you can keep pushing in HD/SSD. And this basic unit is what gets replicated to build clusters. (Little note about SSD rewriting quirk and how cache coherency plays in the overall picture). All this lead to HI-SPADE project (Hierarchy-Savvy Parallel Algorithm Design). The goal is to hide as much as possible, but only expose what is important to tune. Continues with an example of cache misses and how that can be palliated with the right scheduler. Phil then moved to show how that would work on parallel merge sort (solution á la merge first depth search), also compared later with hash join, and again, no free lunch (they run into the usual one worst like a champ, the other fails miserably). The next topic on the presentation when to the quirk of SSD (flash based). The improvements over traditional HD reach a 3 order magnitude. But again, random rewrites in flash are painful, but there is a way to express semi-random algorithms may be help palliate the problem. Shifting gears, pervasive multimedia sensing is the next topic on the arena. Phil start reviewing sensor networks and how the sensors are becoming more powerful, but more important, the numbers are growing and scaling up (exponentially). Again, moving to their example of the IrisNet project, and how pushing computation down helps also with the distribution of the data, pushing their results as feeds (XML) into aggregations nodes (in a tree shape). Once aggregated, they provide the usual distribution, replication, and querying.

[BDCSG2008] Scientific Applications of Large Databases (Alex Szalay)

Wed, 26 Mar 2008 23:38:03 +0000

Alex is opening the talk showing a clear exponential growth in Astronomy (LSST and the petabyte example generation). Data generated from sensors keep growing like crazy. Images have more and more resolution. Hopkin’s databases started with the digital sky initiative, with generated 3 terabytes of data in the 90’s, and the number keep growing up to the point of LSST which will be forced to dump images because is not possible to store all of them. SkyServer is base on SQL server and .NET, serving the 3 terabytes, serving SQL queries reaching 15 millions queries recently. Alex presented a revision of the usage and data delivery. They are planning to anonymize the log and make it publicly available. Then, he switched gears toward the immersive turbulence project that seeks to generate high resolution turbulence images. Again they are storing the information on a SQL server. Moving gears to SkyQuery. A federating web services to build a join query on the fly, but the problem is some join results turn out to be unfeasible. The revision of projects then moved to “Life under your feet”, a non intrusive way to measure environments. The key component is the aggregation and drill down mode for exploring those aggregates down to a sensor level. Another one, OncoSpace, oncology treatment evolution based on the comparison of images of the same patients and their evolutions across time, again implemented on the same SQL server. But all this projects has commonalities, indexing and the need to extract small subsets from larger datasets. MyDQ targets to extract data from other databases and services and leave it on a relational database the user can use. Graywulf, there is no off-the-shelf solution for scientific 1000 TB of data, so the solution is to scale it out. They took the root of fragmentation and create chunks easy to manage, again using SQL server clusters. They also introduce a workflow manager to monitor and the control of it. And this led them to create a Petascale center to play with at John Hopkins University, build on Pan-Starts.

[BDCSG2008] Sherpa: Cloud Computing of the Third Kind (Raghu Ramakrishnan)

Wed, 26 Mar 2008 22:44:14 +0000

Raghu (former professor at Madison Wisconsin, now at Yahoo!) is leading a very interesting project on largely scale storage (Sherpa). Here you can find some of my unconnected notes. Software as a service requires to CPU and data. Cloud computing using assimilated to Map-Reduce grids, but they decouple computation and data. For instance Condor is great for high-throughput computing, but on the data side you run into SSDS, Hadoop, etc. But there is a third one, transactional storage. Moreover SQL is the most largely used parallel programming language. Raghu wonder why can’t we build on the lesson learned on RDBMS for OLTP. Sherpa is aiming not to support ACID models, but massively scalable via relaxation. Updates: creation, or simple object updates. Queries: selection with filtering. The vision is to start in a box, if it needs to scale, that should be transparent. PNUTS is part of Sherpa, and it is the technology for: geographic replication, uniform updates, queries, and flexible schemas. Then he goes and describe the inner parts of PNUTS and the software stack. Some interesting notes, no logging, message validation, no traditional transactions. Lower levels put and get a key, on the top of it ranges and sorted, PNUTS on the top provide the querying facility (insert, select, remove). Flexible schemas, the fields are declared at the table label, but do not need to be present (flexible growth). Records are mastered on different nodes, during the utilization, the masters can migrate depending on the usage of them. The basic consistency model is based on a timeline. Master writes and reads are ordered, others can catch up in time. Load balancing by splitting and migration, and guaranteed by the Yahoo! Message Broker. The goal, simple, light, massively scalable.

[BDCSG2008] “What” goes around (Joe Hellerstein)

Wed, 26 Mar 2008 22:10:01 +0000

Joe open fires saying “The web is big, a lot of monkeys pushing keys”. Funny. The industrial revolution of data is coming. Large amounts of data are going to be produce. The other revolution is the hardware revolution, leading to the question of how we program such animals to avoid the dead of the hardware industry. The last one, the industrial revolution in software, echoing automatic programming. Declarative programs is great, but how many domains, and which ones can absorb it. Benefits: Rapid prototyping, pocket-size code bases, independent from the runtime, ease of analysis and security, allow optimization and adaptability. But the key question is where is this useful? (besides SQL and spreadsheets). His group has rolled out declarative languages for networking. That includes routing algorithms. other networking stacks, and wireless sensor nets. His approach is a reincarnation of DATALOG. It fits the centrality of the graphs and rendezvous in networks. After this initial issues P2 has been used for consensus (paxos), secure networking, flexible data replications, and mobile networks. Currently other applications being build: compilers, natural language, computer games, security protocols, information extraction, modular robotics. The current challenges they are facing include a sound system design, language facing the usage on real world programing, lack of analysis for the languages, and not turing complete, connections to graph theory and algebraic modeling, efficient models for A*. Another challenge is how you do distributed inference and metacompilation to provide hardware runtimes. The data network uncertainty and P2 can help solve the embedding of the routing information, the network routing informations, and the conceptual networks together, and being able to express them together. Evita Raced is the runtime for P2 (a simple wire data flow bootstrapper). More info here.

[BDCSG2008] Mining the Web Graph (Marc Najork)

Wed, 26 Mar 2008 21:34:37 +0000

Marc takes the floor and starts talking about the web graphs (the one generated by pages hyperlinks). Hyperlinks is a key element of the element. Lately webpages has an increase of the number of links, usually generated by CMS (for instance navigation). However, there is a change on the meaning of those hyperlinks. Analytics have different flavors, for example page rank is pretty simple, but others require random access, requiring memory storage (requiring to to huge re graphs in memory). Using their own Microsoft tools, they distribute and replicate it in a cluster, to be able to run some of these analytic algorithms (for instance HITS for page ranking). Sampling can help deal with high a-rity nodes in a graph. He continues presenting the SALSA algorithm (successor of HITS). SALSA requires sampling, and Marc suggest that uniform works pretty well) However, how you evaluate the ranking algorithms? Compile a truth set? Sometime assembled by humans (may not know what the intend of the query was), but another alternative is to use click logs (potentially biassed toward the first results presented). As a field, he claims about the need to collaborate with social sciences to model and better understand the meaning and motivations of hyperlinks.

[BDCSG2008] Algorithmic Perspectives on Large-Scale Social Network Data (Jon Kleinberg)

Wed, 26 Mar 2008 21:07:53 +0000

How can we help social science to do their science, but also how can we create systems from the lessons learned. This topics also include security and sensitivity of the data. He also review from the Karate papers to the latest papers about social networks. Scale changes the way you approach the data. The original studies allowed knowing what each link mean, but large scale networks loses this property. However he is approaching for a language to express some of the analysis of the social networks and processes. Also, how we bind information per user and how can we model users. But the also security policies. Diffusion in social networks and how things are propagated (even locally), but it is hard to measure how people change their minds on the diffusion process. Chain-letter study where the petition and the trace was collected, but they can also be forward to mailing list, but you can trace some some of the traces of the mailing list. The path were messed with mutations (typos) amputations, etc. They generate some algorithms for maximum likelihood of the tree assemble. But the output was unexpected, opposed to the six-degree separation, they found narrow deep trees. Why a chain-letter would run as a deep-first search? Time played a role. Since friends are small searches, and basically the replicated copies where discarded. The model of the trees was able to be replicated following this time dimension. Another element gets throw to the mix is the threshold of the diffusion. Basically, a message gets in, but how many inputs repetitions your require to validated it an pass it along? Results show that the second input the one that boost that threshold. Viral marketing is another example that wants to understand diffusion. All this leads to multiple models and how you integrate them. Privacy and social networks is another key element. How does that play? Is anonymation the way to go? Social network graphs, even if anonymized hints can lead to the deanonymation of the picture. Before the network is release you can add actions to it, and then you have something to roll back from. The idea create a unique pattern, and then ping them to other people. You can compromise a graph with square root of the log of the number of nodes. Jeff final reflections: toward a model of you. Models of human behavior are possible (for instance the model of time to reply email). But computers track more information about your behavior, opening the door to new modeling (something that the DISCUS project has also been postulating for the last 5 years).

[BDCSG2008] Handling Large Datasets at Google: Current Systems and Future Directions (Jeff Dean)

Wed, 26 Mar 2008 20:28:49 +0000

Jeff was the big proposer for map-reduce model–the map-reduce guy. Jeff reviews of the infrastructure and the heterogeneous data set (heterogeneous and at least petabyte scale), their goal: maximize performance by buck. Also data centers, locality, and center are key in the equation. Low cost (not redundant power supplies, not raid disks, running linux, standard network) Software needs to be reliable to failure (node, disks, or racks going dead). Linux on all the production. An scheduler across the cluster to schedule jobs. Cluster wide file system on top, and usually big table cell. The GFS centralized manager that manages metadata and allocations of chunks and replication (talk to the master and then talk to the chunk servers). Big Table helps applications that need a bit more structure storage (key, column, time-stamp). It also provides garbage collection policies. Distribution is break in to tablets (range of rows) and managed by a single machine, the system can split growing tablets. Big Table provides transactions and allow to specify local columns to group together. They allow replications policies across data centers. MapReduce is a nice fitting model for some programs (for instance, the reverse index creation) Allows to move computation to closer to data. Also allows to implement load balancing. GFS works OK for a cluster, but they do not have a global view across the data centers. For instance, they are looking for unique naming of data, and also, if integrated allow data center to keep working if they became disconnected. They are also looking for data distribution policies.

[BDCSG2008] Simplicity and Complexity in Data Systems (Garth Gibson)

Wed, 26 Mar 2008 19:04:32 +0000

Energy community and HPC. It is cheaper to collect a lot of samples and run simulations to decide where to drill (the extremely costly part). Energy community and HPC. It is cheaper to collect a lot of samples and run simulations to decide where to drill (the extremely costly part). Review of several effort one modeling for science making. They also run a collection of failures and maintenance cycles on hardware. Job interruptions becomes linear in number of chips (quite interesting result). Compute power on top 500 still doubles per year, and thus the failure rates. Lost disk becomes more and more painful (since the regeneration takes from 8h currently per terabyte to weeks in prediction). All this claims a change on the design. The approach is to spread data as much possible, and that can help to linearly reduce the reconstruction times. File systems are “object” file systems such as the Google FS and Hadoop FS. pNFS: scalable NFS soon. A key goal: reuse current tools people are using to speed up adaption and make it appealing to the users.

[BDCSG2008] Computational Paradigms for Genomic Medicine (Jill Mesirov)

Wed, 26 Mar 2008 18:37:29 +0000

Jilll is reviewing what is going with data and biology. There has been an explosion on the numbers they are generating data (from volumes to throughput). Simulations has also been common practices, robot operations, etc. more and more data. Some numbers, now their center use 4.8K processors and 1440+ Terabytes of storage. The challenge give the proper tools to biologist (not CS people). The two key topics of the talk: computation paradigms and computation foundations. They heavily rely on genome expression arrays (row patients, column genes, value expression values). A simple example: classify leukemias (example of how can be distinguished using expression arrays). Patient samples, extract messenger RNA and then create the expression signatures (high dimensionality low training sample set). They repeated the same problem for predicted outcome on prognosis on brain cancer, but for this program there was no strong signal to get them accurate enough. Genes work on regulatory networks (sets of genes), and they tried to do the analysis this way—acting as adding background knowledge to the problem—boosting the results and making the treatment possible. But, the problem is that there should be and infrastructure that could be easy to use and able to replicate experiments. Infrastructure should integrate and interact to components. Should be able to support techs and illiterates equally. Two interfaces (visual and programatic). Access to a library of functions, write pipes, language agnostic, and build on web services (scalable from the laptop to clusters). The name GenePattern. They are collaborating with Microsoft working on a tool (word document) to link to pipelines and the data in the data (can run with other version) and append results to the document too.

[BDCSG2008] Clouds and ManyCores: The Revolution (Dan Reed)

Wed, 26 Mar 2008 17:32:17 +0000

Dan Reed (former NCSA director now at Microsoft Research) continues with the meeting presentations. His elevator pitch: the infrastructure need to take into account applications and the user experience. Current trend is that monolithic data consolidation is crumbling under dispersion, changing the traditional picture. The flavors of big data can be explored along two dimensions: (regular/irregular) versus (structured/unstructured). He emphasizes on focusing more on the user experience with big data, and how you can manage resource at any given point. Cloud computing can help organically orchestrate this resources on demand. He also show some examples of Dryad (the Microsoft take on map-reduce architectures) and DryadLINQ. Another interesting comment:

[BDCSG2008] Text Information Management: Challenges and Oportunities (ChengXiang Zhai)

Wed, 26 Mar 2008 17:05:47 +0000

UIUC CS professor Zhai reviews texts information management. ChenXiang start reviewing the importance of text as a natural way to encode human knowledge. His main focus is how he can provide support for different usages of text information, and how they interact to models, applications, systems and algorithms. This allowed him to motivate future research directions on information retrieval. Some of his interesting words:

Future research directions require improvements on IR and NLP (shallow: POS, partial parsing, fragmental semantic analysis), but it is fragile and domain oriented. Machine learning algorithms are still no scalable and not enough training data to satisfy the algorithm requirements. Data mining has lots of algorithms, but only for salient patterns.

[BDCSG2008] Data-Intensive Scalable Computing (Randy Bryant)

Wed, 26 Mar 2008 16:31:00 +0000

Randy opens fire reviewing models of parallelisms and how Google’s Mpa-Reduce model (the core of Yahoo’s Hadoop) is changing the picture. He is emphasizing how data is and integral part of the computational process (which has been greatly unregarded). Map-Reduce model can greatly help because of it fault tolerant capabilities. Now he is reviewing the two traditional parallel programming models (shared model and message-passing model) and how this differ from map-reduce (and how this increases the IO). Initiatives like Hadoop allow to cut-down cost for accessing large scale computing.

Big Data Computing Study Group 2008

Wed, 26 Mar 2008 16:07:49 +0000

I am lucky to attend the Big Data Computing Study Group 2008. The line of speaker is impressive. The event is held at Yahoo! Sunnyvale, and Thomas Kwan (UIUC alumni know at Yahoo!) is helping organize it. I will keep blogging about it the rest of the day.

Generic looping in Python

Wed, 16 Jan 2008 16:00:12 +0000

I just posted on my IlliGAL blog how to implement a generic genetic algorithm (GA) main loop squeezing the dynamic behavior of Python. Pretty sleek, if you have tweaked GA main you main find this interesting ;)

A simple and flexible GA loop in Python

Wed, 16 Jan 2008 15:34:18 +0000

Yes, yes, I know, Python. I just have been playing around to see how much you can squeeze out of it and I am very surprise of how elegant it can be. Just and example. I am pretty sure you have written way to many customize GA loops that you need to tweak every now and then, if so take a look at the Python version below.

def evolve(self):
    """Implements a simple evolution loop"""
    func\_calls = \[\]
    func\_calls.append(self.evaluate)
    if ( self.env.selection!=None ) : func\_calls.append(self.selection)
    if ( self.env.crossover!=None ) : func\_calls.append(self.crossover)
    if ( self.env.mutation!=None ) : func\_calls.append(self.mutate)
    if ( self.env.replacement!=None ) : func\_calls.append(self.replace)
    \[f() for f in func\_calls for i in range(self.env.length/5)\]
    self.evaluate()
    return (self.best\_fitness,self.best\_individual)

Where the self.env object just contains references to the methods implementing each of the functionalities generically implemented in self.evaluate, etc. Pretty sleek.

Profiling Python Code

Thu, 10 Jan 2008 15:36:48 +0000

For the first time in 9 years, this vacation break I have done absolutely nothing. Wow what a coach potato I have become! Well that is not totally true, just for fun I started going over Python and, as usual, for any new language I end writing a simple genetic algorithm. I like the flexibility and compactness of the code (no verbose at all). However when I fire my first run (yes, the good old OneMax problem), I realized that some of my assumptions about coding did not directly transfer. Yes, it was a bit slow. So I started digging for a profiler and, surprise!, it comes with the Python interpreter. Here is an example on how to run the profiling capabilities

GECCO 2008 Deadline Extended to January 30th

Wed, 09 Jan 2008 17:27:31 +0000

I just got an email from the GECCO 2008 organizers extending the deadline till January 2008.

GECCO 2008 Deadline

Tue, 04 Dec 2007 15:15:54 +0000

The deadline for the next Genetic and Evolutionary Computation Conference (GECCO 2008) is approaching. Januray 16, 2008 is the date. Get ready!

Reasoning for the semantic web

Mon, 26 Nov 2007 15:04:04 +0000

Jena 2 Inference Support is a nice introduction to the inference engines provided by the Jena package. Besides standardized reasoning for RDF and a subset of OWL/Lite and OWL/All ontologies, it also provides the mechanisms for creating your own rule-based inference engine using the generic rule-based inference also provided.

Wordpress and your iPhone

Fri, 12 Oct 2007 02:40:28 +0000

WordPress iPhone / Mobile Admin adapts the WordPress admin UI to be more friendly to mobile devices, specifically phones.

Reset the MBR using Linux

Fri, 21 Sep 2007 14:55:45 +0000

Here is a simple trick if you need to clean it. Recently I needed to clean some Rocks Clusters node’s HD to allow it to reinstall dd if=/dev/zero of=/dev/HD count=1 bs=512 This avoid the computer to try to boot from the hard drive. However, if the node was already on the cluster, you need to run # insert-ethers --remove="compute-0-1" To remove it from the cluster, where “compute-0-1” is the name of the node in cluster. Then, you can start insert-ethers, and boot the node and it will start reinstalling itself.

TechnoCrunch 40 Conference 2007

Wed, 19 Sep 2007 23:23:27 +0000

TechCrunch40 Conference 2007 present 40 startups. If you want to procrastinate a bit, check the ideas they are proposing. Maybe they will get you inspired ;)

LCS researcher changes jobs

Wed, 19 Sep 2007 14:37:49 +0000

This is an excerpt from Alwyn Barry’s web page (Thanks Pier Luca for pointing it out)

Most people who know me will be aware that I am changing job shortly. I will be leaving the University of Bath from 30th September 2007, and will no longer be an academic. I am moving to Street in Somerset to become the Pastor of Street Baptist Church. I am still happy to answer any questions relating to my previous research, so do feel free to contact me via my new email address, which is linked from this site.

Embedded Jetty applications

Wed, 19 Sep 2007 14:25:00 +0000

I am currently working on an open source project (do not ask me which since it will surface soon, and I should not talk much about it till it does ;)) that required to provide web access to apps, services, and contents. From my days fighting with Mulgara descriptors I remembered that Jetty (full-featured web server implemented entirely in Java) could be embedded into applications to provide such services. It has been two months now since I started using, and it is a nice, shiny, and slick piece of software. I used Tomcat for most of my stuff, but Jetty is definitely and amazing alternative Below I just pasted one of the ways you can embed Jetty in your app.

Kumara Sastry Ph.D.

Fri, 07 Sep 2007 19:33:13 +0000

Kumara has just defended his Ph.D. thesis “Genetic Algorithms and Genetic Programming for Multiscale Modeling: Applications in Materials Science and Chemistry and Advances in Scalability”. The gallery above shows snapshots of Kumara in action. If you have not taken a look at his excellent work, put it on your todo list, you will not regret it. Congratulations!

The Engineer of the Future Workshop

Wed, 05 Sep 2007 15:19:13 +0000

I am sitting at the workshop on the Engineer of the Future hosted at NCSA. The workshop is part of the ETSI lecture series organized by Michael Loui and David E. Goldberg. The talks so far are very interesting. Hard to highlight all the relevant points, but I would like to mention the experience Sherra E. Kerns is sharing about their experience at Olin College. The engagement with the curriculum that students show is very remarkable. She mentioned current established curricula tend to kill innovation (and engagement). That comment resonate with my personal experience freshmen was always more rewarding and challenging than last year students.

Live from The Engineer of the Future

Wed, 05 Sep 2007 15:13:16 +0000

Installing third-party apps to your iPhone

Wed, 05 Sep 2007 14:38:05 +0000

Matt Cutts has a nice post on how to hack your iPhone to install third-party apps. I am still reluctant to do it.

Sync your iCal and Google Calendar

Thu, 23 Aug 2007 02:22:01 +0000

Did you ever wanted to sync your iCal and your Google Calendar? I just ran into Spanning Sync. I have been testing and it is definitely worth it if you need such capabilities.

Synchronize Google Calendar and Apple iCal. Share calendars between multiple Macs. Share calendars with your coworkers, family, and friends. And while you’re at it, connect Google Calendar to your iPod, mobile phone, and other devices.

Zattoo - TV to Go

Sun, 05 Aug 2007 19:35:46 +0000

I was browsing around today and I ran into Zattoo - TV to Go. It is a P2P IPTV service provided by an University of Michigan at Ann Arbor startup. See below their claim to fame.

Zattoo acquires, transports and presents quick-start, long-play streaming video in one browser for all channels for broadband users anywhere. Zattoo is the only provider of P2P IPTV to provide users with diverse and desired content of the highest possible video quality in one browser, while dramatically reducing cost and increasing reach for broadcasters and enabling advertisers to leverage the best aspects of both web-based and traditional television advertising methods. Zattoo partners with broadcasters and advertisers to serve consumers with a wide selection of content from various sources in one easy-to-use web interface.

E2K blog has moved

Thu, 02 Aug 2007 19:40:55 +0000

The E2K blog has moved. You can reach it at

http://dita.ncsa.uiuc.edu/e2k/

DITA+ALG+DISCUS = VAST contest entry

Sun, 29 Jul 2007 16:19:15 +0000

DITA and ALG at NCSA have joined forces with the DISCUS team to enter the 2007 VAST contest. You can find a podcast of the entry to the contest here, and a description of the VAST contest below.

Visual Analytics is the science of analytical reasoning supported by highly interactive visual interfaces. People use visual analytics tools and techniques to synthesize information into knowledge; derive insight from massive, dynamic, and often conflicting data; detect the expected and discover the unexpected; provide timely, defensible, and understandable assessments; and communicate assessments effectively for action. The issues stimulating this body of research provide a grand challenge in science: turning information overload into the opportunity of the decade.

Loading RDF/XML files into Virtuoso’s metadata store

Fri, 20 Jul 2007 14:19:07 +0000

Bernie just put together this beauty to load small RDF/XML files into Virtuoso’s metadata store (We are using testing the open source version).

DB.DBA.RDF_LOAD_RDFXML(http_get ('URI to the RDF/XML file'),'','Name of the graph in the store');

We have tested loading a 5 Million triple RDF/XML and results are pretty nice (It took around 6 minutes to load into a dual Pentium 4 extreme edition at 3GHz with 4GB of RAM on a slow 7500rpm ext3fs). When pushing to larger files, the stream version of this is a must to reduce memory consumption.

Crawlers on the loose

Fri, 20 Jul 2007 03:01:41 +0000

Recently, our lab web server has been experiencing a great increase of bot visits. Besides the usual Google, Yahoo, MSN, etc. bots a new one caught my attention. It is visiting pretty frequently: Cuill. From the few information they have on their site, one of the co-founders is a former UIUC CS alumni Anna Patterson.

Cuill Inc. (pronounced [kool]) is a startup company that is pioneering a new approach to Search. The company was founded by Anna, Russell, and Tom. Our offices are located on a quiet street in Menlo Park, CA. If you’d like to learn more about Cuill or the people behind it, please get in touch.

Google Notebook

Wed, 18 Jul 2007 22:07:50 +0000

Google Notebook has a pretty cool add-on for Firefox. It allows you to clip, and organize information from across the web in a single online location that’s accessible from any computer.

Back in town and a Virtuoso

Wed, 18 Jul 2007 03:11:47 +0000

Vacations and GECCO were great. I am still trying to catch up. Hope everything gets up to speed soon. Just a quick note to jump start: Dealing with relational databases, metadata stores, DAV servers? If so, take a look at Virtuoso, you won’t be disappointed ;)

GECCO 2007 Hat Trick

Fri, 13 Jul 2007 03:46:07 +0000

Yup, never happened to me before. In John Koza’s words: Hat Trick, best paper in EDA track and two Bronze medals in the HUMIES competition. I just think that I was lucky and blessed to have the pleasure to work with such great people. It would not have happened if it wasn’t for them. Below you can find some of the presentations for the above Hat Trick, as well as some other work I presented on genetics-based machine learning.

HTML to Wiki converter

Sun, 03 Jun 2007 22:45:35 +0000

Seapine {Labs}::HTML to Wiki Converter is an online converter of HTML formated code into a Wiki (MediaWiki) sintax. Very convenient if you want to add already formated material.

Blogging about NCSA PSP

Tue, 22 May 2007 14:14:04 +0000

Yesterday, today, and tomorrow NCSA is running the PSP 2007 meeting. You may find some of my blogging about it here.

Reset user’s password on a MediaWiki

Mon, 14 May 2007 23:31:47 +0000

I needed to reset the password for a user on a MediaWiki site. Luckily, I run into this post “Reset a user password on MediaWiki - Greg’s Postgres stuff” which helps you to do so. The five-cent summary for a MySQL powered site:

UPDATE user SET user_password = md5(CONCAT('123-',md5('newpassword'))) WHERE user_id=123;

An Overview of the DISCUS project

Fri, 11 May 2007 18:41:31 +0000

This presentation covers a general overview of the goals, origins, reasearch and tools currently available for the DISCUS project. For more information please visit the DISCUS project website. A follow up presentation with further details is also available here.

Uniform sampling of a data set

Fri, 11 May 2007 13:20:34 +0000

Sometimes you may need to sample a dataset. You may want to get a uniformly sampled subset out of a datatset stored in a file. The perlscript below does the job for you.

if ( $#ARGV!=1 ) {
        print "Wrong number of arguments\\n\\t".
                "uniform-sampler.pl <file> <sample_proportion>\\n";
}
else {
        srand();
        open(FILE,$ARGV[0]) or die "File $ARGV[0] could not be open";
        while($line=<FILE>) {
                if ( rand()<$ARGV[1] ) {
                                print $line;
                }
        }
        close FILE;
}
1;

Surface 51

Thu, 10 May 2007 18:43:58 +0000

SURFACE 51 is a Urbana-Champaign graphic and web design company. You can check out their portfolio here.

Jon Kleinberg visits UIUC

Tue, 01 May 2007 20:05:50 +0000

Jon Kleinberg is visiting UIUC today and giving a talk “Decentralized Search, Cascading Behavior, and the Structure of On-Line Communities” in the Age of Networks seminar series. The abstract of his talk today:

The rise of large-scale information networks has provided us with systems that are simultaneously social and technological in nature, and in which the human interactions that unfold can be recorded and studied in extraordinary detail. Against this backdrop, a significant amount of recent work has focused on the development of mathematical models that capture some of the qualitative properties observed in large-scale network data; such models have the potential to help us reason, at a general level, about the ways in which large networks are organized, and about the ways in which abstract models can capture the processes we observe.

Package e1071 for R

Sun, 29 Apr 2007 20:29:13 +0000

The package e1071 for R is an interesting add on to your list of R packages. It includes functions for latent class analysis, short time Fourier transform, fuzzy clustering, support vector machines, shortest path computation, bagged clustering, naive Bayes classifier, independent component analysis, and more.

Zotero now talks to MS Word

Sat, 28 Apr 2007 16:56:41 +0000

Zotero has released a first integration plugin for MS Word. Pretty useful if you need to move your bibtex into a Word document.

GE498-ECI, Lecture 14: Enough theory, show me some action

Wed, 25 Apr 2007 21:23:15 +0000

This lecture review some tools for innovation and creativity support developed under the DISCUS project. We reviewed some of the uses of the technologies presented in the course when used on real-world marketing applications.

Principal Component Analysis in R

Wed, 25 Apr 2007 03:58:02 +0000

There are, at least :), two ways to compute the principal component analysis of a data set in R. The first one is from scratch computing eigenvectors and eigenvalues. It works as follows

# 
# From scratch 
# 
cbind(1:10,1:10 + 0.25*rnorm(10)) -> myData 
myData - apply(myData,2,mean) -> myDataZM 
cov(myDataZM) -> cvm 
eigen(cvm,TRUE) -> eCvm 
t(eCvm$vector%*%t(myDataZM)) -> newMyData

This simple code just transforms the data to align it with the principal components obtained. Of couse, the second way to compute them is using some of the functions that R provides in the stats package.

Online Feed Validator

Tue, 24 Apr 2007 03:16:33 +0000

I have been running into some problems with a feed generator I am using (yes, it is the one in WordPress MU, long story). However I found a useful tool, an on-line feed validator for Atom and RSS.

A programming IDE

Fri, 20 Apr 2007 05:17:36 +0000

Need one? Check Eclipse.

Eclipse Metrics Plugins

Fri, 20 Apr 2007 04:00:51 +0000

I have run into a couple of plugins for Eclipse to compute several measure about the code you have in your project. The one I currently use is Eclipse Metrics Plugin.

This Eclipse plugin calculates various metrics for your code during build cycles and warns you, via the Problems view, of range violations for each metric. This allows you to stay continuously aware of the health of your code base. You may also export the metrics to HTML for public display or to CSV format for further analysis.

GE498-ECI, Lecture 13: Social means interaction (II): Influence diffusion in social networks

Wed, 18 Apr 2007 21:23:15 +0000

Continuation of GE498-ECI, Lecture 12: Social means interaction (I): Learning what you like.

Ben Shneiderman at UIUC

Wed, 18 Apr 2007 17:03:44 +0000

You can find my notes for his presentation here.

The Thrill of Discovering (Live from Ben Shneiderman’s talk)

Wed, 18 Apr 2007 16:55:57 +0000

Ben Shneiderman is visiting UIUC today. I am sitting at his talk “The Thrill of Discovery” at room 1040 NCSA. If you miss this one he will be at 126 GSLIS this afternoon at 3pm given another talk Accelerating Discovery and Innovation: Designing Creativity Support Tools". His opening today:

This talk will start by reviewing the growing commercial success stories such as www.spotfire.com, www.smartmoney.com/marketmap and www.hivegroup.com. Then it will cover recent research progress for visual exploration of large time series data applied to financial, Ebay auction, and genomic data (www.cs.umd.edu/hcil/timesearcher).

Is your MacBook Pro running too hot?

Wed, 18 Apr 2007 03:14:39 +0000

The first generations of MacBook Pros tend to run too hot because the low rpm set by default. The new ones have doubled the rpm speed. If you have a first generation on and want to boost the fans a bit to make it run a little cooler, you may find two tool apps to increase the fan speed here.

Machine learning & Statistical Learning in R

Tue, 17 Apr 2007 19:27:57 +0000

Torsten Horthorn maintains a page with a list of packages for machine learning and statistical learning in R.

University of Illinois vs a Mummy

Tue, 17 Apr 2007 02:44:09 +0000

A hidden treasure found in the Urbana Free Library. University of Illinois vs a Mummy is a movie made by students at UofI. If you have never been in UofI you will find this movie pretty hilarious.

PCA & ICA

Mon, 16 Apr 2007 20:54:22 +0000

I found a couple of interesting tutorials. One is on principal component analysis by Lindsay I. Smith and the second one is about independent component analysis by Hyvärinen and Oja. Good introductions if that is what you are looking for.

Visualizing content from metadata stores

Sun, 15 Apr 2007 03:48:33 +0000

Last Friday with ALG and DITA people we put a brief presentation for NCSA’s cyberarch group on our common efforts to create a generic framework for querying and visualizing content stored in metadata stores. Mulgara, SOAP, XLSTs, and custom Java code to render content using Prefuse and JFreeChart. You can download the slides here.

We finally have a group name!

Sun, 15 Apr 2007 01:46:31 +0000

The group was created a while ago to unify the research efforts conducted inside the Automated Learning Group. Michael Welge, Loretta Auvil, and I were sitting in Michael’s office a Monday morning scratching our heads. He generated the initial population, Loretta recombined the ideas, and I just selected what I liked. So, we become Data-Intensive Technologies and Applications :D.

GE498-ECI, Lecture 12: Social means interaction (I): Influence diffusion in social networks

Wed, 11 Apr 2007 21:23:15 +0000

This lecture covered some of the work presented in “Mining Social Network in Message Boards” (the paper PDF may be found here (1Mb)) published in the Sympposium on Conversational Informatics for Supporting Social Intelligence and Interaction (AISB’05, 12-15 April, Hatfield, UK, 2005). The lecture covered how to identify relevant topic and player in an ongoing discussion. You can download the PDF here (2Mb)—or see below.

Wordpress MU plugin to render math using latex commands

Tue, 10 Apr 2007 19:16:28 +0000

I just run into an interesting Wordpress MU plugin to render mathematical equations via LaTeX commands.

More on Wordpress themes

Mon, 09 Apr 2007 15:29:14 +0000

Via wpmudev, I run into a huge pack of themes for Wordpress. Kumara Sastry also have a nice collection of his favorites.

GE498-ECI, Lecture 11: Summarize and drill: Helping to avoid cognitive overload

Wed, 04 Apr 2007 21:18:30 +0000

This lecture covered the basic of how to identify relevant topics and extract summaries out of a conversation. This lecture is based on the paper “Delineating Topic and Discussant Transitions in Online Collaborative Environments” by Yasui, N.I., Llorà, X., and Goldberg, D.E.. You may find the PDF of the paper here.

GE498-ECI, Lecture 10: Innovation and visual concept maps

Wed, 04 Apr 2007 21:12:36 +0000

This lecture covered how to visualizes concepts on an ongoing conversation. The lecture was based on the paper about using interactive genetic algorithms to tune the KeyGraph algorithm for a given document. The paper was published in the First European Workshop on Chance Discovery, EWCD 2004 (Valencia, Spain). The paper may be found here (PDF, 0.7Mb) and the presentation slides here (PDF, 0.5Mb)—or see below.

Paul Borgeson poetry contest at UIUC

Wed, 04 Apr 2007 17:08:40 +0000

There is a poetry contest at UIUC in honor of Paul Borgeson (former UIUC decesed faculty). You still have time to turn your poem in till April 6th. It is never to late to have fun.

Interesting compilation of Firefox add-ons

Tue, 03 Apr 2007 23:49:11 +0000

Kumara Sastry has an interesting blog post about add-ons for Mozilla Firefox.

Java and Open GL

Tue, 03 Apr 2007 22:08:09 +0000

I knew there were some efforts to have access to OpenGL from Java, but I did not check it for a while. The JOGL API project has made a long journey since its inception. You can check the demos easily, since they are Java web started applications.

GE498-ECI, Lecture 9: The Unstructured Data Contains a Clue

Wed, 28 Mar 2007 21:42:41 +0000

Lecture 9 covered a quick introduction to text mining. The slides for this lecture can be found below.

GE498-ECI, Lecture 8: Connectivity Everywhere; Graph Theory 101

Wed, 28 Mar 2007 21:39:29 +0000

This lecture covered some of the basis of graph theory. The slides for this lecture can be found below.

Analyzing Trends in the Blogosphere Using Human-Centered Analysis and Visualization Tools

Mon, 26 Mar 2007 20:23:28 +0000

by Xavier Llorà, Noriko Imafuji Yasui, David E. Goldberg (2006). Proceedings of the International Conference on Weblogs and Social Mining (ICWSM 2007). Also as IlliGAL TR. No. 2006026. Link to the PDF.

Abstract

The blogsphere is a valuable source of information. From simple topic analysis in the blogosphere—what’s hot—to harvesting and analyzing valuable market trends—what product and features are suggested—require a tight integration of computer- and human-based analysis capabilities. Computers can easily assist the processing filtering and visualizing relevant and key elements of the blogosphere, but coupling them with human evaluation and reasoning can provide the final steps to connect pieces of relevant information into better description map of the current trends of the blogosphere. An example of the need for such human-centered analysis was David R. Ellis’ film Snakes on a Plane (2006) which failed to properly translate blogosphere discussions into a successful commercial product—as a clear misalignment of both environments the blogsphere and the final targeted market. In this paper, we present some human-centered visualization and analysis tools that can help users to compare and reason synergies and misalignments revolving around a particular topic.

Communication gap management for fertile community

Sat, 24 Mar 2007 15:17:35 +0000

by Naohiro Matsumura, David E. Goldberg, and Xavier Llorà (2006).

Journal of Soft Computing, Volume 11 , Issue 8, pp. 791–798, ACM press. Link to the ACM portal. Initial work also available as IlliGAL TR No 2005001.

Abstract

In the paper, we first present an approach to extract social networks from message boards on the Internet. Then we propose communication gaps based on structural features of the social networks as an indicator of understanding the state of communication. After we classify 3,000 social networks into three types of communication, i.e., interactive communication, distributed communication, and soapbox communication, we suggest communication gap management to identify the types of communication, the roles of individuals, and important ties, all of which can be used for drawing up a plan for realizing fertile community.

GE498-ECI, Lecture 7: The semantic web

Thu, 15 Mar 2007 01:59:37 +0000

Lecture 7 of GE498-ECI covered basic concepts necessary to understande the semantic web. The slides for this lecture can be found below.

GE498-ECI, Lecture 6: A few enterprise storage concepts

Thu, 15 Mar 2007 01:51:01 +0000

Lecture 6 of GE498-ECI presented the basis of enterprise data storage using relational databases. Lecture slides below.

Photos of Tokyo

Mon, 12 Mar 2007 01:17:31 +0000

Tokyo Spring 2007

This album contains the photos of my recent trip to Tokyo.

A different Akihabara

Sun, 04 Mar 2007 13:33:02 +0000

Yes, what you see up there is the video that mesmerized Nao, Kazu, and me in a Akihabara store while waiting for Noriko to pay for her DVDs.

Seattle and Tokyo

Sat, 03 Mar 2007 05:38:37 +0000

I am traveling again. This time Seattle was the first stop. NCSA was having a private sector partner meeting at Boeing. I was invited to present the work about DISCUS we are doing as part of the LINK alliance endeavor. And from Seattle to Tokyo where Yukio Ohsawa is organizing “Cubes for innovation: Information Systems for Design and Marketing”. The workshop sponsored by Tsukuba University and IEEE is being held today at University of Tokyo. I was invited to go over our work on DISCUS to create IT infrastructure to support innovative and creative processes. You can find the slides of my presentation above (you can also download the PDF).

GE498-ECI, Lecture 5: The schema of a simple web application

Wed, 21 Feb 2007 22:18:55 +0000

Lecture 5 of GE498-ECI presented basic required networking components to understand the basic architecture and functioning of a web application. The lesson slides are available below.

Taking another snow day: Snow cancels UofI classes today too

Wed, 14 Feb 2007 16:12:04 +0000

UofI has cancelled the classes again today. After the blizzard, today is plowing-the-snow day. Most people are stuck at their homes plowing the snow away from driveways so they can uncover their cars from under the snow hills. I guess tomorrow everything should be back to normal. I am planning to go for a walk on the snow latter thought. It looks pretty from my window ;)

The day after the winter 2007 blizzard

Wed, 14 Feb 2007 16:05:09 +0000

The day after the winter 2007 blizzard

Taking a snow day: Snow cancels UofI classes after 40 years

Tue, 13 Feb 2007 16:01:38 +0000

Yup, it is snowing a bit here. The picture above was taken 10 minutes ago from my appartment window. You may not be able to appreciate it, but the visibility is very poor due to the snow fall and heavy gutsy wind. This blizzard has closed UofI. Here you have the chancellor’s e-mail.

All classes have been canceled for Tuesday (Feb. 13) at the University of Illinois at Urbana-Champaign. More than 5 inches of snow fell overnight, up to another foot of snow is possible by Tuesday night and a blizzard warning is in effect. U. of I. employees who have been designated as essential personnel are expected to report to work. This mailing approved by: The Office of the Chancellor

blizzard winter 2007

Tue, 13 Feb 2007 15:57:26 +0000

Blizzard winter 2007. Yup, it is here.

GE498-ECI, Lecture 4: The web with a human spin

Wed, 07 Feb 2007 22:28:33 +0000

Lecture 4 of GE498-ECI covered how the massive adoption of the web as a communication medium has changed the definition of a successful application. Slides below.

GE498-ECI, Lecture 3: Let's play

Wed, 31 Jan 2007 22:12:42 +0000

Lecture 3 of GE498-ECI covered a models of innovation and creativity. From Wallas to Osborne, and then to Goldberg’s innovation intuition. Slides below.

Fourth José Luís Balcazar tournament (2007)

Thu, 25 Jan 2007 15:58:14 +0000

The Jose Balcazar Tournament is the fourth annual unclassified competition at The Point Fencing club. On January 27 at 10 am registration will open for beginners, one year and under (or Fencing III) and close at 10:30. The winner of each years tournament is commemorated on a special display plaque. This is a very special tournament for the point, do not miss it! You may found the entry for last year’s one here.

GE498-ECI, Lecture 2: Innovation and creativity

Thu, 25 Jan 2007 04:53:04 +0000

Lecture 2 of GE498-ECI covered a first scoop to innovation and creativity. The slides used are available below.

Toward routine billion-variable optimization using genetic algorithms

Thu, 18 Jan 2007 22:32:36 +0000

by Goldberg, D. E., Sastry, K., and Llorà X. (2007). Complexity, 12(3), 27—29. Link to the PDF.

Abstract:

The push for better understanding and design of complex systems requires the solution of challenging optimization problems with large numbers of decision variables. This note presents principled results demonstrating the scalable solution of a difficult test function on instances over a billion variables using a parallel implementation of a genetic algorithm (GA). The problem addressed is a noisy, blind problem over a vector of binary decision variables. Noise is added equaling a tenth of the deterministic objective function variance of the problem, thereby making it difficult for simple hillclimbers to find the optimal solution. The genetic algorithm used - the compact GA - is able to find the optimum in the presence of noise quickly, reliably, and accurately, and the solution scalability follows known convergence theories. These results on noisy problem together with other results on problems involving varying modularity, hierarchy, and overlap foreshadow routine solution of billion-variable problems across the landscape of complexity science.

GE498-ECI, Lecture 1: Introduction

Wed, 17 Jan 2007 22:53:35 +0000

We have just finished the first lecture of GE498-ECI. We reviewed the course goals and topics. We also started with a little exercise to foster our creativity. You can find the slides of the lecture today below.

GE498-ECI: Enterprise Collaboration and Innovation Support Systems

Wed, 17 Jan 2007 22:39:53 +0000

Enterprise Collaboration and Innovation Support Systems is a course about creativy and innovation. It is also a course about how to create supporting technologies for human activities, and what could make such technologies “disruptive”.

Topics

Creativity and innovation

Basic concepts
How to model innovation and creativity?
Techniques to help people to be creative

Information technologies

The web has change the way business are conducted
The web has become “the” communication medium
Social patterns of interaction had taken over the web (in less than 10 years!)
Learn just enough of these technologies to make you dangerous out there

Informatics of people connectivity

GECCO 2007 deadline extended

Tue, 16 Jan 2007 20:45:20 +0000

The 2007 Genetic and Evolutionary Computation Conference submission deadline has been extended from Wednesday, January 17 to WEDNESDAY JANUARY 31, 22:00 Pacific Standard Time.

Is a war for easily-sharing-your-stuff-with-your-friends coming?

Tue, 16 Jan 2007 03:22:36 +0000

Lately, I have been running into more and more Slashdot entries about newly released products to support sharing in one form or another. Just two of the latest one that surfaced: I Like (music-based) and Tubes (share anything from any device).

CEC 2007 deadline

Sun, 14 Jan 2007 15:50:31 +0000

The deadline for the Congres on Evolutionary Computation (CEC 2007) deadline for paper submission is March 15, 2007.

iPhone blast

Wed, 10 Jan 2007 15:55:52 +0000

I am sure you all saw it already, but Steve Job’s keynote at MacWorld let me speachless. Besides AppleTV, interesting by itself—now there is a new Catalan TV (Vilaweb.tv) channel just broadcasting over the net that I will be able to watch at home in Urbana easily with one of those—Steve dropped the boom of the new iPhone—a phone+iPod+internet device. Smartphones and blackberries are facing a dark and uncertain future. I remember having a conversation with Kumara Sastry not long ago were he was complaining about not finding anything new on the cell phone market (he wanted to change his cell phone). Well, may be there is something new after all and from a kind of unexpected Apple ;)

Advances at the frontier of LCS: LNCS 4399

Tue, 09 Jan 2007 05:07:54 +0000

The volume Advances at the frontier of Learning Classifier Systems is already in Springer hands for the final stages of editing and printing. The volume is going to be printed as Springer’s LNCS 4399 volume. You can find the tentative table of contents here.

Two movies, Julio Medem, and Daniel Sánchez Arévalo

Sun, 07 Jan 2007 17:06:11 +0000

Julio Medem has finished his new coming movie. The director of “La ardilla roja” (The red squirrel), “Tierra” (Earth), and “Los amantes del círculo polar” (Lovers of the Artich Circle)-–among others—has finished shooting “Caótica Ana” (Cahotic Ana) which should open Spring 2007. I just hope he has forgotten about the boring “Lucía i el sexo” (Sex and Lucía) and he has recovered his fascinating storyteller qualities. Anyway, lots of expectations every time he comes up with a new movie. Here is as summary of the plot of the movie from Julio’s unofficial web site.

Antònia Font

Mon, 01 Jan 2007 16:13:40 +0000

I run into Antònia Font last summer, but I did not have the chance to buy their discs in Girona. I hope I can do it before going back to Urbana in a few days. Below you may find some of their videos on YouTube.

Happy 2007 :D

GECCO 2007 Deadline

Fri, 29 Dec 2006 15:59:19 +0000

GECCO 2007 is approaching. The deadline for paper submission is January 17. You can find more instructions on how to format and submit the papers on the GECCO 2007 papers’ page.

Challenging lectures on-line at TED

Mon, 18 Dec 2006 23:03:57 +0000

Yesterday I was having dinner with a friend. He mentioned that he had become addicted to the lectures posted on the TED (Technology, Entertainment, Design) web site. I found this interesting description of TED on their site:

TED is all about connections. The connections of ideas and the connections of people. It is based on the insight that to truly understand anything, you need to understand a little bit of everything that surrounds it. And that by allowing ourselves to be exposed each year to a diverse group of some of the most remarkable people on the planet, we transplant ourselves out of the one-dimensional mind-set of much of our working lives and into fertile country that will allow us - actually, almost force us - to grow.

Observer-Invariant Histopathology using Genetics-Based Machine Learning

Wed, 13 Dec 2006 20:58:47 +0000

by Xavier Llorà, Anusha Priya, and Rohit Bhargava (2006). To appear in the Special Issue on Learning Classifier Systems of the Natural Computing Journal. Also as IlliGAL TR No. 2006027. Link to the PDF.

Abstract

Prostate cancer accounts for one-third of noncutaneous cancers diagnosed in US men, and it is a leading cause of cancer-related death. Advances in Fourier transform infrared spectroscopy of stained tissue is now able to provide very large data sets describing the chemical properties of the cells forming the prostate tissue. Uniting spectroscopic imaging data and computer-aided diagnoses (CADx), we seek to provide a new approach to pathology by automating the recognition of cancer in complex tissue. The first step toward the creation of such CADx tools requires mechanisms for automatically learn tissue type classification—a key step on the diagnosis process. As we will show, genetics-based machine learning (GBML) can be used to approach such a problem. However, there is an urge for efficient and scalable implementations that enable to process such very large data sets. This paper proposes and validates and efficient GBML technique—NAX—based on an incremental genetics-based rule learner that exploits massive parallelisms—via the message passing interface (MPI)—and efficient rule-matching using hardware-implemented operations. Results show the competence of NAX solving the prostate tissue type prediction and how such and efficient implementation makes it a very powerful tool for biomedical image processing.

Delineating Topic and Discussant Transitions in Online Collaborative Environments

Wed, 13 Dec 2006 18:26:10 +0000

by Noriko Imafuji Yasui, Xavier Llorà, and David E. Goldberg (2006). Illinois Technical Report No. 2006025. Link to the PDF.

Abstract

In this paper, we propose some methodologies for delineating topic and discussant transitions in online collaborative environments, more precisely, focus group discussions for product conceptualization. First, we propose KEE (Key Elements Extraction) algorithm, an algorithm for simultaneously finding key terms and key persons in a discussion. Based on KEE algorithm, we propose approaches for analyzing two important factors of discussions: discussion dynamics and emerging social networks. Examining our approaches using actual network-based discussion data generated by real focus groups in a marketing environment, we report interesting results that demonstrate how our approaches could effectively discover knowledge in the discussions.

Adaptable Extraction of Key Elements from Weblogs

Wed, 13 Dec 2006 15:27:24 +0000

by Xavier Llorà, Noriko Imafuji Yasui, David E. Goldberg (2006)

Abstract

This paper proposes AKEE (Adaptable Key Elements Extraction) algorithm for web-log (blog, for short) mining. AKEE enables us to identify significant information in various blog components (e.g., a blog post, a series of blog posts, a set of series of blog posts, etc.).

Illinois Technical Report No. 2006024. Link to the PDF.

The new DISCUS site is up

Wed, 29 Nov 2006 20:57:08 +0000

The new site is here! Besides moving it to Wordpress, we have been working to engineer a site that will allow us to demonstrate some of the human-centered analytic technologies the DISCUS project has created. For each post, now you can analyze it with a single click using the links provided on the left-hand side. The first analytics tool released is based on KeyGraph techniques. Other DISCUS analysis techniques are already on the pipe and will reach the site soon. We have not stopped at reengineering the DISCUS site. We have also created an extension for Firefox that allow you to analyze the web page you are currently viewing. All DISCUS capabilities integrated in your browser. The DISCUS Firefox extension adds a DISCUS toolbar, some entries on the tools menu and in the right-click contextual menu. Hope you find it useful. Following the site, this toolbar will also grow and incorporate new features and human-centered analytic tools.

Human-Centered Analysis and Visualization Tools for the Blogosphere

Wed, 29 Nov 2006 20:31:08 +0000

by Xavier Llorà, Noriko Imafuji Yasui, Michael Welge, David E. Goldberg (in press, 2007).

To apper in the Proceedings of the Digital Humanities 2007 Conference.Also as IlliGAL TR No. 2006023. Link to the PDF.

Abstract

Blogging has become a new and disruptive communication medium. Blogs have changed the way people and organizations express, interact, and—quite unforeseen—exercise influence. The digital nature of the blog media provides access to an always-expanding corpus of information. It would take more than a lifetime to read all the available blogs necessary to answer questions such as what were the more relevant plots suggested or what key concepts were managed by bloggers in their ideas. However, human-centered analysis and visualization techniques may help users navigate such enormous corpus. This paper presents human-centered analysis and visualization techniques for supporting innovation and creativity can help to identify relevant post portions and to visualize concept relations in the blogosphere.

Metadata, RDF, and the browsers

Tue, 28 Nov 2006 21:31:02 +0000

Recently I have been playing on implementing some extension for Firefox. When I started working on it, I was not aware on how far they have taken the use of metadata—I blogged before on metadata stores here. Actually, now you can query RDF data sources, which contain information ranging from your browsing history or bookmarks—to mention a few. But what makes it more exiting is that when you are writing extensions you can use java script to get that information stored by the browser using the RDF interface to do something useful. For instance, DISCUS is using it to provide on-the-fly analytics for the current page currently viewing. DISCUS will soon release a demo version of such a toolbar. If you want to read more about it, I would recommend the XULplanet.com.

Photoshoot at The Point

Fri, 27 Oct 2006 00:55:44 +0000

The photo above is a snake preview of the 2007 The Point Calendar. This year the calendar has moved into the full-fledge digital era. Digital cameras and Mac were flying everywhere. Stay tuned, because this one is going to be very different from the 2005 and 2006 ones :)

E2K: Evolution to knowledge software

Sat, 21 Oct 2006 15:55:24 +0000

Evolution to Knowledge (E2K) is a set of Data to Knowledge (D2K) modules and itineraries that perform genetic algorithms (GA) and genetics-based machine learning (GBML) related tasks. The goal of E2K is to fold: simplify the process of building GA/GBML related tasks, and provide a simple exploratory workbench for the evolutionary computation community to help users to interact with evolutionary processes. It can help to create complex tasks or help the newcomer to get familiarized and trained with the evolutionary methods and techniques provided. Moreover, due to its integration into D2K, the creation of combined data mining and evolutionary task can be effortlessly done via the visual programming paradigm provided by the workflow environment and also wrap other evolutionary computation software. E2K targets the creation of a common shared framework for the evolutionary computation community. E2K allows users to reuse evolutionary components and, using a visual programming paradigm, connect them to create applications that fulfill the targeted needs. E2K is a project built around the D2K framework developed by the Automated Learning Group at the National Center for Supercomputing Applications. D2K’s dataflow architecture provides users with a simple workbench where they can rapidly create applications visually by just dragging and connecting components (modules) together. E2K modules provide simple computation activities—such as evaluation, selection, and recombination mechanism—that when combined together create complex evolutionary computation algorithms. Due to the module standardization in D2K, it can act as integrator of evolutionary techniques and library—for instance wrapping ECJ or Open BEAGLE components—and also take advantage of the data mining techniques provided with the D2K.

R and Java

Thu, 19 Oct 2006 16:12:38 +0000

I just ran into two nice packages rJava and JRI that allow running R code from Java and Java code from R. The packages are developed by Department of Computer Oriented Statistics and Data Analysis at Universität Augsburg. It just makes my life much easier. If you need scientific computing from Java it is a really nice option to try :D.

The great pumpkin patch

Sun, 15 Oct 2006 16:08:18 +0000

The Great Pumpkin Patch is a seasonal attraction in Arthur, IL (just 40 minutes drive from Urbana). You can find all sorts of pumpkins and Halloween-oriented decorations. The gift shop is culturally interesting, and the home bakery has a wide variety of angel cakes (really soft and nice looking). The Great Pumpkin Patch also has one of the 1900s rural one-room classroom schools—there was at least one school in a 2 miles reach of any farm in Illinois. Children of all ages were sharing one class and teacher, definitely a pedagogical challenge. If you are around, it is a must stop place before it closes its doors till next year.

One talk and a visit to UK

Mon, 09 Oct 2006 01:06:03 +0000

September 21 I was invited to give a talk at the Computer Science Department at UIUC. During the talk “Combating User Fatigue and Contradictions in Subjective-based Optimization Schemes” I reviewed some of the research I have been involved about active interactive genetic algorithms. The PDF of the presentation can be downloaded here. I also gave the same presentation to some of the members of the ASAP research group at the University of Nottingham. Natalio Krasnogor invited me for a visit. The main topic was latest advances on Pittsburgh LCS (Jaume Bacardit is working there on protein folding problems using Pitt-style LCS). I really enjoyed interact with people there—lots of challenges and interesting discussions. Oh, I almost forgot, the three days I was in Nottingham I saw the sun most of the day :).

How big is big?

Thu, 21 Sep 2006 10:53:24 +0000

Recently I have been working on a data-mining problem that requires supervised learning. The problem is not supposed to be big, just a few hundreds of features. The interesting issue is the number of records, around half a million or more. Most of the implementations of supervised learning algorithms available on the web are not designed with such a volume of data. Scalability of the algorithms becomes a clear issue when dealing such a volume of data. For instance, algorithms that scale as n^3 with respect to the data may become prohibitively costly for any feasible approach. Also, algorithms that require global processing, defeating efficient parallelization may not be an option either. For such reasons, I started working last fall on efficient implementations of GBML algorithms focusing on (1) efficient implementations hacking the available hardware, (2) minimizing memory food prints required, and (3) massively exploiting the inherent parallelism of such methods. A few initial steps can be found here and here.

Metadata stores, RDF, Mulgara, MySQL, and Oracle

Tue, 19 Sep 2006 10:48:46 +0000

I have been using Kowari for a while. I am quite happy with it, but I am planning to ditch it in favor of the new release of Mulgara (Open Source for of Kowari) lead by Paul Gearon. If you haven’t check it out yet, take a look at it because is more than worth it. I have also been exploring other options. There is modified implementation of SPARQL for MySQL maintained by Eric Prud’hommeaux, his approach is very interesting one if you rely on MySQL server, it is a must. You can find some of his related papers and presentations and here. The last one I ran into recently is the RDF store implementation by Oracle Spatial. I have not played with it much, but it has Oracle behind, and that is a big name…

Evolving emotional prosody

Sun, 17 Sep 2006 15:57:30 +0000

by Cecilia Ovesdotter Alm and Xavier Llorà (2006).

Proceedings of the Ninth International Conference on Spoken Language Processing (INTERSPEECH 2006), paper 1741. Also as IlliGAL TR No 2006018. Link to the PDF.

Abstract

Emotion is expressed by prosodic cues, and this study uses the active interactive Genetic Algorithm to search a wide space for sad and angry parameters of intensity, F0, and duration in perceptual resynthesis experiments with users. This method avoids large recorded databases and is flexible for exploring prosodic emotion parameters. Solutions from multiple runs are analyzed graphically and statistically. Average results indicate parameter evolution by emotion, and appear best forsad speech. Solutions are quite successfully classified by CART, with duration as main predictor.

Finally I am back up

Sun, 17 Sep 2006 10:23:09 +0000

Vacations were over quite a while ago, but things kept piling on my desk. Finally I succeeded clearing some big chunks of matters out of it. Enough that I could go to the Design Theory and Methodology (DTM 2006) conference in Philadelphia last Tuesday. The conference is held as part of the ASME International Design Engineering Technical Conferences (IDETC 2006) . Yan Jin invited me to join the panel he organized on Intelligent Systems and Innovation. I had a great time joining the DTM people, very interesting work—if you are interested on innovation and creativity.

E2K: Evolution to knowledge

Wed, 13 Sep 2006 20:14:15 +0000

by Xavier Llorà (2006). ACM SIGEvolution, Volume 1 , Issue 3, pp. 10-17. Link to the Journal.

Abstract

Evolution to Knowledge (E2K) is a set of Data to Knowledge (D2K) modules and itineraries that perform genetic algorithms (GA) and genetics-based machine learning (GBML) related tasks. The goal of E2K is to fold: simplify the process of building GA/GBML related tasks, and provide a simple exploratory workbench for the evolutionary computation community to help users to interact with evolutionary processes. It can help to create complex tasks or help the newcomer to get familiarized and trained with the evolutionary methods and techniques provided. Moreover, due to its integration into D2K, the creation of combined data mining and evolutionary task can be effortlessly done via the visual programming paradigm provided by the workflow environment and also wrap other evolutionary computation software.

El dia del senyor

Fri, 04 Aug 2006 14:05:25 +0000

In the mid 90’s I started reading Quim Monzó-–maybe a after everybody did. The funny history is that I never start reading his articles on the Sunday edition of the Avui. I just learned about him when he started showing up in a talk show at Catalunya Radio, together with Sergi Pàmies. They were hilarious and refreshingly new, something not easy back then. In the last twenty years Quim has made his way into one of the most prevalent figures in Catalan contemporary literature. The other day I just run into a collection of his early articles published on the Catalan newspapers. They were compiled into “El dia del senyor” book. It covers the articles from 1982 till 1985. I am just going over it during my vacations and I cannot belief what I read. Twenty years after, they are still about hot topics in Catalonia. I can just change the actors’ and places for current ones, but most of the topics in those articles are in still today’s newspaper headlines. Maybe the tune never changes much in Catalonia…

PhD comics

Wed, 02 Aug 2006 18:00:18 +0000

Piled Higher and Deeper the amazing fun of the PhD life. If the comic strip is not enough, Cecilia’s blog is a must read.

Trendy design, decoration, and architecture blogs

Sat, 29 Jul 2006 18:39:50 +0000

Here are some blogs that I usually track. Sometimes they have fascinating stuff :D

Apartment therapy: Now with NYC, LA, SF, and Chicago, also a Kitchen section! Big one!
Tendir: A never-ending source of home decoration ideas.
Grassroots modern: A small spinoff for affordable ideas.
Funfurde: The first one I tracked, it opened my eyes.
Archidose: A daily dose of architecture.
The Design Weblog: Some interesting entry door to design.

Closed for vacation

Fri, 28 Jul 2006 21:34:23 +0000

Yup, it finally happen. I am back to Girona. Please allow me a slow response latency meanwhile :D

GECCO 2006

Sat, 08 Jul 2006 18:14:50 +0000

Well, it finally happened. GECCO started today at 8:30am. You can find live blogging at the IlliGAL Blogging.

Analyzing active interactive genetic algorithms using visual analytics

Fri, 07 Jul 2006 19:54:53 +0000

by Xavier Llorà, Kumara Sastry , Francesc Alías, David E. Goldberg, and Michael Welge (2006).

Proceedings of the ACM Genetic and Evolutionary Computation Conference (GECCO 2006), pp. 1417–1418, ACM press. Also as IlliGAL TR No 2006004. Link to the PDF.

Abstract

This paper build on active interactive genetic algorithms and introduces visual-analytic techniques to aggregate, summarize, and visualize the information generated during interactive evolutionary processes. Special visualizations of the user-provided partial ordering of solutions, the synthetic fitness surrogates induced, and the model of user preferences were prepared. The visual-analytic techniques proposed point out potential pitfalls, strengths, and possible improvements in a non-trivial case study where the hierarchical tournament selection scheme of an active interactive genetic algorithm is replaced by an equivalent incremental selection scheme. Visual analytics provided an intuitive reasoning environment that unveiled important properties that greatly affect the performance of active interactive genetic algorithms that could not have been easily reveled otherwise.

Fast rule matching for Learning Classifier Systems via vector instructions

Fri, 07 Jul 2006 18:37:55 +0000

by Xavier Llorà and Kumara Sastry (2006, accepted).

Proceedings of the ACM Genetic and Evolutionary Computation Conference (GECCO 2006), pp. 1513–1520, ACM press. Also as IlliGAL TR No 2006001. Link to the PDF.

Abstract

Over the last ten years XCS has become a de facto standard for Michigan-style learning classifier systems (LCS). Since the initial CS-1 work conceived by Holland, classifiers (rules) have widely used a ternary condition alphabet {0,1,#} for binary input problems. Most of the freely available implementations of this ternary alphabet in XCS rely on character-based encodings—easy to implement, not memory efficient, and expensive to compute. Profiling of freely available XCS implementations shows that most of their execution time is spent determining whether a rule is match or not, posing a serious thread to XCS scalability. In the last decade, multimedia and scientific applications have pushed CPU manufactures to include native support for vector instruction sets. This paper presents how to implement efficient condition encoding and fast rule matching strategies using vector instructions. The paper elaborates on Altivec (PowerPC G4, G5) and SSE2 (Intel P4/Xeon and AMD Opteron) instruction sets producing speedups of XCS matching process beyond ninety times. Moreover, such a vectorized matching code will allow to easily scale beyond tens of thousands of conditions in a reasonable time. The proposed fast matching scheme also fits in any other LCS other than XCS.

The χ-ary extended compact classifier system: Linkage learning in Pittsburgh LCS

Fri, 07 Jul 2006 06:00:38 +0000

by Xavier Llorà, Kumara Sastry, David E. Goldberg, and Luis de la Ossa (2006). To appear in the Proceedings of the International Workshop on Learning Classifier Systems (IWLCS 2006). Also as IlliGAL TR No 2006015. Link to the PDF.

Abstract

This paper proposes a competent Pittsburgh LCS that automatically mines important substructures of the underlying problems and takes problems that were intractable with first-generation Pittsburgh LCS and renders them tractable. Specifically, we propose a χ-ary extended compact classifier system (χeCCS) which uses (1) a competent genetic algorithm (GA) in the form of χ-ary extended compact genetic algorithm, and (2) a niching method in the form restricted tournament replacement, to evolve a set of maximally accurate and maximally general rules. The results clearly show that linkage exists in the multiplexer problem which needs to be accurately discovered and efficiently processed in order to solve the problem in tractable time. The results also show that in accordance with the facetwise models from GA theory, the number of function evaluations required by χeCCs to successfully evolve an optimal rule set scales exponentially with the number of address bits (building block size) and quadratically with the problem size.

LCS and other GBML warming up for GECCO 2006

Fri, 07 Jul 2006 00:39:01 +0000

The agenda for the Ninth International Workshop on Learning Classifier Systems (IWLCS’2006) can be found here. The workshop is coming with a list of very interesting papers and topics. We are looking forward to another edition of the workshop crowded of new and exiting ideas. If you are in GECCO, do not let it pass by ;). By the way, if you want to dig a little further into the learning classifier systems (LCS) and other genetics-based machine learning (GBML) world, check the website of the NCSA/IlliGAL Gathering on Evolutionary Learning (NIGEL’2006). It contains the slides and videos of the talks on cutting-edge LCS and GBML research.

Are you stressed out?

Sat, 20 May 2006 03:57:56 +0000

A friend sent this Flash animation link earlier today to one of the e-mail lists I am subscribed. Everybody has gone nuts trying to get farther and farther. The last thing I heard was 577.6.

GALE is back!

Wed, 10 May 2006 15:35:32 +0000

Yes, with the new update of the website GALE was unreachable for a while. Here it is the original code again. Please, take a few minutes to read this post. Hope you find it useful. Enjoy :D

GALE?

GALE (Genetic and Artificial Life Environment) is fine-grained parallel genetic algorithm for data mining. Its main contributions are simplicity and its knowledge-independent model. The simplicity of GALE relies in its fine-grained parallelism based on spreading the population (feasible solutions to the classification task) over a 2D grid. Thus, artificial evolution can be easily modeled in terms of neighborhood relations. These neighborhood relations define GALE as a massive parallel evolutionary model. On the other hand, GALE does not constrain the knowledge representation. It can evolve indistinctly rules, instances, partially defined instances, and decision trees (orthogonal, oblique, and multivariate based on nearest neighbor).

Yes, this was Memphis in May

Tue, 09 May 2006 04:18:21 +0000

As Kumara blogged here, we went to the Beale Street Music Festival that takes place during Menphis in May. Yup, as the name says it happens in May in Memphis, Tennessee. This year was an unique opportunity to see in the same festival names like B. B. King, Little Richard, Bo Diddley, Jerry Lee Lewis, Hubbert Sumlin, and James Brown—to mention a few. The picture above is B. B. King roaring on “How blue can you get”. Breathless. Oh, I almost forgot, we also paid tribute to the other Memphis King. Graceland has become a fascinating monument to the Elvis icon. It has been a long time since I was this fascinated for the eclecticism of a place—maybe roaming the streets of Las Vegas is the closest feeling. I have to admit that 60’s and 70’s iconography has always amused me. To boost the surrealistic feeling during our tour in Graceland we were not aloud to leave the King’s house. We were trap inside because they were filming the American Idol show. Fascinating.

List of papers to be presented at IWLCS 2006

Fri, 05 May 2006 05:18:59 +0000

The list of papers to be presented at the Ninth International Workshop on Learning Classifier Systems (IWLCS 2006) can be found here.

Metadata stores and D2K

Fri, 21 Apr 2006 16:35:29 +0000

Today I gave a presentation for the ALG group in NCSA about metadata stores and how they can be used in D2K. The presentation has two parts. The first one covers and introduction to metadata in general with some examples. The second one, is about how to wrap a metadata store (Kowari, see the previous post here) and make it accessible in D2K. You can find the slides here.

Fast fitness implementation of multiplexer problems for Pittsburgh LCS

Sat, 15 Apr 2006 21:53:12 +0000

by Xavier Llorà (2006). IlliGAL TR No 2006017. Link to the PDF.

Link to the Java code

Abstract

This technical report describes how to compute the fitness of a rule for an arbitrary size multiplexer without doing any instance matching. Pittsburgh-style learning classifier systems require the accuracy and the error of a rule to compute a fitness that promotes maximally accurate and maximally general rules. The accuracy (α) may be computed as the proportion of overall examples correctly classified, and the error (ε) is the proportion of incorrect classifications issued. Once the accuracy and error of a rule are known, the fitness can be computed as f(r)=α(r)*ε(r). This technical note shows how to computed the fitness only by inspecting the rule, requiring a time proportional to number of possible address values O(2^|a|) instead of the O(2^l) that requires a traditional rule matching strategy against all the possible instances. The proposed method makes tractable for Pittsburgh learning classifier systems multiplexer problems larger than the 11-input one.

Palabras para Julia

Thu, 13 Apr 2006 06:12:47 +0000

José Agustín Goytisolo wrote a powerful poem named “Palabras para Julia”. The poem can be found here. Below, the song by Paco Ibáñez sings José’s poem. The mp3 is provided by www.poesia-inter.net.

The endurance: Shackleton’s legendary Antarctic expedition

Mon, 03 Apr 2006 04:05:47 +0000

Sunday night. The first tornado warning of the season is over in Champaign-Urbana. This one came pretty close. The tornado did not hit the town but strong winds touched ground and caused damages. When the tornado alarm went off I was watching “The Endurance: Shackleton’s Legendary Antarctic Expedition”. I was deep into the movie and it took me a little while to realize that it wasn’t a whaler blowing the horn. After the proper safety stop looking for shelter and listening at the radio report, I continued with Endurance. If you have not seen it, just watch it, I am still speechless.

PPSN IX deadline is today!

Mon, 27 Mar 2006 12:00:30 +0000

Yes, today was the deadline for the Ninth International Conference on Parallel Problem Solving from Nature. Submit your paper here. The deadline has been extended till April, 5th.

Metadata stores

Mon, 13 Mar 2006 16:20:27 +0000

The DISCUS project has always supported that intuition that annotation capabilities are a must for knowledge and information exchange. For instance, imaging that you are analyzing the KeyGraph generated from a particular discussion (here you can find an example). You may want to enrich such graph with your analysis, comments, or related information. Basically, you want to add metadata to the KeyGraph. If such a capability is available, a whole new bunch of information will need to be efficiently stored to allow, not only fast and easy retrieval, but allow analysis of the added metadata. The Kowari project is an Open Source, massively scalable, transaction-safe, purpose-built database for the storage, retrieval and analysis of metadata. It provides a simple query language to interact with the metastore (iTQL). If you are familiar with SQL the resemblance will help you get up to speed very fast. The design is oriented to efficiently manage large volume metadata. Informal tests from Joe Frutelle, a NCSA colleague, have convinced me that this metastore can be the way to go for storing the large volumes of metadata that annotation may produce in DISCUS.

Innovation and creativity support via chance discovery, genetic algorithms, and data mining

Wed, 01 Mar 2006 17:28:45 +0000

by Xavier Llorà and David E. Goldberg, Yukio Ohsawa, Naohiro Matsumura, Yuichi Washida, Hiroshi Tamura, Masataka Yoshikawa, Michael Welge, Loretta Auvil, Duane Searshmith, Kei Ohnishi, and Chen-Ju Chao (2006).

New Mathematics and Natural Computation, World Scientific, pp. 2(1):85–100. Link to the Journal publication.

Abstract

Creativity protocols and methodologies tend to be time consuming if applied manually. This paper presents how information technologies can support innovation and creativity for collaborative scenario creation and discussion. The fusion of change discovery, genetics algorithms, and computer-supported collaborative tools provide computational models of innovation and creativity. The proposed technology allows groups of participants in a creative processes to have pervasive access to the analysis of the current scenario in real time. This paper introduces such innovation technologies gathered in the DISCUS project, and summarizes initial successful usages of DISCUS on marketing research workshops.

8 minutes and 37 seconds

Mon, 27 Feb 2006 14:00:37 +0000

Eight minutes and thirty-seven seconds was the fencing time of the semifinal between Peter Habala and David Lidow. By far, this bout was the best fencing at the Illinois Open. I ever seen in my two and half years of foil fencer. Everybody knew that this was the anticipated final. To impressive details: David’s point accuracy was from another world, Peter fleshing could cover in less than half a second around four lethal meters flying in the air. We were all speech less. The 64 tableau for the DE was totally skew since one fencer of my pool did not show up. This lead up to all A fencers to the same quarter. So after I was eliminate by Carla Esteva (A2002), she run into Peter Habala (A2006), and then Peter run into David Lidow (A2006). The other three quarters were not so nasty. Anyway, it was a memorable Illinois Open and I forgot my camera home. Oh, one last thing, the results can be found here.

The innovation pump: Supporting creative processes in collaborative engineering

Sun, 26 Feb 2006 22:45:54 +0000

by Xavier Llorà and David E. Goldberg (2006). IlliGAL TR No 2006011. Link to the PDF.

Abstract

The pervasive expansion of computers and Internet has change the way people collaborate. Terms such as cybercollaboratories are getting traction in day-to-day work. Web boards, blogs, e-mails, and instant messaging have become de facto mainstream communication channels. People scattered across the globe collaborate thanks to such technologies to carry out their daily work. Creative processes—such as collaborative engineering—have also taken advantage of such new communication media. This paper reviews the new framework set after these technologies and presents how collaborative creativity and innovation can be modeled and supported using computational models. The paper continues presenting a innovation-support model based on the usage of genetic algorithms as computational metaphors of human innovation. The paper also briefly discuses the results achieved using the proposed technologies in real-world collaborative creative processes.

All staff meeting at NCSA

Thu, 23 Feb 2006 18:14:49 +0000

I attended the first all staff meeting at NCSA. There a lot of exiting challenges here. You can learn more about the NCSA mission here, but a must-read document is the NCSA 2010: The future of NCSA, which gives a general overview of NCSA strategy for the years to come. Another interesting editorial by NCSA’s director Thom Dunning can be found here.

Illinois Open (Feb 25-26)

Tue, 21 Feb 2006 14:35:20 +0000

The Illinois Open at CAAFC is coming (Feb. 25-26). You can check the registered fencers here. Preregistrations are still possible here here.

LCSWeb creates a LCS and GBML paper database

Mon, 20 Feb 2006 20:11:38 +0000

Jan Drugowitsch in agreement with Tim Kovacs have team up to provide a LCS and other GBML paper database. You can access it here. You can also search the IlliGAL biblography at the LCS and other GBML web site (rigth-hand side search boxes).

New books section on the LCS and GBML web

Sun, 12 Feb 2006 22:21:07 +0000

The LCS and GBML site has a new section about book. It can be found here. The section collects LCS and GBML related books. If you found that relevant LCS and GBML books are missing, please post a comment to the editors of the site and they will take care of it.

El somriure de Burt Lancaster

Tue, 07 Feb 2006 18:49:04 +0000

The smile of Burt Lancaster is small delicious book collecting eleven short tales by Victor Alexandre. The different tales elaborate on eternal human elements. Ambition, despotism, fate, oppression, pride, deception, revenge are some of the few spicing ingredient of his tales. Victor usually writes about socio-political matters, but this book shows an interesting fiction facet of his writing. His tales are powerful and addictive. Once you read one, the next one is waiting and you need to read it. The first tale, “El fabricant de somnis” (The dream maker) gets your attention setting you up for a nice and sweet ending twist.

Preregister for the Illinois Open at CAAFC

Tue, 07 Feb 2006 18:16:32 +0000

The Illinois Open is going to be on February 25 and 26. You can preregister at Ask Fred.

Métaphysique des tubes

Thu, 02 Feb 2006 15:39:05 +0000

This is the first book I read from Amélie Nothomb (translation here).

Daughter of a Belgian ambassador, she was born in Kobe, Japan. The book is an auto-biography of a western girl born in Japan. It fascinated me that it only covers her early years. I felt for her easy fluent, sweet, and charming writing. A fascinating description of the magic Japan she was born in through her young innocent eyes. I cannot erase two episodes from my mind: the struggle to decide the first word to pronounce, and her sadness when she learned that May 5 is Tango no Sekku (the Boy’s festival) but there is no girl’s festival at all. You can find more information about her books here.

GECCO 2006 deadline is today!

Wed, 01 Feb 2006 12:00:08 +0000

Yes, today is the deadline for the ACM 2006 Genetic and Evolutionary Computation Conference. Submit your paper here, you have time till 10pm Pacific time.

Teo

Wed, 01 Feb 2006 07:20:01 +0000

Teo is the main character of a collection of books for kids. It is amazing how Teo’s illustrated books by Violeta Denou are ageless. I still remember spending hours reading again and again “En Teo va en avió” (Teo catches a plane).

CEC 2006 deadline is today!

Tue, 31 Jan 2006 12:00:49 +0000

Yes, today is the deadline for the IEEE 2006 Conference onf Evolutionary Compuations. Submit your paper here. Tomorrow is the deadline for the ACM 2006 Genetic and Evolutionary Computation Conference. However, the submission server will remain open for two weeks after that date, and many of the special sessions have already extended the deadline accordingly.

Chicago, Ask Fred, and Peter Habala

Mon, 30 Jan 2006 04:50:48 +0000

Finally, we decided to go up to Chicago for the CAAFC tournament. Long, long time since the last one I went. Now at CAAFC you can pre-register and check the competition results in Ask Fred. Here you can find the link to the tournament we went. The mixed foil event that I took part in was won by Peter Habala. I could sit for ever watching his fluent fencing. It seems so natural and easy…

2006 José Luís Balcazar Open

Sun, 29 Jan 2006 05:33:20 +0000

The 2006 José Luís Balcazar Open for novice fencers at The Point Fencing club took place on Saturday, January 28th, 2006. The final top four was occupied by:

Cece Borries (1st place)
Judy Vancleeff (2nd place)
Daniel Downey (3rd place)
Stephanie Helle (3rd place)

These are some of the pictures of the fencers and people in the meeting.

Software for fast rule matching using vector instructions

Thu, 19 Jan 2006 20:24:52 +0000

In the last decade, multimedia and scientific applications have pushed CPU manufactures to include native support for vector instruction sets. This software presents how to implement efficient condition encoding and fast rule matching strategies using vector instructions. The paper elaborates on Altivec (PowerPC G4 and G5) and SSE2 (Intel P4/Xeon and AMD Opteron) instruction sets producing speedups beyond ninety times when compared to non-vectorized implementations. The code of this post was used to run the experiments described in the IlliGAL 2006001 technical report “Fast Rule Matching for Learning Classifier Systems via Vector Instructions” by Xavier Llorà and Kumara Sastry. The code for fast rule matching can be downloaded here. Please read the README file for further details and instructions. The code is distributed under GPL license.

Next tournament at CAAFC

Thu, 19 Jan 2006 17:19:39 +0000

The next tournament is an Open tournament at CAAFC on Sunday 29th of January. Epee registration closes at 9:00am, foil at 11:30am, and sabre at 1:3opm. More information can be found at the schedule tab and competition calendar of CAAFC web site.

GECCO-2006 submissions deadline extended to February 1st

Wed, 18 Jan 2006 19:37:40 +0000

The submissions deadline for the ACM 2006 Genetic and Evolutionary Computation Conference to be held July 8-12, 2006 (Saturday-Wednesday) in Seattle, WA has been extended to February 1.

O Zahir

Sun, 08 Jan 2006 20:52:58 +0000

Yup, flying over the Atlantic always provide some spare time to put to good use. That’s what I did reading O Zhair, the latest Paulo Coelho’s book. You can read an extract of the first chapter here.

The Point Fencing Club 2006 Calendars

Mon, 19 Dec 2005 05:42:06 +0000

The calendars for 2006 arrived to the club last Friday during the 3rd Friday Foil Fight. Check them out! This year they come in a cool landscape layout instead of the 2005 portrait one.

A simple UMDAc implementation in Java

Tue, 06 Dec 2005 06:04:55 +0000

Cecilia Oversdotter Alm is working on an adaptation of active interactive genetic algorithms (see here) to her work on speech synthesis and perception of emotions in expressive storytelling. She needs a version of the active interactive genetic algorithm that works on continuous domains. For that reason I coded a version of UMDAc to replace the cGA currently used for discrete domains. The Java implementation of UMDAc can be found here. In order to run it, you need to download the COLT toolkit . The code is distributed under GPL license.

Special issue on chance discovery (I)

Fri, 02 Dec 2005 04:52:48 +0000

The Journal of New Mathematics and Natural Computation is running the first ot two parts of a special issue on chance discovery (volume 1, number 3). The number, besides including two regular papers, contains the first part of this special issue. The journal page can be found here. The table of contents of this first part of the special issue is:

Ruediger Oehlmann, Preface, page 371.
Yukio Ohsawa, Data crystallization: chance discovery extended for dealing with unobservable events, page 373.
Renate Fruchter, Yukio ohsawa, and Naohiro Matsumura, Knowledge reuse through chance discovery from an enterprise design-build enterprise data store, page 393.
Calkin a. s. Montero and Kenji araki, Human chat and self-organized criticality: A chance discovery application, page 407.
Ja-min Koo and Sung-bae Cho, Interpreting chance for computer security by viterbi algorithm with edit distance, page 421.
Edward Tsang, Sheri Markose and Hakan Er, Chance discovery in stock index option and futures arbitrage, page 435.

A biography to read

Thu, 01 Dec 2005 20:13:06 +0000

Note to self (wishful thinking): Read as soon as possible Sir Arthur Conan Doyle’s biography by Daniel Stashower titled “Teller of tales: The life of Arthur Conan Doyle” published by Owl Books.

New blog for LCS and other GBML

Wed, 30 Nov 2005 14:00:11 +0000

Recently, Pier Luca Lanzi and I decided to upgrade the learning classifier systems site he used to maintain. We are greatly thankful to professor David E. Goldberg for kindly accept to host the site at IlliGAL. The new LCS and other GBML site is still in dippers, but do not worry, it will grow soon.

Probabilistic models of text and images

Tue, 29 Nov 2005 15:41:58 +0000

Recently I attended a talk by David Blei about probabilistic models of text and images. His thesis described probabilistic models for the retrieval, organization, and exploration of large information collections, casting this tasks as statistical queries. The abstract of his thesis ca be found here and the thesis itself here.

New club hours at The Point Fencing Club

Tue, 29 Nov 2005 00:57:15 +0000

As of December 1st, the club will no longer be open for free bouting on Thursdays. It will be opened on Thursdays only for clinics and special events as announced. The new club hours are as follows:

Monday, Tuesday, Wednesday, Friday: 6:00-9:30pm
Saturday: noon-4:00pm

Evaluation consistency in iGAs: User contradictions as cycles in partial-ordering graphs

Mon, 28 Nov 2005 01:06:23 +0000

by Francesc Alías, Xavier Llorà, Lluís Formiga, Kumara Sastry, and David E. Goldberg (2006, accepted).

IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2006). May 14-19, 2006. Also as IlliGAL TR No 2005022. Link to the PDF.

Abstract

Active interactive genetic algorithms (aiGAs) rely on actively optimizing synthetic fitness functions. In interactive genetic algorithms (iGAs) framework, user evaluations provide the necessary input for synthesizing a reasonably accurate surrogate fitness function that models user evaluations or, in other words, his/her decision preferences. User evaluations collected via tournament selection only provide partial-ordering relations between solutions. Active iGAs assemble a partial-ordering graph of user evaluations. In such a directed graph, any contradictory evaluation provided by the user introduces a cycle in the graph. This property is explored in this paper to measure the consistency of the evaluations provided by the user along the evolutionary process. The consistency measures are applied to a real-world problem, the weight tuning of the cost function involved in corpus-based text-to-speech synthesis. Results show the usefulness of such measures to identify inconsistent users during the evolutionary tuning process, and successfully the number of evaluations required by more than half.

DISCUS as open source code?

Thu, 24 Nov 2005 21:06:35 +0000

Recently Jack Park from Iris Semantic Network left a comment asking, among other things, if DISCUS was going open source. This is an interesting question. DISCUS is currently under two patent pending processes. However, the current policies at NCSA, one of the DISCUS partners, is to push toward open source. We are currently finishing a first release of DISCUS International with support for English and Japanese. When it passes the release quality standards, some parts of DISCUS will be disseminated, may be as open source code. This is, however, an issue that needs to be ultimately discussed with the Office of Technology Management (the university office handling the DISCUS patent process).

Machine learning & Statistical Learning in R

Thu, 24 Nov 2005 20:09:03 +0000

Torsten Horthorn maintains a page with a list of packages for machine learning and statistical learning in R.

Programming IDEs

Thu, 24 Nov 2005 18:42:57 +0000

My favorite programming IDE.

Another useful one

KDevelop

Data mining tools

Thu, 24 Nov 2005 18:03:07 +0000

These tools are in Java. The first one is developed by the Automated Learning Group at NCSA

D2K
WEKA

Genetic algorithms tools

Thu, 24 Nov 2005 18:01:16 +0000

Check the IlliGAL source code page.

IlliGAL source code page

Database tools

Thu, 24 Nov 2005 18:00:45 +0000

Database engines

Database modeling

Fencing?

Thu, 24 Nov 2005 17:22:16 +0000

No, this is not about the art of placing fences around your garden; this is about fencing, the sport. Jose L. Balcázar introduced Javier, Hector, and me to fencing during his visit to UIUC in July 2003. He is a fencer at SAM (Sala d’Armes de Montjuïc) in Barcelona. Since then, I am a member of The Point Fencing Club in Champaign and join the United States Fencing Association to be able to take part in the Illinois division tournaments. Here hare some useful links

The compact classifier system: Motivation, analysis and first results (Presentation)

Mon, 05 Sep 2005 17:00:30 +0000

This is the presentation for the initial paper on the compact classifier system published in CEC 2005 (Edimburgh., Scotland, September 2005). The presentation slides here (1Mb).

The compact classifier system: Motivation, analysis and first results

Wed, 20 Jul 2005 22:13:47 +0000

by Xavier Llorà, Kumara Sastry, and David E. Goldberg (2006). Proceedings of the Congress on Evolutionary Computation, 1, 596—603. Also as IlliGAL TR No 2005019. Link to the PDF.

Abstract

This paper presents an analysis of how maximally general and accurate rules can be evolved in a Pittsburgh-style classifier system. In order to be able to perform such an analysis we introduce a simple bare-bones Pittsburgh-style classifier systems—the compact classifier system (CCS)—based on estimation of distribution algorithms. Using a common rule encoding schemes of Pittsburgh-style classifier systems, CCS mantains a dynamic set of probability vectors that compactly describe a rule set. The compact genetic algorithm is used to evolve each of the initially perturbated probability vectors. Results show how CCS is able to evolve in a compact, simple, and elegant manner rule sets composed by maximally general and accurate rules. The initial theoretical analysis and results also show that traditional encoding schemes used by Pittsburgh-style classifiers add an extra facet of diffiiculty. Such a bias plays a central role on the overall performance and scalability of CCS and other Pittsburgh-style systems using such encoding schemes.

Combating user fatigue in iGAs: Partial ordering, support vector machines, and synthetic fitness

Tue, 19 Jul 2005 19:38:32 +0000

by Llorà, X., Sastry, K., Goldberg, D.E., Gupta, A., Lakshmi, L. (2005).

Published in the ACM Genetic and Evolutionary Computation Conference (GECCO 2005), ACM press, pp. 1363–1371. Also as IlliGAL TR No 2005009. Link to the PDF

Abstract:

One of the daunting challenges of interactive genetic algorithms (iGAs)—genetic algorithms in which fitness measure of a solution is provided by a human rather than by a fitness function, model, or computation—is user fatigue which leads to sub-optimal solutions. This paper proposes a method to combat user fatigue by augmenting user evaluations with a synthetic fitness function. The proposed method combines partial ordering concepts, notion of non-domination from multiobjective optimization, and support vector machines to synthesize a fitness model based on user evaluation. The proposed method is used in an iGA on a simple test problem and the results demonstrate that the method actively combats user fatigue by requiring 3–7 times less user evaluation when compared to a simple iGA.

The evolutionary path to innovation and creativity

Thu, 26 May 2005 15:30:42 +0000

Keynote speaker talk in Congreso Mexicano de Computación Evolutiva (CONCEV-2005) about how evolutionary models and chance discovery can support human innovation and creativity. The presentation may be download here (PDF 8.5Mb).

Mining social networks in message boards

Tue, 12 Apr 2005 19:49:54 +0000

by Matsumura, N., Goldberg, D.E., Llorà, X. (2005). Published in the Symposium on Conversational Informatics for Supporting Social Intel ligence, The Society for the Study of Artificial Intelligence and the Simulation of Behavior Press, pp. 18–27. Also as IlliGAL TR No 2005001. More info.

Abstract:

In this paper, we first present an approach to extract social networks from message boards on the Internet. Then we show structural features of 3,000 social networks extracted from 3,000 message boards from 15 categories in Yahoo! Japan Message Boards to prove the relationships between the features and the categories. After we classify social networks into three types (interactive communication, distributed expertise communication and soapbox communication), we suggest an approach for mining social networks to identify the types of communication, the roles of individuals, and important ties, all of which can be used to redesign the means communication as well as understand the state of communication.

Mining social networks in message boards (Presentation)

Tue, 12 Apr 2005 17:00:11 +0000

This is the presentation for the paper “Mining Social Network in Message Boards” (the paper PDF may be found here (1Mb)) published in the Sympposium on Conversational Informatics for Supporting Social Intelligence and Interaction (AISB'05, 12-15 April, Hatfield, UK, 2005). You can download the PDF here (2Mb).

KeyGraphs, small-worlds, and interactive GAs

Mon, 23 Aug 2004 22:00:20 +0000

This is the presentation about using DISCUS interactive genetic algorithms to tune the KeyGraph algorithm for a given document. The paper was published in the First European Workshop on Chance Discovery, EWCD 2004 (Valencia, Spain). The paper may be found here (PDF, 0.7Mb) and the presentation slides here (PDF, 0.5Mb).

Chance discovery and marketing

Mon, 23 Aug 2004 21:00:55 +0000

This is the presentation about using DISCUS and chance discovery for marketing purposes. The paper was published in the First European Workshop on Chance Discovery, EWCD 2004 (Valencia, Spain). The paper may be find here (PDF, 0.7Mb) and the presentation slides here (PDF, 0.6Mb).

DISCUS 2.0

Tue, 20 Jan 2004 17:00:38 +0000

Presentation about the pilot version of DISCUS 2.0. The slides may be found here (0.5Mb PDF).