Alexandre's Notebook: 2009

How Randomness Rules Our Lives

Authors @ Google talk with Leonard Mlodinow about his book, "The Drunkard's Walk: How Randomness Rules Our Lives".

I haven't read the book, but very much enjoyed the talk.

Euclid's Elements

Written by the Greek mathematician Euclid in Alexandria circa 300 BC, Euclid's Elements is a collection of mathematical definitions, axioms, theorems and proofs about geometry.

Is the most successful and influential textbook ever written, one of the most influential works in the history of mathematics and is estimated to be second only to the Bible in the number of editions published.
- wikipedia

A proof, from Euclid's Elements:

And there's of course an (English)online version with interactive demonstrations

Functional Programming - Erik Meijer

Enjoying these lectures:

A journey into the world of Functional Programming with functional language purist and high priest of the lambda calculus, Dr. Erik Meijer.

lectures here

3rd European Lisp Symposium in Portugal

The purpose of the European Lisp Symposium is to provide a forum for the discussion and dissemination of all aspects of design, implementation and application of any of the Lisp dialects. We encourage everyone interested in Lisp to participate.

Call for Contributions: we would welcome both papers describing original work, not published elsewhere, and submissions for tutorial sessions.

Details here...

Functional

Here's a few articles from Slava Akhmechet (on www.defmacro.org) that i enjoyed reading:

And there's more.

YQL

The Yahoo! Query Language (YQL) web service allows to query web data in an SQL-like syntax, so you can do things like:

Get all cat pictures from flickr:

select * from flickr.photos.search where text='cat'

Get pictures from flickr that have interestingness starting with word Chinese:

select * from flickr.photos.interestingness where title like 'Chinese%'

Get sushi places in San Francisco that have a rating bigger than 4:

select * from local.search where query='sushi'
and location='san francisco, ca'
and Rating.AverageRating > 4.0

Get the profile of all my friends(my social connections):

select * from social.profile where guid IN
(select guid from social.connections where owner_guid=me)

It allows to combine several different sources in the same query, just like a SQL query that uses several tables, is possible to join results of GEOIP services with restaurant rating services, rss feeds, financial information, weather service, etc etc...

Nice one Yahoo!

Screencast: Introducing YQL

YQL site

The book of Odds

Collection of Odds about everything here (on US population).

The odds a man does not own a pair of blue jeans are 1 in 10 (US, 2001).
The odds a man using a public restroom was observed not washing his hands are 1 in 2.99 (US, 2007).
The odds a woman owns a pair of high heels are 1 in 1.64 (US, 2001).
The odds a person 25 or older with a master's degree has an income of $100,000 or more are 1 in 5.43 (US, 3/2007).
The odds a female 25 or older with a master's degree has an income of $100,000 or more are 1 in 10.98 (US, 3/2007).
more...

Hermosa y Horrible- a.sáxeo

a.sáxeo in a Spanish art gallery

Javascript validator on Emacs (v2)

A Javascript validator that I use frequently, mentioned here for Textmate, now on Emacs:

(defun my-js-validate ()
  (interactive)
  (if (string-match "\\.js$" (buffer-file-name))
      (shell-command
       (concat
        "java -jar /Bins/Javascript/rhino1_7R2/js.jar /Bins/Javascript/lint/rhino_jslint.js "
        (buffer-file-name (current-buffer))))))

(defun my-js-hooks ()
  (progn
    (message "Entered Javascript Mode")
    (add-hook 'after-save-hook 'my-js-validate)))

(add-hook 'javascript-mode-hook 'my-js-hooks)

To force to run: M-x my-js-validate

With the hooks setup, whenever a javascript file is saved then my-js-validate is executed.

(if you use js2-mode or espresso-mode, then you need to change 'javascript-mode-hook accordingly, for example to 'espresso-mode-hook)

less calculus, more probabilities

PT - 2009

re-post: Standard Deviation explained

Why use the standard deviation instead of the more intuitive mean deviation? Found a nice explanation here.

PT bedroom bookshelf

book: Career 2.0

Good career advice.

My Takeaways:

Ultimately the responsibility for your career and your life rests with one and only one person. You.
Be passioned about your work.
Keep on learning, keep on building your intellectual portfolio.
Select your goals and remind them frequently.
Breakdown your goals into weekly and even daily actions.
Sharing knowledge, by writing and speaking publicly, deepens your grasp of the knowledge.

This book is also good reference of public speaking techniques.

Thanks Erki, for the suggestion.

Michael Jackson Dance Tribute - Tallinn

Awesome dancing in one of the most busiest streets of Tallinn.

FlightCaster

“FlightCaster predicts flight delays. We use an advanced algorithm that scours data on every domestic flight for the past 10-years and matches it to real-time conditions. We help you evaluate alternative options and help connect you to the right person to make the change.” check it out

Its a very interesting mix of Clojure, Rails, Hadoop, Amazon EC2 and YCombinator.

Basketball Throws

A little probability simulation, using Clojure and Incanter, hosted on google docs:

book: ReSampling - The New Statistics

This book describes a revolutionary —but now fully accepted— approach to probability and statistics. Monte Carlo resampling simulation takes the mumbo-jumbo out of statistics and enables even beginning students to understand completely everything that is done.
— from books intro

Its statistics made easy, because there's no need to tinker around with theory much, its all about harnessing computer power to do the simulations of the probabilities you'r after. Which i think is just great. You can find the book here.

On the process, the authors(Julian Simon and Peter Bruce) developed also a computer language for performing these simulations described and used in the book.
This has been implemented in a Java application, by John Grosberg, and made available as Statistics101, so you can easily play around and test the code examples from the book.

book: FREE

Chris Anderson, editor in chief of Wired and author of The Long Tail, has published a new book that looks at something of interest to Consumerist: the trend of content and services to slide toward free, especially in the digital world. It's pretty light reading and an interesting look at economics in the digital marketplace in particular, and for now, at least, it's available in multiple formats for free.
— from consumerist

It discusses Free, its meaning, its implications, its misconceptions, its business models, etc, Free is no doubt the trend of the future that has breen brought full-on with the digital age and is having a huge impact in most industries including radically changing money making models.

In its launch is being offered for free, I listened to the Free's free audio version from wired.

Emacs (v2)

Why use Emacs for Clojure

Here's a few of references i found recently that help to get up and running with Emacs and Clojure development:

Iteration I
Started using the Carbon version of emacs. Then take a look at: Emacs Starter Kit(ESK), it provides a saner set of defaults than you get normally with Emacs. For step by step installation instructions, here's a good tutorial: Setting up Emacs & Clojure with Emacs Starter Kit. And finally, a few of more details about developing Clojure setup with the ESK, from ESK author himself.

Iteration II

After spending some time with Carbon Emacs(and learning a bit better how emacs work) I've decided to try out the Aquamacs version.

Aquamacs is, out of the box, more mac friendly because supports default key bindings like copy, paste, selectAll, etc.. that all mac text handling apps normally have.

And thats actually very useful, even after learning a lot of emacs shortcuts. Note that emacs key bindings are also worth learning, is amazing to discover how many applications support them. But i've found that stuff like Ctr-C, Ctr-V, and Selecting text working as you expect just make you move around faster.

One issue i ran into, is that the ESK i mentioned before is not compatible with Aquamacs because of different key bindings schemes.

But that turned out not to be a problem, and ended up learning on how to customize Emacs in the process, Here's a a few references i've used for my current emacs setup:

References:

AquamacsFAQ, for several tips and tricks on the Aquamacs
Emacs Wiki, central point for a lot of emacs packages and tips, essential reference.
ESK, I use this as a reference, for learning on how to customize and go check what packages are used there. I use, for example, the zenburn.el theme from there.

Packages:

ELPA, for installing packages, great to install all you need for Clojure development instantly.
ido, lot of helpful stuff, especially the automatic completion when using Dired.
espresso, for javascript editing.
Balancing your parentheses, to customize the highlight of your parenthesis, even possible to highlight a mismatched parenthesis with red, to give visual hint you closing the wrong ones!

The Power of the Pentatonic Scale

Fun video of Bobby McFerrin engaging an audience in singing a little tune using the pentatonic scale, this music scale is a 5 notes sequence of musical intervals that is found again and again in completelly different music cultures around the world. See in Wikipedia.

book: Super Crunchers

Nice book, giving a lot of reasons on why data crunching is so important.
Enjoyed also the story about the 2 SD(Standard Deviation) Rule, used for point estimation.

Visualizing Sampling Size Accuracy with Pretzels

A common problem when doing data mining is that the data you need to work with, is too big(too many data) and painfully slow to process, especially if you want to run complex mining algorithms on it.

Sampling allows to select randomly, a subset of examples from the total data, and then work with only that smaller subset. It works, because quite often the subset is statistically equivalent to the total data we are looking at.

The tricky part is choosing how big this sample should be, because we need to guarantee that the subset we pick is big enough to represent all the (total)data accurately, and at the same time we want it small as possible, to be easy to work with.

For example, lets look at the mean of the weight of 50 bags of pretzels,

I'm using the language Clojure, with the library Incanter:

We see that the real mean(of total data) is 451.2, so our sample mean should come as close to that values as possible.

For this experiment, i'll use incanter's sample method, that returns a sample of the given size from the given collection.

So, for a sample size of 5, the whole procedure is: pick 5 pretzel bags weights at random, calculate the mean and subtract that with the real mean, the resulting value shows the error(how far we are from the real mean).
Doing that only once, might happen that we pick 5 bag weighs that match exactly the real mean, and thus be fooled by a one time lucky strike, to avoid that lets repeat that procedure 10.000 times.

And lets also use increasingly bigger sample sizes:

As expected, as the sample size gets bigger, less the error and closer to the real mean.

So, how small subset can it really be? Well, it all depends on how accurate you need to get, and thus how big error, you tolerate.
Also note that the error does not decreases linearly. increasing 5->10 samples we decreased error by 1, increasing 10->40 samples also decreased error by 1.

So, best way to have an idea on the error variating with the sample size is to visualize it:

Note that it starts to level off at the end, meaning that adding more samples when the number of samples is already big will have less impact, compared to adding them when the sample size is small. And of course with different data, the error curve will show different pattern, so Visualizing it is a great way to figure out the sample size you need.

Kudos for Incanter, i'm having a great experience using it.

Quoting Björk on a friday morning

from my twitter:

Good Morning World, from Sunny Tallinn. Am enjoying my latte and feeling well! but i know is not for long...

As Björk would say: Its all so peaceful and quiet until you open the eMAIL, bam boom, you blow a fuse and the sky up above is caving in.

re-post: Java, please stop ruining my fun.

Spot-on description of Java hiccups that slow down Clojure development.

To minimize my own pain on this, i've made up a little template project, that I start from, when doing Clojure code, it looks like this:

clj-template/

build.properties

build.xml

clj

lib/

clojure-contrib.jar

clojure.jar

jline-0.9.94.jar

README

src/

myapp/

helper.clj

myapp.clj

test/

myapp/

my_test.clj

myapp_test.clj

I edit build.properties to specify the class of the Main App (myapp), and the class of the tests(myapp-test).
lib/ contains all the jar's needed for the app, just drop a new one into lib/ and is automatically included when running or compiling.
./clj allows to get a repl or run a script if is called with a .clj file as argument. like: ./clj src/myapp.clj of course includes automatically all jar's existing in lib/
build.xml is used by ant(included with all Java distributions), where i have tasks for: compile, create stand-alone jar, run tests, extract documentation.
And then there's a bunch of small details to be aware off: naming of the Class has to match filename, specifying (:gen-class) in main Class to be able to create stand-alone jar, always defining a "-main" as the main entry point for the app, etc...

I see all these hiccups as a small cost of being on Java platform. When you then look at the benefits(of being in the Java platform) they largely overwhelm these costs. But yes they are annoying at times, so if we can fix them please do, after all programming should be about solving algorithmic puzzles not dependencies puzzles.

re-post: Augmented Reality Presentation

via: asaxeo

re-post: How did I not find this language years ago ?

about Lisp

Metric driven site improvements

An explanation of how to improve your site, by testing alternatives and measuring results, its a part of my daily job.

Kudos for Google to make free tools that everyone can use.

Clojure Books

Books that influenced Clojure

via: liebke

Quotes on Less is More

If programmers got paid to remove code from sofware instead of
writing new code, software would be a whole lot better.

- Nicholas Negroponte

Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away.

- Antoine de Saint-Exupéry

The letter I have written today is longer than usual because I lacked the time to make it shorter.

- Blaise Pascal

Lets Do It 2008 - Teeme Ära 2008

via: asaxeo

re-post: RepRap machine

A home printer that can print 3D objects, and can even print a copy of itself.

book: Getting Real - Alone Time

People need uninterrupted time to get things done

37signals is spread out over four cities and eight time zones. From Provo, Utah to Copenhagen, Denmark, the five of us are eight hours apart. One positive side effect of this eight hour difference is alone time.

There are only about 4-5 hours during the day that we're all up and working together. At other times, the us team is sleeping while David, who's in Denmark, is working. The rest of the time, we're working while David is sleeping. This gives us about half of the day together and the other half alone.

Guess which part of the day we get the most work done? The alone part. It's not that surprising really. Many people prefer to work either early in the morning or late at night — times when they're not being bothered.

When you have a long stretch when you aren't bothered, you can get in the zone. The zone is when you are most productive. It's when you don't have to mindshift between various tasks. It's when you aren't interrupted to answer a question or look up something or send an email or answer an im. The alone zone is where real progress is made.

from: 37 signals - getting real

re-post: Markov Chains

nice explanation of Markov Chains

Clojure rocks

lispnyc: Stuart Sierra on implementing AltLaw.org in Clojure

The Soul of a New Start-Up Machine

on Paul Graham's Y Combinator's

Small, Lean, Fast, Less and Cheap are keywords that keep appearing as success factors of very interesting startup's.

re-post: The Way i Work by Matt Mullenweg

nice read

book: Getting Real

From 37signals:

Getting real is less. Less mass, less software, less features, less paperwork, less of everything that's not essential (and most of what you think is essential actually isn't).

Getting Real is staying small and being agile.

Getting Real is about iterations and lowering the cost of change.

Getting Real is all about launching, tweaking, and constantly improving which makes it a perfect approach for web-based software.

Getting Real delivers just what customers need and eliminates anything they don't.

Great Stuff, do check it out, its here for free.

Video - The Vendor Client relationship

Funny

Future Vision on Manufacturing

via: bebetter

Incanter

Incanter is a Clojure-based, R-like statistical computing and graphics environment for the JVM.

oh i like this, very many :)

Again to note, the bliss of being on a huge platform while using a lean language gets you the best of both worlds: nice pleasant development with the possibility of leveraging mature libs to build your own killer app.

Video: Europe according to Estonians

Flavored with Estonian humor and with the right Eesti-English accent. (via JLM)

on the same page

Estonia can get as hot as a southern country :)

re-post: Software for NASA

The group has so much data accumulated about how it does its work that it has written software programs that model the code-writing process. Like computer models predicting the weather, the coding models predict how many errors the group should make in writing each new version of the software. True to form, if the coders and testers find too few errors, everyone works the process until reality and the predictions match.

from here

on Javascript

This Just In: JavaScript is a Real Language, and from a comment of that post:

Of course JavaScript is a real language, in fact you could go so far as to say JavaScript is a real functional programming language! Its native support of closures and higher-order functions puts it in a league that most developers never even considered it to be capable of.

On Douglas Crockford talk JavaScript: The Good Parts (worth checking) he mentions that Brendan Eich, the inventor of Javascript, actually wanted to code in Scheme, but was not allowed at Netscape so he inserted Scheme features into Javascript, thus the existence of closures and higher-order functions in Javascript.

And for the ones about to code:

Here's Douglas, javascript coding guidelines. Luckily, he's also created (and shared) a Javascript validator that we can use to check how correct and safe our javascript is, called jslint.

jslint can be run directly from web, but might be more convenient to integrate it into your work flow. For example, using Textmate (or command line), you can use Rhino(a Javascript Java interpreter) to run it like:


java -jar /dir/rhino1_7R2/js.jar /dir/jslint.js ${TM_FILEPATH}

Also, when using jslint don't forget to check the options to adjust the warnings and errors you want to see, when validating your code.

re-post: A Third Order Quine in Three Languages

This is a Haskell program that outputs a Python program that outputs a Ruby program that outputs the original Haskell program.
from sigfpe blog

How cool is that!

going up the abstraction ladder

The central idea behind functional programming is that of building new functions by composing existing ones. Monads move up a level of abstraction by hiding the plumbing necessary to compose functions whose parameters and return values are different. Monad transformers move up yet another level of abstraction by allowing monads to be combined so that the plumbing that each monad does is combined and abstracted away, but without having to write new code.
from: monads in clojure

And a some references of Monads tutorials:

Harnessing wasted human cycles

nice

clojure, twitter and test-is

Here’s a way, using clojure, to get your tests alerts on twitter.

Go get jtwitter, put it in your classphath and create:

(update "username" and "password" with your twitter login)

Then hook it up to the “test-is” library(test library in clojure): just overwrite the report summary method that by default prints out the summary of the executed tests.

Before:

After:

Just added a line calling the twitter-update method.

And thats it, now every time you run your tests, you will have the failures and errors twittered:

I've created a 2nd account on twitter where i post these automated messages. And have my clojure tests(regression tests actually) running every week, and letting me know if all is good.

This little twitter-update is very easy to use for any kind of alerts and automations.

Google Search Extended

Leo talks with Tim Ferriss

audio interview about productivity, routine and life.

ack is the new grep

This is a great improvement on your toolkit(if you use grep, that is).

Checkout ack, auto-entitled better than grep.

There's also a very nice Textmate plugin to replace the Find in Project. Its so much faster, does not block your textmate when searching and you are able to copy-paste the end results.

ps: I found it, from the tools on the edge article. Go have a peek you might find some more gems for your toolbox.

Work Best Practices

via Giles Bowkett

re-post: Pop Up Guide to the Personal Computer

go see

A Brief, Incomplete, and Mostly Wrong History of Programming Languages

1980 - Alan Kay creates Smalltalk and invents the term "object oriented." When asked what that means he replies, "Smalltalk programs are just objects." When asked what objects are made of he replies, "objects." When asked again he says "look, it's all objects all the way down. Until you reach turtles."

1987 - Larry Wall falls asleep and hits Larry Wall's forehead on the keyboard. Upon waking Larry Wall decides that the string of characters on Larry Wall's monitor isn't random but an example program in a programming language that God wants His prophet, Larry Wall, to design. Perl is born.

1995 - Yukihiro "Mad Matz" Matsumoto creates Ruby to avert some vaguely unspecified apocalypse that will leave Australia a desert run by mohawked warriors and Tina Turner. The language is later renamed Ruby on Rails by its real inventor, David Heinemeier Hansson. [The bit about Matsumoto inventing a language called Ruby never happened and better be removed in the next revision of this article - DHH].

hilarious

Hadoop - The Future of (large) Data Infrastructures?

Check out the PoweredBy list of Hadoop.

Microsoft is using it for search.

HTML2Wiki converter tool

For those who work with Wikis, but already have HTML content. here's a little tool to help converting HTML to Wiki Format. Several Wiki's & Formats supported.

re-post: European Lisp Symposium 2009

programme details

João Pavão Martins and Ernesto Morgado, will be presenting :) Also had them also as teachers in college, together with António Leitão, they'r a very bright bunch. High quality teaching coming out from that Artificial Intelligence group.

Clojure and Selenium

Motivation

I needed a kind of crawler to go around a list of pages, invoke some javascript and collect that output.

Curl or a regular http lib’s don’t do the trick because i need to run javascript on each requested page. For that i can use Selenium, Selenium is a great framework to perform web testing, that uses directly a browser and thus we can run Javascript.

Selenium can be scripted from Java which matches very well with my wish to learn Clojure :)

Solution

Its not really a crawler in the sense that it does not go around automatically following all the links it finds, it actually gets the list of links to check from the site sitemap.xml.

As some sitemaps.xml are huge, i added also a little pick-a-sample function that randomly selects only a subset from all the sitemap.

Code

Im on the process of learning Clojure, so probably a lot of things could be improved.

For Selenium, we need first to start the server, then the client, and then use the client to browse the pages. As is not very elegant to have a “start server” and “start client” on the top of the script and a “stop client” and “stop server” call at the end of the script, so i’ve wrapped it around a macro (one of the major strengths of Lisp like languages).

The whole thing goes like this:

process-sitemap receives a sitemap, transforms it into a map(with xml-to-zip), collects the url links in it, then picks a sample from them(with pick-a-sample) and calls check-pages with them.

check-pages gets a list of urls. It starts by using the macro, obtains a-browser from it, then iterates over the list of urls, calling check-a-page on each url(a-url). Note that at this point the standard output is redirected to a file, so i can log the results from check-a-page.

check-a-page gets a-browser and a-url, so you can guess what it will do :)It opens that url in the browser, calls the javascript, and prints to standard output the return of the js call.

Hope google does not mind to use their site as an example. But do not run this on Google site, its just an example, use it on your own site!

For this to run you will need to have in your classpath a bunch of jar libs, this is how my lib folder looks like:

         lib/
        clojure-contrib.jar
        clojure.jar
        jline-0.9.94.jar
        selenium-java-client-driver.jar
        selenium-server.jar

I called this app “coverager”

Main Code: coverager.clj

And of course tests: coverager_test.clj

Take Aways

Clojure is great! Its my opinion that on the Lisp family of languages the code is more elegant and visually cleaner than the C family.

I don’t care much for working directly with the Java language, but working on the JVM with other languages like JRuby, Clojure, and harnessing all the vast amount of Java libs and infrastructure out there is a major major advantage.

I suspect i will be spending more time with Clojure in future :)

Photo: Lucky Dice

book: Collective Intelligence in Action

From editors site:

Collective Intelligence in Action is a hands-on guidebook for implementing collective-intelligence concepts using Java. It is the first Java-based book to emphasize the underlying algorithms and technical implementation of vital data gathering and mining techniques like analyzing trends, discovering relationships, and making predictions.This book is for Java developers implementing collective intelligence in real, high-use applications.

Overall i liked it and have learned loads, it does have more Java code in it that i'd wanted, but for the ones who intent to use Java language the code in the book is handy, well, its after all a Java book, right? There's also some very useful, class diagrams that will help you to use the code described from JRuby, Clojure, Scala...

Its goes through all the process of: explaining the data structures needed to represent data in your app, how to collect data from web(blogs, etc), theory & implementation of mining algorithms, leveraging open source tools like Weka and Lucene, up to how Amazon, Google News and Netflix do their data mining(to some extent).

Liked also the data mining(machine learning) summaries on different data mining techniques, very compact and nicely done.

Clojure Reference

Great reference (an quite complete, as far as i can tell) to the Clojure language. by Mark Volkmann

And Clojure page from same author here.

re-post: Make your own Fonts

from: asaxeo.blogspot.com

re-post: Almost Viral

Article about online marketing strategies: Paid & Viral.

couple of points:

Almost viral can be safer than full on viral. (specially if your product is new)
The real cost per acquisition should also account for the viral coefficient. And the growth of acquisition is a geometric series.

Great use of math to build a formula that models the (real) cost per acquisition.

re-post: How to Become As Rich As Bill Gates

fun read here

re-post: Shit is not easy. Stuff takes time.

Nice (but slightly harsh) "rant" from Steve Yegge about Complexity: Have you ever legalized marijuana?

book: The 4-Hour Workweek

A favorite of mine.

Some discussed topics and quotes:

80/20 Pareto's rule.
Parkinson's Law.
The need for an "Information Diet" and the excess of information.
Spending a lot of time on something unimportant does not make it important.
Lack of time is actually lack of priorities.
Automations to free you from work.
Virtual Assistants to free you from work.
People don’t want to be millionaires. They want to experience what they believe only millions can buy.
Less is not laziness. Focus on being productive instead of busy.
Slow down and remember this: Most things make no difference.

Thanks José Santos, for the suggestion.

A Google SMS

My geeky mind thinks is kinda cool to get an sms from Google.

(I blanked out the content)

Starting up Clojure simple tips

Here’s a couple of things that i use, when learning about Clojure.Im on Mac and using Textmate.

Screencasts & Tutorial

Here are a series of 10 screencasts that i found to be quite useful Intro to Clojure (youtube playlist)

And there’s also Rich Hickey screencasts http://clojure.blip.tv/, but they are longer, so start the with ones above.

For reading material take a look at Mark Volkmann Clojure very complete notes here. And his clojure references here.

Folder Structure for code


bin/
repl.sh
run.sh
lib/
clojure-contrib.jar
clojure.jar
jline-0.9.94.jar
*.jar
README
src/
*.clj
test/
*.clj

Assuming a structure, is possible to do some automation, see next.

Textmate Builds run.sh and repl.sh

To address the issues of including jar’s you want to use in the Classpath here’s a little ruby script that I trigger from Textmate. It looks into the lib/ dir and creates the repl.sh and run.sh scripts:


#!/usr/bin/env ruby -wKU

project_dir=ENV['TM_FILEPATH'].split("/")[0...-1].join("/")

libs_path = Dir[project_dir + "/../lib/*.jar"].collect

File.open(project_dir+"/../bin/run.sh", "w+") do |fil|
fil.puts "java -cp .:#{libs_path.join(':')} clojure.lang.Script #{ENV['TM_FILEPATH']}"
end

File.open(project_dir+"/../bin/repl.sh", "w+") do |fil|
fil.puts "java -cp .:#{libs_path.join(':')} jline.ConsoleRunner clojure.lang.Repl #{ENV['TM_FILEPATH']}"
end

So if you want to use some new jar’s just place them inside the lib/ and trigger this ruby code that will re-generate the the run.sh and repl.sh scripts.

Existing code

Other thing that i keep close by is existing Clojure code examples, i also use Textmate so that they are just a keyboard shortcut away(they are 3 different commands):

 git fetch
mate '/somewhere-in-your-disk/programming-clojure/'

svn up
mate '/somewhere-in-your-disk/clojure-contrib/'

svn up
mate '/somewhere-in-your-disk/svn_clojure/trunk'

you can get this code onto your machine from:

 Clojure core
http://code.google.com/p/clojure/
svn checkout http://clojure.googlecode.com/svn/trunk/ clojure-read-only

Clojure contrib
http://code.google.com/p/clojure-contrib/
svn checkout http://clojure-contrib.googlecode.com/svn/trunk/ clojure-contrib-read-only

Programming Clojure book:
http://github.com/stuarthalloway/programming-clojure/tree/master

API

Another handy keyboard shortcut is to get the clojure api web page, also directly from Textmate, setup a new command with the code:

open http://clojure.org/api

Google it

A lot of people are starting to explore Clojure, so google it and you’ll find a lot of posts and tips on clojure already out there.

HTML to PDF & Printing

Making HTML into PDF properly is normally mission impossible: tables and images get cut in the middle, titles can appear alone at end of page with the text starting on the next page, headers and footers are normally non-existing or have some strange stuff from the browsers, like the file path to the html file… normally margins make no sense are either too big or too small, etc etc…

At the same time CSS is great to format layouts of documents in a clean, reusable way… Almost seems ideal, but then when you get it onto PDF(or print) it gets all funky…

MultiMarkdown to PDF & Printing

I’m especially interested in this because i am a fan of the so called lightweight markup languages like Markdown(specially the MultiMarkdown extension). Thus avoiding having to go into Microsoft Word.

MultiMarkdown can be easily outputted into HTML and use CSS for formating, but then when trying to get it onto PDF(or printing), trouble starts…

You can either print it(or create PDF) directly from the browser and although it does keep the css formating still has all the troubles described in begin of post… Alternatively, you can transform the MultiMarkdown to LateX and then to PDF, the output is nice but the CSS formatting does not work there, and although the LateX supports formating, its complicated, so in practice i always ended up with the same looking PDF, which ends up being rather boring…

recently i found a 3rd option.

Prince XML

On a comment of a previous blog post on lightweight markup languages, someone suggested an alternative tool: Prince XML

Its a command line application, that gets an html file as input(plus a bunch of other options) and outputs a PDF file. Go check some samples here and example web applications using it here.

I found it so much better from the alternatives i’ve tried, that Im posting it in case someone else is twiddling around the same issues.

You can even find a Google talk about it, with Håkon Wium Lie(who proposed CSS originally and is Opera CTO) and Michael Day (system architect for Prince).

For Textmate users

I’ve created a simple command, so i get my MultiMarkdown directly onto PDF using 1 command, (its originally the Multimarkdown command that creates HTML with a couple of more lines added that produce the PDF using prince), here’s the code:

# Process the MultiMarkdown document into HTML and then PDF.

NAME="${TM_FILEPATH:-untitled}"
BASENAME="${NAME%.*}"

cd "${TM_MULTIMARKDOWN_PATH:-$HOME/Library/Application Support/MultiMarkdown}"
cd bin

#to HML
./multimarkdown2XHTML.pl > "$BASENAME.html"

#to PDF
`prince "$BASENAME.html" -o "$BASENAME.pdf"`

#open PDF
`open "$BASENAME.pdf"`

This assumes you have Prince XML installed in your system, and you can call it from command line.

Multiplication Technique

Tim Ferriss

Tim Ferriss has some amazing ideas on what he calls: Experiments on Lifestyle Design.

re-post: Why i Like PLT Scheme

nice article on: Why i Like PLT Scheme(a Lisp dialect)

re-post: Time Management

Most Time Management Is Rubbish. Here Are Ten Things That Work for Me.

Is Looking At Ya - Part ii

part 1 here

Dad's New Blog

My Dad has a blog: asaxeo.blogspot.com (Portuguese text mostly). How cool is that!

book: The Speed Reading Course

by: Peter Shepherd & Gregory Unsworth-Michell
Very short. It teaches techniques:

to increase reading speed,
to more quickly grasp and digest information(key work highlight, MindMaps, visualization of information, pre-read survey, etc etc..),
to improve memory recall.

Useful tips in today's information overload.

book: The Power of less

How to do less while accomplishing more, learn how to limit yourself to what's important. Full of very practical advices.

"With the countless distractions that come from every corner of modern life, it’s amazing that we’re ever able to accomplish anything. The Power of Less demonstrates how to streamline your life by identifying the essential and eliminating the unnecessary — freeing you from everyday clutter and allowing you to focus on accomplishing the goals that can change your life for the better."

Thanks José Santos, for the suggestion.

book: Predictably Irrational

"Do you know why we sometimes find ourselves excitedly buying things we don’t really need?"

"Do you know why we still have a headache after taking a five-cent aspirin, but why that same headache vanishes when the aspirin costs 50 cents?"

"Do you know why people who have been asked to recall the Ten Commandments tend to be more honest (at least immediately afterward) than those who haven’t? Or why honor codes actually do reduce dishonesty in the workplace?"

"..We are pawns in a game whose forces we fail to understand..."

Is not in any way related to conspiracy theories and the like, is about simple every day things, the book clearly shows(by experiments) that everybody makes predictable mistakes, again and again, we just don't notice it most of the times, thus being aware of them is a first step on how to avoid them.

Viewpoints Research Institute - Alan Kay

An excruciating example of an area that needs more than incremental improvements is programming, both in the large and in the small. Code is too: large, complex, costly, buggy, insecure, segregated, and inexpressive. We have plans to attempt a qualitative reinvention of programming and to start one of the subprojects this year: to make a practical working mathematical model of a complete personal computer system.

promissing stuff here

Why's (Poignant) Guide to Ruby

There's something magical about this introduction to ruby.
This is the book that got me started in the ruby language.

impressive wearable tech

Focus on what's important

Nice article about focusing on whats important for starting a new business: Trying to Earn More Money? Stop Wasting Your Time

relates to my previous rant: Are you spending time with the right stuff?

The Future

iCal as a GTD Tool

iCal

(image borrowed from appeinside.com)

Why?

See what GTD is all about. But basically its a set of guidelines that helps you be more organized and consequently more productive(they call it Getting Things Done), this is specially useful to check out if your work feels chaotic at the moment.
I don’t follow it too strictly, but i do use it a lot.

Tools

Before iCal I was using Things app, but it went recently into paid version. iCal in Leopard improved a lot so i decided to give it a try.
I did try and use for while other GTD apps: iGTD, Chandler, and couple more… but was not fully happy with them.
After a few months of using iCal, its working well for me.

iCal makes it simpler

Because I was already using iCal for the Calendar, doing the TODO list management inside iCal removes the need of having another app getting in the way.
Don’t have to worry more about sync problems between another app.
Create TODO’s directly from email, natively.
Is altogether more integrated into OS.

myCalendars in iCal

Work:

work-now
work-later

Personal:

house
others

Mail:

inbox-tasks

How i use them:

work-now and work-later are the ones that get more frequent daily use.
work-later is for TODO’s that I won’t do now. Either i’m waiting for something to be able to finish it or only needs to be done in some days.
Tasks that are not to be done right now go into the work-later, this way the work-now list gets less cluttered and I’m mostly looking at the work-now list.
When i have the work-now list empty I go into the list of work-later and move the ones with higher priority into the work-now list. Also check the ones in inbox-tasks.
I normally create a TODO item either directly into work-now calendar or from the Mail.app and these go into the inbox-tasks, that i review later on.
When creating TODO items, I use a name format like “proj1: todo1”, “proj1: todo2”, this enables to sort them by name, because sometimes is better to do in batch several tasks from same project.
When creating an item I also set its priority. Priority is another way to sort TODO items. This enables me to see what TODO’s need to done first.
Some TODO’s can’t be done now(thus should go into work-later), but because i want to remember them before deadline, I drag and drop them into the calendar, create a remember alarm and then move them into the work-later.
Use the TODO item “note:” to keep the relevant details about the TODO item. Basically the description of the TODO item exist here: description of the task, plan how to tackle the task, code repository revisions, URL’s to bug, etc etc..
I keep iCap app open all day and use it to guide my work: I look what’s on work-now list with higher priority, pick the first, check if other any tasks can be batched with it, do it and finally tick it off when done.
ICal Settings: un-tick the “Show all Completed Items”, so that when a task is done it disappears from the list.

2nd European Lisp Symposium 2009

Milan, Italy
May 27-29, 2009

And i'm very happy to say that in the organizing committee is professor António Leitão. My Scheme teacher from "introduction to programming" class in college.

Reality Check

Portugal

from: os externos

@Lisbon, 21 July 2006.

When i made this picture little did i know that almost 3 years later I would be blogging it from Estonia.

Not Lisp Again...

a first exposure to Lisp

"I was floored. Here we take the simple, straightforward definition of the derivative of a function. Type it in with essentially no modification and suddenly the computer knew how to do calculus! This was serious magic. I had never seen anything like it before."

Had similar feeling while watching recently SICP video lectures.

the Rosetta Code

Rosetta Code

The idea is to present solutions to the same task in as many different languages as possible. Rosetta Code currently has 264 tasks, and covers 128 languages. A variety of tasks are listed, and visitors to this site are invited to solve the tasks in the language of their choice. The tasks cover everything from the mundane Empty Program to the esoteric Towers of Hanoi.

for example the Monty Hall simulation
explained in wikipedia

How Google and Facebook

... are using R

book: Outliers

This book is an Outlier itself by being a great book.

Outliers explained, by Malcolm Gladwell

Evolution

... of a programmer

On Programming

A series of articles by Sterling “Chip” Camden:

Notes:

The first programming jobs consisted of flipping switches and moving cables around...
Java: "...led to the creation of gratuitous classes merely to enclose functions, and thus to an over-abundance of nouns in the code conversation...often becomes an exercise in verbosity."
History of the scripting languages: from the JCL to Unix Shell to Pearl to Ruby.
Lisp appeared around the same time as Fortran and COBOL the 3rd generation languages.
"...Lisp is the first language to be based on the lambda calculus... It boggles the mind to realize how much more advanced Lisp was (and still is) than its contemporaries...
Author's favorite language is Ruby.

On Artificial Intelligence

Jeff Hawkins talk at the RSA Conference 2008:

Hierarchical Temporal Memory (HTM) in wikipedia

visual svn diff on mac using FileMerge

here

Lem-E-Tweakit and Logic programming

from SICP lectures 8a & 8b.

While watching these, one thing that striked me is that this logic programming seems very similar to kind of things we use SQL for, just better... more flexible.

So why we do use SQL after all? after a bit of googling found this good article: http://search.cpan.org/dist/AI-Prolog/lib/AI/Prolog/Article.pod one of the references says:

So if prolog(read AI) and SQL(DB) are so similar, Why is one so successful commercially and the other deemed A complete failure in terms of scalability?

Databases implement powerful techniques to improve performance:

Indexing
Hashing
reordering goals to reduce backtracking

Where as prolog based systems have very few such Techniques.

So does have SQL have everything needed?

One of the most powerful features of Prolog is recursion.
SQL does not have recursion built into it. This is a severe Handicap. However there is a way to overcome this problem by invoking multiple SQL queries from a host language like C
Or JAVA. SQL3 has begun supporting recursion.

SICP - Structure and Interpretation of Computer Programs

I'm learning loads watching these, and quoting fellow colleague & Lisp wizard Edgar Gonçalves:

Great way to learn about programming, the *right* way by Abelson and Sussman

Why are computer keyboards the way they are?

The most common keyboard layout in use today is called QWERTY, it takes its name from the first six characters seen in the far left of the keyboard's top row of letters.

Why QWERTY? why this layout?
from Wikipedia:

"The QWERTY layout was introduced in the 1860s, being used on the first commercially-successful typewriter, the machine invented by Christopher Sholes. The QWERTY layout was designed so that successive keystrokes would alternate between sides of the keyboard so as to avoid jams. Improvements in typewriter design made key jams less of a problem."

So the mechanical issues of the original typewriters was the main reason for this layout design.

A second popular layout is called Dvorak, designed in the 1930's and it tries to address another problem:

"... the introduction of the electric typewriter in the 1930s made typist fatigue more of a problem, leading to increased interest in the Dvorak layout..."

But even the Dvorak layout is already some years old, so is the typist fatigue the same as now? For example do we write, the same things as in 1930's?

Is there a better layout?

Take a look at the texts you write every day, and imagine if changing the keyboard would make it more practical for you.
For example, for this text i had to write the word WHY a lot, so if i had whyrtu instead of qwerty on my keyboard, would be an improvement...right? Well almost, not that easy, doing this change might make it harder for other frequent words i use.

So we require a more strict and complete process, something like:

counting all the most used words.
decide criteria for best layout, figure out the major annoyances:
many vertical movements, like jumping up and down on keys rows.
many horizontal movements, can cause hitting 2 successive keys with same finger is very inefficient.
frequent use of pinky.
etc... (you can make up your own)
Finally try and evaluate several keyboard layouts and variations.

Imagining to do this by hand, like Dvorak and Sholes(for QWERTY) did, is a pain. Luckily we have computers :) and as a matter of fact someone already played around with this problem (which made think about this text in the first place).

See here for experience: http://klausler.com/evolved.html, and also the final keyboard layout: http://klausler.com/evolved.pdf

Your Own keyboard
But the right layout depends so much on the words you write more frequently, and is easy to see it can change very easily:

different people use words in different ways.
different languages make for completely different words.
15 years old person words are different from a 70 years old person words.
use of computer during working hours compared out-of-work working hours...
etc and so on...

So even for you, an optimal keyboard layout would probably change over time.

But imagine a future intelligent computer that can be all the time analyzing what you write in the keyboard and can auto-adapt itself to give your own very optimal layout. Not too frequently, of course, you don' want your keyboard changing every day. But maybe every 5 years is not impossible to imagine....

Keyboards of the future should come with little blank displays that can be personalized with the key you want.

Every time you login(just by touching, with fingerprint id) into a different computer, your keyboard configuration is pulled from internet.

Maybe extra keys where each key is actually a full word, of your the top 10 most used words, might increase writing speed...

Or have touch screens, and have a keyboard appear on a touch screen. A keyboard on a touch screen is whole new level of layout possibilities, we could make more frequent keys bigger than others... play with different keyboard shapes that can adapt to each person hands sizes and shape for example... etc...

Some serious hacking, see the question "Can you tell me more about your strategies for persistence?".
Who said Scheme didn't got any real use: "...I got to write half of it in Scheme, which probably means that I deployed more Scheme runtime than anybody else on the planet...."
Huge: "..It amounted to a distributed code war on a 4-10 million-node network..."
How safe are we: "...It would have been fairly trivial for me to go spelunking for people’s credit card information or whatever..."

Programming Languages

couple references that got me interested in exploring something new.

I'll be looking at Lisp(because of paip and aima), Scheme and Ioke.