Saturday, January 31, 2009

The danger of algorithms

Sometimes it's very easy to make algorithms that have very odd side-effect. Especially when doing data analysis and classification. I remember when I used to work on classification the amount of crazy things that my algorithm did, like calling a "3 Person Extra-Comfortable Couch - Mango" a Mango (the fruit). So this picture posted by newobj was great:

First time I tried it was already fixed, so probably didn't last very long. But it's easy to see why Google could classify itself as a harmful site: it dynamically links to all sorts of different places, including harmful sites. It's reasonable to believe that if a site has a very low proportion of harmful things, but still has some it might be classified as harmful. So, if they don't remember to filter out content-less search engines they will always end up saying that Google or any other search engine, is dangerous to you.

Maybe the solution is to just stop using them and only work on controlled sources like Wikipedia or Mahalo or Twine or any of the other million of sites and products out there that try to provide you with the ability to discover things in a more human-controlled fashion.

I do believe that that's part of the future of the web (and you can claim that since the early stages of Yahoo it has been the past of the web too): a sea of information with personalized filters. The personalized filter is aware of your social network (i.e., it trusts information approved by your friends more than the information approved by complete strangers) and your current state (I'm on my cellphone in a city I've never been before searching for "gas station" should be very straight-forward what to return). Little by little hardware and software are converging there.

And there I went in a complete tangent from my original post. Probably I just don't want to do what I need to do: work. But today is my only real chance. Tomorrow I have a superbowl party which will consume most of the day.

Tuesday, January 27, 2009

Magic Berry tasting and everything else is just that - everything else

So today I was able to participate on an interesting event: Magic Berry Tasting (also known as Miracle Fruit). It's a little berry that basically turn all acidic things sweet. So you can eat a lemon and it tastes like an orange. Very interesting experience. Not life-changing, but interesting nevertheless. In my opinion, the weirdest things that I've tried were:
  • Lemon: as I've mentioned, it tastes like orange
  • Guiness (yes the beer): it loses all its sour components and has some hints of chocolate and coffee.

As I said, in general it wasn't life-changing. Most things just tasted more ripe, sweeter and without the normal after-tastes. Quite worth the experience, though.

And now I have to get back to life. I've been having problems finishing a project at work. I have most things done, I just don't seem to like the way I did them, so I'm slowly rewriting everything instead of just walking away. And this has been driving me crazy. And this week is quite busy with a lot of evening events, which always makes it harder.

Oh, and I received a curious link today that I'm still not sure what to do about it. It's about a set of challenges launched by one of the most important molecular gastronomy blogs, Khymos: TGRWT (They Go Really Well Together). Somebody comes up with two odd ingredients and people have to suggest recipes that bring those ingredients together. This month's challenge is Malt & Soy Sauce, which is a very curious combination.

The month is almost over, so I won't try to do anything about it. I'll wait until next month and consider it. Probably I should start challenging my culinary abilities more often. Prove to myself once more tha I don't really know how to cook. I just know how to fake well. :-)

Saturday, January 24, 2009

My life in pictures

Sorry, the title is not really right. I just wanted to post a couple of pictures to represent last's post. First with small things that remind me that I now own a house and have to care for it... My first damage:

Snow fell on it and it couldn't handle the weight. Not that I really liked it, but it was a good example of what is to own a house.

Now for my cheese. Before waxing:

After waxing:

More on this in 3 months when I'll give it a try. Hopefully I've waxed it enough so that it won't become a blue cheese.

No new books this week

This is the first week I haven't finished a book this year. Quite amazing! Part of the reason is that I'm reading Richard Powers's "The Time of our Singing", which is a 600+ page book. That will take me some time. I've also been reading the news in the morning and listening to more podcasts. The most interesting one that I've read was:

Daniel Tunkelang talks about Endeca, Search, and ‘Reconsidering Relevance’ from Talis's Nodalities.

It's an interesting interview that discusses things that I can relate more than the usual discussions about SaaS and linked data. As this is not my technical blog, I won't get much into why I found this interesting. Eventually I'll write some notes about it.

Things at work have been a little stressful lately. And it's not really because of work itself, but mostly because I have been a little unfocused. Maybe it's the new house, maybe it's because I'll have visitors in town this weekend, maybe it's because the cleaning people killed my lucky bamboo this week, or maybe it's just the moon, I don't know. I just hope it will go away soon.

What else can I talk about my week here? Well, last weekend I decided to finally do an actual hard cheese. I chose one of the easier ones to start with: cheddar. Last Saturday to Sunday I did the whole preparing the curds, cutting them, heating, and pressing. This week it was drying and today I applied cheese wax to seal it from mold and let it age for at least a couple of months. The trick now is that, without any experience, when will I decide to open it?

Also in two months I should open one of my bottles of plum wine to try. I tried a little when it came out of the clearing up process and it was alright... Won't win any prizes. I've been thinking of starting a new batch of something, but I'm not sure what yet. It's not that hard, it's just such delayed gratification and moderate risk that is tough to be very excited about it. And I think I do need some excitement in my life right now. Just too many things to be worried about that I feel I don't have full control over. Like my poor lucky bamboo.

Monday, January 19, 2009

Fun bus experience to start the day

I was running a little late today (too many things to do this morning and I was only able to do 1/3 of them) and got to the bus stop about a minute after my bus (74) was supposed to show up. Another person that takes the same bus as I do was at the stop too, so I thought I was safe. A minute after that the bus that comes after my bus (75, and does not take me to work) arrives. We ask the bus driver if our bus is late and he doesn't know and says that some buses are not running today (it's Martin Luther King day). So we take the 75 that should take us to a hub where many buses that go to downtown stop. I was convinced that my bus was a little early and that's why we both missed it.

So we get to the other stop and stand there and wait for the next bus to get to downtown. Two show up that take the slow route and I decide not to take them (the other confused person that took the 75 with me go onto one of those buses). Then, after about 10 minutes waiting the 74 shows up! It was late and not early and it takes a slower route to this hub. So I end up getting to work about 5 minutes later than usual.

With all this experience what I learned is something dangerous: I can be a little late, get the 75 and still be able to catch the 74 later in the bus line. The only trick about it is that usually the 74 is full at the time it gets to the hub (not today because of the holiday), so it's not something I would want to do every day, but good (or bad) to know that I have this option.

Oh, and you might be asking: if it's a holiday, what are you doing going to work? Well, in order to "help" businesses, the US government does not impose any holidays onto companies. So companies pick and choose what holidays to give to their employees. Amazon didn't choose MLK as one of these holidays. And it's not that unusual for this holiday to be one of the ones that are ignored by companies.

Thursday, January 15, 2009

Two more books for the year

I'm quickly running out of books to read, I guess. Last week I decided to read something completely random, which ended up being Jim Butcher's Storm Front. Why this book? Well, I was reading on random things, which ended up taking me to an old series that died some time ago, The Dresden Files. Which reminded me that it actually came from a book series. I kind of enjoyed the TV series (although I wasn't very sad when it was gone), so I decided to read the first book of the series. It was mildly entertaining, but probably not enough for me to read more on the series.

That was last week. This week I decided to start reading Malcolm Gladwell's Outliers: The Story of Success. I just finished it on my ride back home today. The book makes very interesting points on why people become successful. The core concept is that everything counts. You can't become successful just based on intelligence, or school, or social background, or cultural background, or luck. You become successful because of the combination of some or all those factors.

I won't get more into the book, as I won't be able to do justice to all the things that he talks about. What is a little depressing about the book is that it sort of shows that unless you focused on something from early on in your life, you'll very likely never be successful. That means that "moving downstream" is not likely going to be a success story. Oh, well...

Saturday, January 10, 2009

An Excel guy playing with Apple's Numbers

I've done a lot of Excel in my life, and still do. I can't say I'm very proud of it, but I have to admit that it's one of the most powerful tools that "normal people" seem to use.

Well, so Apple decided that one of their big Macworld Expo keynote announcement was their new version of iWork, their Office competitor. So I decided to give it a try and see what I thought of it.

As expected, installation is easy and startup is straight-forward. It starts with template choices, which was a surprise for me. I know that pretty much all the office suites have templates, but I think I have repeatedly tried to use them and never found any template that was really helpful to me, so I gave up on them a long time ago. Well, Numbers still has them. And they look pretty with odd things like Home Inventory, Baby Record and even Garden Journal.

But I kept to my normal experience and chose a blank file.

The next thing I notice is that by default it is highlighting the top row and the leftmost column, as if to notice that they should be used in a special way. Excel offers that also, but it's not the default behavior.

Another thing that struck me at first was the ability to add tables to a table. Not only a simple table, but a pre-formatted table, like a checklist or one that already contains the sum calculation in the bottom. And it can go into the same sheet, embedded under the current table.

I found that concept very interesting. Many times I did want to have one sheet, but multiple different pieces of information with different formatting. Many times I wished Excel had this feature and you didn't have to keep adding new sheets or putting everything in long set of columns.

I didn't spend too much time using it, so the only other thing I have to say is that it does have a lot less options than Excel. This could be a very good thing though. People never know even 20% of what Excel can do for them. And when they do, many end up spending a lot of time trying to understand how to do it. For example, what the parameters of v-lookup mean, how to get pivot tables correct, and so on. I can't tell yet if they've removed too much stuff from it to be useless.

I'll continue giving it a try. I have another 29 days left.

And I might eventually look at the other things you get on the suite, like Keynote and Pages, and post something about them. Now it's time to follow X's suggestion and go to sleep. It's barely 12:30, so that's certainly better to check if his way of coping with it works for me too.

Friday, January 09, 2009

8 years in the US

This would have been a much better subject for my 400th post, huh? Well, unfortunately I'm not sure I have much insightful to talk about this one. Most of it has been the theme of this blog: "moving downstream". Just going with the flow, towards the path of least resistance. Yes, you might find a waterfall here and there. But the path seems never to really branch out. Only other paths join in and become one big way... downstream.

It's probably a little too late also to have deep thoughts about this. Lately I've been busy and stressed. Lots of things just don't seem right. Projects are just not complete, ideas flow around and don't get much attention, and new things keep piling up on top of the incomplete things.

For example, I've set up my office here. I have both my computers on, chair, but it's missing my printer, there are still some stacks of papers that I haven't figured out where I'm going to store them, things are inside my drawers, but still wrapped up. It's mostly functional, but just not finished. I have 5 email drafts that I can't seem to finish and send them to friend. Projects at work are almost at the same state. And my mind keeps trying to convince me to do something new. Probably is some escapist need to keep me away from my weaknesses.

At least I'm blogging and being able to finish my posts. So, changing subjects back to the original intent of the post, thanks US for the last 8 years. They were comfortable.

Thursday, January 08, 2009

Spam day today, huh?

This month I decided to aggregate some statistics about spam detection on all my mailing systems (i.e. how many false positives/negatives I get on Yahoo, Gmail and Hotmail). So I'm building this spreadsheet with how many spam messages I get a day on each of those services. With this amount of visibility, it's interesting to see some odd patterns. Like, for example, today I received so far roughly double the average spam I receive a day (which is around 17 messages - today I'm at 33 and the day is not over yet).

What have I found so far besides that? Well, I can't really tell. I receive way too much spam on my gmail account and very little on my other accounts. So I don't have any statistically significant numbers on the other accounts. Current numbers are:

True positives: 159
False positives: 1
False negatives: 0

True positives: 3
False positives: 0
False negatives: 1

True positives: 25
False positives: 0
False negatives: 1

If I was not statistically inclined, I would say that, as I was expecting, gmail seems to err on the side of classifying things are spam when they are not spam, which was the biggest problems on early spam detection systems (which had a much higher false positive number than the 0.6% that I'm seeing here). Hotmail and Yahoo learned with this bad experience and decided to miss more spam messages in order for people to be able to use the "empty" button to get rid of all their spam messages without having to go through each of them.

But, as I am "statistically inclined", I'm not going to conclude anything yet and just wait until I have more evidence.

Oh, yes, and this is my 400th post. Interesting number. This blog has been alive for 4 years and 3 months, approximately. My previous blog that lived for 3 years and 8 months had 1,049 posts. Just a small difference, huh?

Half an hour is a long time!

Warning: this is probably going to be a very boring post about routine change.

Since the move, almost two weeks ago, I had to change my morning schedule. My bus ride is longer, so in order to get to work at about the same time as before (between 8 and 8:30 AM) now I have to leave half hour earlier. I decided not to wake up 30 minutes earlier (which is good as lately I haven't been able to convince me to go to bed before 1:30 AM), so I end up having 30 minutes less at home in the morning (which are now spent in the bus). It's odd how hard it's been for me to adapt to it!

In the past, my morning routine used to be:
  • Wake up
  • Check emails
  • Check favorite blogs
  • Read news
  • Shower
  • Have breakfast
  • Reply to one or two emails (only the time-sensitive ones, usually from my family)
  • Read more news
  • Leave for the bus listening to a podcast
  • Listen to podcast in the bus (except for towards the end of some weeks that I ran out of podcasts to listen to and I would either listen to music or an audiobook)
Now my routine (if a week of doing something can be called a routine) has been:
  • Wake up
  • Check emails
  • Shower
  • Have breakfast
  • Check favorite blogs
  • Leave for the bus listening to a podcast
  • Sit on the bus reading a book
It's a lot of things that were cut out of my morning. When I get back home, I do try to check news, but that makes my evening way longer and less productive (that's part of the reason why I haven't been able to force myself to go to bed before 1:30 AM - I have to feel like I did something productive at home). I tried to read the news on the bus using my Kindle yesterday and today, but it's just not the same thing. "Normal" news have been quite boring for some time. The economy, and Obama setting up his people, and now the Israel-Gaza conflict (which is depressing, not boring at all). And I miss the ability to navigate through news to get more information on things that seemed interesting to me.

Well, I probably need to revisit my plans. Maybe I'll reconsider setting my alarm to wake me up earlier. Or maybe buy a tablet PC with a 3G wireless card so that I can do my news reading with browsing on the bus (no, not an iPhone, or equivalent - the screen is too small for my taste). Or maybe I just need to get used to it and reorganize my evening around it. I've been arriving home earlier (because there are no good buses to get me home that leave work after 6:20 pm)...

Anyway, I said this was going to be a boring post. I'm not sure why I wrote it... And now that I did it, I can't convince myself not to publish it. Oh, well, maybe this will force me to post something better soon to compensate for my personal ramblings. Now onto work!

Friday, January 02, 2009

The first book of 2009

It's kind of cheating to say that I've finished my first book in 2009 today, considering that I've read most of it in 2008 (actually I read 80% of it this week - the joy of longer bus rides), but, well, I'll add to the list anyway.

So, to the book: Firstborn

It's actually the third and last book of the "A Time Odyssey" series, and I thought it was the most interesting of them. Events happen in a much more exciting pace. I bought the series when I heard that Arthur C. Clarke passed away. I was in the bus going to work and bought all the books on my Kindle. Later I found out that I had already bought the hardcover version of the books... Oh, well..

It was a good series. Not Arthur C. Clarke's or Stephen Baxter's best, but quite good anyway. I'm not sure how much I can say without spoiling it for people, but the whole idea of it is this fight with a race of highly advanced beings that seem to not really care about other life forms in the universe. There is no real interaction with this race, just a struggle for the human race to keep alive.

What is next on my list? Well, I'm still reading João Ubaldo Ribeiro's "Viva o povo brasileiro", a 600+-page tome of Brazilian literature. Tough and rewarding read. It's actually interesting to read it and realize how much different it is to read works of literature in Portuguese to English. Not that I'm calling "Firstborn" a work of great literary depth - it wasn't meant to be that way. I'm not drawing the comparison to it, but to the more general English-language literature, like Richard Powers's The Time of Our Singing.

In any way, the Brazilian/Portuguese literature that I miss is much more psychological and, in many ways, metaphorical. Time goes by slowly as the author takes time on the characters' reactions to what is going on more than the event itself. Sometimes you can't even really tell what is going on as it's so small compared to the world inside the mind of the protagonist that draws relationships to other events on their lives or world events and, more disturbingly, keeps replacing the current events to things in their past in a form of "internal metaphor" that you have to keep following in order to understand the book. As I said, tough but rewarding.

And I think that's the main things I have on my list right now. I did buy Malcom Gladwell's Outliers: The Story of Success, but I'm not sure I want to read it right now. We'll see what surprises I'll find once I finish unpacking and organizing my books back onto my shelves.

Thursday, January 01, 2009

Happy new year

It's 2009. Kind of shocking how quickly all those years in this new decade/century/millennium are going. But, well, many things have happened on those years, so it's not that I'm just realizing that I haven't accomplished anything. But I'm not sure I can say the same thing about last year.

Anyway, let's start with the obvious: Happy New Year to all my readers. There aren't many of you out there, I know, but it's not what is important.

So, what are my plans for this blog for this new year? I'm actually not so sure yet. I still have plans of keeping a personal (this) and a technical blog separate. But maybe I'll finally work on my homepage and migrate both blogs there. It's been on my plans for the last year, but didn't get even close to doing anything about it. I think most of it is because I want too many things on that homepage. Things like:
  • Research interest/links to my papers: not that hard as I have stopped writing papers some time ago
  • Technical interest/links to my project designs/alpha implementations: this is a little trickier, because I'm not sure how many of my projects I actually really have a consumable design or implementation. But that's where my other blog would be linked
  • Aggregation of my pictures: that right now are mostly on SmugMug
  • This blog
  • A twitter-like feed of interesting links: It's what I'm trying to accomplish with the FriendFeed side bar, but the current way I find hard to really navigate
It's a good amount of things and integration with things. Especially when getting to my projects up somewhere and running.

Anyway, I also know that a good part of this year is going to be devoted to the new house. The list of things that might need some work is getting long. I'm planning on not starting on anything until I have enough experience with the house and weed out things I want changed not because they are unusable, but because I'm used to something different from my previous houses/apartments. But I'm sure that there will still be a lot of things to do. And right now my goal should be finishing setting up my office, which right now is a sea of boxes with a small island of my chair and a small part of my desk, where I'm sitting and writing this post. It's actually quite impressive how many books and magazines I've collected in the last 8 years. I came to the US with pretty much no books. But now I have a Kindle and the speed of my book collection growth might slow down a little. Hopefully.

Alright, time to start the work today. Happy new year again and wishes to a wonderful year, full of accomplishments, to everybody.