It’s a bit of a boring title, but it actually has been an interesting week.  Although I found myself highly distracted by some unrelated but fascinating things, I still managed to get quite a bit done.

The script that I used to download the SRTM data set and import it into a Postgres database can now deal with all continents and supports uploading a subset of a continent by means of a bounding box. I also put the md5 check sum of every tile in the source code.

Since the App Engine still has some issues, I have revived the Postgres version of my application. It is located at http://altitude-pg.sprovoost.nl and contains most of Europe as far east as Moscow and as far south as Cyprus. It runs on my home computer in The Netherlands, so please be nice to it. I use apache-mod-python for the formal demonstration website and apache-mod-wsgi combined with web.py for the altitude profile server. To make this as painless as possible, I have moved all App Engine and Postgress specific code to their own files and kept as much common functionality as possible in the main file. I can now run the development servers for both Apache and the App Engine from the same source code folder, at the same time.

I have requested more storage space on the App Engine and I am also considering a more efficient storage method. In stead of storing one altitude per record, I could store 100 altitudes per record and zip them. That would drastically reduce the total storage requirement, but at the cost of performance because I often need only about 2 out of these 100 altitudes.

I have also been a bit more active on their mailing list; it feels good to be able to answer peoples questions and at the same time it allows me to verify my own code and design. There are also some interesting albeit more philosophical discussions on the list.

I have signed and fulfilled a pledge to “spend an hour OpenStreetMapping features on Caribbean islands from Yahoo! aerial imagery and […] donate £10 to the OpenStreetMap Foundation but only if 60 other people will do the same.”. I felt like I could really use another jet-lag. The pledge is full, but who knows, if they can rally another 60 people there might be a second ticket?

Those of you who laboriously follow every commit to the OpenStreetMap subversion repository, may have noticed that I am still struggling with git-svn. I got really tired of fixing conflicts, so I unleashed the power of git-svn set-tree:

git-svn set-tree -i trunk 3cb585dca1d7fe10791312ca26125168506b61c1
git-svn set-tree -i trunk 07c9024f5ea4ce60f481b8089b61d4988e7588fa

Even the manual recommends against doing this, and you should make sure nobody else (like your mentor) has committed anything to subversion before you do this.

I find git-svn to be harder to use than it should be. I think it is trying to hard to properly translate between The Git Way and The Subversion Way. I just want the subversion repository to ‘sort of’ track my git repository. I don’t care if it has to represent the history a bit different. Just keep the code up to date. I am looking forward to this command:

git-svn just-do-it

I really think Git would benefit the OpenStreetMap community, because it reflects the decentralized nature of OpenStreetMap. With Git, there is no such thing as a central repository. People can write any code they like without having to live in constant fear of breaking the trunk with their next commit. In stead, when they build something cool or useful, they will tell their friends to pull it in and experiment with it. The person who operates a production website will only pull pieces of code that he or she considers safe and useful enough.

But the reality is that many organizations rely on subversion at the moment and have excellent reasons for not risking their operations by making an instant jump to Git. So people are not going to adopt Git very quickly as long as it is so hard to sync with subversion. But lets wait for a while and see…

I am getting better and better at keeping my git repository synchronized with the osm subversion, but I would not recommend this strategy to others.

I created a project on Google Code Hosting project for the altitude profile. Not to host the code, not even for the wiki, but just to keep a list of issues. I realize I could have applied for a place on the OpenStreetMap Trac, but I want to use Google Code Hosting for my new project: Jobtorrent. This is also the reason most of the issues point to the Git source (I do point to  subversion on the main page and the only reason I do not always point to both is that I am lazy). I will write more about Jobtorrent later; first I need to work on my Summer of Code project you know…

This list of issues should be good for continuity. Because my project does not interact with any OpenStreetMap code at the moment, I am probably the only one in the community who knows how the code works and what needs to be improved. That is a very low bus factor! (“tram factor” would be a better term in Melbourne) Now I really like the OpenStreetMap effort and I will certainly find ways to stay involved in the future, but it might be in a completely different project. Depending on circumstances, I should at least prepare for the possibility that the altitude profile project will be orphaned within a few months.

I use a personal organizing method inspired by the book Getting Things Done (David Allen) and that makes it very easy to transfer everything I am working on or thinking about to the Internet. So that is what I did.

The more difficult part is keeping it synchronized. David recommends that you never share your projects. That is, you should always keep your own lists and let nobody else touch them. Your lists must reflect what you want, or you will start to rebel against them and as a result mess up your system.

So in practice you will end up with a central list (e.g. the list of issues on Google Code) and your local copy of it. They will not be the same. There are a couple of things on my personal list that are not online (nothing ground braking, don’t worry) and my own priorities are not identical to the ones online. The online version reflects what is important for The Project, the offline version reflect what is important for me. At least in theory; as long as I am the only one working on it, it probably reflects my opinion a lot better than it ideally should.

Now I am pretty sure the average recruiter looking for a “true team player” does not like what I just said in the last paragraph.

I have recently started following Umar Haque’s blog.

Umair Haque is Director of the Havas Media Lab, a new kind of strategic advisor that helps investors, entrepreneurs, and firms experiment with, craft, and drive radical management, business model, and strategic innovation.

He’s written a manifesto for the next industrial revolution and vouched to provide free consulting to five web 2.0 start ups that actually do something useful for a change (i.e. change the world).

He started an open discussion around the question whether Google is evil or not. I find this a fascinating question. Many people find great comfort in just answering the question with “yes” and pointing to some good examples. However I’ve met a couple of Googlers here and there, read some blogs and read the book. That convinced me that there are at least some people within the company who firmly believe in the Do No Evil motto and work very hard towards that goal.

Also, saying that Google as a whole is either evil or good implies a conspiracy. It implies that a select group of people with evil and greedy plans (I guess most people would point to the shareholders and managers because they wear suits) is in complete control of the company. Not only that, they deliberately employ naive people and let these naive people blog and talk about doing no evil, so that that they can continue their evil plans unnoticed.

This conspiracy theory implies that Google is a well oiled machine, a super organized corporation. My outside observations convince me that that is far from the case. I would not go as far as to say they are an organizational mess, but they are definitely not organized enough to take over the world in a massive global conspiracy.

That argument also means they can’t be all Good. They are definitely doing things that I do not feel comfortable with, even things that I would probably still disagree with if I knew all the inside info. Just use their website to find examples.

So for those people who love simple answers; just pick the answer like and stick to it, if it makes you feel better. There are always more important things in life than answering this question.

This is my reply to Umar’s question:

Can you think of instances where Google has violated this [do no evil] principle?

Probably. In some cases it is hard to decide because they won’t or can’t release all the internal discussion that went into certain decisions. We just have to take their word for it…

Is Google becoming more evil as it attains more market power?

… which becomes a very unattractive option given their current size. No matter how good their intentions, people (and especially institutions that are supposed to protect us from companies doing evil stuff) are just not going to take their word for it. You already mentioned this in a previous post. Update: no that was an article in the International Herald Tribune titled “Google The New Master of Network Effects”, second last paragraph.

Is the relationship between market power and evil set in stone – will Google inevitably become evil, because that’s what happens to companies (and people) as they grow up?

Nothing is set in stone. Google is just a decade old and probably has a hard time dealing with its own size and growth. They can grow really bad or really good. I also doubt that even Larry and Sergey have enough control over their creation to steer it in any direction, although I hope I’m wrong.

In my opinion the “best” (as opposed to “most evil”) thing Google could do is create complete openness. Ideally, they should open source *everything*, including their secret sauces and also have no more secret projects. I think the best way to keep governments happy, make people feel more comfortable and help the world move forward is to make it as easy as possible for a strong competitor to emerge. Ideally a new market player would just have to buy half a million computers, a couple of engineers and marketing team and be able to run a copy of Google, adsense included.

Now this sounds a lot like economic suicide to me, but if they keep growing like this, then if I was Neelie Kroes I would enforce this at some point in the future anyway. So Google could, like Microsoft, wait for that moment or do it themselves right now.

I am sure Larry, Sergey and a whole lot of people working at Google would love this idea, but the shareholders probably won’t like it and it might not even be legal. Unless someone can find a way to do that and make a profit out of it.

Or we can enter the realm of my even more far fetched ideas: just start abusing your power and let the government help you bypass the shareholders. I guess most people are happy that I’m not a CEO 🙂