Using OTE’s 11888 to provide fast CLID data for FreePBX/Asterisk

For those using FreePBX (a configuration/administrative interface that manages Asterisk) there is a CallerID Lookup module, published by the FreePBX team. The module allows you to interface with data sources using several methods, one of them being http.

Like many companies, we maintain several systems that can be easily integrated with our VoIP PBX system, but we also receive a number of calls from the public, viz. companies interested in advertising on GEO|ADS, adding their business to AthensBook or ThessBook, or asking for a quote for a project. OTE recently revamped its aging whitepages.gr site and incorporated its content into their ‘umbrella’ 11888 business directory operation. The new site is faster, more beautiful and written in a more professional manner, as far as the markup is concerned, which makes it a perfect source of caller id information for a large number of telephone subscribers in Greece. Around one hour of Ruby scripting using Mechanize, Sinatra and some custom transliteration tables to convert the scraped names to their latinised counterparts (Cisco phones don’t seem to like Greek characters all that much) resulted in the following script (github)




Mango / Tears of Steel

Tears of Steel is the fourth film (and the first live-action short) by the blender foundation and a giant leap forward for the community and the software. It is a sci-fi film that showcases recent work on blender, including Compositing, the Cycles render engine and many of the features introduced in the blender 2.6x series, like BMesh.

Like most blender foundation films, this one suffers from the the usual suspects: bad audio, bad accents and now, for the first time, they are complemented by pretty bad acting (the guy with the monocle is borderline unbearable) and a somewhat frustrating script; surprisingly I found the music to be more than decent; yet the true value of this film is the amazing work on blender and the impressive gfx and cgi the artists created for it. If anything, leaving æsthetics aside, Tears of Steel successfully showcases how far blender has come and will hopefully spur more people to contribute and use this amazing piece of open source software.



Visualising geospatial data using SVG, Python and Javascript/d3

Geography is an immensely important tool in the modern business environment. Doing business globally was the first wave of the globalised economy, and the widespread use of internet-based services and e-commerce only accelerated and solidified the notion. The next wave came with mobility in the form of location-based services. Today everyone can create location-based tools that span the globe, and many companies do, but in my view this is akin to a car manufacturer producing a single model for Chinese farmers and European urban commuters alike. Hyperlocality, the concept that each location is worthy of particular attention, that it has unique characteristics and inherent value impossible to tap by going ‘global’ is slowly becoming a major trend. In 2009, while bootstrapping AthensBook, ThessBook and GEO|ADS it was our only choice.

For us geography is everything. We didn’t start global or even regional. We started local, paying attention to the needs of the people in Athens and Thessaloniki and expanded our featureset accordingly; with limited resources and funding. The importance of understanding geography is amplified when your geographic realm is a city, as opposed to a country or region. When we launched GEO|ADS in early 2009, the business world didn’t really know what to do with it. Even today, mobile advertising is still in the process of coming up with standards, conceptual or technical, the world is trying to understand how to use it, how to extract value from it. With GEO|ADS we were the first platform to provide meaningful, consistent high-resolution spatial analytics to our customers in 2009 for Athens and Thessaloniki. We always thought this was a fundamental point where we could contribute valuable, differentiated feedback, compared to the Web or traditional media.

Our spatial analytics reports have long been generated largely automatically as kml files, a de facto XML-based standard that originated alongside Keyhole’s Earth Viewer. Keyhole was a company funded by In-Q-Tel, CIA’s venture capital appendage, and focused on a single (publicly offered) product, the Earth Viewer. The application was very much ahead of its time and it was only through the acquisition of the company by Google in 2004 that turned it into a product enjoyed by the masses: Google Earth. Choosing KML was an easy decision for us, as the format is open and the output usable by everyone — Google Earth is free and infinitely more accessible, usable and engaging than your average GIS application. It is also deeply interactive, as users can zoom, pan and rotate around regions of interest extremely quickly, allowing business development managers and marketers choose locations for campaigns, expansion, targeting more easily and quickly than it would have been possible otherwise. Being XML based meant that we could write a relatively short Python script and leverage all the amazing facilities KML and Google Earth provide.

In addition to customer reports, however, as part of our operations at Cosmical, we often employ spatial analytics for internal use, to understand how our users interact with our applications, identify potential improvements or interesting types of venues to be considered for future versions and, enhance our hyperlocal character by identifying local trends and featured venues. For this purpose we have been using web-based technologies and more specifically SVG in conjunction with CSS and Javascript to create interactive, scalable and æsthetically coherent reports.



» TextMate 2 is now available under GPL3!

Who would’ve thought a few years ago that this day would come! Given the success of TextMate 1.x and the unprecedented delay in releasing TextMate 2, I guess open sourcing it makes sense. But GPL3? Really?



On the Google IO Keynote

So Android announced Android 4.1 (Jelly Bean), the next version of the operating system with much awaited performance improvements and some new (marginal) features, available to Galaxy Nexus users in mid-July and the remaining 99% of the Android ecosystem sometime between a year and never. Along with the new version of Android, Google announced several other products and services, including their Nexus 7″ tablet, which I won’t cover in this post. What I am going to focus on is Nexus Q, the first product designed exclusively by Google, a ‘social’ media player that is, intriguingly, manufactured in the U.S and costs almost $300. This is not representative of a new class of devices, it’s not even functionally all that impressive either: it’s a device that connects to speakers and TVs, like countless before it. The difference in the case of the Nexus Q lies in its impressive industrial design and the fact that 1. it can stream data from Google Play and 2) it can allow local Android devices to ‘share’ the content they hold in their local memory. Google, borrowing from the Apple cue-card of shoving features into products without really caring about what people really want or need, created a product aimed solely at increasing the impact and the revenue of their online media store (Google Play) and added a small cherry on top that makes use of the networked nature, ubiquity and storage capacity of modern smartphones. Of course, even the ‘social’ aspect of the device is not exactly groundbreaking either. The functionality has been available, one way or another, albeit in more ‘technically challenging’ forms for years. But what’s important to note about the Q is that it doesn’t really cover what is probably the number one request (or, if your prefer, the number one pet peeve) people have about for other locked down devices of this kind (yes, Apple TV is a prime example): the ability to stream content stored in existing home networked devices, such as another computers or NAS devices on the local network, using standard or widely-used protocols such as DLNA/uPNP, DAAP/Airplay or even SMB/NFS. With the Nexus Q, Google demonstrates the same denial it’s got with that of the success, or rather lack thereof, of Google+. And in their attempt to convince people that the botched Nexus Q is worth paying the huge premium they’re asking (other devices of the same kind cost 2-3 times less), their enthusiasm for silly features like overlay mustaches in Hangouts, their desire to marry profits with openness and openness, they lose the mark. They fail to deliver the extremely seductive walled-garden, locked-down experience that allows Apple to thrive at the cost of openness and empowering products and services and, at the same time, they lose the geekcred that made Google so affable in the first place. Google largely knows this and they have been performing a balancing act for years: the ‘openness’ of Android, that of ‘the social graph’, their contribution to the public via open source software. The company is cognizant of its differences to other players of the market; yet, there with today’s announcement of the Nexus Q there is a paradox: today we have so much technology available to us, often of high quality and free, in the form of open source software, powerful and affordable hardware. Google, like Apple and everyone else in between, realise this and have been struggling to trap the cat inside their new shiny new boxes. The Nexus Q could have been a great little device, and maybe it will be one as soon as people start ‘tinkering with its internals’. Yet it would have been much better if Google had shipped it like that instead of ‘tolerating’ hackability by the community. By blatantly promoting its own cloud based offerings instead of trying to marry them with locally stored content, Google is crippling its product.

In the end, like Microsoft a few days ago, Google is copying Steve Jobs’s style and strategy; provide a platform and tools, but focus on content and lock-down as a monetisation technique to counter technology commoditisation. Gundotra’s presentation of Google+ Events reeks of vintage early 2000s Jobsian technique, with the preset graphics and the sildeshow features. With everyone and their dog showcasing complete lack of tech culture in the face of a dominant resurgent Apple, the world treads in dangerous territory: a technology monoculture in the corporate space that bases its profitability on polished jailhouses instead of innovation and increased freedoms. In this context, the cloud remains a key technology that can help liberate us from the burdens of backups and local storage, but also imprison us in a world where a few corporations control all of our information.

Which brings us to Glass. The Sergey Brin, Project Glass segment was super fun and very impressive. If anything, it shows that Google is well-humoured and daring and works on interesting technology. Of course the demo and the features they showed are not exactly representative of the product, its potential or its usefulness. The segment which followed the breathtaking landing on Moscone and Brin’s introduction was stale and unconvincing: I doubt a mom would want to ‘make contact’ with her baby looking like the Borg, or that everyone would think twice before relegating their ‘Glass’ to a drawer and never using it, its novelty quickly overshadowed by its unnatural and intruding appearance. It’d be a shame if Google really thought that people want to look like futuristic soldiers from a second tier sci-fi movie. Sure, the concept is nice and perhaps as technology progresses it will be easier to integrate such functionality in more human-friendly prodcuts, like contact lenses or integrated with optical glasses. As it stands, Project Glass is only an unappealing curiosity that serves better as a tech demo than a product right now.


» A Bill Of Rights

I found this article on EFF to be a very concise summary of many of the issues I’ve written (and often talked about) in the past, pertaining to the freedom to use the devices you have paid for and own as you see fit, and the increasingly worrying trend of manufacturer lockdowns that largely define what you can and cannot do with them. While Apple with its popular iOS may be the most well-known (and most successful) ambassador of the lock-down platform, the trend has been on the radar well before Apple managed to escape the threat of extinction in the late 1990s; Microsoft, with Windows RT and the Secure Boot flag in UEFI only manages to actually implement all those technologies they initially developed, studied and proposed more than ten years ago with Palladium/TCPA.

The cat is still out of the box, but technology ages quickly and the threat is quite real: a combination of a cloud abused by the Valley oligopoly, lack of the computing storage ubiquity and locked down devices would be a nightmare scenario that would strip the computer of its fundamental differentiating quality from appliances of yore: its malleability, the power derived from its programmability and its ability to solve countless problems, to achieve infinite different tasks and not perform a single function, as manufacturers would most likely want.



On Device Identifiers.

Mere hours after pressing ‘Publish’ on the previous mini-article concerning walled gardens, an article on TechCrunch, this morning, clarified the situation we have more or less been suspecting for a while now: that Apple, after deprecating UDIDs (one of the things they truly did well in iOS from the beginning), they will start rejecting apps after the backlash caused by lawsuits, noise and a few rogue developers that seemed keen to take advantage of their users and use their private information in ways they didn’t agree (and which are illegal in more ways than one).

The situation with unique device identifiers is an important one. On one hand, user privacy should be the number one concern of platform owners/builders like Apple, Google and Microsoft. It isn’t, for their software can do pretty much whatever it wants with the users’ private information, as we have seen several times these past few years. On the other, developers have many uses for an immutable, unique identifier for devices; from providing metrics for their own use, understanding the patterns of use of their applications, improving ad targeting, enforcing proper use of their applications and communities among others. Of course, it can also be a tool aiding in unsolicited tracking and profiling of users, of a range of personal information violations etc.

When Google came out with Android, they failed to provide any sort of unique device identifier of any significance to their developer community. They did provide several ways for developers to get some seemingly unique identifier, but those were easily modifiable, sometimes were not set at all or set to the same value across all devices sold by an OEM. In addition they would get reset after a factory wipe, etc. Developers resorted to DIY identifiers, scoured and composed from several unique component identifiers available to them by the system, such as the IMEI in phone devices, or the MAC address of the WiFi network interface in others. Then Google released Android 2.3 which included a unique identifier which, while better than the previous ones, was still not 100% robust.

Microsoft has belatedly joined the new-walled-garden era, first with Windows Phone 7 and now with Windows 8. The ‘new’ API and model for applications, Metro, goes one step further by not providing any single unique device identification capability to developers (there are some exceptions, but they are truly exceptional and as of right now undocumented). The only thing close to user/device authentication is ‘Microsoft Account’ (formerly Windows Live, Passport etc. etc.) integration which is probably useless for 99% of the cross-platform applications available out there, that have a need for some sort of unique identification of their users/devices.

It’s the permissions stupid.

The whole situation boils down to botched design in terms of permission control, abuse by advertising, analytics and developers and extremely late regulatory and social reaction to the above, perhaps combined with a pretty simple way to raise barriers to entry to the competition while ‘solving’ the issue of privacy. All platforms have some sort of privacy/permission control, but none have a good one. Android has a pretty comprehensive permission system that assumes that before installing an application each user bothers to read a silly list of permissions (many of which they will probably not understand) and once they accept they will perpetually want to grant all those permissions to said application. There is no fine grained permission control post installation, no possibility to grant or revoke individual permissions to applications before they are launched (something like “I would like to allow App X to use my network connection, but not my location or my address book data”). iOS is also similarly badly designed: there is no explicit permission asked or required for using the network connection, a slew of personal data, several APIs, storage etc., except for location, where iOS does a much better job than Android, probably because of the high-profile exposure that their data-collection ‘functionality’ took a few years ago. At the same time, both platforms actively transmit information gathered by your device, be it nearby BSSIDs (the identifiers of wifi networks, akin to ethernet MAC addresses) or Cell IDs (the unique identifiers of nearby cellular transmitter/antennae) so that they improve their ‘network-based’ geolocation service. Google fares better in this respect, as they allow you to disable this; Apple doesn’t, as far as I know.

Then comes Microsoft, the ailing software behemoth that only recently decided that Balmer’s rhetoric about the iPhone’s failings, the iPad not gaining any significant traction etc. was totally wrong after all, and that they should jump on the tablet bandwagon, not in the way they’ve been trying to do for about a decade, but the way Apple did with their own version of a walled garden, doing away with the desktop paradigm and providing a dumbed down, simpler interface that does away with compatibility, file-systems etc and uses a locked down, app store/marketplace based model to ensure software legitimacy and boost profits. So Windows Phone 7 and Windows 8 provide new sets of APIs and a new ‘application environment’ called Metro. In the Windows 8 version, the æsthetics borrow much more than its name from Windows Phone 7, the company’s revamped operating system for mobile phones that, while a decent effort, doesn’t seem to be doing that great on the market. Metro on Windows 8, however, is not a finished product by any means, and probably won’t be ‘finished’ (that is of a sufficiently high quality) until Windows 9 is released in a few years from now. Metro on Windows 8 also has permissions, like Android, but does away with unique device identifiers and any sort of meaningful API to get any sort of replacement of one. It also allows the user to revoke a permission (say, for the location), but only after the application has be executed, which kind of defeats the purpose.

My experience with the ‘next-generation’ platforms I have programmed on until now strongly suggests that the companies and people designing them have no idea about the implications of their work. They are experimenting, releasing APIs, platforms and products without thinking them through, or the impact their software has on the users, developers building applications using them or the overall social effect of their design decisions. In the case of Android, many more developers have access to IMEIs, MAC addresses and other, arguably much more sensitive information about devices and their users than they would have, had Google paid some attention and provided a unique, immutable pseudo-random unique device identifier from day one. It is also surprising how bad their permissions system is, given that they at least went through the trouble of designing one in the first place. In the case of Microsoft, the complete lack of such a mechanism, may eventually play its part in hurting the company’s efforts to enter the game (they already are extremely late). And finally, Apple, the market leader that did so many things right in the first place, is risking pissing off everybody from small independent companies that helped build the platform, to its greatest non-platform owning competitors that can see through the excuse of legal heat from regulators and the government, their hypocrisy on protecting the users’ privacy and who may call their action as an excuse to block them out of their platform. At the end of the day, the three big players in this market still get all your information, and their expansion into advertising, mobile payments, e-commerce and every single part of the software ecosystem possible means that they have the greatest incentive to (ab)use it.

In the end, all of the privacy problems that location, unique device identification and access to other personal information may give rise to are easily solvable by a modern, smart permission system that gives the user the power to deny, revoke or grant permissions to individual applications post installation, including system software/applications, thus creating a level playing field where the user would decide what kind of access to provide to whom. That would be a clear demonstration, on the platform owners’ part, that they truly care about users’ privacy and not just creating barriers to entry to the competition and their bottom line.


» Independent companies

WSJ: Before Steve Jobs of Apple Inc. died, he approached you with a buyout offer. Why did you turn it away?

Mr. Ferdowsi: The problem that we’re trying to solve is a problem that only an independent company can solve. We want to let you use a Mac, or Windows PC, or iPad, or Android, without having to think about any of the technical details. It isn’t a problem any of those larger companies is going to be as inclined to solve in the same way we are.

A very very pertinent point, seeing that we’re experiencing a renaissance of massive, vertical closed systems, walled gardens and a childish desire to lock people into proprietary platforms that try to offer everything. Look at how Google, Facebook, Apple and now Microsoft are heavily promoting their respective ‘authentication’ platforms, playing the game of ignoring_the_competition. Facebook would certainly like you to use their APIs to authenticate your users, but they don’t have to try much because they have the most powerful database right now. Microsoft heavily promotes their ‘Microsoft Account’ (previously known by half a dozen names) and will do even more in Windows 8, while Apple makes ever increasing use of their Apple ID, across their products and services. Google, in lieu of their recent privacy terms update, needs no introduction I think with Google+ and every other service tied to a single Google account. The fact that Dropbox fully supports practically every single system platform I can think of using is reason enough for me to prefer it from competing services (Ubuntu One, Microsoft Skydrive, iCloud etc) and a refreshingly sane choice they made contrasted heavily by that of the established market leaders who fear of inadvertently promoting their competition.


DigitalOcean. Affordable, Fast, SSD VPS