Touching the tip of the iceberg — a look back at Canada’s first “big data” conference

Last week, Edmonton was the site of Canada'€™s first '€˜Analytics, Big Data and the Cloud'€™ conference, hosted by ABCtech, the University of Alberta School of Business, and Athabasca University. This was an international event, with delegates from the USA, India, Israel, and across Canada. Those who came were a mix of entrepreneurs wanting to find out what was behind the hype of big data, and researchers wanting to connect and collaborate.

On Monday, a Cybera team of Everett Toews and myself conducted two hands-on cloud building workshops. Participants were taken on a quick tour of the Amazon cloud before rolling up their sleeves to build an OpenStack cloud. The goal of the workshop was to give people enough knowledge about OpenStack to be able to repeat the exercise on their own, allowing them to study a working cloud in greater depth. This was done by building the OpenStack cloud inside an Amazon virtual machine. Using this method, anyone can build a small cloud suitable for experimentation for about $0.16 per hour; and can do so before finishing their morning coffee.

On Tuesday, Everett moderated the "Big Data Issues & Implications" panel. To focus the group, Everett turned to the wisdom of one man: DevOps_Borat.

Culled from his wise words, the following phrases helped structure the panel discussion:

  • '€œ1 TB of Big Data is much more than 1024 GB of small data.'€
  • '€œDevops prediction for year 2012: Cloud, NoSQL and Big Data are final get press they are deserve.'€
  • '€œIf you want of do Big Data and you not know good math all you can able achieve is Medium Data.'€

These phrases roughly translate to big data's issues with:

  • Definition
  • Hype
  • Skills

An interesting panel discussion ensued, with great questions from the audience and informative answers from the panelists.

During "The Cloud: Searching for Meaning" panel, professor Eleni Stroulia of the Computing Science department at the University of Alberta gave an illuminating talk on the use of Hadoop. She elegantly described the Map/Reduce algorithm that is at the core of Hadoop, as well as explaining the rest of the Hadoop ecosystem. She then spoke about her research group's work on big data using Hadoop, and how they had migrated to the tool. At the end of her session, Eleni encouraged people from industry to experiment with Hadoop on their data sets or, if they have an absence of time and resources to do so, partner with her department to allow students to experiment on the (anonymized) data sets.

On Wednesday, Randy Goebel, Vice President of Alberta Innovates Technology Futures, chaired the panel on '€˜Analytics Centres: Innovating Industrial Applications'€™. Here, three panelists described their efforts to create collaborative centres in which analytics can thrive. It was noted that, although Alberta universities produce a good number of world-class data scientists and analysts, these people promptly leave the province due to a lack of community. All panelists agreed that in order to retain these experts, a sort of centre-of-gravity is required, towards which experts from around the world can be attracted.  Just as Stanford University and Menlo College acted as incubators for Silicon Valley, so too can the universities and centres in Alberta act as incubators for local industry and research.

Jurji Paraszaak of IBM gave the Wednesday afternoon keynote address, speaking on '€˜The Beauty of Data Models Managing Smarter Cities'€™. Paraszaak encouraged an approach to analytics that breaks down the barriers between silos of data. This allows a more holistic view of the system, yielding new insights into the world.

This has only recently become possible thanks to the vast amounts of data now available that is cheap to gather, rich in content, and easy to access and combine. For example, new building designs could incorporate the improved energy efficiency of the building itself, as well as ongoing changes to city infrastructure requirements, changes to traffic patterns and local population density, changes to intellectual capital of the city, and so on. This encompassing approach to analysis is one step to smarter, more efficient cities.

I confess, I found the conference to be long on hyperbole, and short on practical advice. Too often, I heard about the benefits of big data and analytics, inspiring case studies of the results from applying analytics. But I heard very little advice about the steps one might take to achieve such results for themselves. I could see the destination, I just couldn'€™t see the path.

Perhaps I'€™m being a little unfair, here. This was, after all, only the first such conference in Canada. If its only achievement was to show what is possible — to reveal a larger world —€” then it has performed a valuable service.