Free Webcast: Performance to Cost Index & my personal history with “Big Data”

NOTE: This fun review of “big data” was inspired by a recent presentation I gave on behalf of Tableau Software at the Big Data Conference in Chicago. You can find the 2nd part of this 3 part webcast here, ““Big Data” in US History, Exploring the 1790 US Census”. Part 3 is here, “Big Data” on your laptop, fast, informative and at your command.

Many people ask me, what is “big data”?   For most of them, the right answer is that big data is any data that is difficult to use or understand (yes, I know the official, “correct” answers, which often vary and typically include topics like Hadoop and Cloudera.)

In this presentation, I share my experience with the Commodore 64, the PS/2, DEC Stations, VAX servers, Solaris Servers, PC’s and a MacBook Pro.   Products and languages covered include BASIC, FORTRAN, SAS, Oracle, Teradata and Tableau.

It is truly astonishing when you consider the Performance to Cost Index  (records per minute divided by system price) and how that has changed since I first started programming in BASIC.   Starting at a PCI of 3 with the Commodore 64 to a PCI of 50 with the DEC Stations in the 90’s, to a PCI of 10,000 with the Solaris Server in 2011 to an incredible PCI of 350,000 with Tableau running on the Mac (using VMWare/Windows 7.)

NOTE: some people may experience problems viewing this webcast in Internet Explorer.   If this occurs, we recommend   Firefox or Chrome to watch the video.   The alternative is to see if your Flash Player is properly installed (sorry, we are unable to help diagnose IE video problems beyond this shared link.)  

Look for parts 2 and 3 of this series in future blog posts here at Freakalytics.

Please note that all historic values are estimates based on my best recollection.   No warranty or guarantees of performance are implied or offered.   Please note that databases and servers can perform at radically different speeds based on indexing, optimization, memory, loads, data structures and many other factors.   Future releases of compared systems may have radically different performance results.

My MacBook Pro has been enhanced with an SSD drive, available for $200 at Amazon.   SSD drives greatly accelerate any type of disk-based access, which benefits the Tableau Data Engine, a file-based data access system.   The small SSD drive price was included in my PCI for the MacBook Pro.