logoalt Hacker News

didgetmastertoday at 12:06 AM1 replyview on HN

Creating interactive pivot tables from large relational tables.

Many people know that a handy data analysis feature in Excel is to create a pivot table from a spreadsheet. But spreadsheets are limited to just a million rows. You can get around this limit by jumping through a bunch of hoops.

My system lets you easily create tables with thousands of columns and hundreds of millions of rows. (Just drop a CSV, Json, or other file on a window to create a table.)

Now you can create a pivot table from it with just a few clicks of the mouse. It is fast (I created a pivot table against an 8.5 million row table of Chicago crime data in less than a second.)

The resulting pivot table is interactive. Each cell (row/column intersection) has all the row keys mapped to it. Double-click on any cell and it will instantly show you all the rows in the original table that were used to calculate the cell. You can then analyze those rows further.

It also works well against much larger tables. I have tested it out against 25M, 50M, 100M, and 200M+ row tables.


Replies

misirtoday at 12:15 AM

How are you planning to sell it given the market dominance of Excel? The people that would be most willing to pay for spreadsheets are also the people who are already paying for Excel.

Not trying to discourage you, I am curious as to see how you are planning to enter the market as that was something I couldn’t answer when considering working on spreadsheet tools of various kinds or even an excel alternative.

show 1 reply