Olivier Travers

Free cash flow for the win
Home > Archives > 2002 > December > 5 > More Desktop Computing Power is Useless, Blah, Blah, Blah
More Desktop Computing Power is Useless, Blah, Blah, Blah

If you want to put your GHz+ Athlon or P4 CPU on its knees, ask Excel to import 50,000 rows out of an XML query. Enjoy (not!) CPU 100% load for several minutes. My tip: split the XML file into smaller chunks, query them once at a time, and remove the query reference from "beneath" the data (i.e. just keep flat data in Excel after each import).

Yeah, yeah, I know, it's dumb to use Excel as a database. Even dumber though is to confuse the browser for a data crunching and charting tool. Nothing beats Excel's pivot tables and charts. A best practice would probably to dump my search logs (I'm logging search queries done by users on my SF & fantasy site) into a database and plug Excel into SQL Server, not a huge (as in 8MB+) text file. Or maybe use the XML file as an intermediary data store (for resiliency and ease-of-use purposes), "replicate" it to SQL, then query SQL from Excel? For reference: XML and Databases.

I'm wondering whether Excel 11 will be able to parse binary XML (I guess using SAX or something, if I actually knew what I'm talking about.) Binary might make sense in direct-from-the-Internet queries (as opposed to downloading the file first through ftp, or accessing it on a LAN.) Am I making sense at all?

A related query leads me to this search result page (incidentally, I'm going to work on SciFan's serp later today) with a nice way to show/hide summaries for the whole list or at the entry level. I'm kinda obsessed with how to improve list/detail web display.


Category(s): digital lifestyle ·
Post a comment






Remember personal info?
Your e-mail address is used to send you future comments to this entry, but I won't use it for any other purpose and it won't appear on the site. I prefer you comment using your real identity, thanks.
Email this Story to a Friend




This form is used only to email this story to your friend. I won't save the email addresses you type in, or use them in the future in any way.


About
Contact



Web Feed

Powered by Movable Type

My profiles: