If you want to put your GHz+ Athlon or P4 CPU on its knees, ask Excel to import 50,000 rows out of an XML query. Enjoy (not!) CPU 100% load for several minutes. My tip: split the XML file into smaller chunks, query them once at a time, and remove the query reference from "beneath" the data (i.e. just keep flat data in Excel after each import).
Yeah, yeah, I know, it’s dumb to use Excel as a database. Even dumber though is to confuse the browser for a data crunching and charting tool. Nothing beats Excel’s pivot tables and charts. A best practice would probably to dump my search logs (I’m logging search queries done by users on my SF & fantasy site) into a database and plug Excel into SQL Server, not a huge (as in 8MB+) text file. Or maybe use the XML file as an intermediary data store (for resiliency and ease-of-use purposes), "replicate" it to SQL, then query SQL from Excel? For reference: XML and Databases.
I’m wondering whether Excel 11 will be able to parse binary XML (I guess using SAX or something, if I actually knew what I’m talking about.) Binary might make sense in direct-from-the-Internet queries (as opposed to downloading the file first through ftp, or accessing it on a LAN.) Am I making sense at all?
A related query leads me to this search result page (incidentally, I’m going to work on SciFan’s serp later today) with a nice way to show/hide summaries for the whole list or at the entry level. I’m kinda obsessed with how to improve list/detail web display.
