Pandas: or why I need to stop rolling my own tabular data parsers in Python.

I don’t know how many times I’ve written the following Python code:

f = [x.split('\t') for x in open('tab_delimited_data.out').read()]

to read tab delimited data. With Pandas, the python data manipulation library, it’s as easy as R:

f = pd.read_csv('tab_data.out', sep='\t', index_col=0)

and now I have a data frame, with a billion more useful functions than my 2D list I usually would have.

Advertisements

Book Review: Against the Fall of Night

I just finished Arthur C. Clarke’s Against the Fall of Night, one of his earliest works (1948). It’s a beautiful, poignant, almost desperately sad story of the human race, billions of years in the future, set against a fading galaxy in the last human city of Diaspar.

One of the things about Clarke’s stories, or at least this one in particular, is the elegance of their future technology. Somehow, writing over 60 years ago, he can make it feel dishearteningly primitive to sit back down at a modern computer or mobile device. The technology in Against the Fall of Night is seamless and eternal- it was designed to work virtually forever. The descriptions are sparse and perfect; one of the most beautiful inventions is a solid path that moves rapidly in the center, like a moving walkway, yet seamlessly comes to a halt at its terminus- all without moving plates or parts.

Clarke is famous for the saying “any sufficiently advanced technology is indistinguishable from magic,” but there is a certain realistic grace in the “magic” behind the city of Diaspar in the book that exceeds most other fantasy or science-fiction worlds I’ve read. It makes one discouraged and unsatisfied with how cumbersome and flawed our current technology really is: power grids with innumerable weak points, computers not significantly more advanced than their first ancestors, a hideously inefficient system of transportation and living and working and programming and government.

Even though the story is so poignant and bittersweet, one of the most amazing things about Arthur C. Clarke’s writing is how optimistic he was. It is something of an inspiration to read about these beautiful worlds: a reminder that things worth doing are worth doing well.