Wednesday, October 30, 2013

Who's war on science?

"I would avoid them. I would avoid them until there's more science that actually, you know, really says if it's safe or not. Right now there's a lot of contradictory information out there as to whether it's safe or it's not safe."

Sounds a lot like a republican complaining about some climate change policy, but in fact its a food activist in California campaigning against genetically modified (GMO) food. The NPR article points out (citing Michael Pollan of all people) that the scientific consensus is in fact that GMOs are safe.

Yes conservatives often find it convenient to ignore scientific consensus when it suits their purposes. but there are plenty of examples of liberals doing the same thing.

Monday, October 21, 2013

Big Data Travails

Lots have been said recently about Big Data. So I figured I'd try it out. If anyone wants to commiserate, I've spent the last few months wrestling with a dataset that has 47 billion observations. Simple tasks like downloading the data took time to learn Amazon's s3 interface to program, weeks to run, and then, just transferring the files to a different computer has caused me to curse that the only compatible file system between macs and windows is fat32 which can't possibly handle the number of files I have. And simple zipping and unzipping requires shell and perl and python programming and days to run, and fault tolerant code that can handle the inevitable crashes that comes with running things that take days to complete. Doing all that while keeping the data encrypted and secure. Anyway, all that said, if anyone needs help with Big Data questions, I may have some relevant experience.