I had a fantastic idea just now, but it requires Windows to do it:
Damn, being a die-hard Linux user is hard, and I need your help!
This weekend, docs.jquery.com was offline, and I was really annoyed, because its hard to program for an API when you can’t get to the documentation.
So, I did the thing that bad internet users do, and crawled the entire site and stored it locally. Its 138MB uncompressed, and only 5.4MB as a compressed .tar.bz2. I looked briefly at using the MediaWiki tools to create a mirror of it, but it just seemed way to easy to fire off a big wget (-E –mirror -k -p).
Let me know if you want a copy and I can e-mail it to you or put it up on my other server.
When I put source code in a <code> block in WordPress:
I guess I should look around for a source-code syntax-highlighting plugin for WordPress? Anyone know of a good one?
After a brief conversation and thought today, I realized that my previous efforts on parallel sorting on the bash commandline may have fallen a bit short of their potential.
So, I started to craft a new implementation. This one would emulated a parallel qsort algorithm, but using python, bash, and the standard UNIX sort command.
The basis behind this design is that you can eliminate the final merging step if you first ‘pivot’ the input file into pieces that don’t overlap. (I’m using qsorts ‘pivot’ term here). So, I wrote a short pivot program in python:
#!/usr/bin/python2.4
from sys import stdin, argv
outputs = []
numargs = 0
buffer_size = 1024 * 64
for arg in argv[1:]:
outputs.append(open(arg, ‘a’))
numargs += 1
pivot = stdin.readline(buffer_size)
outputs[0].write(pivot)
s = “empty”
while s:
s = stdin.readlines(buffer_size)
for line in s:
if line <= pivot:
outputs[0].write(line)
else:
outputs[1].write(line)
So, with that, one can ‘pivot’ an input file into 2 pieces that are known to not overlap, sort them independently, and then just catenate the final outputs together. No post-merging is necessary, and the pivot step is actually easier than merging (!).
So, here’s an extremely simplified parallel sorting driver script, using the ‘pivot.py’ quoted above:
#!/bin/bash
mkfifo a b c d
cat $1 | pivot.py a b &
sort a -o c &
sort b -o d &
cat c d
And, I compared a script like this with the vanilla ’sort’ UNIX sort command on a file of 58 million random character strings. This new pivot-based sorter was 50% faster than the vanilla sort, completing the task in 2 minutes, compared with 3 minutes for sort.
Of course, with any qsort implementation, choosing a good pivot is important, and this algorithm greatly suffers from poor pivot choices (which are derived from the first line in the file).
Aw, this is a bit harsh, isn’t it? Unfortunately, we seem to have come into a world where hardware and software are bundled & sold as a single monolithic entitiy. In other words, you can buy the iPhone (or TiVo) but the only software you can run on it is the one approved by the manufacturer. I’m not sure how exactly I feel about that restriction…
Tivoization is the creation of a system that incorporates software under the terms of a copyleft software license, but uses hardware to prevent users from running modified versions of the software on that hardware. Richard Stallman coined the term and believes this practice denies users some of the freedom that the GNU General Public License GNU GPL was designed to protect.[1] The term came about in reference to TiVo’s use of GNU GPL licensed software on the TiVo brand digital video recorders DVR.
So, the crap on the Drudge Report is just rumor mongering?
Three days before Democrats open their convention in Denver, officials said the Obama campaign had taken the trouble to print material bearing the names of several potential ticket mates. The result was to minimizing the significance of a report that one company was churning out signs bearing Bayh’s name.
The Associated Press: Obama VP intro to be Saturday; Kaine, Bayh are out
Rabbit — somewhat disturbing, but I really liked it…
Every day on NPR I hear some news story about how McCain wants to “increase the energy supply” and that Obama “doesn’t think we need more energy”, or some paraphrase of crap like that.
I’m disgusted that we, as a society, continue to fall for these simple word substitutions. There’s plenty of “energy” in the world, we’re just not doing anything with it. For example, the american southwest receives enough solar radiation to power the entire country, but we just let it bake the sand.
The 3 common types of energy use in the US are:
When they say “there’s an energy shortage” what they’re saying is “there’s a shortage of conventional fuel” or, “fuel supply is lower than demand, and therefore, prices have risen.” What they neglect to mention is my point #4 above: Energy does not always come from fuel! We have become far too focused on conventional means of energy production!
So, every time you hear the word “energy” in the news, you should substitute it in your head with “fuel”, and you should think about how many people profit when you pump a tank full of light sweet crude into your car.
When you hear “energy prices are high” you should think “fuel prices are high” which is a more correct characterization of the situation, and focuses on the crux of the matter: Fuel used for transportation of goods. (Gasoline and Diesel, to be exact). These are the main commodity prices that have risen. Costs for other sources of energy (solar, wind, etc.) are actually falling. In other words, alternative good, conventional bad.
What McCain wants is more oil, and I wish the news agencies (including NPR) wouldn’t fall for his simple substitution language. I’m just waiting for the day that McCain says something like “we need to drill for more energy.” You don’t “drill for energy”, you drill for oil, which you then burn to turn into energy, and this is not the only source of energy. Get it straight.
Tilty Snake uses the accelerometer in a Monome 64 to create a new interface for the old mobile phone game Snake.
10 Futuristic User Interfaces | Monday Inspiration | Smashing Magazine
I added the emphasis there. Where was this person in 1979 when I was 7 years old and loading “Snake” (distributed by Sears software) off casette tape into my Atari 800? bleeeep. bleeeep. bleeeep. bleeeep. Jeeze, I listened to that sound way too much.
My Microblog