Bookmarks for January 30th through February 4th

These are my links for January 30th through February 4th:

  • Op-Ed Contributor – Microsoft’s Creative Destruction – NYTimes.com – Unlike other companies, Microsoft never developed a true system for innovation. Some of my former colleagues argue that it actually developed a system to thwart innovation. Despite having one of the largest and best corporate laboratories in the world, and the luxury of not one but three chief technology officers, the company routinely manages to frustrate the efforts of its visionary thinkers.
  • Leonardo da Vinci’s Resume Explains Why He’s The Renaissance Man For the Job – Davinci – Gizmodo – At one time in history, even da Vinci himself had to pen a resume to explain why he was a qualified applicant. Here's a translation of his letter to the Duke of Milan, delineating his many talents and abilities. "Most Illustrious Lord, Having now sufficiently considered the specimens of all those who proclaim themselves skilled contrivers of instruments of war, and that the invention and operation of the said instruments are nothing different from those in common use: I shall endeavor, without prejudice to any one else, to explain myself to your Excellency, showing your Lordship my secret, and then offering them to your best pleasure and approbation to work with effect at opportune moments on all those things which, in part, shall be briefly noted below..The document, written when da Vinci was 30, is actually more of a cover letter than a resume; he leaves out many of his artistic achievements and instead focuses on what he can provide for the Duke in technologies of war.
  • jsMath: jsMath Home Page – The jsMath package provides a method of including mathematics in HTML pages that works across multiple browsers under Windows, Macintosh OS X, Linux and other flavors of unix. It overcomes a number of the shortcomings of the traditional method of using images to represent mathematics: jsMath uses native fonts, so they resize when you change the size of the text in your browser, they print at the full resolution of your printer, and you don't have to wait for dozens of images to be downloaded in order to see the mathematics in a web page. There are also advantages for web-page authors, as there is no need to preprocess your web pages to generate any images, and the mathematics is entered in TeX form, so it is easy to create and maintain your web pages. Although it works best with the TeX fonts installed, jsMath will fall back on a collection of image-based fonts (which can still be scaled or printed at high resolution) or unicode fonts when the TeX fonts are not available.
  • Josh on the Web » Blog Archive » Abusing the Cache: Tracking Users without Cookies – To track a user I make use of three URLs: the container, which can be any website; a shim file, which contains a unique code; and a tracking page, which stores (and in this case displays) requests. The trick lies in making the browser cache the shim file indefinitely. When the file is requested for the first – and only – time a unique identifier is embedded in the page. The shim embeds the tracking page, passing it the unique ID every time it is loaded. See the source code.

    One neat thing about this method is that JavaScript is not strictly required. It is only used to pass the message and referrer to the tracker. It would probably be possible to replace the iframes with CSS and images to gain JS-free HTTP referrer logging but would lose the ability to store messages so easily.

  • Panopticlick – Your browser fingerprint appears to be unique among the 342,943 tested so far.

    Currently, we estimate that your browser has a fingerprint that conveys at least 18.39 bits of identifying information.

    The measurements we used to obtain this result are listed below. You can read more about the methodology here, and about some defenses against fingerprinting here

Bookmarks for May 30th through May 31st

These are my links for May 30th through May 31st:

Bookmarks for May 21st from 06:07 to 22:34

These are my links for May 21st from 06:07 to 22:34:

Bookmarks for May 3rd through May 4th

These are my links for May 3rd through May 4th:

  • Dilbert comic strip for 05/04/2009 from the official Dilbert comic strips archive. – Secretary to Pointy Haired Boss: "I live in a rented trailer and all of my money is in my checking account. Your investments are worthless and your mortgage is underwater. My net worth is higher than yours now. I guess promiscuity and a G.E.D. was a pretty good strategy after all." Reminded me of a thought I had earlier this year, that much of Western Civilization is built on valuing delayed gratification, which hasn't worked out so well recently as opposed to immediate consumption in many cases.
  • Without Warning, Twitter Kills StatTweets (Businesses Beware) – StatSheet.com ChangeLog – Owner of StatTweets post regarding his network of sports-related Twitter handles being banned. They had several hundred accounts, one for stats for each team. This makes sense for users, given the way Twitter works, but they don't like mass account creation. Interested to see how this sorts out, there seem to be at least a few similar Twitter networks with team/region/topic-specific handles.
  • Dooley Online: What URL Shortener Should I Use? – Comparison of features and some usage data for URL shorteners such as tinyurl and bit.ly used on twitter and other services.
  • Obesity and Overweight: Trends: U.S. Obesity Trends 1985-2007 | DNPAO | CDC – During the past 20 years there has been a dramatic increase in obesity in the United States. This slide set illustrates this trend by mapping the increased prevalence of obesity across each of the states. In 2007, only one state (Colorado) had a prevalence of obesity less than 20%. Thirty states had a prevalence equal to or greater than 25%; three of these states (Alabama, Mississippi and Tennessee) had a prevalence of obesity equal to or greater than 30%. The animated map below shows the United States obesity prevalence from 1985 through 2007.
  • Why text messages are limited to 160 characters | Technology | Los Angeles Times – A look back to the beginnings of SMS in 1985 – Would the 160-character maximum be enough space to prove a useful form of communication? Having zero market research, they based their initial assumptions on two "convincing arguments," Hillebrand said. For one, they found that postcards often contained fewer than 150 characters. Second, they analyzed a set of messages sent through Telex, a then-prevalent telegraphy network for business professionals. Despite not having a technical limitation, Hillebrand said, Telex transmissions were usually about the same length as postcards.

Bookmarks for April 30th through May 2nd

These are my links for April 30th through May 2nd:

  • FusionCharts Free – Animated Flash Charts and Graphs for ASP, PHP, ASP.NET, JSP, RoR and other web applications – Flash charting component that can be used to render data-driven & animated charts for your web applications and presentations. It is a cross-browser and cross-platform solution that can be used with PHP, Python, Ruby on Rails, ASP, ASP.NET, JSP, ColdFusion, simple HTML pages or even PowerPoint Presentations to deliver interactive and powerful flash charts. You do NOT need to know anything about Flash to use FusionCharts. All you need to know is the language you're programming in.
  • Raphaël—JavaScript Library – Raphaël is a small JavaScript library that should simplify your work with vector graphics on the web. If you want to create your own specific chart or image crop and rotate widget, for example, you can achieve it simply and easily with this library. Raphaël uses the SVG W3C Recommendation and VML as a base for creating graphics. This means every graphical object you create is also a DOM object, so you can attach JavaScript event handlers or modify them later. Raphaël’s goal is to provide an adapter that will make drawing vector art compatible cross-browser and easy.
  • A Really Gentle Introduction to Data Mining | Regular Geek – List of data mining blogs and related resources.
  • BlackBerry SSH Tutorial: Connect to Unix Server using MidpSSH for Mobile Devices – Notes on using MidpSSH on Blackberry for remote access to servers. Seems to work, although big network lag on my BlackBerry Bold / AT&T.
  • Country Reports on Terrorism 2008 – U.S. law requires the Secretary of State to provide Congress, by April 30 of each year, a full and complete report on terrorism with regard to those countries and groups meeting criteria set forth in the legislation. This annual report is entitled Country Reports on Terrorism. Beginning with the report for 2004, it replaced the previously published Patterns of Global Terrorism.
  • DIY: How To Find Authoritative Twitter Users Plus 100 To Get You Started | Ignite Social Media – Some comments on recommendation metrics for Twitter, trying to use "favorites" mark as an indicator.
  • SIGUSR2 > The Power That is GNU Emacs – "If you've never been convinced before that Emacs is the text editor in which dreams are made from, or that inside Emacs there are unicorns manipulating your text, don't expect me to convince you."

Bookmarks for March 16th through April 2nd

These are my links for March 16th through April 2nd:

Bookmarks for March 12th through March 16th

These are my links for March 12th through March 16th:

Bookmarks for March 9th through March 12th

These are my links for March 9th through March 12th:

Bookmarks for February 28th through March 1st

These are my links for February 28th through March 1st:

  • Community Data – Swivel – User contributed datasets, for visualization and graphs with Swivel
  • Obamameter – Map visualization of economic stimulus outlays. "Keep tabs on the the US economy, the global economy and the stimulus through our dashboard for the economy."
  • recovery.gov.pdf – Slide presentation on data sources and construction of initial Recover.gov site in Jan 2009, from talk at Transparency Camp.
  • Virtual Hoff : DoxPara Research – Slides from Dan Kaminsky's talk at CloudCamp Seattle on network and application security issues in cloud and virtualized computing environments.
  • Can You Buy a Silicon Valley? Maybe. – from Paul Graham – "If you could get startups to stick to your town for a million apiece, then for a billion dollars you could bring in a thousand startups. That probably wouldn't push you past Silicon Valley itself, but it might get you second place. For the price of a football stadium, any town that was decent to live in could make itself one of the biggest startup hubs in the world."
  • Berkshire Hathaway 2008 shareholders letter (PDF) – Warren Buffet reviews the state of the financial markets, his worst year ever, and the outlook for 2009.
  • White House 2: Where YOU set the nation’s priorities – Not the actual White House, but an interesting experiment in collaborative input for setting government agenda.
  • Python for Lisp Programmers – Peter Norvig examines Python. "(Although it wasn't my intent, Python programers have told me this page has helped them learn Lisp.) Basically, Python can be seen as a dialect of Lisp with "traditional" syntax (what Lisp people call "infix" or "m-lisp" syntax). One message on comp.lang.python said "I never understood why LISP was a good idea until I started playing with python." Python supports all of Lisp's essential features except macros, and you don't miss macros all that much because it does have eval, and operator overloading, and regular expression parsing, so you can create custom languages that way. "

Bookmarks for February 14th through February 15th

These are my links for February 14th through February 15th:

North Korea tests a nuclear bomb?

North Korea has been threatening to test a nuclear weapon recently, and may have done so a couple of hours ago.

The test is “unconfirmed” at the moment, but South Korea says it detected seismic activity measuring 3.5 on the Richter scale at 0136GMT, or 10:36AM Korea local time. The presumed test site is underground, in a coal mine in Gilju.

It’s surprising to me that, given the advance warning, there isn’t an official confirmation that there was a nuclear test or not. There’s probably no shortage of equipment set up to monitor the situation, and I would expect a different signature for a nuclear explosion than from setting off a huge pile of RDX at the bottom of a mine.

There’s no shortage of countries that could build nuclear weapons if they wanted. South Korea and Japan in particular come to mind at the moment. More problematic would be Kim Jong-Il making a deal with Iran’s Mahmoud Ahmadinejad (or someone similar) to trade oil and hard currency for nuclear weapons technology.

More on the America Online search query data

The search query data that America Online posted over the weekend has been removed from their site following a blizzard of posts regarding the privacy issues. AOL officially regards this as “a screw up”, according to spokesperson Andrew Weinstein, who responded in comments on several sites:

All –

This was a screw up, and we’re angry and upset about it. It was an innocent enough attempt to reach out to the academic community with new research tools, but it was obviously not appropriately vetted, and if it had been, it would have been stopped in an instant.

Although there was no personally-identifiable data linked to these accounts, we’re absolutely not defending this. It was a mistake, and we apologize. We’ve launched an internal investigation into what happened, and we are taking steps to ensure that this type of thing never happens again.

I pulled down a copy of the data last night before the link went down, but didn’t get around to actually looking it over until this evening. In a casual glance at random sections of the data, I see a surprising (to me) number of people typing in complete URLs, a range of sex-related queries, (some of which I don’t actually understand), shopping-related queries, celebrity-related queries, and a lot of what looks like homework projects by high school or college students.

In the meantime, many other people have found interesting / problematic entries among the data, including probable social security numbers, driver’s license numbers, addresses, and other personal information. Here’s a list of queries about how to kill your wife from Paradigm Shift.

More samples culled from the data here, here, and here.

#479 Looks like a student at Prairie State University who like playing EA Sports Baseball 2006, is a White Sox fan, and was planning going to Ozzfest. When nothing else is going on, he likes to watch Nip/Tuck.

#507 likes to bargain on eBay, is into ghost hunting, currently drives a 2001 Dodge, but plans on getting a Mercedes. He also lives in the Detroit area.

#1021 is unemployed and living in New Jersey. But that didn’t get him down because with his new found time, he’s going to finally get to see the Sixers.

#1521 like the free porn.

Based on my own eclectic search patterns, I’d be reluctant to infer specific intent based only on a series of search queries, but it’s still interesting, puzzling, and sometimes troubling to see the clusters of queries that appear in the data.

Up to this point, in order to have a good data set of user query behavior, you’d probably need to work for one of the large search engines such as Google or Yahoo (or perhaps a spyware or online marketing company). I still think sharing the data was well-intentioned in spirit (albeit a massive business screwup).

Sav, commenting over at TechCrunch (#67) observes:

The funny part here is that the researchers, accustomed to looking at data like this every day, didn’t realize that you could identify people by their search queries. (Why would you want to do that? We’ve got everyone’s screenname. We’ll just hide those for the public data.) The greatest discoveries in research always happen by accident…

A broader issue in the privacy context is that all this information and more is already routinely collected by search engines, search toolbars, assorted desktop widget/pointer/spyware downloads, online shopping sites, etc. I don’t think most people have internalized how much personal information and behavioral data is already out there in private data warehouses. Most of the time you have to pay something to get at it, though.

I expect to see more interesting nuggets mined out of the query data, and some vigorous policy discussion regarding the collection and sharing of personal attention gestures such as search queries and link clickthroughs in the coming days.

See also: AOL Research publishes 20 million search queries

Update Tuesday 08-08-2006 05:58 PDT – The first online interface for exploring the AOL search query data is up at www.aolsearchdatabase.com (via TechCrunch).

Update Tuesday 08-08-2006 14:18 PDT – Here’s another online interface at dontdelete.com (via Infectious Greed)

Update Wednesday 08-09-2006 19:14 PDT – A profile of user 4417749, Thelma Arnold, a 62-year-old widow who lives in Lilburn, GA, along with a discussion of the AOL query database in the New York Times.

AOL Research publishes 20 million search queries

More raw data for search engineers and SEOs, and fodder for online privacy debates – AOL Research has released a collection of roughly 20 million search queries which include all searches done by a randomly selected set of around 500,000 users from March through May 2006.

This should be a great data set to work with if you’re doing research on search engines, but seems problematic from a privacy perspective. The data is anonymized, so AOL user names are replaced with a numerical user ID:

The data set includes {UserID, Query, QueryTime, ClickedRank, DestinationDomainUrl}.

I suspect it may be possible to reverse engineer some of the query clusters to identify specific users or other personal data. If nothing else, I occasionally observe people accidentally typing in user names or passwords into search boxes, so there are likely to be some of those in the mix. “Anonymous” in the comments over at Greg Linden’s blog thinks there will be a lot of those. The destination URLs have apparently been clipped as well, so you won’t be able to see the exact page that resulted in a click-through.

Haven’t taken a look at the actual data yet, but I’m glad I’m not an AOL user.

Adam D’Angelo says:

This is the same data that the DOJ wanted from Google back in March. This ruling allowed Google to keep all query logs secret. Now any government can just go download the data from AOL.

On the search application side, this is a rare look at actual user search behavior, which would be difficult to obtain without access to a high traffic search engine or possibly through a paid service.

Plentyoffish sees an opportunity for PPC and Adsense spammers:

Google/ AOL have just given some of the worlds biggest spammers a breakdown of high traffic terms its just a matter of weeks now until google gets mega spammed with made for adsense sites and other kind of spam sites targetting keywords contained in this list.

I think it’s great that AOL is trying to open up more and engage with the research community, and it looks like there are some other interesting data collections on the AOL Research site — but I suspect they’re about to take a lot of heat on the privacy front, judging from the mix of initial reactions on Techmeme. Hope it doesn’t scare them away and they find a way to publish useful research data without causing a privacy disaster.

More on the privacy angle from SiliconBeat, Zoli Erdos

See also: Coming soon to DVD – 1,146,580,664 common five-word sequences

Update – Sunday 08-06-2006 20:31 PDT – AOL Research appears to have taken down the announcement and the log data in the past few hours in response to a growing number of blog posts, mostly critical, and mostly focused on privacy. Markus at Plentyoffish has also used the data to generate a list of ringtone search keywords which users clicked through to a ringtone site as an example of how this data can be used by SEO and spam marketers. Looks like the privacy issues are going to get the most airtime right now, but I think the keyword clickthrough data is going to have the most immediate effect.

Update Monday 08-07-2006 08:02 PDT: Some mirrors of the AOL data

Hello India, we’re still here…

…but other sites are apparently blocked.

There are a fair number of readers here from India, where some ISPs have started blocking many blogs, including all of Typepad, Blogspot, Geocities. So you might have thought this site was also blocked if you came by yesterday, since you would have gotten something like “Connection refused” or a similar error message.

Fortunately / unfortunately, it’s just Dreamhost having some hardware and network problems, which took down many of their clients for several hours yesterday, and is still behaving badly today.

Summer internship at Freedom House North Korea


Passing this along for friends who may have an interest in human rights in North Korea, from The Korea Liberator:

Jae Ku, Director of Freedom House’s North Korea program, sends:

Dear Friends, I am in need of a Korean speaking intern (native, read and write) for this summer. This is a paid internship, to commence immediately. If you know of anyone, please have that person send me his/her resume. I am looking for someone who is mature and responsible. It is helpful but not necessary to have a background in human rights or North Korean issues.

Thank you,

Jae

Jae H. Ku, Ph.D.
Director, Human Rights in North Korea Project
1319 18th Street, NW
Washington, DC 20036
O) 202-747-7048
ku@freedomhouse.org

Links:

Maria moves the markets

I sometimes have CNBC on in my office, sort of as background noise. This afternoon I heard Maria Bartiromo describing her weekend conversation with Federal Reserve chairman Ben Bernanke.

The media and the markets basically got it wrong last week in speculating that the Fed is done raising interest rates.

I’m thinking to myself, “Hmm. That’s going to make some people uncomfortable with their positions.” Especially since this was pretty much the opposite of what people were thinking after last week’s Fed testimony.

Here’s the AP summary:

Last Thursday, Bernanke told a congressional panel the central bank could pause — but not necessarily stop — its string of rate hikes while it keeps a close watch on the economy’s health. However, according to CNBC, Bernanke said future increases will depend mostly on economic data; that stand was troubling to an interest rate-sensitive market.

A spokeswoman for the Federal Reserve, Michelle Smith, declined to comment on the CNBC report

Well, that was fun. Some quick thoughts:

  • If this was intended as clarification of Fed policy, why not say so directly, instead of indirectly? And why not officially comment when given the opportunity?
  • Why did CNBC sit on this all day, instead of reporting pre-open, when they already had the story in hand?
  • Who profited from this? This would have been handy information to have in one’s pocket this morning…

You can currently watch the clip on the CNBC site (requires IE6).

Update 05-02-2006 1030PDT: Trader Mike has a thorough roundup of comments from various blogs.

Better Information Isn’t Always Beneficial

In today’s WSJ, David Wessels outlines some systemic social problems that can arise as information becomes widely available at lower costs.

I like the description of the problem in a quote by Kenneth Arrow: “socially useless but privately valuable” information, which can provide individual benefits, but at an overall cost to society at large.

This is the inverse of the dynamics driving “web 2.0″, which thrives on the sharing of “privately useless but socially valuable” information such as click streams, tagging, location awareness, presence awareness, etc.

Computer and communications technology is making more and better information available ever more quickly. This is a good thing — usually.
But there are some things we don’t want to do more efficiently. Doing them better adds neither to the U.S. national psyche nor to the gross domestic product. Figuring out which is which is a growing challenge to society as technology makes gathering and analyzing information easier and cheaper.

This issue predates computers. “The contrast between the private profitability and the social uselessness of foreknowledge may seem surprising,” the late economist Jack Hirshleifer wrote in 1971. But there are instances, he argued, where “the community as a whole obtains no benefit … from either the acquisition or the dissemination (by resale or otherwise) of private foreknowledge.”

Imagine a place with uncertain weather where food is plentiful in rainy spots, but not in others. Residents, in essence, buy insurance. The lucky feed the unlucky. No one starves. Then it becomes possible to buy accurate weather forecasts. One who buys the forecast knows whether he needs insurance or not; he profits. But the total amount of food available is unchanged. And if everyone buys the weather forecast, the insurance market becomes impossible. “There is a double social loss — the resources used unnecessarily in acquiring information and the destruction of a market for risk sharing,” Mr. Arrow said when he posed this example in 1973.

Update 09-23-2005 10:31 PDT: Discussion on this topic at Slashdot (noisy, but some interesting comments).