Ho John Lee's Weblog

These are my links for January 17th through January 20th:

• PG&E Electrical System Outage Map – This map shows the current outages in our 70,000-square-mile service area. To see more details about an outage, including the cause and estimated time of restoration, click on the color-coded icon associated with that outage.
• Twitter.com vs The Twitter Ecosystem – Fred Wilson comments on some data from John Borthwick indicating Twitter ecosystem use = 3-5x Twitter.com directly.

  "John's chart estimates that Twitter.com is about 20mm uvs a month in the US (comScore has it at 60mm uvs worldwide) and the Twitter ecosystem at about 60mm uvs in the US.

  That says that across all web services, not just AVC, the Twitter ecosystem is about 3x Twitter.com. And on this blog, whose audience is certainly power users, that ratio is 5x."

• Chris Walshaw :: Research :: Partition Archive – Welcome to the University of Greenwich Graph Partitioning Archive. The archive consists of the best partitions found to date for a range of graphs and its aim is to provide a benchmark, against which partitioning algorithms can be tested, and a resource for experimentation.

  The partition archive has been in operation since the year 2000 and includes results from most of the major graph partitioning software packages. Researchers developing experimental partitioning algorithms regularly submit new partitions for possible inclusion.

  Most of the test graphs arise from typical partitioning applications, although the archive also includes results computed for a graph-colouring test suite [Wal04] contained in a separate annex.

  The archive was originally set up as part of a research project into very high quality partitions and authors wishing to refer to the partitioning archive should cite the paper [SWC04].

• Twitter’s Crawl « The Product Guy – "A list of incidents that affected the Page Load Time of the Twitter product, distinguishing between total downtime, and partial downtime and information inaccessibility, based upon the public posts on Twitters blog.

  http://status.twitter.com/archive

  I did my best to not double count any problems, but it was difficult since many of the problems occur so frequently, and it is often difficult to distinguish, from these status blog posts alone, between a persisting problem being experienced or fixed, from that of a new emergence of a similar or same problem. Furthermore, I also excluded the impact on Page Load Time arising from scheduled maintenance/downtime – periods of time over which the user expectation would be most aligned with the product’s promise of Page Load Time. "

• Soundboard.com – Soundboard.com is the web's largest catalog of free sounds and soundboards – in over 20 categories, for mobile or PC. 252,858 free sounds on 17,171 soundboards from movies to sports, sound effects, television, celebrities, history and travel. Or build, customize, embed and manage your own

These are my links for June 3rd through June 4th:

• MIT Database Systems (6.830) TA Course Notes – marcua’s blog – Fall 2008 course notes on database systems, spanning history of databases and data structures up through mapreduce style systems.
• Ray Ozzie’s cloud hangs over the Valley | Beyond Binary – CNET News – Notes from Ray Ozzie talk on Microsoft and the cloud at Churchill Club, June 2009. Would be interesting to hear more of his thoughts on Google Wave, given his past work on Notes and Groove.
• Matasano Chargen » Blog Archive » Typing The Letters A-E-S Into Your Code? You’re Doing It Wrong! – An epic discussion of shortcomings in various common encryption implementations for web applications, presented in the form of a somewhat random screenplay.
• Is Twitter A Substitute For Set Top Box Data? – The post describes a little experiment Simulmedia did to analyze TV channel surfers using only Twitter posts as their data source. Simulmedia explains why they turned to Twitter data for this work:
• NILFS: A File System to Make SSDs Scream | Linux Magazine – A log structured (as opposed to journaling) file system with some advantages for write performance and shorter crash recovery times. Good for large working file systems?

These are my links for June 1st through June 2nd:

• jqPlot – Pure Javascript Plotting – jqPlot is a plotting plugin for the jQuery Javascript framework. jqPlot produces beautiful line and bar charts with many features including: Numerous chart style options. Date axes with customizable formatting. Rotated axis text. Automatic trend line computation. Tooltips and data point highlighting. Sensible defaults for ease of use.
• New Twitter Research: Men Follow Men and Nobody Tweets – Conversation Starter – HarvardBusiness.org – "Although men and women follow a similar number of Twitter users, men have 15% more followers than women. Men also have more reciprocated relationships, in which two users follow each other. This "follower split" suggests that women are driven less by followers than men, or have more stringent thresholds for reciprocating relationships. This is intriguing, especially given that females hold a slight majority on Twitter: we found that men comprise 45% of Twitter users, while women represent 55%."
• Shirky: Power Laws, Weblogs, and Inequality – 2003 article on popularity / traffic on blogs, which was then the latest emerging social media format. "Once a power law distribution exists, it can take on a certain amount of homeostasis, the tendency of a system to retain its form even against external pressures. Is the weblog world such a system? Are there people who are as talented or deserving as the current stars, but who are not getting anything like the traffic? Doubtless. Will this problem get worse in the future? Yes. "
• well-formed.eigenfactor.org : Visualizing information flow in science – Some nice visualization ideas using hierarchical clustering to explore patterns in citation networks.
• Bing API, Version 2.0 – Updated API documentation for Microsoft Bing (formerly Live Search) web services.

These are my links for May 30th through May 31st:

• Scaling Twitter: Making Twitter 10000 Percent Faster | High Scalability – Collection of links to presentations and interviews regarding Twitter's architecture, implementation plans, and performance issues, from spring 2009.
• The Last Psychiatrist: The Difference Between An Amateur, A Scientist, And A Genius – An amateur is full of wonder and speculation, tinkering towards the truth but suffering from a lack of knowledge and idleness; he's not even sure if someone else has already made these discoveries. "Is this a worthwhile pursuit?"

  A scientist performs experiments to confirm or disprove a hypothesis, and in that way he grinds out the truth.

  A genius has three abilities, which are actually the union of amateur and scientist: 1. to know the state of the art, what is known and what is not known. 2. To be able to think "out of the box". 3. To be disciplined enough to concentrate on the tedium of a formal investigation of his wondrous speculations.

• PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing – Research paper on sort of "super healing brush" for manipulating digital images, allows splicing together different sections of the image and automatically selecting similar textures to make the seam transitions work better.
• Light Blue Touchpaper » Blog Archive » Attack of the Zombie Photos – Social networking and sharing sites have challenges implementing and managing access control policies at large scale, and content delivery networks add another wrinkle.
• Map of all Google data center locations | Royal Pingdom – Where in the world is your search being served from? An attempt to assemble a list of known Google data centers worldwide.

These are my links for May 3rd through May 4th:

• Dilbert comic strip for 05/04/2009 from the official Dilbert comic strips archive. – Secretary to Pointy Haired Boss: "I live in a rented trailer and all of my money is in my checking account. Your investments are worthless and your mortgage is underwater. My net worth is higher than yours now. I guess promiscuity and a G.E.D. was a pretty good strategy after all." Reminded me of a thought I had earlier this year, that much of Western Civilization is built on valuing delayed gratification, which hasn't worked out so well recently as opposed to immediate consumption in many cases.
• Without Warning, Twitter Kills StatTweets (Businesses Beware) – StatSheet.com ChangeLog – Owner of StatTweets post regarding his network of sports-related Twitter handles being banned. They had several hundred accounts, one for stats for each team. This makes sense for users, given the way Twitter works, but they don't like mass account creation. Interested to see how this sorts out, there seem to be at least a few similar Twitter networks with team/region/topic-specific handles.
• Dooley Online: What URL Shortener Should I Use? – Comparison of features and some usage data for URL shorteners such as tinyurl and bit.ly used on twitter and other services.
• Obesity and Overweight: Trends: U.S. Obesity Trends 1985-2007 | DNPAO | CDC – During the past 20 years there has been a dramatic increase in obesity in the United States. This slide set illustrates this trend by mapping the increased prevalence of obesity across each of the states. In 2007, only one state (Colorado) had a prevalence of obesity less than 20%. Thirty states had a prevalence equal to or greater than 25%; three of these states (Alabama, Mississippi and Tennessee) had a prevalence of obesity equal to or greater than 30%. The animated map below shows the United States obesity prevalence from 1985 through 2007.
• Why text messages are limited to 160 characters | Technology | Los Angeles Times – A look back to the beginnings of SMS in 1985 – Would the 160-character maximum be enough space to prove a useful form of communication? Having zero market research, they based their initial assumptions on two "convincing arguments," Hillebrand said. For one, they found that postcards often contained fewer than 150 characters. Second, they analyzed a set of messages sent through Telex, a then-prevalent telegraphy network for business professionals. Despite not having a technical limitation, Hillebrand said, Telex transmissions were usually about the same length as postcards.

These are my links for April 30th from 05:57 to 07:10:

• SIGUSR2 > The Power That is GNU Emacs – "If you've never been convinced before that Emacs is the text editor in which dreams are made from, or that inside Emacs there are unicorns manipulating your text, don't expect me to convince you."
• How To Be A Successful Evil Overlord – 100 remedies for the fatal flaws exhibited by famous evil overlords of the past. Also some business executives, I think.
• Google Could Have Caught Swine Flu Early | Wired Science – Google’s search data may have been able to provide an early warning of the swine flu outbreak — if the company had been looking in the right place. Last week, at the request of the Centers for Disease Control, Google took a retroactive look at its search data from Mexico. And there the team found a pre-media bump in telltale flu-related search terms (you know, “influenza + phlegm + coughing”) that was inconsistent with standard, seasonal flu trends.
• What Twitter Looks Like For Twitter Employees (SCREENSHOTS) – Some screen shots of current admin tools at Twitter for managing user accounts, blocks, whitelisting, suspensions, and user stats such as # follow attempts, # updates, #directs, etc
• Twitter Aggregator Sawhorse Media Raises Seed Round, Launches Pets, Celeb Sites | paidContent.org – "Channelized" feeds from curated lists of twitter sources.

These are my links for April 24th through April 27th:

• Gallery: Flickr users make accidental maps – tech – 27 April 2009 – New Scientist – Building map visualizations using geotagged photos from Flickr.
• Does Hewlett-Packard Need a Dose of Anarchy? – NYTimes.com – When your organization is built on lean, predictable operating excellence (cost cutting, procurement, scale), where do speculative, high risk, growth opportunities come from?
• JavaScript for hackers – Opera Developer Community – On Javascript parser and some eval examples that lead to XSS vulnerabilities
• Processing.js – Javascript port of Processing graphics library, requires Canvas.
• Pittsburgh Pattern Recognition – Face recognition technology startup from CMU, with a demo of tagging faces in video of all episodes of Star Trek (the original series).

These are my links for April 15th through April 17th:

• Paul Buchheit: Make your site faster and cheaper to operate in one easy step – Compress text files with gzip to reduce file size/bandwidth, the incremental cpu cost is usually low relative to the performance gain from lower network cost. Friendfeed uses nginx in front of main web servers for this.
• Jabbify – Free Comet web service and browser client for simple chat and streaming status applications.
• TinEye Image Search Engine – Idée Inc. – The Visual Search Company – Finds references to images online, starting with an original image. Attempts to use image analysis to be independent of scaling, cropping, and other common manipulations.
• All That Twitters Isn’t Gold: A Popular Web Application in Search of a Business Plan – Knowledge@Wharton – Business school take on Twitter and high growth, non-revenue consumer web startups.
• Almost Viral: A Hybrid Acquisition Strategy – "By being almost viral you can grow very cheaply, control your rate of growth and demographics, and get enough traffic to conduct meaningful experiments. Need to grow more slowly? Just decrease your daily ad spend. Need statistically significant results more quickly? Increase your daily ad spend. With a viral coefficient of 0.9 you’ve dealt with your acquisition risk. Rather than going fully viral and dealing with the operational difficulties, it might be worth your time to deal with other market risks: retention, engagement, and monetization. "

These are my links for April 13th through April 15th:

• On Netflix’s video streaming security framework – A black box look at Netflix video streaming implementation for "Watch Instantly" , built with Microsoft Silverlight for Windows/Mac or various hardware clients, and various authentication and DRM services + HTTPS
• Nati Shalom’s Blog: Designing a Scalable Twitter – A look at using in-memory data grid vs memcached architecture for a simplified Twitter-like service with goals of 10 billion tweets/day, 100 GB stored/day (with 10:1 compression), immutable tweets i.e., there are no updates, only inserts, client applications limited to 70 requests per hour.
• agile approach | World Bank Open API 2.0 Launched – Economic development indicators for 200 countries from World Bank. 1000 API calls/day free for non-commercial use, info at developer.worldbank.org
• "We think you’d also like…" and the Math of Suggestion – Teil 2 | Gründerszene – Simplified overview of recommendation algorithms used for suggesting alternative shopping choices.
• 15 Firefox add-ons for Web developers | Webware – CNET –

These are my links for April 9th from 08:07 to 17:53:

• IP address geolocation SQL database – IP address geolocation with MySQL by Marc-Andre Caron. He's done all the necessary legwork to solve this problem, putting together a free, monthly-updated MySQL dataset that will allow you to derive country, region, city, zip, latitude, and longitude from an IP address.
• Del.icio.us Finally Gets Some Respect from Yahoo – Probably Too Late – ReadWriteWeb –
• In the Event That You Have Accidentally Swallowed the Higgs Boson by Michael Rottman – The Morning News – "7. Do you feel protons decaying? Grand Unification may be occurring near your vital organs. "
• FT.com / Companies / UK companies – Dotcom veterans in Twitter ‘brains trust’ – "Mr Read has brought together a “brains trust” of advisers to Twitter Partners, including Brent Hoberman and Martha Lane Fox, founders of Lastminute.com; Saul Klein, a partner at Index Ventures, the London venture capitalists; and Toby Coppel, the former European vice-president at Yahoo."
• byteonic.com » What you cannot do using Java in Google App Engine – List of some restrictions on Java code running on GAE

These are my links for February 27th through February 28th:

• jurriaanpersyn.com — Database Sharding at Netlog, with MySQL and PHP – Discussion and slides from a presentation at FOSDEM 2009 on database sharding and practical considerations implementing high traffic web services at Netlog.
• The New York Times Explains Twitter…to the New York Times | Peter Kafka – An introductory overview to Twitter for the New York Times, from the New York Times. Includes two embedded slide decks.
• Microsoft teams and products on Twitter | Adam Kinney, Continuum Explorer –
• The Big List Of Search Engines & Their Employees On Twitters –
• The size of social networks | Primates on Facebook | The Economist – How many friends can you manage at once? Extrapolating from the brain sizes and social networks of apes, Dr Dunbar suggested that the size of the human brain allows stable networks of about 148. Rounded to 150, this has become famous as “the Dunbar number”.

These are my links for February 18th through February 19th:

• Single Google Query uses 1000 Machines in 0.2 seconds – Google Fellow Jeff Dean says from 1999-2009, while both search queries and processing power have gone up by a factor of 1000, latency has gone down from around 1000ms to 200ms. Crawler updates now take minutes compared to months in 1999. 1000 machines handle a single query, all in memory.
• Government 2.0: Tweeting the Talk, Walking the Walk « Adriel Hampton – List of twitter users in various government organizations.
• The Absurdly Artificial Divide Between Pure and Applied Research – Olivia Judson – NYTimes.com – I used to explain myself as an "applied research" guy, small "r", not big "R" pure research. Love theory and analysis but want to see it get used for something eventually.
• Amazon Web Services Developer Community : Load data into S3 via hard drives? – Amazon asks for feedback regarding the FedEx option for bulk data transfer. "We have heard a number of requests about sending hard drives to AWS to load into S3. If such a service would benefit your business, we’d like to learn more about your use case."
• Local Media in a Postmodern World, Part XCI, Advertising Loses Its Balance – On the shifts in supply and demand, buyers and sellers in advertising markets as media moves from 1-to-many to niche-oriented, many-to-many and sellers take control of their own online media and advertising campaigns

I ran into Tom Conrad at Barcamp Block a few weeks ago, which reminded me to go check out Pandora again. I’d been an early adopter when they introduced it to the folks at the very first Barcamp, but accidentally stopped using it a while back when I changed out the computer in my office that I’d been running it on. I recently swapped in another system, and among other things I have it running Pandora again.

I like being able to launch a station with a single suggestion and get a few hours of “more stuff like that”, especially when it turns up something I haven’t heard before.

Here’s a sample “station” that I’m listening to at the moment, constructed starting with a track from Rick Braun, which has turned up selections from Larry Carlton, Jeff Golub, Joyce Cooling, and the Brecker Brothers on its own.

It’s been a while since I’ve come across something I haven’t seen before online. Ms. Dewey fits the bill. It is a Flash-based application combining video clips of actress Janina Gavankar with Windows Live search.

As a search application, it’s fat, slow, and the query results aren’t great. However, as John Batelle observes, “clearly, search ain’t the point.” This is search with an flirty attitude, where the speed and quality of the results aren’t at the top of the priority list.

As short-attention-span theater goes, it’s quite entertaining.

If you can’t think of anything to search for, Ms. Dewey will fidget for a while and eventually reach out and tap on the screen. “Helloooo…type something here…”

It’s far more interesting to try some queries and check out the responses. I spent over half an hour typing in keywords to see what would come up, starting with some of the suggestions from Digg and Channel9. The application provides a semi-random set of video responses based on the search keywords, so you won’t always get the same reaction each time.

The whip and riding crop don’t always appear when you’d think, the lab coat seems to be keyed to science and math (try “partial differential equation”), and I’m not sure what brings on the automatic weapons.

“Ms. Dewey” also has a MySpace page with more video clips. The way the application is constructed, they can probably keep updating and adding responses as long as they want to.

I briefly tried using Ms. Dewey in place of Google, as a working search engine, but it takes too long to respond to a series of queries (have to wait for the video to play) and the search results aren’t great (Live is continuing to improve, though). At the moment this is a fun conceptual experiment.

I wonder if we’ll see a new category of search emphasizing style (entertainment, attitude, sex) over substance (relevance, speed, scope). Today’s version might already work for the occasional search user, but imagine Ms. Dewey with faster, non-blocking search results, a better search UI, and Google’s results. It all vaguely reminds me of a William Gibson novel.

Apple showed their “iTV” box at today’s media event. It’s intended to connect downloaded video to home televisions. Today was a pre-announcement, for a planned release next spring, so you can’t go out and buy one.

In any case, I can tell already that the iTV would probably pass the spouse test in our household. We have gone through an assortment of roll-your-own video and media server projects over the past few years, but it usually ends up that I’m the only one that can get BeyondTV / MythTV / Freevo / whatever running, which generally defeats the purpose.

By the spring, there will probably be competing form factors based on Vista as well. In any case, I’m sure they will be far more acceptable to the rest of our household than the DIY approach.

Update 09-12-2006 13:19PDT: Apparently it’s unclear whether existing video files can be added for use with the iTV (for example, DVDs ripped to disk or personal DV files). If it can only be used with iTunes purchases, this isn’t going to be very interesting.

Today SpiralFrog announced a free subscription-based music service. Subscribers will be able to download music to their music playing devices, but will need to listen to advertising presented on the SpiralFrog site periodically, to keep the music authorized. It sounds like the downloaded music would be WMA files, using Microsoft Windows Media DRM.

A couple of days ago, Engadget pointed at FairUse4WM, a Windows Media DRM 10 and 11 removal utility with a user friendly interface.

This FT article says that iPod+iTunes has the largest market share for legally authorized music at 80%. At the same time, it notes the growing number of non-iPod MP3 and other music players coming to market. I suspect it won’t be long before there’s a one-click utility to remove the Windows Media DRM, transcode the WMA file to MP3, and import them into iTunes so subscribers can listen on their iPod or whatever device. It probably won’t be from SpiralFrog, though.

The upcoming Zune music / video players from Microsoft are likely to have similar issues, whatever their online media network turns out to be.

I think it’s great that the music publishers are trying different business models, in this case advertising. On the other hand, I find I use services like Pandora for casual listening and finding new music, then buy the actual CDs of music I want rather than purchasing from iTunes, just so I have a clean, portable DRM-free audio file that can be shipped around the house and across whatever device happens to be convenient. I’d rather just buy clean, portable bits, without needing the physical CD. Where’s the service for that? (Other than allofmp3.com).

More on SpiralFrog from BoingBoing, TechCrunch

Update Tuesday 08-29-2006 21:16 PDT – I see that Microsoft is working on patching WinDRM to block FairUse4WM. (Good luck with that.) And on the iPod front, it looks like jHymn has been getting updates so it can work with iTunes 6 to remove the FairPlay DRM, making those files portable to non-iPod devices.

…but other sites are apparently blocked.

There are a fair number of readers here from India, where some ISPs have started blocking many blogs, including all of Typepad, Blogspot, Geocities. So you might have thought this site was also blocked if you came by yesterday, since you would have gotten something like “Connection refused” or a similar error message.

Fortunately / unfortunately, it’s just Dreamhost having some hardware and network problems, which took down many of their clients for several hours yesterday, and is still behaving badly today.

I’m amazed by the volume of discussion about Amanda Congdon, Andrew Baron, and the history and future (or not) of Rocketboom. I’m looking forward to seeing what either or both of them do going forward, like everyone else, but have nothing to add to the discussion other than best wishes.

However…the flap is also having the side effect of showing that just about everyone I “know” online has been watching Rocketboom. Check out the Technorati search for pointers to Amanda’s departure post and see how many names you recognize. Who knew?

Update 07-06-2006 16:20 PDT: Rocketboom the comic
Update 07-12-2006 21:42 PDT: Rocketboom is back on the air with new host Joanne Colan, here’s her debut.

I sometimes have CNBC on in my office, sort of as background noise. This afternoon I heard Maria Bartiromo describing her weekend conversation with Federal Reserve chairman Ben Bernanke.

The media and the markets basically got it wrong last week in speculating that the Fed is done raising interest rates.

I’m thinking to myself, “Hmm. That’s going to make some people uncomfortable with their positions.” Especially since this was pretty much the opposite of what people were thinking after last week’s Fed testimony.

Here’s the AP summary:

Last Thursday, Bernanke told a congressional panel the central bank could pause — but not necessarily stop — its string of rate hikes while it keeps a close watch on the economy’s health. However, according to CNBC, Bernanke said future increases will depend mostly on economic data; that stand was troubling to an interest rate-sensitive market.

A spokeswoman for the Federal Reserve, Michelle Smith, declined to comment on the CNBC report

Well, that was fun. Some quick thoughts:

• If this was intended as clarification of Fed policy, why not say so directly, instead of indirectly? And why not officially comment when given the opportunity?
• Why did CNBC sit on this all day, instead of reporting pre-open, when they already had the story in hand?
• Who profited from this? This would have been handy information to have in one’s pocket this morning…

You can currently watch the clip on the CNBC site (requires IE6).

Update 05-02-2006 1030PDT: Trader Mike has a thorough roundup of comments from various blogs.

The proposal on the table is to split Time Warner into four pieces, undoing years of mergers and acquisitions. The (massive) report from Carl Icahn’s investment banking team at Lazard is worth a look for anyone with an interest in online or traditional media businesses or who simply lived through the dot-com boom and crash. I’ve only skimmed through it so far, but it’s practically a textbook on the evolution and current state of the media industry.

TWX is at the center of the storm that has and will continue to jolt American industry. Technology, regulation and competition are changing at an accelerated pace. The markets are increasingly rewarding companies—across all industries—with a well-defined vision, as shareholder expectations on transparency, capital returns, appreciation and corporate governance increase. Against this backdrop, anticipating and harnessing change is critical for success.

If you want to get a quick look at market sizes, margins, and fees, this is a fascinating read. It’s packed with details comparing the financial and operating performance and market reach of AOL with Google, Yahoo, MSN, and other online properties, television properties such as HBO, CNN, Cartoon Network, Court TV, and others, print publishing for People, Time, and dozens of magazines, the Time Warner cable system, and the Warner Brothers movie business.

The proposed restructuring would create four new businesses: AOL, a television and film media business, a print publishing business, and Time Warner’s cable distribution business. I have no stake in TWX, but if I were a long time shareholder, I’d be wondering why I’m getting a lower return than holding cash, while I see hugely successful franchises (The Matrix, Harry Potter, AOL, HBO, People) operating in the various business units. Icahn only holds about 3% of the company, so the proposal doesn’t seem likely to succeed soon, but this is a pretty major prod for the TWX management team.

The 300-something-page report, along with various SEC filings, is available for free download from enhancetimewarner.com

More from Bloomberg, Business Week.

See also:

• Kara Swisher’s There Must Be A Pony In Here Somewhere
• Nina Munk’s Fools Rush In: Steve Case, Jerry Levin, and the Unmaking of AOL Time Warner