Ho John Lee | February 11th, 2010 | Comments are closed
These are my links for February 4th through February 11th:
Schneier on Security: Interview with a Nigerian Internet Scammer – "We had something called the recovery approach. A few months after the original scam, we would approach the victim again, this time pretending to be from the FBI, or the Nigerian Authorities. The email would tell the victim that we had caught a scammer and had found all of the details of the original scam, and that the money could be recovered. Of course there would be fees involved as well. Victims would often pay up again to try and get their money back."
PeteSearch: How to split up the US – Visualization of social network clusters in the US. "information by location, with connections drawn between places that share friends. For example, a lot of people in LA have friends in San Francisco, so there's a line between them.
Looking at the network of US cities, it's been remarkable to see how groups of them form clusters, with strong connections locally but few contacts outside the cluster. For example Columbus, OH and Charleston WV are nearby as the crow flies, but share few connections, with Columbus clearly part of the North, and Charleston tied to the South."
Redis: Lightweight key/value Store That Goes the Extra Mile | Linux Magazine – Sort of like memcache. "Calling redis a key/value store doesn’t quite due it justice. It’s better thought of as a “data structures” server that supports several native data types and operations on them. That’s pretty much how creator Salvatore Sanfilippo (known as antirez) describes it in the documentation. Let’s dig in and see how it works."
Op-Ed Contributor – Microsoft’s Creative Destruction – NYTimes.com – Unlike other companies, Microsoft never developed a true system for innovation. Some of my former colleagues argue that it actually developed a system to thwart innovation. Despite having one of the largest and best corporate laboratories in the world, and the luxury of not one but three chief technology officers, the company routinely manages to frustrate the efforts of its visionary thinkers.
Ho John Lee | February 4th, 2010 | Comments are closed
These are my links for January 30th through February 4th:
Op-Ed Contributor – Microsoft’s Creative Destruction – NYTimes.com – Unlike other companies, Microsoft never developed a true system for innovation. Some of my former colleagues argue that it actually developed a system to thwart innovation. Despite having one of the largest and best corporate laboratories in the world, and the luxury of not one but three chief technology officers, the company routinely manages to frustrate the efforts of its visionary thinkers.
Leonardo da Vinci’s Resume Explains Why He’s The Renaissance Man For the Job – Davinci – Gizmodo – At one time in history, even da Vinci himself had to pen a resume to explain why he was a qualified applicant. Here's a translation of his letter to the Duke of Milan, delineating his many talents and abilities. "Most Illustrious Lord, Having now sufficiently considered the specimens of all those who proclaim themselves skilled contrivers of instruments of war, and that the invention and operation of the said instruments are nothing different from those in common use: I shall endeavor, without prejudice to any one else, to explain myself to your Excellency, showing your Lordship my secret, and then offering them to your best pleasure and approbation to work with effect at opportune moments on all those things which, in part, shall be briefly noted below..The document, written when da Vinci was 30, is actually more of a cover letter than a resume; he leaves out many of his artistic achievements and instead focuses on what he can provide for the Duke in technologies of war.
jsMath: jsMath Home Page – The jsMath package provides a method of including mathematics in HTML pages that works across multiple browsers under Windows, Macintosh OS X, Linux and other flavors of unix. It overcomes a number of the shortcomings of the traditional method of using images to represent mathematics: jsMath uses native fonts, so they resize when you change the size of the text in your browser, they print at the full resolution of your printer, and you don't have to wait for dozens of images to be downloaded in order to see the mathematics in a web page. There are also advantages for web-page authors, as there is no need to preprocess your web pages to generate any images, and the mathematics is entered in TeX form, so it is easy to create and maintain your web pages. Although it works best with the TeX fonts installed, jsMath will fall back on a collection of image-based fonts (which can still be scaled or printed at high resolution) or unicode fonts when the TeX fonts are not available.
Josh on the Web » Blog Archive » Abusing the Cache: Tracking Users without Cookies – To track a user I make use of three URLs: the container, which can be any website; a shim file, which contains a unique code; and a tracking page, which stores (and in this case displays) requests. The trick lies in making the browser cache the shim file indefinitely. When the file is requested for the first – and only – time a unique identifier is embedded in the page. The shim embeds the tracking page, passing it the unique ID every time it is loaded. See the source code.
One neat thing about this method is that JavaScript is not strictly required. It is only used to pass the message and referrer to the tracker. It would probably be possible to replace the iframes with CSS and images to gain JS-free HTTP referrer logging but would lose the ability to store messages so easily.
Panopticlick – Your browser fingerprint appears to be unique among the 342,943 tested so far.
Currently, we estimate that your browser has a fingerprint that conveys at least 18.39 bits of identifying information.
The measurements we used to obtain this result are listed below. You can read more about the methodology here, and about some defenses against fingerprinting here
Is Twitter A Substitute For Set Top Box Data? – The post describes a little experiment Simulmedia did to analyze TV channel surfers using only Twitter posts as their data source. Simulmedia explains why they turned to Twitter data for this work:
NILFS: A File System to Make SSDs Scream | Linux Magazine – A log structured (as opposed to journaling) file system with some advantages for write performance and shorter crash recovery times. Good for large working file systems?
When Korean Culture Flourished – WSJ.com – In the geography of the Metropolitan Museum of Art, the gallery devoted to Korea acts as a sort of land bridge between China and South Asia that all too often serves as passage rather than destination. The first in a series of shows to be held over the next 10 to 15 years, "Art of the Korean Renaissance, 1400-1600" may change this. With only 47 objects(!), the exhibition explores a fertile 200-year period in Korea's cultural history, revealing as much through its choice of works as it does through the order in which it displays them. The show's modest size makes the point that, sadly, little has survived from this period, when the Joseon — or Fresh Dawn — dynasty (1392-1910) united the Korean peninsula militarily, established Confucianism as the national ideology and introduced a phonetic alphabet.
Axiis : Data Visualization Framework – Axiis provides both pre-built visualization components as well as abstract layout patterns and rendering classes that allow you to create your own unique visualizations. Axiis is built upon the Degrafa graphics framework and Adobe Flex 3.
Mathematical Atlas: A gateway to Mathematics – "The Mathematical Atlas is a collection of articles about aspects of mathematics at and above the university level, but (usually) not at the level of current research. The goal of this collection is to introduce the subject areas of modern mathematics, to describe a few of the milestone results and topics, and to give pointers to some of the key resources where further information is to be found. Like any good atlas, we try to present several ways to look at each area and to show its relationship with neighboring areas and sub-areas. "
Annals of Innovation: How David Beats Goliath: Reporting & Essays: The New Yorker – Malcolm Gladwell, with a reporter at large on Vivek Ranadivé and his NJB girls basketball team, employing asymmetric strategies to overcome conventionally stronger teams, and a broader look at the history of insurgent strategies from David and Goliath, T.E. Lawrence, George Washington, etc.
James Smith • loopj.com » Blog Archive » jQuery Plugin: Tokenizing Autocomplete Text Entry – Looks handy – "This is a jQuery plugin to allow users to select multiple items from a predefined list, using autocompletion as they type to find each item. You may have seen a similar type of text entry when filling in the recipients field sending messages on facebook."
Behind The Business Plan Of Pirates Inc. : NPR – It takes around $250K to fund a Somali pirate operation. About 20 percent goes to pay off officials who look the other way. About 50 percent is for expenses and payroll. The leader of an attack makes $10,000 to $20,000 (the average Somali family lives on $500 a year). The initial investor — who put in $250,000 of seed capital — gets 30 percent, sometimes up to $500,000.
Dilbert comic strip for 05/04/2009 from the official Dilbert comic strips archive. – Secretary to Pointy Haired Boss: "I live in a rented trailer and all of my money is in my checking account. Your investments are worthless and your mortgage is underwater. My net worth is higher than yours now. I guess promiscuity and a G.E.D. was a pretty good strategy after all." Reminded me of a thought I had earlier this year, that much of Western Civilization is built on valuing delayed gratification, which hasn't worked out so well recently as opposed to immediate consumption in many cases.
Without Warning, Twitter Kills StatTweets (Businesses Beware) – StatSheet.com ChangeLog – Owner of StatTweets post regarding his network of sports-related Twitter handles being banned. They had several hundred accounts, one for stats for each team. This makes sense for users, given the way Twitter works, but they don't like mass account creation. Interested to see how this sorts out, there seem to be at least a few similar Twitter networks with team/region/topic-specific handles.
Obesity and Overweight: Trends: U.S. Obesity Trends 1985-2007 | DNPAO | CDC – During the past 20 years there has been a dramatic increase in obesity in the United States. This slide set illustrates this trend by mapping the increased prevalence of obesity across each of the states. In 2007, only one state (Colorado) had a prevalence of obesity less than 20%. Thirty states had a prevalence equal to or greater than 25%; three of these states (Alabama, Mississippi and Tennessee) had a prevalence of obesity equal to or greater than 30%. The animated map below shows the United States obesity prevalence from 1985 through 2007.
Why text messages are limited to 160 characters | Technology | Los Angeles Times – A look back to the beginnings of SMS in 1985 – Would the 160-character maximum be enough space to prove a useful form of communication? Having zero market research, they based their initial assumptions on two "convincing arguments," Hillebrand said. For one, they found that postcards often contained fewer than 150 characters. Second, they analyzed a set of messages sent through Telex, a then-prevalent telegraphy network for business professionals. Despite not having a technical limitation, Hillebrand said, Telex transmissions were usually about the same length as postcards.
These are my links for April 30th through May 2nd:
FusionCharts Free – Animated Flash Charts and Graphs for ASP, PHP, ASP.NET, JSP, RoR and other web applications – Flash charting component that can be used to render data-driven & animated charts for your web applications and presentations. It is a cross-browser and cross-platform solution that can be used with PHP, Python, Ruby on Rails, ASP, ASP.NET, JSP, ColdFusion, simple HTML pages or even PowerPoint Presentations to deliver interactive and powerful flash charts. You do NOT need to know anything about Flash to use FusionCharts. All you need to know is the language you're programming in.
Raphaël—JavaScript Library – Raphaël is a small JavaScript library that should simplify your work with vector graphics on the web. If you want to create your own specific chart or image crop and rotate widget, for example, you can achieve it simply and easily with this library. Raphaël uses the SVG W3C Recommendation and VML as a base for creating graphics. This means every graphical object you create is also a DOM object, so you can attach JavaScript event handlers or modify them later. Raphaël’s goal is to provide an adapter that will make drawing vector art compatible cross-browser and easy.
Country Reports on Terrorism 2008 – U.S. law requires the Secretary of State to provide Congress, by April 30 of each year, a full and complete report on terrorism with regard to those countries and groups meeting criteria set forth in the legislation. This annual report is entitled Country Reports on Terrorism. Beginning with the report for 2004, it replaced the previously published Patterns of Global Terrorism.
SIGUSR2 > The Power That is GNU Emacs – "If you've never been convinced before that Emacs is the text editor in which dreams are made from, or that inside Emacs there are unicorns manipulating your text, don't expect me to convince you."
site admin | April 30th, 2009 | Comments are closed
These are my links for April 30th from 05:57 to 07:10:
SIGUSR2 > The Power That is GNU Emacs – "If you've never been convinced before that Emacs is the text editor in which dreams are made from, or that inside Emacs there are unicorns manipulating your text, don't expect me to convince you."
How To Be A Successful Evil Overlord – 100 remedies for the fatal flaws exhibited by famous evil overlords of the past. Also some business executives, I think.
Google Could Have Caught Swine Flu Early | Wired Science – Google’s search data may have been able to provide an early warning of the swine flu outbreak — if the company had been looking in the right place. Last week, at the request of the Centers for Disease Control, Google took a retroactive look at its search data from Mexico. And there the team found a pre-media bump in telltale flu-related search terms (you know, “influenza + phlegm + coughing”) that was inconsistent with standard, seasonal flu trends.
What Twitter Looks Like For Twitter Employees (SCREENSHOTS) – Some screen shots of current admin tools at Twitter for managing user accounts, blocks, whitelisting, suspensions, and user stats such as # follow attempts, # updates, #directs, etc
joshua’s blog: on url shorteners – Joshua Schachter (founder of del.icio.us) summary on the state of URL shorteners (tinyurl, bit.ly, etc), and issues with 3rd party redirects, link sharing through twitter, etc.
Stochastic Models of User-Contributory Web Sites – Tad Hogg, Kristina Lerman 31 Mar 2009 Abstract: We describe a general stochastic processes-based approach to modeling user-contributory web sites, where users create, rate and share content. These models describe aggregate measures of activity and how they arise from simple models of individual users. This approach provides a tractable method to understand user activity on the web site and how this activity depends on web site design choices, especially the choice of what information about other users' behaviors is shown to each user. We illustrate this modeling approach in the context of user-created content on the news rating site Digg.
site admin | March 16th, 2009 | Comments are closed
These are my links for March 12th through March 16th:
Aig Systemic 090309 – AIG memo outlining systemic risks of their various businesses to the global economy.
FT.com / Companies / Insurance – AIG publishes counterparty list – AIG caved in to political pressure Sunday and released a list of some of the financial counterparties that benefited from its $160bn US government rescue, including some of Europe’s largest banks.
site admin | March 1st, 2009 | Comments are closed
These are my links for February 28th through March 1st:
Community Data – Swivel – User contributed datasets, for visualization and graphs with Swivel
Obamameter – Map visualization of economic stimulus outlays. "Keep tabs on the the US economy, the global economy and the stimulus through our dashboard for the economy."
recovery.gov.pdf – Slide presentation on data sources and construction of initial Recover.gov site in Jan 2009, from talk at Transparency Camp.
Virtual Hoff : DoxPara Research – Slides from Dan Kaminsky's talk at CloudCamp Seattle on network and application security issues in cloud and virtualized computing environments.
Can You Buy a Silicon Valley? Maybe. – from Paul Graham – "If you could get startups to stick to your town for a million apiece, then for a billion dollars you could bring in a thousand startups. That probably wouldn't push you past Silicon Valley itself, but it might get you second place. For the price of a football stadium, any town that was decent to live in could make itself one of the biggest startup hubs in the world."
Python for Lisp Programmers – Peter Norvig examines Python. "(Although it wasn't my intent, Python programers have told me this page has helped them learn Lisp.) Basically, Python can be seen as a dialect of Lisp with "traditional" syntax (what Lisp people call "infix" or "m-lisp" syntax). One message on comp.lang.python said "I never understood why LISP was a good idea until I started playing with python." Python supports all of Lisp's essential features except macros, and you don't miss macros all that much because it does have eval, and operator overloading, and regular expression parsing, so you can create custom languages that way. "
Beatles Unknown "A Hard Day’s Night" Chord Mystery Solved Using Fourier Transform – Four years ago, inspired by reading news coverage about the song’s 40th anniversary, Jason Brown of Dalhousie’s Department of Mathematics decided to try and see if he could apply a mathematical calculation known as Fourier transform to solve the Beatles’ riddle. The process allowed him to decompose the sound into its original frequencies using computer software and parse out which notes were on the record.
Recipe for Disaster: The Formula That Killed Wall Street – Felix Salmon writes about David Li, the Gaussian copula function, and its widespread (mis)use in pricing derivatives based on correlation only, leading to parts of today's financial crisis.
FRONTLINE: inside the meltdown: watch the full program – "On Thursday, Sept. 18, 2008, the astonished leadership of the U.S. Congress was told in a private session by the chairman of the Federal Reserve that the American economy was in grave danger of a complete meltdown within a matter of days. "There was literally a pause in that room where the oxygen left," says Sen. Christopher Dodd"
The Dark Matter of a Startup – "Every successful startup that I have seen has someone within their ranks that just kinda “does stuff.” No one really knows specifically what they do, but its vital to the success of the startup."
Why I Hate Frameworks – "A hammer?" he asks. "Nobody really buys hammers anymore. They're kind of old fashioned…we started selling schematic diagrams for hammer factories, enabling our clients to build their own hammer factories, custom engineered to manufacture only the kinds of hammers that they would actually need."
Mining The Thought Stream – Lots of comments around what is Twitter good for and how will it make money, revolving around real/near-time search, analytics, marketing, etc.
Understanding Web Operations Culture – the Graph & Data Obsession … – Comparison of traffic at Flickr, Google, Twitter, last.fm during the Obama inauguration. "One of the most interesting parts of running a large website is watching the effects of unrelated events affecting user traffic in aggregate."
site admin | February 16th, 2009 | Comments are closed
These are my links for February 15th through February 16th:
Berkeley cloud report gets mixed reviews | The Wisdom of Clouds – CNET News – James Urqhardt commentary on UCB paper, "The paper begins by setting a definition of Cloud Computing that will be considered controversial by many, as it is firmly in the "there is no cloud computing inside enterprise data centers" camp."
What really happened at Ma.gnolia and lessons learned – Video podcast with Larry Halff describing how Ma.gnolia was implemented (Ruby on Rails), its ongoing operation leading up to the failure of the (1/2 TB) MySQL database a few weeks ago.
Ho John Lee | November 14th, 2007 | Comments are closed
Another day, another subprime-related fiasco. Today GE Asset Management announced that one of its not-quite-money-market short bond funds, the Enhanced Cash Trust, took a loss from subprime holdings, and is offering customer redemptions at 96 cents on the dollar. Normally these funds are considered to be a higher-yielding version of a money market fund. This would make you pretty unhappy if you were looking for 5%-ish stable returns while waiting for the stock market to settle down.
Along these lines, here are British comedians John Fortune & John Bird chatting about the state of the banking system, Northern Rock, and subprime in another interview of “George Parr, investment banker” from last month.
Here’s a look at today’s market heat map after the close. There’s a lot of green, but this was pretty unenthusiastic.
I wasn’t too impressed with John Thain on CNBC today explaining yesterday’s weirdness with the NYSE trading and reporting systems. The official story seems to be that the systems that compute the DJIA got backlogged with transactions, and separately an internal messaging system for floor traders also got backlogged, so they’re upgrading their servers. Today they had to delay closing a number of stocks to allow transaction queues to clear at the end of the day. Thain mentioned a volume of over 20,000 msgs/second. This is all plausible, but not reassuring. It sounds like something that would happen to a growing e-commerce site, not one of the world’s largest stock exchanges. I wonder how much reserve throughput they can actually deploy. If we get a “real” market crash, they’re going to have to handle a lot more than the 2.4 billion shares they traded yesterday.
It’s interesting to observe that diversification across asset classes and markets didn’t help you today. All 30 Dow stocks closed down. 99 of 100 Nasdaq-100 stocks closed down. Nearly all of the S&P 500 closed down. Oil, gold, and other commodities closed down. Emerging markets closed down. Basically, equities and commodities got sold, and the proceeds went to cash and bonds. (Update – here’s the summary from today’s Worden Report: “Zero Industry groups advanced while 239 declined. There was one winner in the Nasdaq-100, two in the SP-500 and zero in the Dow. HalfPoint+ Movers were seven against 2174. The Leadership Index was 34 versus 2228.”)
One nominal trigger for today’s selling was a 9% drop on the Shanghai exchange, but there have been any number of reasons to be concerned and raise a little cash for a while.
In general, diversifying an investment portfolio across asset classes and markets reduces overall risk for an equivalent level of returns. This works because the price behavior for different markets is supposed to be relatively uncorrelated over time. Lately, disparate markets have been more correlated than in the past, mostly going up. Today the risk was clearly to the down side, making it likely that your investment portfolio closed lower today, unless you were in cash or bonds. (I’m pleased to have reduced my trading positions in India and China over the past couple of weeks.)
Another unexpected systemic risk exposed today was in the odd behavior of the NYSE around 3pm. The new hybrid (electronic and open outcry) trading system was apparently getting backed up due to heavy order flow this afternoon. The DJIA appeared to gap down by 180 points when the backlog was cleared. Anyone trading intraday off a NYSE data feed probably had some problems. (Update – here’s TraderMike with more detail)
I think the price action today is overdone, but I’m also happy to have exited many of my positions in India, China, and other emerging markets over the past few weeks. This is a good time to think about where to invest after the dust settles, or focus on short term and day trading. (Some of you may be interested in using the Ultra and Ultrashort ETFs.)
Two of the most important skills for long term investment success are capital allocation and risk management.
Capital allocation tends to get most of the attention in the press, which publishes headlines like “Where To Put Your Money Now”, or “10 Hot Stocks To Watch”, or perhaps a list of recently funded startups. This is especially true when markets are good and everything is going up.
Risk management isn’t nearly as interesting to the average person. It’s much more interesting to hear about the 800% return in titanium stocks over the past year than the possibility of losing money.
The past couple of weeks have reacquainted investors with market risk. For new investors who haven’t seen a down market in action, this is especially painful. The Indian stock market has been popular with emerging market investors in the US, as well as in India. It had been up around 50% since the beginning of the year but dropped 10% in a single session, and around 30% in a few days last week, triggering margin calls and placing police on suicide watch. The decline in US markets has been tame by comparison.
This afternoon the Enron trial came to an end, with Jeff Skilling and Ken Lay found guilty of fraud, conspiracy, and other charges in the collapse of what was once a $68 billion dollar company.
It wouldn’t have prevented you from losing money, but it would have kept you solvent. The buy-and-hold employee and retiree shareholders probably weren’t so lucky.
I recently became aware of a risk I hadn’t thought of in a while – platform risk. Yesterday I had a profitable short term trade locked in with a protective stop which failed to trigger when the share price turned and passed through the stop level. I asked Schwab to investigate, and it turns out they may be having intermittent problems with certain trailing stops entered through the trading window rather than the custom alerts window in Streetsmart Pro, their client application. They adjusted my trade to reflect the correct stop value, which was logged on their server but not executed.
Trading with unreliable stops in the current market is like driving a car with brakes that don’t always work. I appreciate Schwab making good on the trades, but I hadn’t considered the possibility of correctly entered stops not being executed when hit before yesterday. Kind of like people hadn’t considered the possibility of widespread fraud at Enron before it blew up.
Asset allocation is important, but risk management can keep you in the game when the unexpected inevitably happens.
