Bookmarks for January 20th through January 23rd

These are my links for January 20th through January 23rd:

  • Data.gov – Featured Datasets: Open Government Directive Agency – Datasets required under the Open Government Directive through the end of the day, January 22, 2010. Freedom of Information Act request logs, Treasury TARP and derivative activity logs, crime, income, agriculture datasets.
  • All Your Twitter Bot Needs Is Love – The bot’s name? Jason Thorton. He’s been humming along for months now, sending out over 1250 tweets to some 174 followers. His tweets, while not particularly creative, manage to be both believable and timely. And he’s powered by a single word: Love.

    Thorton is the creation of developer Ryan Merket, who built him as a side project in around three hours. Merket has just posted the code that powers him, and has also divulged how he made Thorton seem somewhat realistic: the bot looks for tweets with the word “love” in them and tweets them as its own.

  • Building a Twitter Bot – "Meet Jason Thorton. To people who know Jason, he is a successful entrepreneur in San Francisco who tweets 4-5 times a day. But Jason has a secret, he’s not really a human, he’s the product of my simple algorithm in PHP

    Jason tweets A LOT about the word “love” – that’s because Jason actually steals tweets from the public timeline that contain the word “love” and posts them as his own

    Jason also @replies to people who use the word “love” in their tweets, and asks them random questions or says something arbitrary

    It took me about 3 hours to code Jason, imagine what a real engineer could do with real AI algorithms? Now realize that it’s already a reality. Sites like Twitter are full of side projects, company initiatives, spambots and AI robots. When the free flow of information becomes open, the amount of disinformation increases. Theres a real need for someone to vet the people we ‘meet’ on social sites – will be interesting to see how this market grows in the next year

  • Website monitoring status – Public API Status – Health monitor for 26 APIs from popular Web services, including Google Search, Google Maps, Bing, Facebook, Twitter, SalesForce, YouTube, Amazon, eBay and others
  • PG&E Electrical System Outage Map – This map shows the current outages in our 70,000-square-mile service area. To see more details about an outage, including the cause and estimated time of restoration, click on the color-coded icon associated with that outage.

Bookmarks for January 17th through January 20th

These are my links for January 17th through January 20th:

  • PG&E Electrical System Outage Map – This map shows the current outages in our 70,000-square-mile service area. To see more details about an outage, including the cause and estimated time of restoration, click on the color-coded icon associated with that outage.
  • Twitter.com vs The Twitter Ecosystem – Fred Wilson comments on some data from John Borthwick indicating Twitter ecosystem use = 3-5x Twitter.com directly.

    "John's chart estimates that Twitter.com is about 20mm uvs a month in the US (comScore has it at 60mm uvs worldwide) and the Twitter ecosystem at about 60mm uvs in the US.

    That says that across all web services, not just AVC, the Twitter ecosystem is about 3x Twitter.com. And on this blog, whose audience is certainly power users, that ratio is 5x."

  • Chris Walshaw :: Research :: Partition Archive – Welcome to the University of Greenwich Graph Partitioning Archive. The archive consists of the best partitions found to date for a range of graphs and its aim is to provide a benchmark, against which partitioning algorithms can be tested, and a resource for experimentation.

    The partition archive has been in operation since the year 2000 and includes results from most of the major graph partitioning software packages. Researchers developing experimental partitioning algorithms regularly submit new partitions for possible inclusion.

    Most of the test graphs arise from typical partitioning applications, although the archive also includes results computed for a graph-colouring test suite [Wal04] contained in a separate annex.

    The archive was originally set up as part of a research project into very high quality partitions and authors wishing to refer to the partitioning archive should cite the paper [SWC04].

  • Twitter’s Crawl « The Product Guy – "A list of incidents that affected the Page Load Time of the Twitter product, distinguishing between total downtime, and partial downtime and information inaccessibility, based upon the public posts on Twitters blog.

    http://status.twitter.com/archive

    I did my best to not double count any problems, but it was difficult since many of the problems occur so frequently, and it is often difficult to distinguish, from these status blog posts alone, between a persisting problem being experienced or fixed, from that of a new emergence of a similar or same problem. Furthermore, I also excluded the impact on Page Load Time arising from scheduled maintenance/downtime – periods of time over which the user expectation would be most aligned with the product’s promise of Page Load Time. "

  • Soundboard.com – Soundboard.com is the web's largest catalog of free sounds and soundboards – in over 20 categories, for mobile or PC. 252,858 free sounds on 17,171 soundboards from movies to sports, sound effects, television, celebrities, history and travel. Or build, customize, embed and manage your own

Bookmarks for May 8th through May 12th

These are my links for May 8th through May 12th:

Bookmarks for May 6th through May 7th

These are my links for May 6th through May 7th:

Bookmarks for April 20th through April 23rd

These are my links for April 20th through April 23rd:

Genius, in search of lab coat

hjl-signtific-lab-profile-top

Didn’t attend ETech this week, but thanks to a Twitter pointer from Gene Becker,  I did take a few breaks to participate in a collaborative future forecasting experiment at the event, organized by Institute For the Future / Signtific Labs. The general idea is to enlist game players to offer Twitter-like short notes with outlier ideas regarding a scenario under discussion, in this case the consequences of inexpensive ($100) 1kg microsatellites (“CubeSats”) capable of high speed networking and remote sensing. The same game framework could be used for any scenario, though. Bonus points are awarded to “Super-Interesting” ideas and ideas that result in additional discussion, which helped me out on the scoreboard.

Gene (“ubik“) won a “Feynman” award on the first day, and I managed to end up with a high score at ETech, thus winning a lab coat to go with my “Genius” label.

Some of my favorite future forecast contributions from “What will you do when space is as cheap and accessible as the Web is today?” (slide summary here):

Jurisdiction-free data haven built with csats full of rad-hard flash memory, hbase-style distributed replication across multiple nodes. Subpoena-proof anonymizers, for better or worse. Alternative, universal internet currency evolves, outside any government’s central bank control. Following forced disclosure of banking client list, Swiss government recognizes anonymous cSat net IDs, followed by Cayman, Bermuda etc.

CSats deorbited in vacant areas of oceans as impulse input to passive sonar imaging. Oceanographers get great maps, submarines lose stealth. Depending on how accurately you can drop a CSat, you can effectively “ping” a region and listen to the return signal through existing arrays. This really messes with strategic deterrence since now subs are vulnerable to first strike. But CSat deorbit is cheap WMD for all. On the positive side, detailed acoustic propagation data leads to new insights on ocean dynamics – bathymetrics, thermoclines, currents, etc. A similar version of dropping CSats on land might yield useful seismic imaging. But these would all be surface impulse, not at depth.

Csat data networks circumvent the Great Firewall of China and other govt access controls, leading to broader/safer citizen engagement online

CSat operating interface is marketed as a toy, like Tamagochi. Recharge, collect interesting data, avoid mean csats, team with friends. Organizations might post cash prize/rewards for things like locating missing ships, oil/trash dumping at sea, smokestack emissions, etc

Commodity traders are early adopters of CSat operator networks. Looking for crop yield data, mine production volumes, freight shipments etc. Among other things, CSat observations could give a more accurate estimate of “floating” oil parked in tankers as well as ongoing demand. Similarly, you’d get a decent idea of iron ore production by watching BHP’s railway in Australia, and the demand side in China, Korea etc. CSat data could improve the market visbility into supply/demand. But one might start creating Potemkin mining/farming operations etc… Sadly, credit derivative risk is not observable via CSat.

Ubiquitous, near real time satellite surveillance. No more privacy outdoors. But really good Google Maps. Ultra high resolution terrain maps of the world synthesized from multiple satellite passes/viewing aspects. Long term studies of effects of erosion, farming, development, earthquakes, flooding, drought, etc. Insurgents, militias, and terrorists get real time tactical data feeds, make use of homebrew UAVs, sensors, and in-field dispatch from afar. Turf wars among poppy and marijuana growers who now know where each other’s fields are. All vehicles – car, truck, rail, container, airplanes, etc – get a sky-facing ID plate. Maybe these should just be really big QR codes with an authoritative registry to foil car thieves from painting on bogus “plates”.

Now I need to figure out how to collect that lab coat.

Bookmarks for March 9th through March 12th

These are my links for March 9th through March 12th:

Bookmarks for March 6th through March 8th

These are my links for March 6th through March 8th:

Bookmarks for March 1st from 22:05 to 22:33

These are my links for March 1st from 22:05 to 22:33:

Bookmarks for February 26th through February 27th

These are my links for February 26th through February 27th:

Women in film

On the occasion of the Academy Awards, an interesting video montage featuring faces of famous actresses. via Instapundit 

I’ve noticed that over the past several years, I basically don’t see movies in theaters, except for kids movies. Everything else is either on airplanes, cable, DVD, or online.

Hacking the stock trading contests

Both CNBC and TheStreet.com have separately reported problems with their respective stock trading contests.

The CNBC contest site apparently had a coding flaw that allowed traders to open an order entry window before market close, then wait until after hours to actually enter the order, which would be executed the closing price.

The more interesting comment from CNBC is this:

In addition, there have been allegations that one or more contestants may have engaged in illegal market manipulation to affect actual prices of stocks represented in their contest portfolios.

With a million dollars at stake, this seemed like a plausible scenario even before the contest started. It’s unclear to me whether “illegal market manipulation” here actually means “illegal under contest rules” or “illegal under SEC regulations”, but it seems quite feasible.

Along similar lines at TheStreet.com, which offered $100,000 in their contest:

The final results of the game indicate that players employed trading strategies to achieve returns that could not be duplicated in the real world, thereby depriving other contestants of an equal chance to win.

I don’t know why these large contests don’t just partner with a retail brokerage that already has a paper trading (“demo”) mode instead of building their own. Wouldn’t solve the market manipulation problem though.

See also:

Reliably predict the future of the Shanghai market!

Since last February’s market gymnastics, I regularly take a glance at the wacky Shanghai market in overnight trading to see if that particular financial asteroid will be landing in the morning. This evening I noticed that the free quotes on the US Yahoo site lag the quotes on the China Yahoo site:

Here’s the Shanghai market as seen on the US site:

Here’s the Shanghai market as seen on the Chinese site:

In both cases, they’re free, delayed quotes, and are correctly time stamped. I never noticed that the overseas Yahoo sites provide more current data for the local markets before, though. Now you can all impress your non-trading friends with your nearly psychic ability to call intraday turns in the Shanghai market.

Not bad for real money, but not enough for paper trading


It appears I didn’t win the CNBC stock contest, ending up at +30%. The leading portfolios were something like $4-5MM, so there is a huge spread in that top 2% bracket.

Insufficient volatility to win


I’m currently in the top 4% at 14746 in the CNBC contest. Oddly, they don’t publish the portfolio values on their website, but did show the current portfolio values and standings on TV last Friday. The top positions are clustered around $2M, or a total return of around 100%. It would be interesting to know what the distribution of 1-week returns looked like. That’s probably the best opportunity for placing at this point.

In either case, I haven’t hit any 25% to 50% movers, which looks like a prequisite for placing in the standings so far.

Moving up in the standings, but not for long


It’s hard to find high volatility stocks that are likely to move up in this market. I made a little progress on the CNBC stock contest, but this is going to be a lot lower tomorrow when they update the results (seeing how the markets had another abysmal day). It would be a lot more interesting if you could take short positions. Everyone would probably be in NEW and LEND.

As an aside, the rules for the trading contest apparently don’t prohibit multiple accounts owned by the same person. Consequently, someone named “Nancy Beaumont” has entered hundreds of portfolios. By the rules, they can only win once, but having multiple high-volatility portfolios in the running significantly improves your chance of ending up with one or two successful outliers, sort of like buying multiple lottery tickets, but with much better odds. At the moment they’re occupying 8 of the top 20 rankings.

How to win a million dollars


CNBC is running a stock trading contest starting today, with a prize of one million dollars for the best performance by May 25th. Signing up is free. Each participant gets a notional $1,000,000 to trade. There are a few non-obvious rules:

  • All trades are priced at end-of-day – no intraday trades, and no GTC limit orders.
  • No ETFs – this means no indexes, countries, sectors, or commodities, and no inverse trades
  • No mutual funds – again, no indexes, country, sector, commodity, or inverse vehicles
  • No options, futures, or other derivatives
  • Minimum market cap of $500 million as of the starting date – this is probably to keep people from winning by manipulating a thinly traded microcap stock.

In order to win this type of contest, you pretty much have to treat it like a free lottery ticket and make your selections accordingly. There is no risk-adjusted return to consider, and no downside to taking extreme losses. So on a short timeframe, and in a long-only US stock portfolio, you’re looking for something highly speculative that’s going to move a lot.

Before I read the rules, I was thinking that with a million real dollars at stake, someone would game the system by thrashing a microcap issue back and forth, but the $500MM market cap requirement makes it harder. Not out of the question, but probably not worthwhile for anyone in a position to control the stock price.

Since we can’t trade the easily manipulated microcap stocks, I’d look for low float, heavily shorted small cap stocks that have some positive opportunity for event or news driven movement. Some general areas to look:

  • Biotech development companies that have regulatory approval or conferences coming up
  • Despised companies that are potential buyout candidates
  • Disaster / event driven companies, such as avian flu, anti-terrorism, or whatever you can think of
  • Fad driven trades. Last year it was ethanol and energy drinks. It might be ethanol and alt-energy again this year

These are all exactly the opposite of good investment or trading approaches, but the contest has a binary outcome – either you win or you don’t. By taking on extreme levels of portfolio risk, you’re trying to increase the variance of portfolio returns during the contest period, in hope of coming up with the highest outlier by the end.

A contrarian approach might be to stay in cash or very conservative stocks, and hope that everyone else makes speculative bets that crash and burn.

Hmm. I just looked at the rules again, and they also have $10,000 prizes for each week’s best 1-week return. This actually increases the incentive for finding a more-or-less binary, news-driven trade each week (such as earnings, court ruling, regulatory approvals). If it works out, your portfolio goes up. If it goes down, it doesn’t affect your 1-week return for the following week.

I was going to run a scan for this over the weekend, but perhaps I should try to dig something up for this week in case something works out.

Needless to say, this isn’t how one should invest or trade with real money.

Venn diagram humor


Indexed” features sketches on 3×5 index cards, heavy on graphs and Venn diagrams. If you’re reading this, there’s probably something in there for you. (via Korean Jurist)

See also Gaping Void (blogging / tech cartoons drawn on business cards) and xkcd (math / grad student humor).

Three years in 3 minutes – a self portrait movie


This popped up on YouTube this afternoon – filmmaker Ahree Lee shot an image of her face every day for three years starting in November 2001, then concatenated them into a fascinating short movie called “Me”. The project is set to music by Nathan Melsted, which give it a hypnotic, X-file-ish feeling.

A slightly longer and sharper version is posted at AtomFilms. There’s also a short related project built on lots of different faces, called “Everyone”.

Star Trek and the Knights of the Round Table


“Knights, I bid you welcome to your new home…Camelot!”

An amazing mashup of Monty Python’s “Knights of the Round Table” song, with singing and dancing by the cast of the original Star Trek.

If this doesn’t leave you rolling around on the floor, it probably means you’re completely baffled and/or younger than 30 or so.

Try naming the episodes for extra credit…

via The Big Picture

Page 1 of 212