Ho John Lee | January 23rd, 2010 | Comments are closed
These are my links for January 20th through January 23rd:
Data.gov – Featured Datasets: Open Government Directive Agency – Datasets required under the Open Government Directive through the end of the day, January 22, 2010. Freedom of Information Act request logs, Treasury TARP and derivative activity logs, crime, income, agriculture datasets.
All Your Twitter Bot Needs Is Love – The bot’s name? Jason Thorton. He’s been humming along for months now, sending out over 1250 tweets to some 174 followers. His tweets, while not particularly creative, manage to be both believable and timely. And he’s powered by a single word: Love.
Thorton is the creation of developer Ryan Merket, who built him as a side project in around three hours. Merket has just posted the code that powers him, and has also divulged how he made Thorton seem somewhat realistic: the bot looks for tweets with the word “love” in them and tweets them as its own.
Building a Twitter Bot – "Meet Jason Thorton. To people who know Jason, he is a successful entrepreneur in San Francisco who tweets 4-5 times a day. But Jason has a secret, he’s not really a human, he’s the product of my simple algorithm in PHP
Jason tweets A LOT about the word “love” – that’s because Jason actually steals tweets from the public timeline that contain the word “love” and posts them as his own
Jason also @replies to people who use the word “love” in their tweets, and asks them random questions or says something arbitrary
It took me about 3 hours to code Jason, imagine what a real engineer could do with real AI algorithms? Now realize that it’s already a reality. Sites like Twitter are full of side projects, company initiatives, spambots and AI robots. When the free flow of information becomes open, the amount of disinformation increases. Theres a real need for someone to vet the people we ‘meet’ on social sites – will be interesting to see how this market grows in the next year
Website monitoring status – Public API Status – Health monitor for 26 APIs from popular Web services, including Google Search, Google Maps, Bing, Facebook, Twitter, SalesForce, YouTube, Amazon, eBay and others
PG&E Electrical System Outage Map – This map shows the current outages in our 70,000-square-mile service area. To see more details about an outage, including the cause and estimated time of restoration, click on the color-coded icon associated with that outage.
Ho John Lee | January 20th, 2010 | Comments are closed
These are my links for January 17th through January 20th:
PG&E Electrical System Outage Map – This map shows the current outages in our 70,000-square-mile service area. To see more details about an outage, including the cause and estimated time of restoration, click on the color-coded icon associated with that outage.
Twitter.com vs The Twitter Ecosystem – Fred Wilson comments on some data from John Borthwick indicating Twitter ecosystem use = 3-5x Twitter.com directly.
"John's chart estimates that Twitter.com is about 20mm uvs a month in the US (comScore has it at 60mm uvs worldwide) and the Twitter ecosystem at about 60mm uvs in the US.
That says that across all web services, not just AVC, the Twitter ecosystem is about 3x Twitter.com. And on this blog, whose audience is certainly power users, that ratio is 5x."
Chris Walshaw :: Research :: Partition Archive – Welcome to the University of Greenwich Graph Partitioning Archive. The archive consists of the best partitions found to date for a range of graphs and its aim is to provide a benchmark, against which partitioning algorithms can be tested, and a resource for experimentation.
The partition archive has been in operation since the year 2000 and includes results from most of the major graph partitioning software packages. Researchers developing experimental partitioning algorithms regularly submit new partitions for possible inclusion.
Most of the test graphs arise from typical partitioning applications, although the archive also includes results computed for a graph-colouring test suite [Wal04] contained in a separate annex.
The archive was originally set up as part of a research project into very high quality partitions and authors wishing to refer to the partitioning archive should cite the paper [SWC04].
Twitter’s Crawl « The Product Guy – "A list of incidents that affected the Page Load Time of the Twitter product, distinguishing between total downtime, and partial downtime and information inaccessibility, based upon the public posts on Twitters blog.
http://status.twitter.com/archive
I did my best to not double count any problems, but it was difficult since many of the problems occur so frequently, and it is often difficult to distinguish, from these status blog posts alone, between a persisting problem being experienced or fixed, from that of a new emergence of a similar or same problem. Furthermore, I also excluded the impact on Page Load Time arising from scheduled maintenance/downtime – periods of time over which the user expectation would be most aligned with the product’s promise of Page Load Time. "
Soundboard.com – Soundboard.com is the web's largest catalog of free sounds and soundboards – in over 20 categories, for mobile or PC. 252,858 free sounds on 17,171 soundboards from movies to sports, sound effects, television, celebrities, history and travel. Or build, customize, embed and manage your own
neo4j open source graph database – Sounds interesting for social media graph hacking. "Neo4j is a graph database. It is an embedded, disk-based, fully transactional Java persistence engine that stores data structured in graphs rather than in tables. A graph (mathematical lingo for a network) is a flexible data structure that allows a more agile and rapid style of development."
EJW.COM || SETTING THE EXAMPLE – Thoughts on URL shorteners from Eric James Woodward (tr.im / Nambu). tinyurl.com, bit.ly, tr.im etc know what everyone is shortening right now; is 15 minutes of lead time useful? Click throughs depend on audience size and can be gamed, etc.
Mathematical Atlas: A gateway to Mathematics – "The Mathematical Atlas is a collection of articles about aspects of mathematics at and above the university level, but (usually) not at the level of current research. The goal of this collection is to introduce the subject areas of modern mathematics, to describe a few of the milestone results and topics, and to give pointers to some of the key resources where further information is to be found. Like any good atlas, we try to present several ways to look at each area and to show its relationship with neighboring areas and sub-areas. "
Annals of Innovation: How David Beats Goliath: Reporting & Essays: The New Yorker – Malcolm Gladwell, with a reporter at large on Vivek Ranadivé and his NJB girls basketball team, employing asymmetric strategies to overcome conventionally stronger teams, and a broader look at the history of insurgent strategies from David and Goliath, T.E. Lawrence, George Washington, etc.
site admin | April 23rd, 2009 | Comments are closed
These are my links for April 20th through April 23rd:
What I’ve Learned from Hacker News – Paul Graham on social dynamics and managing Hacker News, user submitted comments and ranking (voting up/down) , editorial intervention and moderators, project goals.
Didn’t attend ETech this week, but thanks to a Twitter pointer from Gene Becker, I did take a few breaks to participate in a collaborative future forecasting experiment at the event, organized by Institute For the Future / Signtific Labs. The general idea is to enlist game players to offer Twitter-like short notes with outlier ideas regarding a scenario under discussion, in this case the consequences of inexpensive ($100) 1kg microsatellites (“CubeSats”) capable of high speed networking and remote sensing. The same game framework could be used for any scenario, though. Bonus points are awarded to “Super-Interesting” ideas and ideas that result in additional discussion, which helped me out on the scoreboard.
Gene (“ubik“) won a “Feynman” award on the first day, and I managed to end up with a high score at ETech, thus winning a lab coat to go with my “Genius” label.
Jurisdiction-free data haven built with csats full of rad-hard flash memory, hbase-style distributed replication across multiple nodes. Subpoena-proof anonymizers, for better or worse. Alternative, universal internet currency evolves, outside any government’s central bank control. Following forced disclosure of banking client list, Swiss government recognizes anonymous cSat net IDs, followed by Cayman, Bermuda etc.
CSats deorbited in vacant areas of oceans as impulse input to passive sonar imaging. Oceanographers get great maps, submarines lose stealth. Depending on how accurately you can drop a CSat, you can effectively “ping” a region and listen to the return signal through existing arrays. This really messes with strategic deterrence since now subs are vulnerable to first strike. But CSat deorbit is cheap WMD for all. On the positive side, detailed acoustic propagation data leads to new insights on ocean dynamics – bathymetrics, thermoclines, currents, etc. A similar version of dropping CSats on land might yield useful seismic imaging. But these would all be surface impulse, not at depth.
Csat data networks circumvent the Great Firewall of China and other govt access controls, leading to broader/safer citizen engagement online
CSat operating interface is marketed as a toy, like Tamagochi. Recharge, collect interesting data, avoid mean csats, team with friends. Organizations might post cash prize/rewards for things like locating missing ships, oil/trash dumping at sea, smokestack emissions, etc
Commodity traders are early adopters of CSat operator networks. Looking for crop yield data, mine production volumes, freight shipments etc. Among other things, CSat observations could give a more accurate estimate of “floating” oil parked in tankers as well as ongoing demand. Similarly, you’d get a decent idea of iron ore production by watching BHP’s railway in Australia, and the demand side in China, Korea etc. CSat data could improve the market visbility into supply/demand. But one might start creating Potemkin mining/farming operations etc… Sadly, credit derivative risk is not observable via CSat.
Ubiquitous, near real time satellite surveillance. No more privacy outdoors. But really good Google Maps. Ultra high resolution terrain maps of the world synthesized from multiple satellite passes/viewing aspects. Long term studies of effects of erosion, farming, development, earthquakes, flooding, drought, etc. Insurgents, militias, and terrorists get real time tactical data feeds, make use of homebrew UAVs, sensors, and in-field dispatch from afar. Turf wars among poppy and marijuana growers who now know where each other’s fields are. All vehicles – car, truck, rail, container, airplanes, etc – get a sky-facing ID plate. Maybe these should just be really big QR codes with an authoritative registry to foil car thieves from painting on bogus “plates”.
Now I need to figure out how to collect that lab coat.
Geek And Poke – Mostly twitter and cloud computing themed cartoons.
Official Google Blog: Here comes Google Voice – GrandCentral makes a comeback, after disappearing into Google a while back. Now with voice transcription, SMS folders, and integration with GMail address book.
Amazon Web Services Blog: Announcing Amazon EC2 Reserved Instances – AWS introduces pricing structure for longer term, reserved capacity. Upfront payment, plus a (lower) incremental hourly charge, net savings for continuous 24×7 clients, and guaranteed availability of instances for backup or surge capacity.
If browsers were women – Sharenator.org – "[Chrome] Extremely skinny, but very cool and friendly. However, when it comes to the bedroom, she is very inexperienced and has little to offer. [IE] For most, she's the first woman they tried. She's really easy but can get you infected." etc etc
FT.com | Tech Blog | How many computers does the world need? – According to Microsoft research chief Rick Rashid, around 20 per cent of all the servers sold around the world each year are now being bought by a small handful of internet companies – he named Microsoft, Google, Yahoo and Amazon.
The New Hot Cuisine: Korean – WSJ.com – Korean food is slowly making its way into mainstream awareness, both high end (French Laundry, Le Bernardin) and everyday (CPK, Kogi BBQ).
Beatles Unknown "A Hard Day’s Night" Chord Mystery Solved Using Fourier Transform – Four years ago, inspired by reading news coverage about the song’s 40th anniversary, Jason Brown of Dalhousie’s Department of Mathematics decided to try and see if he could apply a mathematical calculation known as Fourier transform to solve the Beatles’ riddle. The process allowed him to decompose the sound into its original frequencies using computer software and parse out which notes were on the record.
On the occasion of the Academy Awards, an interesting video montage featuring faces of famous actresses. via Instapundit
I’ve noticed that over the past several years, I basically don’t see movies in theaters, except for kids movies. Everything else is either on airplanes, cable, DVD, or online.
Both CNBC and TheStreet.com have separately reported problems with their respective stock trading contests.
The CNBC contest site apparently had a coding flaw that allowed traders to open an order entry window before market close, then wait until after hours to actually enter the order, which would be executed the closing price.
The more interesting comment from CNBC is this:
In addition, there have been allegations that one or more contestants may have engaged in illegal market manipulation to affect actual prices of stocks represented in their contest portfolios.
With a million dollars at stake, this seemed like a plausible scenario even before the contest started. It’s unclear to me whether “illegal market manipulation” here actually means “illegal under contest rules” or “illegal under SEC regulations”, but it seems quite feasible.
Along similar lines at TheStreet.com, which offered $100,000 in their contest:
The final results of the game indicate that players employed trading strategies to achieve returns that could not be duplicated in the real world, thereby depriving other contestants of an equal chance to win.
I don’t know why these large contests don’t just partner with a retail brokerage that already has a paper trading (“demo”) mode instead of building their own. Wouldn’t solve the market manipulation problem though.
Since last February’s market gymnastics, I regularly take a glance at the wacky Shanghai market in overnight trading to see if that particular financial asteroid will be landing in the morning. This evening I noticed that the free quotes on the US Yahoo site lag the quotes on the China Yahoo site:
Here’s the Shanghai market as seen on the US site:
Here’s the Shanghai market as seen on the Chinese site:
In both cases, they’re free, delayed quotes, and are correctly time stamped. I never noticed that the overseas Yahoo sites provide more current data for the local markets before, though. Now you can all impress your non-trading friends with your nearly psychic ability to call intraday turns in the Shanghai market.
It appears I didn’t win the CNBC stock contest, ending up at +30%. The leading portfolios were something like $4-5MM, so there is a huge spread in that top 2% bracket.
I’m currently in the top 4% at 14746 in the CNBC contest. Oddly, they don’t publish the portfolio values on their website, but did show the current portfolio values and standings on TV last Friday. The top positions are clustered around $2M, or a total return of around 100%. It would be interesting to know what the distribution of 1-week returns looked like. That’s probably the best opportunity for placing at this point.
In either case, I haven’t hit any 25% to 50% movers, which looks like a prequisite for placing in the standings so far.
It’s hard to find high volatility stocks that are likely to move up in this market. I made a little progress on the CNBC stock contest, but this is going to be a lot lower tomorrow when they update the results (seeing how the markets had another abysmal day). It would be a lot more interesting if you could take short positions. Everyone would probably be in NEW and LEND.
As an aside, the rules for the trading contest apparently don’t prohibit multiple accounts owned by the same person. Consequently, someone named “Nancy Beaumont” has entered hundreds of portfolios. By the rules, they can only win once, but having multiple high-volatility portfolios in the running significantly improves your chance of ending up with one or two successful outliers, sort of like buying multiple lottery tickets, but with much better odds. At the moment they’re occupying 8 of the top 20 rankings.
CNBC is running a stock trading contest starting today, with a prize of one million dollars for the best performance by May 25th. Signing up is free. Each participant gets a notional $1,000,000 to trade. There are a few non-obvious rules:
All trades are priced at end-of-day – no intraday trades, and no GTC limit orders.
No ETFs – this means no indexes, countries, sectors, or commodities, and no inverse trades
No mutual funds – again, no indexes, country, sector, commodity, or inverse vehicles
No options, futures, or other derivatives
Minimum market cap of $500 million as of the starting date – this is probably to keep people from winning by manipulating a thinly traded microcap stock.
In order to win this type of contest, you pretty much have to treat it like a free lottery ticket and make your selections accordingly. There is no risk-adjusted return to consider, and no downside to taking extreme losses. So on a short timeframe, and in a long-only US stock portfolio, you’re looking for something highly speculative that’s going to move a lot.
Before I read the rules, I was thinking that with a million real dollars at stake, someone would game the system by thrashing a microcap issue back and forth, but the $500MM market cap requirement makes it harder. Not out of the question, but probably not worthwhile for anyone in a position to control the stock price.
Since we can’t trade the easily manipulated microcap stocks, I’d look for low float, heavily shorted small cap stocks that have some positive opportunity for event or news driven movement. Some general areas to look:
Biotech development companies that have regulatory approval or conferences coming up
Despised companies that are potential buyout candidates
Disaster / event driven companies, such as avian flu, anti-terrorism, or whatever you can think of
Fad driven trades. Last year it was ethanol and energy drinks. It might be ethanol and alt-energy again this year
These are all exactly the opposite of good investment or trading approaches, but the contest has a binary outcome – either you win or you don’t. By taking on extreme levels of portfolio risk, you’re trying to increase the variance of portfolio returns during the contest period, in hope of coming up with the highest outlier by the end.
A contrarian approach might be to stay in cash or very conservative stocks, and hope that everyone else makes speculative bets that crash and burn.
Hmm. I just looked at the rules again, and they also have $10,000 prizes for each week’s best 1-week return. This actually increases the incentive for finding a more-or-less binary, news-driven trade each week (such as earnings, court ruling, regulatory approvals). If it works out, your portfolio goes up. If it goes down, it doesn’t affect your 1-week return for the following week.
I was going to run a scan for this over the weekend, but perhaps I should try to dig something up for this week in case something works out.
Needless to say, this isn’t how one should invest or trade with real money.
“Indexed” features sketches on 3×5 index cards, heavy on graphs and Venn diagrams. If you’re reading this, there’s probably something in there for you. (via Korean Jurist)
See also Gaping Void (blogging / tech cartoons drawn on business cards) and xkcd (math / grad student humor).
This popped up on YouTube this afternoon – filmmaker Ahree Lee shot an image of her face every day for three years starting in November 2001, then concatenated them into a fascinating short movie called “Me”. The project is set to music by Nathan Melsted, which give it a hypnotic, X-file-ish feeling.
A slightly longer and sharper version is posted at AtomFilms. There’s also a short related project built on lots of different faces, called “Everyone”.
