Ho John Lee's Weblog

These are my links for February 4th through February 11th:

• Schneier on Security: Interview with a Nigerian Internet Scammer – "We had something called the recovery approach. A few months after the original scam, we would approach the victim again, this time pretending to be from the FBI, or the Nigerian Authorities. The email would tell the victim that we had caught a scammer and had found all of the details of the original scam, and that the money could be recovered. Of course there would be fees involved as well. Victims would often pay up again to try and get their money back."
• xkcd – Frequency of Strip Versions of Various Games – n = Google hits for "strip <game name>" / Google hits for "<game name>"
• PeteSearch: How to split up the US – Visualization of social network clusters in the US. "information by location, with connections drawn between places that share friends. For example, a lot of people in LA have friends in San Francisco, so there's a line between them.

  Looking at the network of US cities, it's been remarkable to see how groups of them form clusters, with strong connections locally but few contacts outside the cluster. For example Columbus, OH and Charleston WV are nearby as the crow flies, but share few connections, with Columbus clearly part of the North, and Charleston tied to the South."

• Redis: Lightweight key/value Store That Goes the Extra Mile | Linux Magazine – Sort of like memcache. "Calling redis a key/value store doesn’t quite due it justice. It’s better thought of as a “data structures” server that supports several native data types and operations on them. That’s pretty much how creator Salvatore Sanfilippo (known as antirez) describes it in the documentation. Let’s dig in and see how it works."
• Op-Ed Contributor – Microsoft’s Creative Destruction – NYTimes.com – Unlike other companies, Microsoft never developed a true system for innovation. Some of my former colleagues argue that it actually developed a system to thwart innovation. Despite having one of the largest and best corporate laboratories in the world, and the luxury of not one but three chief technology officers, the company routinely manages to frustrate the efforts of its visionary thinkers.

These are my links for January 30th through February 4th:

• Op-Ed Contributor – Microsoft’s Creative Destruction – NYTimes.com – Unlike other companies, Microsoft never developed a true system for innovation. Some of my former colleagues argue that it actually developed a system to thwart innovation. Despite having one of the largest and best corporate laboratories in the world, and the luxury of not one but three chief technology officers, the company routinely manages to frustrate the efforts of its visionary thinkers.
• Leonardo da Vinci’s Resume Explains Why He’s The Renaissance Man For the Job – Davinci – Gizmodo – At one time in history, even da Vinci himself had to pen a resume to explain why he was a qualified applicant. Here's a translation of his letter to the Duke of Milan, delineating his many talents and abilities. "Most Illustrious Lord, Having now sufficiently considered the specimens of all those who proclaim themselves skilled contrivers of instruments of war, and that the invention and operation of the said instruments are nothing different from those in common use: I shall endeavor, without prejudice to any one else, to explain myself to your Excellency, showing your Lordship my secret, and then offering them to your best pleasure and approbation to work with effect at opportune moments on all those things which, in part, shall be briefly noted below..The document, written when da Vinci was 30, is actually more of a cover letter than a resume; he leaves out many of his artistic achievements and instead focuses on what he can provide for the Duke in technologies of war.
• jsMath: jsMath Home Page – The jsMath package provides a method of including mathematics in HTML pages that works across multiple browsers under Windows, Macintosh OS X, Linux and other flavors of unix. It overcomes a number of the shortcomings of the traditional method of using images to represent mathematics: jsMath uses native fonts, so they resize when you change the size of the text in your browser, they print at the full resolution of your printer, and you don't have to wait for dozens of images to be downloaded in order to see the mathematics in a web page. There are also advantages for web-page authors, as there is no need to preprocess your web pages to generate any images, and the mathematics is entered in TeX form, so it is easy to create and maintain your web pages. Although it works best with the TeX fonts installed, jsMath will fall back on a collection of image-based fonts (which can still be scaled or printed at high resolution) or unicode fonts when the TeX fonts are not available.
• Josh on the Web » Blog Archive » Abusing the Cache: Tracking Users without Cookies – To track a user I make use of three URLs: the container, which can be any website; a shim file, which contains a unique code; and a tracking page, which stores (and in this case displays) requests. The trick lies in making the browser cache the shim file indefinitely. When the file is requested for the first – and only – time a unique identifier is embedded in the page. The shim embeds the tracking page, passing it the unique ID every time it is loaded. See the source code.

  One neat thing about this method is that JavaScript is not strictly required. It is only used to pass the message and referrer to the tracker. It would probably be possible to replace the iframes with CSS and images to gain JS-free HTTP referrer logging but would lose the ability to store messages so easily.

• Panopticlick – Your browser fingerprint appears to be unique among the 342,943 tested so far.

  Currently, we estimate that your browser has a fingerprint that conveys at least 18.39 bits of identifying information.

  The measurements we used to obtain this result are listed below. You can read more about the methodology here, and about some defenses against fingerprinting here

These are my links for January 23rd through January 30th:

• Leonardo da Vinci’s Resume Explains Why He’s The Renaissance Man For the Job – Davinci – Gizmodo – At one time in history, even da Vinci himself had to pen a resume to explain why he was a qualified applicant. Here's a translation of his letter to the Duke of Milan, delineating his many talents and abilities. "Most Illustrious Lord, Having now sufficiently considered the specimens of all those who proclaim themselves skilled contrivers of instruments of war, and that the invention and operation of the said instruments are nothing different from those in common use: I shall endeavor, without prejudice to any one else, to explain myself to your Excellency, showing your Lordship my secret, and then offering them to your best pleasure and approbation to work with effect at opportune moments on all those things which, in part, shall be briefly noted below..The document, written when da Vinci was 30, is actually more of a cover letter than a resume; he leaves out many of his artistic achievements and instead focuses on what he can provide for the Duke in technologies of war.
• jsMath: jsMath Home Page – The jsMath package provides a method of including mathematics in HTML pages that works across multiple browsers under Windows, Macintosh OS X, Linux and other flavors of unix. It overcomes a number of the shortcomings of the traditional method of using images to represent mathematics: jsMath uses native fonts, so they resize when you change the size of the text in your browser, they print at the full resolution of your printer, and you don't have to wait for dozens of images to be downloaded in order to see the mathematics in a web page. There are also advantages for web-page authors, as there is no need to preprocess your web pages to generate any images, and the mathematics is entered in TeX form, so it is easy to create and maintain your web pages. Although it works best with the TeX fonts installed, jsMath will fall back on a collection of image-based fonts (which can still be scaled or printed at high resolution) or unicode fonts when the TeX fonts are not available.
• Josh on the Web » Blog Archive » Abusing the Cache: Tracking Users without Cookies – To track a user I make use of three URLs: the container, which can be any website; a shim file, which contains a unique code; and a tracking page, which stores (and in this case displays) requests. The trick lies in making the browser cache the shim file indefinitely. When the file is requested for the first – and only – time a unique identifier is embedded in the page. The shim embeds the tracking page, passing it the unique ID every time it is loaded. See the source code.

  One neat thing about this method is that JavaScript is not strictly required. It is only used to pass the message and referrer to the tracker. It would probably be possible to replace the iframes with CSS and images to gain JS-free HTTP referrer logging but would lose the ability to store messages so easily.

• Panopticlick – Your browser fingerprint appears to be unique among the 342,943 tested so far.

  Currently, we estimate that your browser has a fingerprint that conveys at least 18.39 bits of identifying information.

  The measurements we used to obtain this result are listed below. You can read more about the methodology here, and about some defenses against fingerprinting here

• Benlog » Don’t Hash Secrets – If I tell you that SHA1(foo) is X, then it turns out in a lot of cases to be quite easy for you to determine what SHA1(foo || bar) is. You don’t need to know what foo is. because SHA1 is iterative and works block by block, if you know the hash of foo, then you can extend the computation to determine the hash of foo || bar

  That means that if you know SHA1(secret || message), you can compute SHA1(secret || message || ANYTHING), which is a valid signature for message || ANYTHING. So to break this system, you just need to see one signature from SuperAnnoyingPoke, then you can impersonate SuperAnnoyingPoke for lots of other messages.

  What you should be using is HMAC: Hash-function Message Authentication Code. You don’t need to know exactly how it works, just need to know that HMAC is specifically built for message authentication codes and the use case of SuperAnnoyingPoke/MyFace. Under the hood, what’s approximately going on is two hashes, with the secret combined after the first hash

• Data.gov – Featured Datasets: Open Government Directive Agency – Datasets required under the Open Government Directive through the end of the day, January 22, 2010. Freedom of Information Act request logs, Treasury TARP and derivative activity logs, crime, income, agriculture datasets.

These are my links for December 31st through January 17th:

• Khan Academy – The Khan Academy is a not-for-profit organization with the mission of providing a high quality education to anyone, anywhere.

  We have 1000+ videos on YouTube covering everything from basic arithmetic and algebra to differential equations, physics, chemistry, biology and finance which have been recorded by Salman Khan.

• StarCraft AI Competition | Expressive Intelligence Studio – AI bot warfare competition using a hacked API to run StarCraft, will be held at AIIDE2010 in October 2010.
  The competition will use StarCraft Brood War 1.16.1. Bots for StarCraft can be developed using the Broodwar API, which provides hooks into StarCraft and enables the development of custom AI for StarCraft. A C++ interface enables developers to query the current state of the game and issue orders to units. An introduction to the Broodwar API is available here. Instructions for building a bot that communicates with a remote process are available here. There is also a Forum. We encourage submission of bots that make use of advanced AI techniques. Some ideas are:
  * Planning
  * Data Mining
  * Machine Learning
  * Case-Based Reasoning
• Measuring Measures: Learning About Statistical Learning – A "quick start guide" for statistical and machine learning systems, good collection of references.
• Berkowitz et al : The use of formal methods to map, analyze and interpret hawala and terrorist-related alternative remittance systems (2006) – Berkowitz, Steven D., Woodward, Lloyd H., & Woodward, Caitlin. (2006). Use of formal methods to map, analyze and interpret hawala and terrorist-related alternative remittance systems. Originally intended for publication in updating the 1988 volume, eds., Wellman and Berkowitz, Social Structures: A Network Approach (Cambridge University Press). Steve died in November, 2003. See Barry Wellman’s “Steve Berkowitz: A Network Pioneer has passed away,” in Connections 25(2), 2003. It has not been possible to add the updating of references or of the quality of graphics that might have been possible if Berkowitz were alive. An early version of the article appeared in the Proceedings of the Session on Combating Terrorist Networks: Current Research in Social Network Analysis for the New War Fighting Environment. 8th International Command and Control Research and Technology Symposium. National Defense University, Washington, D.C June 17-19, 2003
• SSH Tunneling through web filters | s-anand.net – Step by step tutorial on using Putty and an EC2 instance to set up a private web proxy on demand.
• PyDroid GUI automation toolkit – GitHub – What is Pydroid?

  Pydroid is a simple toolkit for automating and scripting repetitive tasks, especially those involving a GUI, with Python. It includes functions for controlling the mouse and keyboard, finding colors and bitmaps on-screen, as well as displaying cross-platform alerts.
  Why use Pydroid?

  * Testing a GUI application for bugs and edge cases
  o You might think your app is stable, but what happens if you press that button 5000 times?
  * Automating games
  o Writing a script to beat that crappy flash game can be so much more gratifying than spending hours playing it yourself.
  * Freaking out friends and family
  o Well maybe this isn't really a practical use, but…

• Time Series Data Library – More data sets – "This is a collection of about 800 time series drawn from many different fields.Agriculture Chemistry Crime Demography Ecology Finance Health Hydrology Industry Labour Market Macro-Economics Meteorology Micro-Economics Miscellaneous Physics Production Sales Simulated series Sport Transport & Tourism Tree-rings Utilities"
• How informative is Twitter? » SemanticHacker Blog – "We undertook a small study to characterize the different types of messages that can be found on Twitter. We downloaded a sample of tweets over a two-week period using the Twitter streaming API. This resulted in a corpus of 8.9 million messages (”tweets”) posted by 2.6 million unique users. About 2.7 million of these tweets, or 31%, were replies to a tweet posted by another user, while half a million (6%) were retweets. Almost 2 million (22%) of the messages contained a URL."
• Gremlin – a Turing-complete, graph-based programming language – GitHub – Gremlin is a Turing-complete, graph-based programming language developed in Java 1.6+ for key/value-pair multi-relational graphs known as property graphs. Gremlin makes extensive use of the XPath 1.0 language to support complex graph traversals. This language has applications in the areas of graph query, analysis, and manipulation. Connectors exist for the following data management systems:

  * TinkerGraph in-memory graph
  * Neo4j graph database
  * Sesame 2.0 compliant RDF stores
  * MongoDB document database

  The documentation for Gremlin can be found at this location. Finally, please visit TinkerPop for other software products.

• The C Programming Language: 4.10 – by Kernighan & Ritchie & Lovecraft – void Rlyeh
  (int mene[], int wgah, int nagl) {
  int Ia, fhtagn;
  if (wgah>=nagl) return;
  swap (mene,wgah,(wgah+nagl)/2);
  fhtagn = wgah;
  for (Ia=wgah+1; Ia<=nagl; Ia++)
  if (mene[Ia]<mene[wgah])
  swap (mene,++fhtagn,Ia);
  swap (mene,wgah,fhtagn);
  Rlyeh (mene,wgah,fhtagn-1);
  Rlyeh (mene,fhtagn+1,nagl);

  } // PH'NGLUI MGLW'NAFH CTHULHU!

• How to convert email addresses into name, age, ethnicity, sexual orientation – This is so Meta – "Save your email list as a CSV file (just comma separate those email addresses). Upload this file to your facebook account as if you wanted to add them as friends. Voila, facebook will give you all the profiles of all those users (in my test, about 80% of my email lists have facebook profiles). Now, click through each profile, and because of the new default facebook settings, which makes all information public, about 95% of the user info is available for you to harvest."
• Microsoft Security Development Lifecycle (SDL): Tools Repository – A collection of previously internal-only security tools from Microsoft, including anti-xss, fuzz test, fxcop, threat modeling, binscope, now available for free download.
• Analytics X Prize – Home – Forecast the murder rate in Philadelphia – The Analytics X Prize is an ongoing contest to apply analytics, modeling, and statistics to solve the social problems that affect our cities. It combines the fields of statistics, mathematics, and social science to understand the root causes of dysfunction in our neighborhoods. Understanding these relationships and discovering the most highly correlated variables allows us to deploy our limited resources more effectively and target the variables that will have the greatest positive impact on improvement.
• PeteSearch: How to find user information from an email address – FindByEmail code released as open-source. You pass it an email address, and it queries 11 different public APIs to discover what information those services have on the user with that email address.
• Measuring Measures: Beyond PageRank: Learning with Content and Networks – Conclusion: learning based on content and network data is the current state of the art There is a great paper and talk about personalization in Google News they use content for this purpose, and then user click streams to provide personalization, i.e. recommend specific articles within each topical cluster. The issue is content filtering is typically (as we say in research) "way harder." Suppose you have a social graph, a bunch of documents, and you know that some users in the social graph like some documents, and you want to recommend other documents that you think they will like. Using approaches based on Networks, you might consider clustering users based on co-visitaion (they have co-liked some of the documents). This scales great, and it internationalizes great. If you start extracting features from the documents themselves, then what you build for English may not work as well for the Chinese market. In addition, there is far more data in the text than there is in the social graph
• mikemaccana’s python-docx at master – GitHub – MIT-licensed Python library to read/write Microsoft Word docx format files. "The docx module reads and writes Microsoft Office Word 2007 docx files. These are referred to as 'WordML', 'Office Open XML' and 'Open XML' by Microsoft. They can be opened in Microsoft Office 2007, Microsoft Mac Office 2008, OpenOffice.org 2.2, and Apple iWork 08. The module was created when I was looking for a Python support for MS Word .doc files, but could only find various hacks involving COM automation, calling .net or Java, or automating OpenOffice or MS Office."

These are my links for June 11th through June 12th:

• Using Google Search to find interesting Twitter Users | Social Media Stuff | Justin Parks – Ideas for queries on Google to find Twitter users / content.
• David Ziegler’s Blog – A Python Script to Automatically Extract Excerpts From Articles – Some notes on cleaning up web page content to extract a text excerpt of the intended content.
• Papers on PageRank you should read | Science for SEO – A list of interesting PageRank and search relevance ranking papers. June 2009.
• aicoder: TunkRank Scoring Improvement – On influence ranking models for Twitter.
• datawrangling’s trendingtopics at master – GitHub – This repository contains the full source code for Trendingtopics.org, built by Data Wrangling to demonstrate how Hadoop & EC2 can power a data driven website. The trend statistics and time series data that run the site are updated periodically by launching a temporary EC2 cluster running the Cloudera Hadoop Distribution. Our initial seed data consists of the raw wikipedia database content dump along with hourly traffic logs for all articles collected from the wikipedia squid proxy (curated by Domas Mituzas). We made the first 7 months of this hourly data for all articles available as an Amazon Public Dataset. The current trend calculations are run with Hadoop Streaming and Hive. The output produced by these Hadoop jobs is loaded into MySQL and indexed to power the live site.

These are my links for June 9th through June 10th:

• Announcing the Yahoo! Distribution of Hadoop (Hadoop and Distributed Computing at Yahoo!) – Yahoo releases its internal version of Hadoop, a source-only distribution of Apache Hadoop tested and used in production at Yahoo.
• Google Fusion Tables FAQ – Sort of like extra-large Google Docs spreadsheets, up to 100MB per table, 250MB per user. One interesting wrinkle is that it doesn't actually delete your dataset when you "delete" it, so the data is still available for derived tables that other users have built.
• Filesystem Performance from a Database Perspective – Presentation on performance benchmarks on linux filesystems (ext2, ext3, reiserfs, xfs, etc)
• What Assumptions Make: Filesystem I/O from a database perspective – Slide presentation comparing linux file system performance across various formats (ext2, ext3, etc), RAID configurations, readahead buffer sizes
• MySQL – Common Queries Tree – A collection of common queries implemented in MySQL

These are my links for June 3rd through June 4th:

• MIT Database Systems (6.830) TA Course Notes – marcua’s blog – Fall 2008 course notes on database systems, spanning history of databases and data structures up through mapreduce style systems.
• Ray Ozzie’s cloud hangs over the Valley | Beyond Binary – CNET News – Notes from Ray Ozzie talk on Microsoft and the cloud at Churchill Club, June 2009. Would be interesting to hear more of his thoughts on Google Wave, given his past work on Notes and Groove.
• Matasano Chargen » Blog Archive » Typing The Letters A-E-S Into Your Code? You’re Doing It Wrong! – An epic discussion of shortcomings in various common encryption implementations for web applications, presented in the form of a somewhat random screenplay.
• Is Twitter A Substitute For Set Top Box Data? – The post describes a little experiment Simulmedia did to analyze TV channel surfers using only Twitter posts as their data source. Simulmedia explains why they turned to Twitter data for this work:
• NILFS: A File System to Make SSDs Scream | Linux Magazine – A log structured (as opposed to journaling) file system with some advantages for write performance and shorter crash recovery times. Good for large working file systems?

These are my links for June 1st through June 2nd:

• jqPlot – Pure Javascript Plotting – jqPlot is a plotting plugin for the jQuery Javascript framework. jqPlot produces beautiful line and bar charts with many features including: Numerous chart style options. Date axes with customizable formatting. Rotated axis text. Automatic trend line computation. Tooltips and data point highlighting. Sensible defaults for ease of use.
• New Twitter Research: Men Follow Men and Nobody Tweets – Conversation Starter – HarvardBusiness.org – "Although men and women follow a similar number of Twitter users, men have 15% more followers than women. Men also have more reciprocated relationships, in which two users follow each other. This "follower split" suggests that women are driven less by followers than men, or have more stringent thresholds for reciprocating relationships. This is intriguing, especially given that females hold a slight majority on Twitter: we found that men comprise 45% of Twitter users, while women represent 55%."
• Shirky: Power Laws, Weblogs, and Inequality – 2003 article on popularity / traffic on blogs, which was then the latest emerging social media format. "Once a power law distribution exists, it can take on a certain amount of homeostasis, the tendency of a system to retain its form even against external pressures. Is the weblog world such a system? Are there people who are as talented or deserving as the current stars, but who are not getting anything like the traffic? Doubtless. Will this problem get worse in the future? Yes. "
• well-formed.eigenfactor.org : Visualizing information flow in science – Some nice visualization ideas using hierarchical clustering to explore patterns in citation networks.
• Bing API, Version 2.0 – Updated API documentation for Microsoft Bing (formerly Live Search) web services.

These are my links for May 24th through May 27th:

• Formulas and game mechanics – WoWWiki – Your guide to the World of Warcraft – Formulas and game mechanics rules and guidelines for developing role playing games
• Manchester United’s Park Has the Endurance to Persevere – NYTimes.com – Korean soccer player Park Ji-Sung – On Wednesday night in Rome, Park is expected to become the first Asian player to participate in the European Champions League final when Manchester United faces Barcelona.
• mloss.org – Machine Learning Open Source Software – Big collection of open source packages for machine learning, data mining, statistical analysis
• The Datacenter as Computer – Luiz André Barroso and Urs Hölzle 2009 (PDF) – 120 pages on large scale computing lessons from Google. "These new large datacenters are quite different from traditional hosting facilities of earlier times and cannot be viewed simply as a collection of co-located servers. Large portions of the hardware and software resources in these facilities must work in concert to efficiently deliver good levels of Internet service performance, something that can only be achieved by a holistic approach to their design and deployment. In other words, we must treat the datacenter itself as one massive warehouse-scale computer (WSC). We describe the architecture of WSCs, the main factors influencing their design, operation, and cost structure, and the characteristics of their software base."
• Geeking with Greg: The datacenter is the new mainframe – Pointer to a paper by Googlers Luiz Andre Barroso and Urs Holzle on the evolution of warehouse scale computing and the management and use of computing resources in a contemporary datacenter.

These are my links for May 22nd through May 23rd:

• Improve MySQL Insert Performance – Summary – use LOAD DATA INFILE
• Scratch | Home | imagine, program, share – Scratch is designed to help young people (ages 8 and up) develop 21st century learning skills. As they create and share Scratch projects, young people learn important mathematical and computational ideas, while also learning to think creatively, reason systematically, and work collaboratively
• Alice.org – Programming language environment for teaching kids, built on Java, geared toward a story telling approach.
• Jason R Briggs | Snake Wrangling for Kids – “Snake Wrangling for Kids” is a printable electronic book, for children 8 years and older, who would like to learn computer programming. It covers the very basics of programming, and uses the Python 3 programming language to teach the concepts.
• Benchmarking BDB, CDB and Tokyo Cabinet on large datasets – CDB comes out significantly faster. (It's for unchanging data though, so not totally surprising) Benchmark data for 11M key-value pair dataset stored in Berkeley DB, CDB, and Tokyo Cabinet.

These are my links for May 21st from 06:07 to 22:34:

• Data.gov Launches to Mixed Reviews | Wired Science | Wired.com – Commentary and initial reactions to data.gov. Good news – government is taking steps to improve access to public data. Bad news – limited catalog of datasets to start (47). But it's going in the right direction.
• IEEE Spectrum: The Million Dollar Programming Prize – Robert M. Bell, Jim Bennett, Yehuda Koren, and Chris Volinsky. A look at collaborative filtering and other predictive clustering systems used by one team competing for the Netflix prize, in which the goal is a 10% improvement over the original Cinematch recommendation system.
• bb: 21may2009 – (opinion) Graphic of the gap between how people (and companies) succeed, and how people think they succeed.
• Seventeen things that people are actually saying when they retweet others – meish dot org: life, unfolding – A look at various reasons why messages are retweeted on Twitter.
• Expanding the Cloud: Moving large data sets into Amazon S3 with AWS Import/Export. – All Things Distributed – More on shipping data to S3 via hard drive from Werner Vogels
• Amazon Web Services Blog: AWS Import/Export: Ship Us That Disk! – Tanenbaum-style bulk transfer for AWS/S3! "Customers with AWS storage requirements at the terabyte and petabyte level often ask us if they can sidestep the internet and simply send us a disk drive, or even a 747 full of such drives. "

These are my links for May 20th from 19:50 to 22:03:

• PicFog Displays the Strength of Real-Time Image Search – More real time social search prototypes, this one for images shared on twitter. Fun to play with.
• bits done properly – 7 TwitPic alternatives – A list of alternative photo sharing sites suitable for use with Twitter.
• Twitter Data – A simple, open proposal for embedding data in Twitter messages – Home – "Twitter Data is a simple, open, semi-structured data representation format for embedding machine-readable, yet human-friendly, data in Twitter messages. This data can then be transmitted, received, and interpreted in real time to enable powerful new kinds of applications to be built on the Twitter platform."
• Announcing TweetMotif for summarizing twitter topics with a dash of NLP – Brendan O’Connor’s Blog – TweetMotif is an experiment in using natural language processing to identify trending topics.
• OneRiot Announces API & Real-Time Search Partnerships – "Real-time social search outfit OneRiot today announced their API and partnership program for adding real-time search capabilities to browser add-ons, desktop applications, social websites and other services" Screenshots from initial app TwitterBar (browser extension)
• Mozilla Labs » Blog Archive » Introducing Jetpack, Call for Participation – API for Firefox extension development

These are my links for May 19th from 08:04 to 19:24:

• List of Really Useful Free Tools For JavaScript Developers | W3Avenue –
• When Korean Culture Flourished – WSJ.com – In the geography of the Metropolitan Museum of Art, the gallery devoted to Korea acts as a sort of land bridge between China and South Asia that all too often serves as passage rather than destination. The first in a series of shows to be held over the next 10 to 15 years, "Art of the Korean Renaissance, 1400-1600" may change this. With only 47 objects(!), the exhibition explores a fertile 200-year period in Korea's cultural history, revealing as much through its choice of works as it does through the order in which it displays them. The show's modest size makes the point that, sadly, little has survived from this period, when the Joseon — or Fresh Dawn — dynasty (1392-1910) united the Korean peninsula militarily, established Confucianism as the national ideology and introduced a phonetic alphabet.
• Axiis : Data Visualization Framework – Axiis provides both pre-built visualization components as well as abstract layout patterns and rendering classes that allow you to create your own unique visualizations. Axiis is built upon the Degrafa graphics framework and Adobe Flex 3.
• Report: Mint Considers Selling Anonymized Data from Its Users – ReadWriteWeb – A lot of people would be interested in that dataset. Tricky to balance data exposure with consumer privacy.
• Lendingclub.com: A De-anonymization Walkthrough « 33 Bits of Entropy – Step by step look at de-anonymizing a consumer data set. Given alternate sources, you can fill in a lot of gaps.

These are my links for May 14th through May 15th:

• Congratulations, Google staff: $210k in profit per head in 2008 | Royal Pingdom – Google had $209,624 in profit per employee in 2008, which beats all the other large tech companies we looked at, including big hitters like Microsoft ($194K), Apple ($151K), Intel ($64K) and IBM ($30K).
• Statistical Data Mining Tutorials – A nice collection of presentations reviewing topics in data mining and machine learning. e.g. "HillClimbing, Simulated Annealing and Genetic Algorithms. Some very useful algorithms, to be used only in case of emergency." These include classification algorithms such as decision trees, neural nets, Bayesian classifiers, Support Vector Machines and cased-based (aka non-parametric) learning. They include regression algorithms such as multivariate polynomial regression, MARS, Locally Weighted Regression, GMDH and neural nets. And they include other data mining operations such as clustering (mixture models, k-means and hierarchical), Bayesian networks and Reinforcement Learning.
• Dare Obasanjo aka Carnage4Life – Why Twitter’s Engineers Hate the @replies feature – Looking at the infrastructure overhead required for Twitter's attempted change to @reply behavior.
• Scratch Helps Kids Get With the Program – Gadgetwise Blog – NYTimes.com – On my candidate list for 7th grade introductory programming and analysis. "Scratch, an M.I.T.-developed computer-programming language for children, is the focus of worldwide show-and-tell sessions this Saturday. "
• jLinq – Javascript Query Language – For manipulating data sets in Javascript, sort of like jQuery

These are my links for May 13th from 06:26 to 22:36:

• Zen»Blog Archive » See Which Twitterers Don’t Follow You Back (and vice-versa) In Less Than 15 Lines of…Perl – Short Perl script for comparing friends/followers, motivated by the Ruby version.
• See Which Twitterers Don’t Follow You Back In Less Than 15 Lines of Ruby | blogt✪sk1 – Short follower/friend comparison script in Ruby.
• HornetsEye – Computer Vision for the Robotic Age – HornetsEye – "HornetsEye is a Ruby real-time computer vision extension running under GNU+Linux and Microsoft Windows. HornetsEye is maybe the first free software project providing a solid platform for implementing real-time computer vision software in a scripting language. The platform potentially could be used in robotic applications, industrial automation, unmanned aerial vehicles as well as in image and video processing, microscopy, materials science, and medical research."
• Announcing the PHP SDK for Windows Azure – Port 25: The Open Source Community at Microsoft – PHP for using Microsoft Azure services, plus pointers to related projects.
• Just Landed: Processing, Twitter, MetaCarta & Hidden Data | blprnt.blg – Map visualization of people's travel using Processing with data inferred from Twitter messages, e.g "just landed at heathrow"

These are my links for May 12th from 10:52 to 21:56:

• An unthinking programmer’s guide to the new C++ • The Register – Must be better for templating or something. Double ampersands and empty curly braces? int&& r2 = foo(); // foo() returns an int ; int i = {}; // i = 0;
• Announcing the Map/Reduce Toolkit – Open Blog – NYTimes.com – Package for making it easier to use mapreduce for batch processing, from NYTimes.
• jQuery-gestures- Image gallery – Demonstration of using jQuery to build a gesture-based user interface (in this case a photo gallery application)
• Dolores Labs Blog » The Programming Language with the Happiest Users – Interesting sentiment analysis based on Twitter messages. Cobol = least happy, Perl = most happy. Some judgement calls here, sentiment was done using 3 people on Mechanical Turk.
• U.S. starting lineup if top American jocks played soccer – Greg Lalas – SI.com – Trying to get my head wrapped around Kobe and Lebron with a ball at their feet.
• Dept. of Science: Don’t!: Reporting & Essays: The New Yorker – The ability to wait (delayed gratification) is difficult, but learnable. Stanford study of pre-school kids at Bing Nursery School waiting for 1 marshmallow now or 2 later, the kids who were able to wait often ended up more successful in school then career. But some impatient kids learned to wait, now some proposals to teach self control in school.

These are my links for May 8th through May 12th:

• 15 Fascinating Ways to Track Twitter Trends – List of trend tracking sites, plus more in the comments.
• ToorCamp 2009, A Hacker Conference Inside An Abandoned Titan-1 Missile Silo – This looks like fun!
• neo4j open source graph database – Sounds interesting for social media graph hacking. "Neo4j is a graph database. It is an embedded, disk-based, fully transactional Java persistence engine that stores data structured in graphs rather than in tables. A graph (mathematical lingo for a network) is a flexible data structure that allows a more agile and rapid style of development."
• One Div Zero: A Brief, Incomplete, and Mostly Wrong History of Programming Languages – A sample – "1972 – Dennis Ritchie invents a powerful gun that shoots both forward and backward simultaneously. Not satisfied with the number of deaths and permanent maimings from that invention he invents C and Unix."
• EJW.COM || SETTING THE EXAMPLE – Thoughts on URL shorteners from Eric James Woodward (tr.im / Nambu). tinyurl.com, bit.ly, tr.im etc know what everyone is shortening right now; is 15 minutes of lead time useful? Click throughs depend on audience size and can be gamed, etc.

These are my links for May 4th through May 5th:

• Influential Nodes in a Diffusion Model for Social Networks (icalp05-inf.pdf) – Kempe, Kleinberg, Tardos. Algorithm for greedy approximation of most influential nodes in social network (63% of optimal) under various conditions.
• Maximizing the Spread of Influence through a Social Network (kdd03-inf.pdf) – Kempe, Kleinberg, Tardos. Maximizing propagation by selecting most influential nodes is NP-hard, but a greedy approximation can work well (63% of optimal) under various conditions.
• Notification Strategies for Social Networks – Discussion on approaches to maximizing use of a limited number of notifications within social networks e.g. Facebook
• James Smith • loopj.com » Blog Archive » jQuery Plugin: Tokenizing Autocomplete Text Entry – Looks handy – "This is a jQuery plugin to allow users to select multiple items from a predefined list, using autocompletion as they type to find each item. You may have seen a similar type of text entry when filling in the recipients field sending messages on facebook."
• Google Code FAQ – Using cURL to interact with Google data services – Step by step tutorial on using curl with Google data APIs.
• Behind The Business Plan Of Pirates Inc. : NPR – It takes around $250K to fund a Somali pirate operation. About 20 percent goes to pay off officials who look the other way. About 50 percent is for expenses and payroll. The leader of an attack makes $10,000 to $20,000 (the average Somali family lives on $500 a year). The initial investor — who put in $250,000 of seed capital — gets 30 percent, sometimes up to $500,000.

These are my links for April 30th through May 2nd:

• FusionCharts Free – Animated Flash Charts and Graphs for ASP, PHP, ASP.NET, JSP, RoR and other web applications – Flash charting component that can be used to render data-driven & animated charts for your web applications and presentations. It is a cross-browser and cross-platform solution that can be used with PHP, Python, Ruby on Rails, ASP, ASP.NET, JSP, ColdFusion, simple HTML pages or even PowerPoint Presentations to deliver interactive and powerful flash charts. You do NOT need to know anything about Flash to use FusionCharts. All you need to know is the language you're programming in.
• Raphaël—JavaScript Library – Raphaël is a small JavaScript library that should simplify your work with vector graphics on the web. If you want to create your own specific chart or image crop and rotate widget, for example, you can achieve it simply and easily with this library. Raphaël uses the SVG W3C Recommendation and VML as a base for creating graphics. This means every graphical object you create is also a DOM object, so you can attach JavaScript event handlers or modify them later. Raphaël’s goal is to provide an adapter that will make drawing vector art compatible cross-browser and easy.
• A Really Gentle Introduction to Data Mining | Regular Geek – List of data mining blogs and related resources.
• BlackBerry SSH Tutorial: Connect to Unix Server using MidpSSH for Mobile Devices – Notes on using MidpSSH on Blackberry for remote access to servers. Seems to work, although big network lag on my BlackBerry Bold / AT&T.
• Country Reports on Terrorism 2008 – U.S. law requires the Secretary of State to provide Congress, by April 30 of each year, a full and complete report on terrorism with regard to those countries and groups meeting criteria set forth in the legislation. This annual report is entitled Country Reports on Terrorism. Beginning with the report for 2004, it replaced the previously published Patterns of Global Terrorism.
• DIY: How To Find Authoritative Twitter Users Plus 100 To Get You Started | Ignite Social Media – Some comments on recommendation metrics for Twitter, trying to use "favorites" mark as an indicator.
• SIGUSR2 > The Power That is GNU Emacs – "If you've never been convinced before that Emacs is the text editor in which dreams are made from, or that inside Emacs there are unicorns manipulating your text, don't expect me to convince you."

These are my links for April 30th from 05:57 to 07:10:

• SIGUSR2 > The Power That is GNU Emacs – "If you've never been convinced before that Emacs is the text editor in which dreams are made from, or that inside Emacs there are unicorns manipulating your text, don't expect me to convince you."
• How To Be A Successful Evil Overlord – 100 remedies for the fatal flaws exhibited by famous evil overlords of the past. Also some business executives, I think.
• Google Could Have Caught Swine Flu Early | Wired Science – Google’s search data may have been able to provide an early warning of the swine flu outbreak — if the company had been looking in the right place. Last week, at the request of the Centers for Disease Control, Google took a retroactive look at its search data from Mexico. And there the team found a pre-media bump in telltale flu-related search terms (you know, “influenza + phlegm + coughing”) that was inconsistent with standard, seasonal flu trends.
• What Twitter Looks Like For Twitter Employees (SCREENSHOTS) – Some screen shots of current admin tools at Twitter for managing user accounts, blocks, whitelisting, suspensions, and user stats such as # follow attempts, # updates, #directs, etc
• Twitter Aggregator Sawhorse Media Raises Seed Round, Launches Pets, Celeb Sites | paidContent.org – "Channelized" feeds from curated lists of twitter sources.