Chasing the Long Tail, Part II

In my last post, I introduced the term "Long Tail" and hinted that a large catalog of web pages could generate substantial traffic, regardless of the quality of the actual content. I also mentioned an experiment I'm conducting to test this theory. I'm not going to go into specifics here as to where I'm conducting this experiment (the web address) because I don't want a flood of traffic from here ruining my traffic statistics. I want those stats to reflect traffic resulting from the test as much as possible. I should also point out that I'm no scientist and there's no real controls here. This is just an informal test.

For my experiment I need pages, lots of pages. I also need to make sure that those pages wouldn't generate lots of traffic of their own under normal conditions (read as boring, unpopular content) and that the web site I'm using for the experiment doesn't already have traffic. Here's what I did to satisfy those three requirements: I picked one of my unestablished domain names I registered and wrote a script. The script fetches Wikipedia articles and stores them locally (totally legal through the GFDL). It also rewrites all the links in the article so that they point internally. In other words, when a search engine indexes the site, a link to Mona Lisa would be to http://(www.mysite.com)/Mona_Lisa rather than Wikipedia's version. When the search engine follows that article link it will, again, cause the new page to be stored locally as well. The general idea is to have the search engine spider all 2,332,000 + articles at Wikipedia, but have them all be at my site instead. That gives me the ungodly amount of content I need for my experiment, but because my site is not popular the content would otherwise never be found in search results.

Being buried in search results is no problem. That's what I want. I want to test the idea (Long Tail) that given a substantially large content catalog, you'll end up with traffic anyway, even if you're not popular. Where this traffic comes from, who knows, but I sometimes click page 10 or more when I search just for fun, sort of pulling from the bottom of the deck in a search result. Plus search engine results are based on a combination of keywords rather than a single word, and it's possible one might have just the right combination the searcher is looking for, a combination that wouldn't be in the original content, and thus drives the page to the top of the results. Who knows, who cares. The experiment is just to see if a lot of traffic comes naturally from having a large catalog of content, regardless of the quality of the content. We're testing quantity over quality.

Here's the results thus far:

I started my experiment in mid-March, with a small handful of links to articles at the test site posted in a blog. The first step was to see if the search engine would even follow the links and index the subsequent pages. They did. They followed my handful of links, which led to more links, and more, and they indexed those too. It's been slow going. To date, with a "seed" of links less than 10, they have indexed approximately 1,200 pages. It appears that it is exponential growth as well. In the first few days, it was one or two pages indexed at a time. Towards the latter end, it's been hundreds of articles at a time. In the last week or two, the amount of indexed pages doubled in size. Cool. 'Cause there's plenty more left to go. Ultimately I'd like to see all two million plus articles indexed because then I have some serious data to examine.

The traffic stats would seem less than noteworthy to someone who doesn't see the significance of the increase. With roughly 1,000 articles indexed, the number of people who visit the site has grown by 5 people per day. That's nothing to write home about, but the point is there was an increase. It's evidence that the idea is sound, and that things are working as they should.

The greater thing to get giddy about is that if we assume a traffic increase of 5 people per 1,000 pages, and if all 2 million pages are covered, then that is a total daily visitor count of 10,000 per day. That is definitely something to write home about. One could earn some serious extra income off a site that generates approximately 300,000 visits per month. Assuming an average of four page views per visit, we're looking at 1,200,000 page views per month, and that's only using simple, uninteresting content borrowed from some other site. The content's not augmented with other services that might cause a visitor to return, or see the page as a useful resource.

If augmented with other services, who knows? Guess we'll have to find out as the experiment continues.

Help me crack a safe

Roughly three years ago my parents bought a three-story historic building located on Second Street in downtown Maysville, Kentucky. It was built sometime towards the end of the 1800s, judging by old photos of the downtown area, and in the Italianate style, with six cast iron Greek styled Corinthian columns out front. It's still in decent shape and solid, with huge brick columns supporting the underbelly (very cool basement). The building has survived several floods, including the Great Flood of 1937. There's a high-water mark plaque on the wall inside the cafe on the first floor, and trust me, that was all underwater.

I plan on leasing the second floor as an office, but that's not the subject of this post.

The building was part of the Maysville Bank from the 1890s until 1925, and later a drug store. Both businesses are notable because it goes to what this post is actually about. In the back room of the building, tucked away for who knows how long, is a safe. It was there when my parents bought the building, probably because it weighs a ton, but there's really no telling how long it has been there. The safe is locked tight. No one knows the combination, or what's in it. There's some evidence at the bottom that suggests it was in place during the flood, and the web's helped me identify the make and model as a Sargent and Greenleaf safe, dating to the early 1900s.

At this point I have all sorts of wild fantasies about what's inside. Cash! or drugs. Who knows? Chopped up Jimmy Hoffa? A few empty bottles like the Al Capone vault? Boring receipts or other papers? Could be anything. Could be nothing.

I've tried to crack the safe myself, but I suck as a safe cracker. A downloaded PDF ebook on safe cracking apparently doesn't make you a master thief. That's me in the photos with my stethoscope. That's me learning that stethoscopes don't help. I've considered calling in professionals to blowtorch it, but there's a risk of ruining any papers that may be inside (including cash), and apparently old safes like this one sometimes had poison vials installed to prevent just this sort of brute force entry. It also ruins an otherwise excellent antique safe, worth something even if it's empty. If you have the combo, that is.

So that's what I need, the combination. Thousands (possibly millions) of number combinations and I'm not imaginative enough to come up with them all. Being the clever lazy bastard that I am, I'm crowd-sourcing.

Effective immediately, I am willing to try any combination that is emailed to me. I want this safe open. Whatever's inside, we can split (if it's legal to do so, read as I'm not going to turn over absinthe or human remains). Think of it as a pseudo-lottery. Which brings me to my next point: Send me your combination idea now before I get ambitious and start selling chances on eBay for $2 a pop. If there's any lawyers out there, is that legal?

The five bucks butterfly effect

I don't remember exactly where I got this idea, but I think it comes from a book project I read about in a magazine a few years back. If someone finds a link to it, please shoot me an email. It's enough to say that it's not an original idea, just a project I've been wanting to do for some time, but can't afford just yet.

The basic idea for this project stems from the butterfly effect in chaos theory, that the flapping of butterfly's wings might ultimately lead to a tornado appearing miles away. It's the idea that the tiniest action can cause a ripple effect leading to big results. In this application, we're looking to cause one person to smile, which causes another to smile, and another, and so on — leading to a viral goodness that goes on and on. It doesn't take a system's theorist to realize that if someone cuts you off on the way to work, the chances of you having a good day are slimmed, and that your bad day is infectious to those around you. This is meant to counteract that.

It's a five dollar bill and a self-addressed stamped postcard left in a phone booth, on a park bench, a bathroom stall, or tacked to the community bulletin board at the grocery store. The card reads, "Smile, you're five dollars richer. Blow it on whatever you want but please drop a note to let us know how it goes." I want to put 100 - 200 of these around town in the oddest of places. Part of it is to brighten people's day, the other part is to just do something odd. It's not an easter egg hunt. They'll be conspicuous. The idea, though, is to go for the unexpected.

Imagine sitting on the bench at a bus stop and noticing a five dollar bill and postcard next to you. What would you spend it on? I'd probably consider it a good fortune and buy five scratch off lottery tickets myself. Then again, you are at a bus stop. Bus fare maybe? Cappucino? A magazine? Taco Bell? Five dollars doesn't go very far these days, but hopefully it's enough to buy a smile.

I don't know. That's the idea behind the postcards. I'd love to hear how it's actually spent, or what effect it causes. It's worth the five hundred to a thousand dollars to find out. The article I recall getting this idea from was about collecting these stories and compiling them into a book. They put out thousands of these bill/postcard combos. I can't afford their budget, but when I have a grand to blow, I'm definitely doing this. Maybe I'll post the responses I get back to a blog or something. Even if I don't get anything back, that's fine too. I'm fairly confident that it'll brighten someone's day, and who can put a real price tag on that? Five bucks butterfly effect might be worth billions in good moods.

Building a RC tank that's actually web controlled

About a year ago I attached a video camera to the top of my remote control PT Cruiser and drove it around town just for fun. This got me thinking of different ways to remote control a camera equipped vehicle, and naturally my thoughts gravitated towards the web. I had heard about a number of web controlled devices but never followed up on how exactly they were done. Nevertheless, armed with the desire to create a web controlled camera equipped device, and a bit of knowledge on how the web works, I drew up a plan on how I could create a web controlled tank that could navigate an obstacle course and snap photos of various objects as a game.

The basic architecture of the web is server/client. A server houses information and clients access the server to retrieve and post information. That's the really simplified schematic of most web systems. You have P2P, of course, where each server is a client and vice versa, but that's still a server-client system. Knowing this, and knowing a bit of server programming, I realized that it'd actually be very easy (at least in concept) to create a web controlled device if the server is embedded in the vehicle.

The idea is to take a remote controlled tank and mash it up with an old laptop I have that's still good, despite not being powerful enough to do much more than light Internet surfing. The body of the tank will be modified to make room for the modified laptop. Attached to the turret of the tank will be a stripped down web cam wired to the laptop. The motors of the RC tank will be replaced with stepper motors wired to the laptop as well. This allows for a software control of the tank device and greater precision in movement. Both the motors and the web cam can be controlled through server software running on laptop.

So, everything's controlled through the server, and that's how we get remote. The laptop server can be accessed through the Internet the same as any other server, for example those that serve up web pages or games. The server on the laptop is connected to a main local area network through the laptop's built in Wi-Fi card, and the WLAN is connected to the Internet through an ISP. The web cam provides for remote viewing of where the tank is heading, so the user at the controls can steer the tank. The controls for operating everything are built into a web page. In short, through software a remote user can control the internal laptop devices (motors, cams, etc), and it is possible for someone in India to control the tank and run it through an obstacle course I set up in my United States yard, snapping photos of targets I set up. It can be someone as far off as India, or it can be me sitting on my back porch surfing the web wirelessly. It's nonlocal. The range of where this tank can go is only limited by battery power and the extent of the Wi-Fi signal, which can extend pretty darn far. This whole thing can be moved to one of those cities that are entirely covered by wireless signals and that's a pretty big area to play.

Doesn't that just sound like fun? There's so many different ways to have fun with this. Two tanks could even be set up and pitted against each other. If I could adequately protect the interior electronics, I could probably even rig up paintball guns to the tanks. Web controlled stepping motors open a lot of doors. Like I said, a number of interactive games could be built upon this idea.

Similar devices have been built by other people, of course, and were built long before I came up with this idea, but as a thought project I purposely didn't want to look at them until after I had mapped out my own project to see if I could devise something on my own. After I completed my plans, I realized it's very similar to how that Texas group did their highly controversial online hunting (mine's just a game where nothing gets hurt) and that Bradley University students in Illinois created almost exactly the same thing as a senior project. When completed, my tank will hopefully look like the Magnus Ivarssons tank (pictured left), which is exactly what I had in mind before realizing he'd already made one. To my knowledge his tank is the first web controlled tank ever built. Clever Swedish people! I'm never upset on finding that someone else came up with an idea first, as long as I independently came up with the idea on my own. It shows that I'm not always a quick thinker, but I can still figure things out : )

At some point I am going to tackle this project completely. I haven't yet because I'm still collecting parts. Sufficiently large RC tanks can get pretty costly, so I might just model my own. Everything's complete on paper and the parts aren't all that expensive compared to the fun that awaits. The only thing putting this project on hold is the build time.