The end of the Internet is near

Well, maybe not the end of the Internet, but the end of the IP Address space, as we know it, is near. You know, IP addresses are the 4 blocks of number separated by periods, i.e. 10.0.0.1; well, there are only 4 billion of them. Sometime next year, 2012, Internet Assigned Numbers Authority (IANA) will have no more IP addresses to give to the Regional Internet Registries (RIRs). About a year after that, the RIRs will have no more to give out to Internet Service Providers (ISPs), such as Cox Communication and Go Daddy.

Read more

Hack to make vipers-video-quicktags detect iPad and use HTML5 Video Tag

I use this WordPress plugin, vipers-video-quicktag, in a couple sites I maintain. I use it to display FLV video. I’ve found if I create .mp4 videos with the H.262 video and AAC audio codecs, it’ll play in the Flash player, as well as on iPhone, iPad and iPod. This is neat, a single video format that works on all 3.

Problem is, with vipers-video-quicktag, it doesn’t detect iPhone/iPod/iPad on the Flash video option and just tries to display it with the Flash player, which we all know doesn’t work on iPhone/iPod/iPad.

The trick is to detect iPhone/iPod/iPad and instead of putting in the Flash video, instead put in a HTML5 Video tag.

WordPress: http://wordpress.org/extend/plugins/vipers-video-quicktags/
Author’s Page: http://www.viper007bond.com/wordpress-plugins/vipers-video-quicktags/

If you know how to read a diff file, here’s how I hacked it in. Remember, this is only going to work universally, if your videos are .mp4 with H.264/AAC encoding.

# diff -Naur vipers-video-quicktags.php.orig vipers-video-quicktags.php
--- vipers-video-quicktags.php.orig     2010-05-23 22:15:11.000000000 -0700
+++ vipers-video-quicktags.php  2010-05-23 22:19:22.000000000 -0700
@@ -3663,7 +3663,9 @@
                //$content .= "// <![CDATA[\n";
 
                foreach ( $this->swfobjects as $objectid => $embed ) {
-                       $content .= '   swfobject.embedSWF("' . htmlspecialchars( $embed['url'] ) . '", "' . $objectid . '", "' . $embed['width'] . '", "' . $embed['height'] . '", "9", vvqexpressinstall, ';
+# dkhack, replace this line with HTML5 Video tag when it's iPhone, iPad, iPod:
+                       # $content .= ' swfobject.embedSWF("' . htmlspecialchars( $embed['url'] ) . '", "' . $objectid . '", "' . $embed['width'] . '", "' . $embed['height'] . '", "9", vvqexpressinstall, ';
+                       $content .= 'if ((navigator.userAgent.indexOf(\'iPhone\') != -1) || (navigator.userAgent.indexOf(\'iPod\') != -1) || (navigator.userAgent.indexOf(\'iPad\') != -1)) document.getElementById("'.$objectid.'").innerHTML = \'<video width="'.$embed['width'] .'" height="'.$embed['height'].'" poster="'.preg_replace("/\.[^\.]{3}$/", ".jpg",  $embed['flashvars']['file']).'" src="'. $embed['flashvars']['file'].'" controls />\'; else  swfobject.embedSWF("' . htmlspecialchars( $embed['url'] ) . '", "' . $objectid . '", "' . $embed['width'] . '", "' . $embed['height'] . '", "9", vvqexpressinstall, ';
 
                        if ( empty($embed['flashvars']) || !is_array($embed['flashvars']) ) {
                                $content .= 'vvqflashvars';

Basically, what it’s doing is detecting iPhone/iPod/iPad in JavaScript and writing this:

<video width="'710" height="400" poster="http://host.tld/path/to/video.jpg" src="http://host.tld/path/to/video.mp4" controls />

One more thing to note…. When I tested this with my iPad, it didn’t work at first. Something is wrong with the iPad – it doesn’t want to see the movie when it’s at it’s default location in WordPress after upload. Is there a 300 level redirect in there somewhere that iPad doesn’t like? Maybe something with .htaccess. Workaround: What fixed it for me was creating a new directory in my DOCUMENT_ROOT called “videos” and then I put symlinks in there to the physical files, and then used these http://host.tld/videos/my_video.mp4 URLs and it works fine.

Dave.

Spirit Jailbreak for iPad

Download the Spirit Jailbreak here http://spiritjb.com/ run and presto, your iPad is JailBroken.

First problem, the MobileTerminal app in Cydia crashes. Good workaround: I already had the iSSH app installed from App Store. I think it was $10. I installed openSSH on my iPad from Cydia, then created a configuration in iSSH for localhost at 127.0.0.1. Now I can use iSSH as a Terminal, using username root and password alpine (first login). Works. I can also SSH to my iPad when my laptop and iPad are on the same WiFi network.

If you do this, make sure to change the default root password from alpine to something else. Don’t want some random person to break into your iPad when you’re on a public network. Also change the password for the “mobile” user:

Dave-Laptops-iPad:~ root# passwd
Changing password for root.
New password:
Retype new password:
Dave-Laptops-iPad:~ root# passwd mobile
Changing password for mobile.
New password:
Retype new password:

How many people already have IPv6 connectivity?

We need to give a lot of thanks to Eric Vyncke for this fine innovation….

 

Put a 1-pixel image on your web site that lives on an IPv4/IPv6 dual stack host. This 1-pixel image is actually a PHP script that uses a special trick to figure out if the client has IPv6, whether it’s coming from a teredo tunnel, v4 or v6 preferred, and correlates it with OS and Browser.

 

It’ll be very interesting to see how the tides change as we near the feared IPv4 depletion date, which as of today is predicted to be Sept 2011.

 

You can put the 1-pixel image on your site and let your site’s traffic help feed the data.

 

Here are Eric’s stats:
http://www.vyncke.org/countv6/stats.php

 

Here are my stats:
http://ipv6poc.com/countv6/stats.php

 

I put these 1-pixel images at the footer of my blog pages:

<img src="http://ipv6poc.com/countv6/image4.php" width="1" height="1" border="none">
<img src="http://www.vyncke.org/countv6/image4.php" width="1" height="1" border="none">

 

If you want your site to feed this experiment, just drop the above two images just before the </body> tag on your site.

 

If your site gets more than 25,000 page hits a day, please reach out before putting these images on your site.

 

Oh, and you can get the source code for this here: http://sourceforge.net/projects/countv6/

 

To answer the question, here are the percentages of people that visit Eric’s site that have v6 over the last 2 years:

 

 

Interesting that it’s above 10%. It was on an upward trend till about Oct 2009, then took a sudden dive.

 

DaveK.

Load Balancing Techniques

Load balancing is a term that describes a method to distribute incoming socket connections to different servers. It’s not distributed computing, where jobs are broken up into a series of sub-jobs, so each server does a fraction of the overall work. It’s not that at all. Rather, incoming socket connections are spread out to different servers. Each incoming connection will communicate with the node it was delegated to, and the entire interaction will occur there. Each node is not aware of the other nodes existence.

Why do you need load balancing?
Simple answer: Scalability and Redundancy.

Scalability

If your application becomes busy, resource limits, such as bandwidth, cpu, memory, disk space, disk I/O, and more may reach its limits. In order to remedy such problem, you have two options: scale up, or scale out. Load balancing is a scale out technique. Rather than increasing server resources, you add cost effective, commodity servers, creating a “cluster” of servers that perform the same task. Scaling out is more cost effective, because commodity level hardware provides the most bang for the buck. High end super computers come at a premium, and can be avoided in many cases.

Redundancy

Servers crash, this is the rule, not the exception. Your architecture should be devised in a way to reduce or eliminate single points of failure (SPOF). Load balancing a cluster of servers that perform the same role provides room for a server to be taken out manually for maintenance tasks, without taking down the system. You can also withstand a server crashing. This is called High Availability, or HA for short. Load balancing is a tactic that assists with High Availability, but is not High Availability by itself. To achieve high availability, you need automated monitoring that checks the status of the applications in your cluster, and automates taking servers out of rotation, in response to failure detected. These tools are often bundled into Load Balancing software and appliances, but sometimes need to be programmed independently.

How to perform load balancing?

There are 3 well known ways:

  1. DNS based
  2. Hardware based
  3. Software based

Read more

Parallel Distributed Computing Example

You may have seen article, Hadoop Example, AccessLogCountByHourOfDay. This is a distributed computing solution, using Hadoop. The purpose of this article is to dive into the theory behind this.

To understand the power of distributed computing, we need to step back and understand the problem. First we’ll look at a command line java program that will process each http log file, one file at a time, one line at a time, until done. To speed up the job, we’ll then look at another approach: multi-threaded; we should be able to get the job done faster if we break the job up into a set of sub tasks and run them in parallel. Then, we’ll come to Hadoop, distributed computing. Same concept of breaking the job up into a set of sub tasks, but rather than running with one server, we’ll run on multiple servers in parallel.

At first you’d think that Hadoop would be the fastest, but in our basic example, you’ll see that Hadoop takes isn’t significantly faster. Why? The Hadoop overhead of scheduling the job and tracking the tasks is slowing us down. In order to see the power of Hadoop, we need much larger data sets. Think about our single server approach for a minute. As we ramp up the size and/or number of files to process, there is going to be a point where the server will hit resource limitations (cpu, ram, disk). If we have 4 threads making use of 4 cores of our CPU effectively, we may be able to do a job 4 times faster than single threaded. But, if we have a terabyte of data to process and it takes say 100 second per GB, it’s going to take 100,000 seconds to finish (that’s more than 1 day). With Hadoop, we can scale out horizontally. What if we had a 1000 node Hadoop cluster. Suddenly the overhead of scheduling the job and tracking the tasks is minuscule in comparison to the whole job. The whole job may complete in 100 seconds or less! We went from over a day to less than 2 minutes. Wow.

Please note: the single thread and multi-threaded examples in this article are not using the Map/Reduce algorithm. This is intentional. I’m trying to demonstrate the evolution of thought. When we think about how to solve the problem, the first thing that comes to mind is to walk through the files, one line at a time, and accumulate the result. Then, we realize we could split the job up into threads and gain some speed. The last evolution is is the Map/Reduce algorithm across a distributed computing platform.

Let’s dive in….

Read more

Inspiring MapReduce lectures by Google

Watched a set of 3 lectures run at Google, by Aaron Kimball, on MapReduce was inspiring to me. I feel like I have a much more solid grasp on MapReduce after watching these. I really liked how it started out with some basic functional programming and socket theory, then moved into how MapReduce builds on the basic principle of Map and Fold. Well worth the 3 hours!

http://www.youtube.com/watch?v=yjPBkvYh-ss

Thank you Google and Aaron Kimball!