Moving!

Moving!

Over the past three months I've written 31 blog entries! I figured that by now I would have lost interest in writing. But contrary to my own self-doubting I've become increasingly fascinated with writing!

I've become so interested in writing (and with the thought that others are gaining some benefit/entertainment from my writing) that I decided to help foster the writing skills of my talented (much more than me for sure) friends.

I've rented out a server, and spent a few hours setting up the free blogging software, and voila!

www.scroggles.com

As such, this blog will be moving to http://greentheo.scroggles.com

Come browse and enjoy reading all my talented friends writings!

Theo

More Reasons that Google is Really Cool

More Reasons that Google is Really Cool.

http://blog.topix.com/archives/000016.html

This article points out some of the various reasons why Google is a computing monster.

First of all, they have an incredible, almost organic computing structure which allows them not only to do massive computations on massive datasets (100 TB+ a day) but also to continue scaling the ability of those services to potentially no end.

Secondly, Google has one of the world's largest computers which is composed of thousands of smaller commodity computers. The brains of the collective computer allow each individual component-computer to fail without having to be immediately replaced. By analogy, our very own brains redirect neurons to make connections with other neurons when the situation calls for it. Also, our blood vessels can grow around damaged tissue, restoring blood flow back to normal.

Thirdly, Google has recently added a cadre of solar panels to their computing facility in order to cut back on costs and save the world from greenhouse gases!

This brings up my next question...is Google in fact transforming to become the first artificial life? It has it's own source of power (sun) and has massive computing resources much of it driven by artificial intelligence....hmm....could be!

On the Seventh Day God Rested

On the Seventh Day God Rested

...So reads the end of the first creation narrative in Genesis of the Bible.

I have often glanced over the Sabbath passages in the Bible with very little notice, except as it suited me to take Saturdays off for "religious" reasons :-)

But as I was watching a History Channel documentary on the Bible, a very interesting nuance of the Sabbath was presented.

The documentary claims that pre-Hebrew cultures had no notion of a universal day off from work (universal in that everyone from slave to King got the day off). In fact, it said, many of the cultures stood careful watch over their slaves all seven days of the week - whip in hand - to make sure that there was no slacking or rebellion. It also mentioned that the gods in these cultures did the same...always on watch 24/7.

This is where the God of the Hebrews and Christians distinguishes Himself. His day off from the work and responsibilities that an Almighty Creator is charged with is a way for Him to state His confidence in the completeness of His work/creation and providence. Furthermore, it signifies that He need not watch over us like an anxious parent ready to catch us in the act of rebellion or sin (as the other non-Hebrew gods might), because He has given us the choice of doing so and provided a framework for redemption in case we did.

To me this is a very interesting trait of God's personality brought to light by one small (in)action...resting.

Life Straw

Life Straw

This article in Newsweek (suggested by Malia) reports on a new device that works as a portable water filter that will only cost $3 and filter about 185 gallons (1 years worth of drinking water).

Most of the LifeStraw's users will never drink anything fancier than plain water through the device. But its impact on their lives can't be overstated. More than 1 billion people worldwide lack access to safe drinking water, and 6,000 people die each day of waterborne diseases like typhoid, cholera and dysentery. In regions like sub-Saharan Africa, half of most people's water consumption takes place outside the home—either while they're working, or walking to and from school. Vestergaard Frandsen S.A.—which also produces mosquito nets and plastic sheeting coated with insecticide to fend off malaria—hopes that the $3 LifeStraw will drastically lessen their chances of getting sick.

Reminds me of the fog harvesting article I put up a while ago... neat stuff.

MapReduce or How Google Rocks

MapReduce or How Google Rocks

This last week I did a presentation on Google's home grown parallel processing algorithm MapReduce which can be found here.

Google has hundreds of Terrabytes (1TB = 1,000 GB = 1,000,000 MB) of information to process every day. They have to rank, sort, and search billions of web pages across the world. And they have to do it as cheaply and quickly as possible.

Since Google has opted to use thousands of cheap rack servers instead of supercomputers to do all their heavy lifting, their software solution must fit this model. Google also has a dislike for Windows and uses only Linux computers, which are based on Unix. But Unix, as anyone who has used it before knows, has very little facility for parallel processing. (update: Perhaps I should say that Unix has little built in functionality for parallel processing and most parallel processing has to be customized to suit your individual needs/constraints.)

Thus, Google created a platform called MapReduce to distribute their giant processing tasks across thousands of cheap computers.

Essentially MapReduce is a two step process carried out by a myriad of workers and one master controller.

Pseudo Algorithm

1. Master gathers the location of the worker computers and assigns them to either the Mapping group, or the reducing group.
2. Master locates all portions of the dataset to be crunched (usually spread across hundreds of servers)
3. The dataset is parsed and chunked into 64 MB digestible portions
4. The Map workers pull their chunk of the data and emit <key, value> to the reduce worker
5. The reduce workers reduce their <key, values> to the desired output and store the final dataset on more file servers

Example - MapReduce Grep

Grep is a Unix command that searches through each line of a file for a specified piece of information and returns the lines that contain that information.

1. Master gathers workers and datasets
2. Map workers search through their chunks for the specified search string and emit <lineidentifier, 1> for each line that contains the info.
3. Reduce workers take all lines emitted, trace back the "line identifier" to the original dataset, pull the line from the file and copy it to the new output file.

Results

Given 1,800 intel Xeon 3 Ghz servers with 4 GB RAM each, MapReduce can search through 1 TB of data and return a new dataset in about 2.5 minutes. How fast is that? Well in comparison my laptop would most likely take about 2-3 weeks to complete the whole task.

Google has a lot of cool technology. I will be posting more work from their repository of research in the next couple of weeks as I will soon be working for Google so stay tuned!

The Importance of Getting out of Debt

The Importance of Getting out of Debt

In my last post I urged you all to start saving for retirement. I even did some math to show that one of the best things you can do is to start saving in an investment account as soon as possible. Even if you have debt, I explained, starting the Roth IRA now will lead to great advantages upon retirement.

However I would like to expand my analysis a bit.

My friend Will has recently turned me on to Dave Ramsey. Dave Ramsey is one of the leaders of the financial ideology that promotes the Judeo-Christian-Muslim idea of living a debt free life. Will and Dave Ramsey have pointed out that there are several benefits to first getting completely out of debt and then saving for retirement. I think they would also agree with my post against Credit Cards.

Hedge Against Job Loss

First, Will and Dave point out that if one pays off debt first and builds up a little emergency fund then a hedge against job and income loss is formed. The essential thing is that while times are good, there is plenty of money to pay back loans and monthly expenses. However when times are bad, there is no money to pay back debts which always have to be paid back. As Dave notes: Most Americans are about three pay checks away from bankruptcy.

Let No Man Be Your Master

Second, Will and Dave point out that when you are indebted to someone, you are also enslaved to them in some fashion. Have you ever thought to yourself, "Gosh I would sure love to do X, but in order to pay back my debts I have to have a job and I can't do X and have a job at the same time :-/ " ? If you have, you know what it means to be a slave to your creditor.

Psychological Freedom

While it may be true that some folks can borrow and borrow and eventually go into bankruptcy and never feel any remorse, guilt or shame, most of us feel a sort of a shadow hanging over us when we go into debt. I personally have anxiety and much doubt when I go to sign for a loan or put a good bit of cash on credit card. This psychological/spiritual burden is so prominent that Dave Ramsey actually has his callers who have just completed their debt payments shout at the top of their lungs, "I'm debt freee!!!"

So while it may be true that one can end up with slightly more money at retirement by first investing in retirement and then paying off debt, there are several great advantages to being out of debt as fast as possible: hedging against income loss, an end to financial slavery, and psychological freedom.

To Save or not to Save: A Tale of Why I love the Roth IRA

To Save or not to Save: A Tale of Why I love the Roth IRA

As an applied mathematician I feel the constant need to prove my worth by working on practical problems for the good of the general public, like a mathematical superhero of sorts. So far I have solved problems/questions such as "How much of the earth needs to be covered in Solar panels to power the world", "What is the probability of getting a meaningful 4 letter acronym from the subtitle of your blog?". I also did "The Mathematics of Credit Cards" in which I ranted against the wretchedness of the credit card industry.

Sustainability and Responsibility

Most people see sustainability as relates to industry and the environment, I see it also in terms of societal and familial preservation. In this article I hope to give a demonstration of how investing in a Roth IRA retirement account can lead to a sustainable retirement in which neither your children, nor your fellow citizens will be taxed by your failure to think about the future when you were younger.

First, how much will it cost to retire? Assuming that you retire at 65 and live to be 85 you will need at least $30,000 dollars per year (assuming you own your house and cars). This totals about $600,000 and leaves little room for error, or inflation. In fact, if we account for inflation, we will need to save $850,000 (3.5% annual inflation). And that's if we retired today! If, like me, you retire in 40 years or so, only spending $30,000 a year means you'll need about $3.3 million due to inflation!

If I made the equivalent of $50,000 a year with a %3.5 annual raise for the rest of my life then I'll have to save about 80% of each pay check in order to reach $3.3 million!

But that's impossible! Who in the world beside Tiger Woods and Bill Gates can save 80% of their pay check?

Off in the distance we see our good friend Roth IRA charging towards us on his white steed. "I'll help you!" he says!

"But how?" we ask.

"By the power of growth funds, stocks, bonds and other compound gains financial instruments....and it's all tax free growth too!"

"But Roth IRA, we heard that the stock market is volatile, and that we could lose all our money if terrorists collapse and the government fails. Shouldn't we invest in Gold?"

"No!" a voice booms. Oh look, it's Dave Ramsey, the guy who keeps telling me to get out of debt. "Gold averages about 2% return on investment since the 1900's. And in time of crisis, it's no where near as liquid as you think! Besides, starting in 1900, even with the Great Depression, the stock market averages a 12% return per year!"

Wow, thank you surprise visitors! So if we want to avoid capital gains taxes (to the tune of 15% of all gains made!) we simply chip in money to our Roth IRA. The only limitation is that you can only put up to $4000 per year into it. Luckily the maximum contribution is adjusted bi-yearly for inflation (every two years I'll get to invest more into it).

So let's see, if I chip in only about 8% of my $50,000 salary, with only 10% growth on average, I will be left with $3.3 million by the time all my hair turns gray! Sweet! That's exactly what I need!

"But wait!" Dave Ramsey exclaims. "Shouldn't you pay off your debts first, and then start to invest in retirement? After all, the credit card companies and students loans are charging you an average interest rate of 15%!"

Well let's do the math. If I have $1000 at the end of every month which I can pay towards debt, my Roth IRA, or a regular (taxable) investment account how much will I have at the end of my 40 year working career?

Let's assume I have $50,000 in assorted debt at 15% interest, and that my IRA earns 10% a year. Furthermore, my regular investment account also earns 10% a year, but I make trades (and thus pay capital gains taxes) on a third of my stock thus limiting my growth to 5%.

Scenario 1

If I pay off all my debt first with my extra money, and then begin to save in the Roth IRA and the regular investment account I will end my career with about $4.5 million.

Scenario 2

However, if I first contribute the maximum amount to my Roth IRA, and then take the leftovers to pay back the debt, and after the debt is paid back continue investing in my regular account I will have about $4.8 million in the bank.

The difference? If you start investing in the Roth IRA at the very beginning it will take you 3 years longer to pay back that $50,000 debt, but during that time you will have been able to contribute about $30,000 to your Roth IRA which will grow to be about $300,000 by the time you retire.

Whether you follow Dave Ramsey's or my suggestion one thing is clear, the sooner you start to save on that Roth IRA the better. Every year earlier that you start saving will be a year that when you retire you will thank your younger, wiser self.

If you would like to start a Roth IRA you can use the same company that I use, Edward Jones. In fact if you give my good friend Ryan Russell a call I'm sure he would appreciate your business!

Note: I have a script saved on my computer for calculating retirement savings, if you would like to see your projected retirement savings email me your financial assumptions and I'd love to help you out!

My Wife

My Wife

Malia and I have been married for nearly 6 months now. It seems like just yesterday we were getting engaged atop St. Mary’s glacier. Then we were planning our wedding in what felt like hardly any time.

We spent many nights up late planning and talking and waiting until the last minute to say goodbye for the evening.

And now we’re married and go to sleep by 10:30 or 11:00 pm at night. And it’s wonderful!

I’ve learned so much about Malia and about myself. I know that she feels the same. And it’s only been 6 months…how much more will we get to learn in the coming years?

She’s learned to live with my dreaming/ranting about Solar Parabolas and how we should never pay for electricity again. She understands that I have a world changing idea every other day, only to be replaced by a new one soon after.

And she’s learned technical terms like PHP and Matlab. She even enjoys watching documentaries with me!

And though it’s only been six months, and we’re still naïve newlyweds I can say without a doubt that marrying Malia has been the best and most rewarding thing I’ve ever done.

As it says right there on the side of this blog…I love my wife!

Machine Learning - Introduction

Machine Learning - Introduction

After my latest blog on "Why Robots are the way of the future" I realized I have not really written about one of my greater passions...Machine Learning/Artificial Intelligence.

Machine Learning and Artificial Intelligence are not quite the same thing. Usually Machine Learning falls under the category of Computational Science, Mathematics, Computer Science and Statistics. Artificial Intelligence usually belongs to the Computer Science, Cognitive Science, Psychology and perhaps Mathematical Philosophy departments.

Machine Learning is generally the field of study related to the question: How can we teach a machine to perform a certain task as well as it can be performed?

However, Artificial Intelligence is generally an answer to the question: How can we simulate (or reproduce) the cognitive functions of humans/animals/intelligent beings? A lot of Artificial Intelligence seeks to model the human mind, which doesn't always do things optimally.

Both fields of study are important for developing algorithms and systems that are able to interact, aid, and enhance our lives. For instance, Machine Learning was used to create software to control a refinery. Up until the 1980's most refineries and their myriad pipes, valves, and gauges were controlled by humans. Any potential catastrophes had to be averted by alert workers. However, ML systems were developed that could optimally monitor and control refineries...even better than humans can! Furthermore, as conditions in the refinery change over time the software adapts and retrains itself without having to be reprogrammed over and over.

Interestingly, many of the Machine Learning algorithms are fairly simple in their approach. For instance, a basic classification algorithm is as follows:

Given a set of descriptions x and their associated objects/predictions y which are part of a set of classification categories {y_1, y_2, ... , y_n},

Build a model (a brain)

- For each unique classification y:

-take mean(x) for all x associated with the classification categories y_n

Make Predictions

- take an unclassified description x and evaluate the distances to each of the category means

- the unclassified description x will be classified according to the closest mean

Enhance the model with new examples

-given additional information x and y, recalculate the mean(x)'s with the new information...the model is now enhanced.

For example: Bobby stands by the road and for every vehicle that passes by quickly measures the length of the vehicle. Bobby also writes down whether the vehicle was an "18-wheeler" or "other". Using the above algorithm he calculates the mean "18-wheeler" length to be 20.4 feet, and the mean "other" vehicle length to be 10.9 feet. A blind girl named Jane comes along and says that she has just measured a vehicle that is 16.9 feet long, but can't tell if it's an "18-wheeler" or not. Bobby says that since the unobserved vehicle is closer in length to an "18-wheeler" it must be one!

This may not seem like a very intelligent algorithm (and it's not) but it does demonstrate one key feature of an intelligent algorithm, the ability to form it's internal computing algorithm via external data.

In the next few articles on Machine Learning I'd like to discuss some other more intelligent algorithms such as Support Vector Machines, Neural Networks and Random Forests...all three of which are some of the coolest and most effective AI/ML algorithms. Until then, see if you can think up your own intelligent algorithm...or use for an algorithm. I'd love to hear about it!

Robots are the Way of the Future

Robots are the Way of the Future

I have this really cool robot called Roomba made by iRobot. It vacuums the floor for Malia and I. All we have to do is push the clean button and away he goes zipping around the carpet in a semi chaotic pattern until it's all clean. And it works too!

This is a great development for 2 reasons

1.) It's just cool.

2.) It really does save us time.

Statements like Robots are the way of the future may seem kind of cheesy and cliche. However, I would like to say that there are plenty of cool robots out there. And as Artificial intelligence and Machine Learning develop, robotics will only become more and more useful.

A few months ago on PBS's Nova a program about the Great Robot Race , sponsored by DARPA, was aired in which many vehicles competed to be the first completely machine driven vehicles to make a trek across a desert in California/Nevada. The winner was a small SUV driven without the help of GPS (which is amazing!). It operated on stereo vision, laser vision and a few other sensors, all controlled by a computer with Machine Learning software. Essentially the researchers who created the small intelligent SUV taught the car to drive in the dessert, and it succeeded on the gruelling 150 mile course with an average speed of more than 25 mph!

The significant thing about robots like the winner of the DARPA challenge and Roomba, is that they are actually learning the task set out for them. By learning I mean that the robot's own internal algorithms are being modified as the robot carries out it's task in response to poor or good performance. If you think about it, we learn this way most of the time. In fact, our whole public education system falls under the category that these robot's education falls under: Supervised Learning. (Well ideally at least, a lot of education falls under the category of the 'Expert System', where the teacher poses as an expert and relays his wisdom and knowledge to the student via memorization.)

Supervised Learning is the process where a student is presented a set of examples (descriptions and their corresponding object), and asked to perform a similar identification task on some previously unseen examples. The performance on the unseen examples is evaluated by a supervisor, and the feedback given to the student. The student takes the feedback and reshapes her internal processing algorithms. The process repeats until the supervisor is satisfied with the student's performance.

Robots can even become so good at learning, that they can interface with our brain's functionality! In this story, a man has a prosthetic arm attached to his body and nervous system. Over time, he and the robotic arm are learning to communicate with each other...now that's cool! Star wars is no longer the fantasy world.

I love Machine Learning and Artificial Intelligence, not only because we can use it to make prosthetic arms and automatic carpet cleaners, but because it teaches us something about ourselves. When we explore the world of learning we also question how it is that we know things. We question what knowledge is, how we obtain it, and ultimately what reality is. And at the end of those questions we will always have more, but that's the point of learning!

They Took My Toothpaste!

They Took My Toothpaste!

Last night my Mom flew to Houston from Denver. As she passed through the TSA security check point, the guards notified her that she would have to relinquish her recently purchased black cherry jam. You can imagine the disapointment she must have felt since black cherries don't really grow in Houston.

I experienced a similar disappointment last year when flying to Cozumel. I had just bought some new toothpaste. In my excitement to get to Mexico I forgot all about having packed in my carry on baggage. I forgot about it until, that is, I was notified that I would have to hand over my toothpaste. Doh!

To me it was a feeling of disappointment and bewilderment, akin to having my bike stolen a few years ago. This was further compounded by the regret that I could have saved my toothpaste from the grubby paws of the TSA had I put it in a ziploc bag. Why a ziploc bag anyway? It must have something to do with explosives. But it borders on absurd to think that somehow would-be-bombers are thwarted by putting their liquid explosives in plastic bags!

The question arose in my mind, "What do they do with all the contrband anyway?" My mother and I can't be the only ones who have forgotten to put our jams, toothpastes and other dangerous fare in protective zip-loc bags.

It turns out that the airports are turning a tidy profit on these items! http://www.cbsnews.com/stories/2006/08/12/national/main1890143.shtml . This article reports that the Pennsylvania airport is making about 100-200 thousand dollars a year auctioning off those confiscated pocket knives and scissors on eBay!

And as for the toothpastes and black cherry jams of the world? The fortunate (or unfortunate) community of homeless get those. While I don't mind that the homeless are getting a break at my expense, I do mind that the TSA apparently has so little concern for their safety that they are shipping the homeless hundreds of potential bombs and chemical warfare agents!

But you'd have to be nearly subhuman to be that blatently calous and uncaring for the poort. No, there's probably an easier explanation. The real reason they ship my toothpaste to the homeless is that the TSA has very little conviction that my toothpaste and my mother's black cherry jam is anything other than what they are supposed to be. Which reminds me....why did they confiscate my toothpaste again?

Airplane's are Awesome!

Airplane's are awesome!

Last Wednesday I got to go flying with my good friend Josh. He and his instructor, allowed me to come along with them as Josh practiced his short runway and grassy field landings.

Shortly after 9:30 AM we were taxiing down runway 11R, getting ready for our turn on the tarmac. The air was buzzing with the sound of the propeller and a few of the other neighbouring plane's engines. Luckily Josh got me a pair of airplane head phones which both block out all engine noise and with the attached microphone allow everybody in the airplane to talk to each other!

Finally our turn came. We were positioned at the beginning of the runway facing south. There was a wind blowing from the North (I think). Josh gave the engine it's full power and kept the brakes on. As the plane rocked a bit he finally released the brakes and off we jumped. Within a few feet, the nose was off the ground.

As Josh concentrated on keeping the nose in the air and the tail from smashing into the ground we wiggled down the run way. As I was to later learn, the wiggling, or more precisely, the yawing, was due to the fact that the ultra concentration of our brave pilot caused a lack of concentration on the rudder. Thus as we took off I realized that riding in a Cessna 172 is nothing like riding in the comfort of a 737!

We climbed high into the air and I could see all of Boulder, Broomfield, Superior, Louisville and some of Lafayette. Since it had snowed in the higher elevations the previous night, the Front Range mountains were quite majestic!

After about 7 minutes, the Erie Airport came into view. We made some preparations and then we landed...but we didn't stop! Nope, we just sped up and kept on flying. Once in the air we circled back around and did the same thing about 5 more times.

Finally our pilot and the instructor had grown tired of terrorizing me and decided that it was time to fly back to Jeffco for a real landing. After making one pass and being rejected by the air traffic controller, we were finally allowed to make a final safe landing.

Flying around the Boulder area in a little plane was a really wonderful experience. It still amazes me that human beings have managed to get themselves off the ground and into the air at such great heights. The thrill of flying and the wonder that it inspires have created a great desire in me to one day (hopefully) soon acquire a private pilot's license!

Thanks Josh for your wonderful airplane skills!

But How Can I be Sure my Widget is Actually Working?

But How Can I be Sure my Widget is Actually Working?

Welcome to the world of statistics, where we try to convince ourselves and our employers/clients/friends/wives that the projects we have spent hours and hours on are actually worthwhile! We will see by the end of this article just how you can too can demonstrate with reasonable certainty that your very own predictor/estimator/widget is having a positive (or negative effect) on the process it is being applied to.

Yes, we're talking about hypothesis testing!

What hypothesis?

The hypothesis that results from the question about whether our widget has a positive (or negative) effect on the process, of course ! If we want to know whether the widget has a positive or negative effect then we must hypothesize the opposite (that there is no effect, or that the widget and the normal process are the same) and try to prove it wrong! This may sound counter-intuitive but one limiting factor about statistics is that you can't prove a statement true, you can only use it to reject a statement (though your're not disproving it).

For example, for my stock market predicting widget, I want to know how I can be 95% confident in my belief that my estimated prediction accuracy of 57% is not just random chance (luck). If you've ever studied the stock market you've probably heard the cliche about how someone let monkeys pick stocks, or threw darts at a dartboard and the resulting random selection of stocks did better than "such and such" a famous money manager. Well, I don't want to end up being the money manager that gets beat by a monkey, that's for sure!

First, in statistics, we are always estimating parameters. In this case we are only estimating the ability of my stock predictor to make correct stock predictions. Because I can't, or wont, test the predictor on every stock ever I can't know it's true 100% ability to make predictions. But this is where the beauty of mathematical statistics comes in...we don't have to! I can estimate the stock's accuracy on a relatively small number of days within the stock's history and then use the principals of confidence intervals to establish a level of confidence or belief that my predictor is better or at least not eqaul to a random predictor.

Secondly, we want to know something about how we expect the process to function without the help of our widget. For my stock widget, I want to know if my predictor is significantly different than a random predictor, i.e. a monkey throwing a dart at a dart board filled with stock predictions (if monkeys could do that).

So how do we know what our process will do without our widget? Easy! We simply sample the output! Sometimes we can do this theoretically, as in the case of my stock market predictor.

My stock predictor only makes predictions on whether the stock should be bought or sold. I can say that the accuracy of my stock predictor, which comes in the form n correct predictions out of N attempts, looks a lot like a Binomial distribution! Actually, Binomial distributions look a lot like Normal/Gaussian distributions ...otherwise known as the Bell curve. A Binomial distribution can be produced with the following 4 steps:

1.) flip a coin 100 times and write down how many times you got heads - we'll call this a "coin flip trial" with a heads population of "p-heads".

2.) we do 1000 "coin flip trials" - yes it will take a while

3.) after the "coin flip trials" are finished we make a chart and plot each unique value of "p-heads", and the number of times we got each one of the "p-heads".

4.) We stand back and marvel at our new representation of the Binomial Distribution - it should look this.

This is what would happen if I randomly chose whether the stock would go up or down. It would achieve 50% accuracy on average.

If you get an evenly balanced coin you will notice that your most popular choice for "p-head" is 50. You will also notice that most of the values fall within the range 45 and 55.

Now, let's say that the evenly balanced coin is a random stock predictor. It will only predict half of all possible stock moves correctly... i.e. you lose as much as you make over time. But when I run my stock predictor on a random sample of the market it tells me that I got 57% of the predictions correct!. Since this estimated accuracy is only on a small random portion of the market, how do I know that I didn't just get lucky? How do I know I'm not a monkey (this may be an altogether different question :-)?

Going back to my hypothesis, I need to evaluate the claim that (Accuracy_random = Accuracy_mystockwidget). But this is the same as evaluating (0 = Accruacy_mystockwidget - Accuracy_random) Now, we know that the accuracy of the random predictor will be distributed according to the picture above. Incredibly, my stock predictor should actually follow the same pattern, only shifted over to the right slightly. But here's the even more incredible thing, if we subtract the two variables, and plot the results the associated relative frequencies, it will also look like the above!

If the resulting distribution contains 0 in a centered 95% selection of it's values about its mean, then we will be forced to make the satement, "We fail to reject the hypothesis that Theo's stock widget is equal to a random monkey predictor." However, if 0 is not found about this center 95% of the distribution, then I can proudly say that I reject the claim that they are equal in favor of the claim that my predictor is different from a coin-flippining monkey (and I will later go on to say that of course it is much better!). So what do the numbers say? Let's take a look at another sweet chart!

And so we see that from a statistical standpoint, my stock predicting widget is significantly different from a coin-flipping-dart-throwing monkey predictor!

Isn't statistics cool?

note: I haven't yet acheived 57% accuracy yet, so hold your horses and your money until I do :-)

This weekend at the Barbecue

This weekend at the Barbecue

I love grilling meat. There's just something very satisfying about meat cooked over an open flame. Perhaps it is my Texan nature. Perhaps it just tastes good.

Whatever the reason, Malia and I ate grilled food twice this weekend...and it sure was good!

On Sunday we had some nice barbecued chicken and grilled asparagus. Grilled asparagus is one of those surprising foods. You never really think it's going to be that good, but it always sort of is. It's not really mind-blowingly good, but just good enough for you to say..."Hmm...that's pretty decent." Furthermore, I think eating asparagus makes you feel morally good, and like a responsible adult. I always think that I'm staving off cancer by eating things like asparagus, broccoli and artichokes.

On Monday, my lovely wife and I decided to have a Memorial day celebration with as many friends that I could think to invite. I couldn't think of everyone though, and unfortunately forgot a few folks. Oh well. If you are reading this and you didn't get to come and eat flame cooked meat with us then I invite you to come over whenever you would like for some good grillin' fun!

In addition to our fare of meat and the requisite potato chips I decided to make some sweet potato fries. They ended up as sweet potato soggy chips, but they were still quite tasty. I found some spices from Chef something-or-other on sale at Albertsons which made my yam-fries quite delicious! If you are interested in making these delicious and nutritious fries just slice up a sweet potato or two, put some salt, pepper and whichever spice you like on them, put them in a baking pan with olive oil and cook them at about 375 F for 20-30 minutes. So delicious!

Fact: Sweet potatoes have more beta Carotene than carrots. Fact: Olive oil is "good for you"...therefore you can use as much of it as you want when you cook. Fact: the previous statement is probably not true.

At Malia's prompting we all played a cool game called celebrity. I wont explain all the rules but it can be played by any number of people and involves the guessing of various famous people or cartoons through word hints and acting as if you are the person or character. I failed miserably at acting out Snoopy. Our friend Barry did a find job at acting out Ronald Reagen however. And I learned from Kevin that St. Thomas Aquinas is also referred to as "The Prince of the Scholastic Philosophers".

Alas, the weekend of glorious barbecuing and fellowship came to an end. Now it's Tuesday and grilling will have to wait until next weekend when we hike into the wilderness and have a real caveman-style barbecue somewhere in the mountains!

A Quasi Philosophical Blog

A Quasi Philosophical Blog

At the request of my brother Arthur this blog entry is going to be long and philosophical in it's essence. Incidentally this blog entry comes not only at the request of my brother Arthur, but also inspired by a 3 part documentary called The Trap by the BBC that he found floating around the Internet. Arthur is an expert at finding quality material on the Internet to stimulate and inspire the mind :-)

These incredible documentaries can be found at:

Google Video - The Trap Part 1

Google Video - The Trap Part 2

Google Video - The Trap Part 3 (note, Part 3 is broken into 3 parts...this link is to part 1 of 3. The other 2 parts are easily accessed from this link.)

More than anything, these documentaries have impressed upon me the notion that philosophy is not an idle academic subject, but a vibrant and nearly living thing. At the very least, the conclusions that we have drawn about the world, the way it works, and it's ultimate purpose have dramatically shaped our government and societal structures. This documentary covers the brief span of history from Post WWII to present regarding socio-political philosophies of the Western world, their consequences and possible future ramifications.

What the film makers saw as they studied the last fifty years of history was something that in philosophy we call Post Modernism. In the film the claim is that after WWII governments, politicians, and the average educated elite began to believe and follow the economic/mathematical theories of the human person as a rational agent in a very large and complicated game. Human beings, the economists and mathematicians theorized, are little more than rational agents acting on behalf of their self interest. Under this "efficient market" theory, human beings will rationally process all publicly available information and make decisions based on that information. The mathematical branch of Game Theory along with some questionable psychology was then used to further "prove" their theories. (See Note: 3)

The economists and government planners then postulated that under efficient market theory, government should step out of the way and let the markets be as efficient as possible. For in this worldview, maximal happiness, and societal equilibrium (i.e. a classless society) must occur when the rational agents are left to their own devices.

Government and a society then are the sum of the many free willed and self-centered agents. More or less we would all become cogs in the machine of humanity...equal cogs...free cogs!

Unfortunately for the brilliant mathematicians and economists (like John Nash who shaped Cold War era government policy, and who was also suffering from paranoid schizophrenia) our country, and certainly not our world, has failed to live up to the democratic-utopian ideal. Furthermore, feelings of alienation and purposelessness seem to have increased (my anecdotal opinion). And unfortunately rather than becoming cogs in the glorious humanity machine, the majority of modern Americans have become cogs in the corporate/consumerist machine (from this fact: we have record amounts of credit card debt mostly because we "need" so many products). The middle and lower classes have not had any real increase in the share of wealth, and thus classes remain as defined as ever (Wikipedia , see note 1). Good health care is still for us relatively wealthy in the world. And the vast amount of technology we have developed has failed to relieve us of a 40 hour work week. We in fact work more than 40 hours a week (Wikipedia)!

And thus we have arrived at our Post Modern, cynical and disillusioned, Prozac-society. We no longer trust government. We no longer really trust each other. We are less educated (graduation from high school rates may be high, but I don't see that as more educated). It seems that we have regressed!

So what went wrong?

To quote my good friend Edward, the major policy shapers of the 20th century started with a false Anthropology.

What are the consequences of holding such a false Anthropology?

In short, the consequences of such a belief can be seen (as the documentary hints at) in the Stalins, Lenins, Mao Tse Tungs and Pol Pots of the world. The massive amount of death and destruction in the 20th century is a result of the philosophical attitude that man is nothing more than machine, and thus just as expendable. The utlimate conclusion of such belief can only be the Nietszchean "Will to Power" in which the Uber Mensche (Super or Powerful Man) wins. Another consequence of the man-as-machine philosophy is purposelessness. As Nietzsche even pointed out, the Uber Mensche is one who realizes the purposelessness of it all, even power and fame, and still has the will to carry on, to pursue power and rulership! Unfortunately, he also saw that the great masses of "weak" (i.e. normal) people would sort of flounder sans meaning. (See Note 2)

There is a progression in the philosophy of the west that I see.

1.) Aristotle, Aquinas - truth can be known by asking enough questions. God is real because of cause and effect (Prime mover). If God is real then we are real, and important, for he created us. Knowing God is real and important. Aquinas' era is the apex of Christian influence in Western Culture

2.) Machiavelli, Hume and Kant - Morality is relevant to your position in life. Cause and effect is not provable. A priori truths (e.g. universal truths) cannot be proven outside of a formal system (Mathematics, formal languages). God is a hypothesis which may not be provable (though they still maintained that He was real).

3.) Nietzsche, Kierkegaard, Darwin- The implications of Kant and Hume are that absolute truths, moralities and God cannot be absolutely known (unless somehow experienced). "God is dead" - Nietzsche. This is also the beginning of the end for Christianity as the major cultural ideology. Evolutionary biology adds that man is in no way unique, just a bi-product of the randomness of our universe. There can be no core essence of man, just a very complex machine composed of neurons and chemicals. (Hitler with his campaign of blood and soil is the perfect embodiment of this philosophy)

4.) Jean Paul Sartre - wrote a book called The Nauseum. He said that life is no better or worse than vomit. Pol Pot loved Sartre. Pol Pot killed millions of his countrymen.

5.) Modern Philosophers/Scientists - since nothing is provable/knowable but Mathematics, then Mathematical Science is the best way to describe our world, even ourselves! Thus the rise of economics in place of socio-political theory.

Because of our massive shift in philosophical understanding of the human person, namely that we are better described by numbers than by adjectives, our society has become what it has become: kind of depressed. The human being in our world has come to believe that he/she doesn't even really exist, except in a possibly material sense. And if we don't really exist, then what the heck is all this for anyhow?

Many many more pages could be filled with the ideas and thoughts of many many more philosophers. But as I stated in the beginning the main thing that I really walked away with from these documentaries is just how effective and powerful ideas and philosophies can be. Our country, and its people, is shaped to a great extent by its understanding of the world, the universe and human nature. A good life and a good country then starts with none other than a correct Anthropology/Philosophy, and that makes it one of the most important subjects to study!

note 1: Furthermore consider that the average household now has 2 income earners, and only makes about 30% more...what a rip off!

note 2: This is my favorite passage of Nietzsche's, this is a summary of it. I love it because it really captures the spirit of the age and the astounding consequences of a society without an immediate and active God.

note 3: a small (or rather large) hitch in that theory is that most markets are demonstrably inefficient...

Credit Card Mathematics

Credit Card Mathematics

As the last couple of mathematically related articles I've written have been kind of useless here is something eminently practical: periodic interest...i.e. the highway robbery of Credit Card companies.

First a brief history about credit cards. Prior to 1978, credit card/revolving loan providers were limited to about 12% interest rates. Most states capped interest rates under "Usury" laws. Usury is the term used by Christianity and Islam to describe the lending of money at an interest rate. It has now come to mean loaning money at rates that are excessive, or loan-sharkish. Only Islam remains steadfast in their commitment to ban the loaning of money at any interest rate. Admirably, at least to me, Muslim bankers prefer to make business loans in which they become a stake holder in the business. This has the advantage of ensuring that the bankers are business partners who are intimately tied into the success of the business and community, not just profiteers. That's pretty cool (and fits into my off-grid idealist perspective).

However in 1978 in Marquette vs. First Omaha National bank the Supreme Court ruled that it was permissible for a nationally chartered bank to "export" their interest rates. Basically, a loan originating in New York no longer had to adhere to a 12% loan. The banks could now charge the maximum interest allowed in the state of origination of the credit card. Delaware and South Dakota had the least amount of usury protection (for the consumer) and thus the major credit cards and lenders relocated their loans to these states. Thus credit card users can now be charged any amount of interest up to about 30%. Furthermore, almost any type of fee can be charged, and have almost no limits to their maximum amount.

Last, laws protecting the consumer from rate variance have significantly changed. For nearly any reason whatsoever, the credit card companies can change the terms of the loan, whenever they want. Imagine if mortgages worked that way...that would be frightening indeed! I wonder who thought that loosening controls on banks was a good idea anyway?

Anyhow, onto the math.

Interest is an easy calculation.

Let P = principal.

Let d = number of days that interest accrues before a payment is made.

Let r = daily periodic rate (an APR of 4% has a corresponding Daily Periodic rate of 4%/365)

Then your balance at the end of the month for a credit card is:

Balance = P*(1+r)^d

So for a credit card with 10,000 dollars on it, and an APR of 18% after one month we will accrue a balance of

10,000*(1+.18/365)^30 = 10149.01 dollars.

Or we end up paying 150 dollars in interest! After a year, if we pay 200$/month on the card we will still have a balance of...9335.371! Man that stinks!

At the rate of paying off 200$ a month it will take us more about 8 years to pay it off, and will have paid more than 7000$ in interest! Man that stinks!

In summary, if you have to use credit cards, then do so...but if not...then stay away! It's a bad deal. But the real question is, how can paying such high rates to banks be at all good for the economy? If we're all paying loads of interest to the banks we can't spend money on real goods. If we can't buy real goods, how can the economy provide jobs? If there aren't any jobs how can we pay back the interest?

For now it seems that wages are keeping up with loan repayments. What happens when that changes? How many people have to default on loans before the house of cards collapses? I suppose we'll see in the near future!

It seems that the banks may be pulling the rug out from under their feet in the long run... and definitely hurting the average working American in the short run.

An estimate of the Probability of getting a 4 Letter Acronym Formed by Blog Subtitles

Kent at The Digression blog asks: What is the probability of a blog's subtitle's first letter of each word producing a meaningful acronym?

Apparently the subtitle for my blog makes the acronym MPEG. MPEG is the acronym for a file format that contains movies or motion pictures.

Well be careful what you ask for, cause here's my answer! :-)

First, let's simplify this by asking, what is the probability of producing a meaningful 4 letter acronym given that the blog has a 4 word subtitle.

The probability of getting a meaningful 4 letter acronym given that the blog has a 4 word subtitle is: P(4 letter meaningful Acronym blog has 4 word subtitle).

Using Bayes rule of conditional probability, we can say that P(4A blog4subtitle) = P(blog4subtitle 4A) x P(4A) / P(blog4subtitle).

The P(4A) = #meaningful 4 letter Acronyms / #of Possible 4 letter Acronyms

The P(blog4subtitle) = #of blogs with 4 word subtitles / #of blogs, the probability that out of all blogs, the chosen blog has a 4 word subtitle.

The conditional probability P(blog4subtitle 4A) = Probability of getting a blog with 4 words in the subtitle given that it has a 4 letter meaningful acronym) = 1

So then P(4A blog4subtitle) = 1xP(4A) / P(blog4subtitle).

Let's proceed shall we?

A rough estimation of the #of meaningful 4 letter acronyms is... well that's kind of hard. Ok so here's where we can get all statistical.

On Wikipedia we can find a list of all acronyms known to wikipedia . Sampling the population of A acronyms I can count that for each section of the A page (26 sections, the acronyms are broken down into the AA, AB, AC...AZ sections) there are about 10 4-letter acronyms. Let's assume that the actual number per section is distributed according to a normal distribution ~ N(10, 2)...i.e. The number of 4 letter acronyms per section is 10+-about 2 per section. We can do this because for large n, the binomial distribution is approximated by a normal distribution. There are 26 sections per page...and 26 pages...thus 676 sections. Taking this into account, we can say that the number of 4 letter acronyms in existence has a sampled distribution of ~N(676*10, 676*2) which means that there will be on average an estimated 6760 4 letter acronyms based on our small sample distribution of 1 section of the A acronyms on wikipedia.

On the other hand computing the number of possible 4 letter acronyms is easy.... 26X26X26X26 = 456976.

Estimating the number of blogs with 4 letter subtitles is also difficult. But again, let's say that the number words in the subtitle of a blog is distributed according to a binomial distribution with mean 6. Assuming that the max number of words in the subtitle is 20, the probability of getting a 4 word subtitle can be approximated by 20!/[4! x 16!] x .3^4 x (.7)^16 = .13 This means that obtaining a blog with 4 letters would be the probability of obtaining a 4 letter blog times the number of blogs available.

The available #of blogs is 66 million...according to BlogHerald .

So(P(blog4subtitle) = .13 x 66million/66million = .13

Thus our final estimation of the probability of getting a meaningful acronym given a four word subtitle is distributed according to a N(6760/456976/.113, 1352/456976/.113) distribution. (It is a distribution because I had to estimate the number of 4 letter acronyms in order not to have to count them.) Thus I can't be 100% sure what the real probability is. However the mean probability from my estimation is about 11.3% with a variation of 2%. And I can say with 99% confidence that the true probability of obtaining a 4 letter meaningful acronym given my 4 word subtitle lies between 6% and 16%.

That was fun wasn't it? With statistics anything is possible to estimate!

Why I Want to Live Off the Grid - Part III Producing Your Own Food

Why I Want to Live Off the Grid - Part III Producing Your Own Food

It seems that lately organic food and natural lifestyles have emerged on the National scene as never before (see note 1). More and more Americans are taking hold of the belief that food grown without chemicals, preservatives and antibiotics is better for your overall health. I think that this trend is really cool.

However, the problems with organic produce remains: it is substantially more expensive, and it is shipped across long distances. This puts a great strain on our environment in many ways. It also puts a great strain our budgets :-)

A benefit to getting off the grid then is the ability to obtain locally (in your backyard locally!) produced, cheap, organic vegetables and fruit. And if you have a little bit of land, you could even get some fresh meat out of it.

Let me first say that there are few things that will taste and satisfy hunger like fresh meat and vegetables. There is nothing like chomping down into an all-natural-grass-fed-beef hamburger when you're hungry!

Some quick math: Seeds to grow your food 1 $/packet, mini sprouting greenhouses 3 $/house, various gardening tools 30 $, fertilizer 10 $/year, water roughly 0 $. Thus to grow 10 different vegetables in your backyard would cost approximately 10+3x3+30+10 = 60$ the first year and 20$ the second year. I don't know your budget, but with my budget this would save us about 50 dollars a month = 300-600 $ savings over a year's time! Now what to do with all my money!

Starting a garden in your backyard, balcony, or even just a sunny window will bring hours of fun, "fruitful" labor, and much delicious produce free of chemicals and unwanted possible side effects. And if you take the extra step of learning how to naturally preserve food, it can feed you and your family throughout the year with the added benefit of naturally occurring pro-biotics which have been proven to aid digestion and keep your gut and thus body in great condition!

A small portion of my backyard. When filled, it should provide vegetables to feed my wife and I nearly year round (I'll still need red meat of course).

First, you'll want to select a nice sunny area of your backyard that you can water easily. A sunny window will also do, just attach a nice little window box and plant whatever you would like in that.

Second, you must pluck the weeds and till the soil prior to planting. Tilling mixes up the soil, helps it to breathe, and to be soft enough for the new roots to grab in!

Third, you'll want to sprout your seeds at least 6 weeks prior to planting. Stores like Lowe's and Home Depot make these great miniature green houses which are well suited to sprouting. The mini green houses have these great circular disks of peat which hold the fledgling plants until they are ready for the great outdoors.

Poor quality picture demonstrating the mini-greenhouses I've been using to sprout my garden. Here in Colorado the potential for snow and freezing lasts until about May 15, so sprouting indoors prior to planting is nearly imperative.

Fourth, once the seeds have sprouted and have obtained a decent strength it's time to plant them in the soil. Plant them in the soil and cover the remaining soil with weed cloth. The weed cloth will let the soil breath, get water and keep the weeds from choking your plants!

Fifth, water ever day when the plants are young. Make sure the roots don't dry out.

Sixth, when the fruits/vegetables get big enough, pluck 'em and eat 'em!

Seventh, if you can't eat them all, preserve them! Cucumbers can be made into pickles, and tomatoes can be made into ketchup. Cabbage can become Sauerkraut, and raspberries can become jam.

Eighth, let your soil rest and get ready for next year!

Well I hope that after reading this you too will grow a garden full of healthy and delicious vegetables. And remember, gardens can be grown both in the fall and late Spring. Enjoy!

note 1: It could be said that prior to the 60's and 70's food was predominately Organic. Thus the trend towards organic is really a reversal of the highly processed and unnatural agriculture of the late 20th century.

A Cool Fact about Uniform Distributions

A Cool Fact about Uniform Distributions

On Monday I received the problem of the week from Doubleclick Technical Group guru, Andrew. The problem was as follows:

If I have a hat with all the numbers from 0 to 1 in it, and I start randomly choosing numbers from it...how many times will the number I draw be the maximum value for the numbers I've already drawn when I draw a total of n numbers from the hat? It should be noted that the question is asking: on average how many times will the maximum value change.

I know the answer is obvious to most people, but I had to do a little bit of work to get it. In the end I discovered a cool fact...that Uniform distributions and Beta distributions are related!

First what we really have here is a process that selects a random variable (r.v) from a Uniform(0,1) distribution. A uniform(0,1) distribution has mean .5 and variance .25. Uniform distributions can be used in many cool ways, the least of which is simulating a coin flip.

A depiction of the probability density of a Uniform distribution

Secondly, when we talk about the maximum of a set of r.v, max{X1, X2,....,Xn}, we can also refer to the last element in the sorted, or ordered, set {X(1), X(2), .... , X(n)<----this would be the maximum}. In this case the sort goes in ascending order.

Here is where it gets interesting! (ok interesting if your into this sort of thing, and I am...so....) When we sort the random variables produced from a Uniform distribution, the ordered set takes on the appearance of a Beta distribution! Furthermore, each element in the set has it's own specific Beta distribution....i.e. the k-th element in the set {X(1), ... , X(k), ... , X(n)} has the distribution Beta(k, k+1-n)! Wow!

These are various depictions of Beta probability density functions...neat-o!

Now, we know that the Beta(k, k+1-n) distribution has a mean value of k/(k+k+1-n). This is quite useful, because in our problem above we are asking the question: how many times does the maximum change, or rather, what is the sum of the probabilities for each draw to be a maximum. We can conclude, with some thinking, that on average after n draws the maximum of a set of uniformly drawn r.v.'s is 1 - the mean value of the n-th Uniform r.v. from the ordered set. Thus the Probability that our nth draw is the new maximum is Pr(Xn = max) = 1 - mean(Beta(n, n+1-n)) = 1 - n/(n+1)! It should be noted that I am interchanging the actual value of the n-th draw with it's probability because for the r.v. to be a max it only has to fall within the previous max's value and 1....making the probability of the draw being the max and the mean value of the Beta(n, n+n+1-n) the same thing.

So... if X0, X1, ... , Xn-1 are our draws...we have the following:

Pr(X0 = max) = 1 -0 (first draw must be the maximum)

Pr(X1 = max) = 1- 1/2 (1/(1+1))

Pr(X2 = max) = 1- 2/3 = 1/3

Pr(X3 = max) = 1- 3/4 = 1/4

.

.

.

Pr(Xn-1 = max) = 1-n-1/(n)

And by summing up these probabilities, we get the expected or average amount of times that the number drawn is the maximum...

sum(1-j/(j+1)) [for j=0 to n-1] !

So for n=10,000,000 we expect to obtain about 14 new maximums along the way!

And you thought Math was useless :-) Next week if you're lucky I will discuss how most natural processes can be described by a Normal distribution...including the stock market!

My First Trip to NewYork

My First Trip to NewYork

These past few days here in New York have been my first exposure to the Big Apple. While I was a bit nervous about spending time in a large fruit it turns out that the big city is very interesting and an experience that all who count themselves cultured should have.

Sunday afternoon I caught a flight out of Denver International Airport. If you've ever been to DIA, then you know that it is a gigantic and very nice airport. You even have to take a train from the entrance to 1 of 3 terminals in order to fly out! Thus it came as a great surprise to me when the plane finally touched down at LaGuardia Airport in New York. I had thoughts of being greeted by fanciness and class. Alas Laguardia was, well, like a well worn shoe :-)

Nevertheless I obtained my bags and walked out to stand in a 20 minute long line for a cab where I struck up a conversation with a very nice man from 'Queens'. He broke my stereotype of New Yorkers as gruff and uncaring people (and as I would later discuss with my new Manager Garrett....New York is all about diversity and breaking stereotypes!).

After checking in to my hotel I decided to take a walk and get something to eat. I was a little afraid of getting robbed (stereotype #2) so I was walking with an alert eye and ready to defend myself at any moment. I kept remembering all the horror stories from my old Jiu-Jitsu instructors from New York. I was ready to perform an Asha Garuma (leg wheel throw) followed by an arm or wrist lock to any potential villains along the way. Eventually my mind was settled that New York City was acceptably safe after all.

I eventually found a pizza shop on Times Square. But the pizza wasn't great. I was disappointed that stereotype #3 about New York wasn't as true :-/ (Later I gave another pizza shop a shot and their pizza was great...stereotype #3 = true!)

The next two days of my trip involved meeting with Warner Bros and making sure that the creative shops knew how to use Motif AdKit to its fullest capacity. Though I wont go over all the details of the meetings, WB and the creative shops we met with are now further on board with Motif.

Aside from the meetings, however, Monday night we took Warner Bros out to dinner at a restaurant called Japonais (http://www.japonaischicago.com/ this is the Chicago version...the NY one is similar).

IT WAS SOOOOOOOOOOOOOOOOOO GOOD!!!!!!!!! First we had about 10 rounds of Sushi and Sashimi. Nothing beats a master sushi chef with fresh out-of-the-water-fish.

But the highlight of Japonais was the Kobe Wagyu style Sirloin steak. I nearly died of happiness when I took the first bite. Being from Texas and all, I really thought that I knew what good steak was. Well, those red meat eating Texans could learn a thing or two from the Japanese that's for sure!

Oh and I also saw a small fraction of Central Park that afternoon. We also made a whirlwind tour through MoMA (the museum of modern art).

Waking up a bit tired on Tuesday I proceeded to some more meetings with WB and their creative shops. Again we presented our material, the creative shops listened and the meetings adjourned without any wrestling matches.

After the meetings I had the pleasure of meeting Fan Zhang and Garrett Cronin in person for the first time! Fan was kind enough to take me around the office and introduce me to all the Adapt folks there. I even learned that in fact there are two Theo's that work at DoubleClick! Luckily I work in Colorado and the other Theo is New York.

Another dinner with WB and another 15 block walk back to the hotel. I decided that I would take as few cab rides as possible in order to prevent all those appetizers and desserts from going straight to my 'Buddha' (my own slang for my ever more rounding belly).

As I headed for the DoubleClick office Wednesday morning I stopped for a bagel and put a check mark next to another New York stereotype: awesome Bagels!

Lunch found Garrett, Fan and I eating fish products and banana pudding at a little restaurant in Greenwich Village called Mary's Fish House. It was very delicious! Our conversation turned to marking out the differences between Colorado and New York. And the difference is....diversity (well that and mountains, way less people, clean air, real estate that you can afford and not be the size of a shoebox...etc)! New York's vast range of ethnicities, cultures and languages is really cool. I think I heard at least 5 different languages as I walked along the streets.

Finally it was time to go. I got my bags, walked out to the street put up my hand and was soon picked up by a Taxi who then drove me at 1000 mph to LaGuardia Airport. It was like being in an F-16 fighter jet! So now here I am back in the Rocky Mountains, with fresh air, clean streets and not a honking horn or siren to be heard. Home sweet home!

Entrepreneurship Or What do I Know About Anything Anyways

Entrepreneurship

To become a successful entrepreneur it is imperative that one adhere to the following rules....

Okay, I'm just joking about that. If there were a set of rules to follow to become a successful entrepreneur than every body would be one. Who wouldn't want to be the guy in charge?

So what does it take to run a successful business? In the end, I really couldn't say. I've never run a "successful" business. However, in my own defense, I have run some nearly successful businesses which I believe were limited only by my own lack of stick-to-it-iveness. In the end I think that this may be the real key to any successful business: sticking to it, and always remaining optimistic.

A Short History of My Entrepreneurial Career

My very first business ever was a lawn mowing business named, "The Lawn Boys". My friend down the street had a riding lawn mower. My father had the desire to get me out of the house for the summer (and for me to become financially independent at 13). And last I had the desire make a little money to buy a sweet new guitar! At our peek we pulled in 50 dollars a week! Alas in the end, neither my buddy nor I could get psyched about lawn mowing....and we eventually threw in the towel.

My second business was a band aptly named Theology. Being a rock and roll musician required much dedication to practicing, getting gigs, and songwriting. My good friend Will and I were dedicated to all three. But unfortunately it became increasingly hard to find a solid and reliable bass player and 2nd guitarist. Soon rock and roll lost out to college study. We did however produce a very solid and nice sounding album on an old computer and a super cheap sound card.

Near the end of college I started another business with my good friend Will. This time we used our brains instead of our ears to make money. We started a small IT business to build websites and construct networks for small businesses. Things were going well, but unfortunately I moved to Colorado to learn about very advanced topics in Mathematics.

My last business was about to get off the ground and rolling, when my good friend and business partner Ray moved back to China. It's still possible for us to do some business, especially as the developed countries turn to the underdeveloped countries for outsourcing. We still exchange emails and ideas every now and then. It may take a while, but I think Ray and I will eventually be able to pull off a transcontinental business.

Currently, like a crouched tiger, I am waiting for the next opportunity to present itself. There are many possibilities, but few real opportunities. And so I'm waiting, thinking, and of course....blogging!

Though there is much to be said about starting a successful business, I have more questions than answers. But that is one of the main reasons I have decided to blog. Perhaps writing my ideas and questions out will bring constructive criticism and also clarify and sharpen my vision. Who knows? Perhaps the next business I start will be less of a warm-up/practice run and more of a multinational conglomerate! :-)

Stock Market Analysis Tools

Stock Market Analysis Tools

For the last 5 months (basically since I finished my Master's thesis) my stock market prediction software has been on hold. I was making great progress on it and thought that I had finally cracked the problem last September. However after running a real time trial that essentially paper traded all the stocks on the S&P 100 on a daily basis, my theoretical accuracy of 60-70% prediction rates turned out to be a bit of a wash. In short, after three months I neither made money, nor lost money. But I did spend about 1000 dollars a day in trading fees (err... virtual dollars that is).

The upper line is my theoretical profit on the stock CAT using predictions...the lower line is the value without predictions. The triangles are predicted short sells, while the circles are predicted long buys, see why I was excited?

I racked my brain and poured through my code. After a month of searching through the code and comparing the real time predictions to the simulated predictions my conclusion is that my framework for building and testing my learning models is sound. However the data that I was using to train the models was not. Tracing it down even further I began to realize that the functions I had written to compute the technical analysis indicators (see note 1) were incorrectly programmed and brought inconsistencies to the data. Thus the data for the training sets was inconsistent with the live data... my models were learning Portuguese while the stock market was speaking Spanish.

Not wishing to rewrite all those functions again I set out looking for some open source software to compute the technical indicators. Amazingly, I could not find a single Matlab, Python, Java or any language package that did such a simple task. I put it on the back burner and pursued other things for a while.

But all is not lost! Yesterday while searching through packages on the statistical computing language R I finally found one that did it! So now the task is for me to write some functions to take my data, use the preexisting functions in Rmetrics to transform it, and then rerun it back through my prediction engine....and start another real time trial! The website that this cool package can be found is... Rmetrics .

Enjoy, I know you all are just dying to try out both R and Rmetrics!

Note 1 - Technical Analysis Indicators are nothing more than non-linear transformations of the price data which are meant to bring out certain aspects of the price. For instance, a 200 day moving average indicator attempts to show a trend in the data using a 200 day smoothing transformation (actually just an average). For SVM's (the basis of my prediction algorithm) non-linear transforms are like glasses...they really bring clarity to an otherwise very fuzzy data set.

Sahara Desert Not a Viable Source of Solar Power

Sahara Desert Not a Viable Source of Solar Power

In one of my last posts I talked about the math behind using solar power. In the end I concluded that the Sahara Desert could power the earth 211 times over. After sending out this link to a lot of family and friends the number question/objection I received back was, "But putting solar panels in the Sahara Desert is really impractical." I would first like to say that the Sahara Desert was just a comparison point to show that even with a small fraction of the available land mass (all continents have unused deserts) we can power the world.

Now it is quite true that putting a lot of solar panels in the Sahara Desert is very impractical. As my Uncle Steve said, "Who the heck are you going to get to work in the middle of the blazing hot desert?!" And as my sister said, "But how are you going to get all the power all the way back to America?"

But this brings up a very interesting and critical point: this is exactly what we are doing in Iraq, Iran, Kuwait, the UAE and Saudi Arabia this very day. There are apparently many people willing to work in the blazing hot deserts to set up oil wells, and refining stations. Furthermore we are shipping millions of barrels of the oil all the way across the ocean in giant ships. I will also point out that several ships have unfortunately crashed or been sunk in the recent past releasing their cargo onto the beaches of Spain and Alaska (to name a few) creating some of the worst ecological disasters on record.

So even though I do believe that covering the Sahara in solar panels is a bad idea, I also think it's paradoxical that most of the public would react with a similar objection, "It's just too impractical!" We are already going to greater trouble to get oil out of the ground than it would take to install panels in the Sahara and run cables across the Atlantic. Clearly, the best solution is to install high efficiency solar capturing devices on businesses and homes across the world thus eliminating the need for a large scale electrical grid and placing the "power" back in the hands of the people.

update: Since posting the original article 5 minutes ago I read the comments from my friend Ray who says that in China they have cool technology such as: electic bicycles, solar hotwater (he hasn't paid a dime for hot water since he moved there he says), and toilets that flush with the option of 1 liter (for a #2) and half a liter (for a #1). Go China!

A False Dilemma Between Water and Energy?

A False Dilemma?

This article in the Christian Science Monitor points out the growing concern in the US Southwest about water shortages. While droughts in the desert are nothing new, the concern is that water usage by electric utilities accounts for about 60 percent of all non-farm water usage. In a region that is growing by leaps and bounds this could be a problem. As more people move in, more electricity is demanded, thus more water to produce the electricity is required. But let's not forget that we drink water and bathe in it, wash our cars and water our lawns with it too.

So how do we produce enough electricity to maintain our lifestyles while keeping enough water available to drink, shower, bath and keep all the little animals who also need water happy? Well, if you read my previous post, you will know my opinion. Solar power! The sun provides us very close to free energy. All we have to do is catch it!

The Mathematics of Free Energy

The Mathematics of Free Energy

Some would note that I am a bit idealist about getting off the grid and Green energy. Some might say it's impossible and rather impractical. However, I believe that the mathematics behind it really "adds up". Here's a look at the arithmetic behind it all.

First the energy usage of the world per year: from ecoworld.com we have the following estimated energy usage in 1995:

316,000,000,000,000,000 BTU = 92,614,302,461,899 KW hrs

The math: The sun pumps out 1.3 KW hrs/m^2 of energy in outer space. The actual sunlight hitting the earth's surface varies throughout the year but let's say it averages 40% of the outer space value (which is a little on the low end). The earth has a land area of about 148,939,100 km^2. If we can harness that energy at 50% efficiency (high for existing technologies), then the following tells us how much energy we can catch in one hour on average.

1.3 (kW hrs/m^2)*1000000 (m^2/km^2)*148,939,100 (km^2) * .50 (efficiency) * .4 (amount reaching earth) = 38,724,166,000,000 kW hrs

More simple math shows that in only 2 hours the earth receives enough energy to power it for the rest of the year.

However let's say that we wished to avoid covering the whole earth in solar panels (after all I do like the green grass and nice trees...the minimum land area that we would need to produce to catch the energy required in one year would be:

Lmin = 92,614,302,461,899 KW hrs/(1.3 (KW hrs/m^2)*1000000 (m^2/km^2)* .50 (efficiency) * .4 (amount reaching earth) *24 (hrs/day)*365 (days/year)) = 40663 km^2

Thus with only .02 % of the earth's available land (or presumably oceanic) surface we could generate enough energy to totally power the earth for a year. Even if you factor in an extra 50% energy usage growth since 1995 and another 50% for future usage we would still only need to cover less than .05% of the earth in solar panels.

To put this in perspective, covering the otherwise unused Sahara desert (86,000,000 km^2) with solar panels would generate enough power each year to power 211 earths.

The proof is in the numbers, there is absolutely no reason to keep paying for oil, nuclear or anything else. Let's put up some solar panels, or windmills or invent something even better!

Why I want to Live off the Grid - Part II, Not Living in the Stone Age

Why I want to Live off the Grid - Part II, Not Living in the Stone Age

When it comes to the practicalities of living off the grid there are many options, and just as many enthusiasts willing to help. The Internet is home to a growing community of DIY green enthusiasts who's advice the off-gridder will most appreciate. A Good starting place is Treehugger.com.

Thoreau's explains in his classic book Walden that the bare minimum a person needs to live a comfortable life is enough food to survive, clean water and a basic shelter that provides protection from the environment. All else, he says, merely allows someone to die in excess comfort. To Thoreau's requirements I add that it is imperative to have an energy resource. For off-grid purposes it must be renewable and as independent as possible.

But where to start? Let's start with a basic inventory our modern lifestyle.

Electricity - Electricity the activity of electrons that is wired into our houses at 110-120 Volts, and alternates current at a rate of 60 times per second. In England and most the world, electricity is set to 220-230 Volts, and alternates current 50 times per second. Solar Panels generally produce electricity at either 12 or 24 Volts and does not alternate current. In order to transform Solar Powered electricity to household electricity an inverter is necessary. Inverters are complicated, see wikipedia.org. Interestingly electrons in electricity move at the speed of light, however the common notion that electricity "flows" through your house is actually incorrect. Electrons can drift through the wires, but for the most part actually stay put in the wires. More or less electrons actually sort of vibrate around with greater or lesser intensity.

Water - Clean water is a necessity. Imagine trying to make a good cup of coffee with muddy water...bleh! Aside from the clarity of water it is imperative that it contains no cysts, bacteria, fecal matter, protozoa or other dangerous and alive organisms. Water is still pumped into our houses the old fashioned way, pipes and valves from a main source. However, if you live in a house off the water grid you will be responsible for pumping it out of the ground, or purifying it from rain...or if the conditions are right collecting it from the air via fog collectors !

Heat - House heating is usually accomplished via fossil fuels. Generally fossil fuels are burnt as efficiently as possible and the resulting warmth is blown throughout the house. Lately there is a resurgence of wood and wood pellet stoves for heating a house. Aside from being more earth friendly than fossil fuels, they are much more rustic and Romantic!

Insulation - Our houses are insulated well..keeping the hot air in in the winter and the cool air in in the summer. This is a key ingredient to a comfortable house.

Air Conditioning/Cooling - In the summer most houses get really warm, thus some form of cooling is required. In Dry Climates evaporative coolers use the principle that hot dry air can be made cool be absorbing the heat in water. In more humid climates Freon is expanded and contracted in order to capture heat from the air; our vehicles use this method as well. There are more methods for cooling air including acoustic cooling!

Food - Most of the food we consume is produced en mass from farms far away from our houses. Food however can be grown efficiently and with much benefit in our own back yard!

Transportation - Nothing defines America like cars. We love cars and we love going places. And we love cheap gas. Unfortunately some of that may need to be rethought as gas prices keep on climbing. Interestingly, the largest producers of oil in the middle east are some of the largest investors in renewable energy...hmmm....interesting.

Appliances - I love appliances. I love to be able to wash my clothes, and dry them quickly, wash my dishes with little effort, put 2 cups of raw rice in the rice cooker and have steamed rice 20 minutes later. I love being able to have cold ice-cream in the refrigerator. I love not having to plug my computer in to use the Internet. Unfortunately all appliances especially washers and dryers use incredible amounts of water and electricity.

Communication - The Internet is perhaps the best invention in Modern History. In fact, the guy who invented packet switching, which makes the Internet possible, was recently inducted into the inventors hall of fame. All devices that enable us to talk and communicate of long distances are not only incredible, but incredibly useful.

Entertainment - Nothing says humanity like entertainment. In a way, entertainment is the lifeblood of our culture. Entertainment is really just story telling. And story telling tells us who we are and what we are doing.

These are the basics of our modern lifestyle. Certainly one could find many more essentials to our modern life, but without the above, we would hardly recognize our lives as modern.

In the next few articles I will explore each topic in more detail and offer some practicle suggestions.

Links to the first two entries in this series

Why I want to Live off the Grid - Part I, Interdependence
Why I want to Live off the Grid - Introduction

Why is there something, instead of nothing?

Why is there something, instead of nothing?

Philosophers are always pondering such questions like why is there something instead of nothing (thanks Jason Fraze for the question prompt) I don't think that this is really an answerable question.

However I would like to ask the question is there such a thing as nothing? In other words is there a place in the universe that is totally devoid of an existent thing?

After spending much too long on the subject I have concluded that no, nothing cannot exist. (In my proof I will prove that non-beings cannot exist, nothing is a subset of the non-beings set).

Proof:

Suppose that there are actually two possible classes of things...beings and non-beings. Assume that a set of non-beings exists. These non-beings are defined in that they are the compliment of the set of all things that are beings (In other words non-beings have no existence). But the set of non-beings is definable. And all things that are definable have a reality (however limited their reality is). But this is a contradiction in that non-beings must not have a reality for they are the compliment of all things real. Therefore by the law of non-contradiction, a non-being, or a nothing cannot be said to have existence or reality.

So what does this mean in practical reality? I'm not sure, but it brings a few thoughts to my mind that there cannot be such a point in history that nothing existed. So at the very least the Big-Bang theory falls short in explaining the universe. For even prior to the Big Bang, there was something. And before that there was something, so on ad infinitum. What is the answer to this "infinite regress" as Aquinas called it? God.

Why I Want to Live off the Grid - Part I, Interdependence, or how the TV is the Devil

Why I Want to Live off the Grid - Part I, Interdependence, or how the TV is the Devil

In High School I quickly became a teachers pet for a paper I wrote called "Television: The Work of the Devil?". Most people who know me, know that I am a well-meaning Christian, and I must first say that this paper was not about the moral evils of television content (though indeed I do find most television content unsavory). It was rather a look at how television has influenced our culture, our selves and our sense of well being. But first and foremost it looked at the ways in which television is a mind and creativity destroyer.

Being on the grid necessarily implies dependence on a larger framework for sustenance. Now to be clear we must reaffirm that one can never be fully independent. No man is an island to quote John Donne. In fact let's review his poem here; it speaks volumes about the very core of humanity.

"All mankind is of one author, and is one volume; when one man dies, one chapter is not torn out of the book, but translated into a better language; and every chapter must be so translated...As therefore the bell that rings to a sermon, calls not upon the preacher only, but upon the congregation to come: so this bell calls us all: but how much more me, who am brought so near the door by this sickness....No man is an island, entire of itself...any man's death diminishes me, because I am involved in mankind; and therefore never send to know for whom the bell tolls; it tolls for thee." - Meditation XVII

Knowing that we are all interdependent and connected means that in some way we have a very sincere and natural need for human interaction. How sad is it then that the average American spends about 5-7 hours watching television a day? In fact, if we sleep for 7-8 hours and work for 8-9 hours, that leaves us with about 1-3 hour of totally free interaction with our friends, our spouses, people we don't know or haven't yet met, much less our grandparents and other distant relatives. To put this in perspective, 5-7 hours a day of television watching a day works will end up being 20-30% of one's lifetime.

At this point, it strikes me that when we talk about off the grid living, we are really talking about a way to become more interconnected with life, humanity, and the environment. An off the grid lifestyle is paradoxical then in that our images of an off-the-gridder is a solitary person in a cabin, by himself, who is kind of nutty and has a very long and disgusting beard (if he is a male of course). However, on the grid people seem to live in nice suburbs with all the modern conveniences and are quite content with the status quo. Thus the paradox is that not having to depend on the modern infrastructure (independence) will lead us to greater interdependence (an independence conditional on other more immediate human beings). However the dependency on modern conveniences has led to a great deal of Independence in the on-grid person...so much so that they often lead lives of quiet desperation and loneliness.

I believe that the first step to getting off the grid of mental dullness, the grid of contentment with the status quo, the grid of normality and blandness is to get rid of our televisions...or at the very least to get rid of cable television and to remove the television from the prominent spot in our living rooms. I think that real life has enough challenges, adventure and mystery to never get bored.

I hope in the next few posts about getting off the grid that we will begin to further understand what it means that get off the grid, and why it is the best thing we can do. We will further understand that becoming independent will lead to interdependence because, in essence, getting off the grid means changing our dependencies to a much more local set of people...something known as interdependence.